Shy Beginner (looking for a starting point)

Hi Richard,

Thank you for your comments.  We realize that as a product as complex as SyntaxEditor grows and grows in features and functionality, it can get difficult to make sure the documentation and samples are still approachable for new customers.  We are happy to work with you to adjust our documentation so that there is a better onboarding experience.

Our SyntaxEditor Getting Started series of samples (in the main samples project under /ProductSamples/SyntaxEditorSamples/QuickStart/GettingStartedXX) is intended to be a step-by-step introduction to building a feature-rich syntax language.  It starts off with basic syntax highlighting and adds features like parsing, code outlining, automated IntelliPrompt, etc. as each step progresses.

What enhancements to the documentation would help most to get you pointed to that Getting Started series of samples so that it's more clear on how to get going?  In the meantime, feel free to ask questions if you can't find related documentation or samples and we can answer them.

Thank you very much for your reply.

Either by pure coincidence or by my «complaint» triggering something in my mind, few minutes after posting my message something «unlocked» and I succeeded in writing a very elementary project (bare-bone editor, «Simple» language loaded in it, completion list popping up when typing «this.», as instructed by your manual). This strongly feels like a beginning, after a long «frozen» time spent trying to find a thread to pull. Before posting my next question I’ll make sure it’s not already clearly (to me) answered in either your documentation or samples.

 Best regards. R.

Good evening.

Preliminaries:

I have faithfully copied a snippet shown in SyntaxEditor’s manual ( Text/Parsing Framework - Tagging --> Taggers and Tagger Providers --> Sample: ITagger<ISquiggleTag> Implementation ), which I repeat here:

public class CustomSquiggleTagger : TaggerBase<ISquiggleTag>
    {

        public CustomSquiggleTagger(ICodeDocument document) : base("ActiProPatternBasedSquiggle", null, document, true) { }

        public override IEnumerable<TagSnapshotRange<ISquiggleTag>> GetTags(NormalizedTextSnapshotRangeCollection snapshotRanges, object parameter)
        {
            if (snapshotRanges != null)
            {
                foreach (TextSnapshotRange snapshotRange in snapshotRanges)
                {
                    string text = snapshotRange.Text;

                    MatchCollection matches = Regex.Matches(text, @"My Test Sentence", RegexOptions.IgnoreCase);

                    if (matches.Count > 0)
                    {

                        foreach (Match match in matches)
                        {
                            SquiggleTag tag = new SquiggleTag();
                            yield return new TagSnapshotRange<ISquiggleTag>
                                (TextSnapshotRange.FromSpan(snapshotRange.Snapshot, 
                                snapshotRange.StartOffset + match.Index, match.Length), tag);
                        }
                    
                    }
                }
            
            }
        }

    }

 I have replaced your test string @"\bActipro\b" with @"My Test Sentence".

My question:

How do I effectively «glue» (is «register» the correct name for this?) my editor (a SyntaxEditor control instance named myeditor on the WPF form) to this CustomSquiggleTagger object, so that typing «My Test Sentence» actually squiggles it?

Further information:

This is what happens in MainWindow.xaml.cs:

public partial class MainWindow : Window
    {
        public MainWindow()
        {
            InitializeComponent();

            // the method defined below is called
            InitEditor(myeditor);

        }

        static void InitEditor(SyntaxEditor myeditor)
        {

            SyntaxLanguageDefinitionSerializer serializer =
                new SyntaxLanguageDefinitionSerializer();               

            ISyntaxLanguage language =
                serializer.LoadFromFile("Simple-Basic.langdef");        

            myeditor.Document.Language = language;                      

            
            myeditor.DocumentTextChanged += Prompter;        /* Prompter(s, e) is a method I use
							     to show a completion list */
        
        }
}

 Thank you very much for your attention, kind regards.

Hello Richard,

Please check out the language class in the AdornmentsSquigglesIntro QuickStart in the samples.  That shows you how you can register a tagger like this with the language.  Then any SyntaxEditor that uses that language on its document will get the tagged used.

I followed your advice and everything worked perfectly. I found inspiration in this line:

this.RegisterService(new CodeDocumentTaggerProvider<CustomSquiggleTagger>(typeof(CustomSquiggleTagger)));

 (courtesy of the CustomSyntaxLanguage.cs file included in the project AdornmentsSquigglesIntro you pointed out)

Hence in my MainWindow’s code I added:

language.RegisterService(new CodeDocumentTaggerProvider<CustomSquiggleTagger>(typeof(CustomSquiggleTagger)));

and now every occurrence of the string My Test Sentence gets squiggled as soon as it’s typed.

Thank you very much for two reasons: 1) your prompt and enlightening help in this specific issue; 2) the overall increasing confidence I’m gaining in what I can find in the already-made sample files, with my «sight» becoming clearer while I peruse them.

Gratefully, R.

Great, you got it exactly right.  We tried to design SyntaxEditor so everything is more or less done in syntax language services that are registered.  That way the syntax language is portable and can be used in multiple SyntaxEditors at the same time, bringing all the functionality with it.

We try to make a QuickStart for every feature area that focus specifically on some small-ish task in that area.  They can be very helpful starting points for various feature areas.

Good evening. My question is about the «3b - Language Code Generation» sample code. I’ve perused those files but I cannot actually spot a simple detail, i.e. where the editor knows that its language is going to be the Simple Advanced Language. Whilst in the previous sample we can find

editor.Document.Language = SyntaxEditorHelper.LoadLanguageDefinitionFromResourceStream("Simple-Advanced.langdef");

  in MainControl.xaml.cs, in this sample this statement is omitted in the same file.  Unless I’m very blind, the only anwser I came up with is the following. MainControl.xaml.cs has the following property:

<editor:EditorDocument.Language>
   <sample:SimpleSyntaxLanguage />
</editor:EditorDocument.Language>

and, in the same namespace, the generated

public partial class SimpleSyntaxLanguage

can be found, and the editor automatically detects it. Is this correct?

Thank you very much for your attention.

Hi Richard,

Yes, that's it.  In the 3a QuickStart, we have the editor use the syntax langauge from the .langdef file.  In the 3b QuickStart, we had the Language Designer write out code-behind files instead of a .langdef file.  This allows us to have a type for our language (SimpleSyntaxLanguage) and to be able to reference it directly in XAML, which is handy.

The "sample" XML namespace is defined as:

xmlns:sample="clr-namespace:ActiproSoftware.ProductSamples.SyntaxEditorSamples.QuickStart.GettingStarted03b"

Which is the .NET namespace where that language is.

Good evening, thank you very much for your previous answer.

My next question is related to this topic:

Language Designer Tool > AST Nodes Configuration Pane > AST Node Configuration > Adding AST Node.

There’s no practical example of a concrete entry, hence I don’t understand how, through this wizard, the created node’s properties relate to other elements in the current language’s grammar, such as classification types. After loading a predefined language (C# in my case) I tried to create AST Nodes with both arbitrary names and names from the classification types list, and the only properties I can set are ID, Key and Inherits. Either way, the «Properties on selected AST Node» pane remain blank.

In this topic:

Text/Parsing Framework > Parsing > AST Nodes > Type-Specific AST Nodes

an example of default AST node is shown, then, without showing the alternative type-specific node (just «ClassDeclaration»?), it is explained that it would be achieved by the following class:

​public class ClassDeclaration : AstNodeBase {
    public string Name { get; set; }
    }

But this «naked» example — which assumes some «ClassDeclaration», maybe as classification type — is another piece among missing pieces of information. In the same paragraph it is also stated that the Language Designer is supposed to take over this class developing job, thus it’d be interesting to see an «extended» version of the previous example, specifically how the AST Node generating this class would look like in the designer.

The issue I’m trying to highlight (which is, of course, also an issue about my understanding of AST Nodes) boils down to:

1. Could you please give me an example (with any language, e.g. the Simple Language) of AST Node entry in the Language Designer?
2. In order to increase my understanding of AST Nodes, could you also give me an example of inherited AST Node in the Language Designer?
3. What exactly the «Properties on selected AST Node» pane is supposed to show?

Thank you very much for your attention and support.

Hi Richard,

First, classification types are related to tokens and help map how syntax highlighting should be applied.  Classification types don't really related to grammars.

When the top node declarations list is focused in the "AST Nodes" tab, the Properties tool window will show Id, Key, and Inherits for the selected nodes.  You generally leave Id blank.  You want Key to be the class name generated for the node.  And Inherits lets you have one AST node inherit another one.  To better clarify this, we will add this verbiage to the documentation topic section you referenced:

For instance, if you are building a set of expression-related AST nodes for a syntax language, you might start with adding an Expression base AST node. Then add other AST nodes with keys like LiteralExpression, ParenthesizedExpression, UnaryExpression, and so on and set each of their Inherits values to be the base Expression AST node. That way, any productions in your grammar that deal with general expressions can return an AST node of type Expression and any of the expression implementation AST node types can be returned since they inherit Expression.

The greatest benefit from your kind answer is mainly understanding that:

1. I have to study more (of course: that’s the required background for well-posed questions against wrong assumptions);
2. I’m maybe seeing the light about one point (one more wrong assumption envolved): your Language Designer has nothing to do with the construction of a grammar and its productions, except for the AST Nodes, objects that a grammar may use.

Thank you very much for your patience and help.

Hi Richard,

Correct, the Language Designer helps code generate language services for lexers / syntax highlighting, and type-specific AST nodes that can be used with a grammar.  But it doesn't help code the grammar itself.  It does have a debugger that can load up a LL(*) Parser Framework grammar and can help debug problems with it though.

Good afternoon. The following question is about your Language Designer Tool, Lexer tab. Using the Simple-Advanced language project as my starting point, I tried to add a new lexical state (the original states are single comment and multiline comment) such that, if the user types a string like this in the editor:

:    word    {

(hence colon + whitespace + word without spaces + whitespace + opening curly brace)

then « word » should receive a special formatting, namely a different color and an italic typeface. For example, this is what Sublime Text editor does when, editing a C# file, we add inheritance to a class name (class or interface inheritance).

I thought that the following setting could be one way to go (I’m learning!):

Lexical state definition

But, alas, when I try to build the language, prior to live testing, I get this:

Exception when building

Something is very not welcome in my «lookaround» patterns (assuming they are correct as regards the desired result!). Can I ask you a few nuggets of wisdom to diagnose this situation? A (compressed) copy of my project is here . 

Thank you very much.

Hi Richard,

Thank you for the detail.  Several things here...

1) While the regex had an error, this particular regex exception slipped through our exception handling and caused the app to close.  We're fixing that for the next maintenance release.

2) The start pattern group's look-behind should be ": {Whitespace}".  Note that since we use "{...}" to delimit macros, they also are used for regex ranges.  Since there's no space after your ":" in your pattern, it thinks it's a range instead.  Adding a space in between should allow it to parse.

3) The end pattern group's look-ahead should be "{Whitespace} \{".  Since "{" is a special character, you need to escape it.

4) The main pattern for each pattern group should be "{Alpha}+" instead of just a single "{Alpha}", which would only match one letter.

I hope that helps!

Dear sir, please consider the very first and the very last line in my message (in every message from me, actually) to be «thank you» for both your kind attention and big help.

Of course, after applying your advice ( here  ), no more crashes occurred and Live Test does indeed reveal that, after typing a colon and a space, the desired lexical state is accessed: here .

By the way, I must have messed up something else in my lexical state definition — something I could not spot before, while the (now fixed) exception stood in my way — which apparently prevents the code from exiting the state. If, after the string « XN12», I type a space and an opening curly brace, I’m expecting to get out of the state, on the contrary coloring and Designer’s tooltips show that my lexical state persists: here .

This state’s pattern group is this (simply anything which is not an opening curly brace). I escaped the curly brace here too, as you suggested.

Do you think that the faulty cog in the machine should be investigated in the two scope patterns or rather in the inner pattern?

Thanks again!

Dear sir, I think I’ve solved the «exiting the state» issue. These are the settings I’m using now and now this happens. Of course I cannot say whether this pattern logic is accurate and perfect as to the desired result (entering and exiting this state and its special formatting) but of course I consider this issue solved. Thank you very much for your attention and support.  

Hi Richard,

Sorry I should have made this suggestion too.  If you are only going to be coloring a single word and you just want to use look-behind and look-ahead patterns for contextual verification, then using lexical states is overkill.  Lexical states want you to consume some pattern to enter the state, possibly consume other patterns while in the state, and then consume another pattern to exit the state.  Which it doesn't seem like you need to do here.

For your scenario, you could probably just get away with a normal regex pattern in your default state.  Add the look-behind and look-ahead patterns to its pattern group.  That will probably do what you need and woudl be much simpler. 

Plus I suspect your last update will fail to leave the state if there's no whitespace between the word and the "{".  You generally want any whitespace to be optional, so using a "*" instead of a "+".

Sir, this is brilliant and of course I hadn’t thought about that: sometimes being too busy mimicking the existing material belonging to more advanced topics causes losing sight of the simpler ideas. Thank you very much.

Good morning. The following question relates to the Language Designer’s LL Parser Debugger and a «minimal coding» scenario. Another summary of my question might be: «How far could a developer go while testing a language without generating the .g. files and sticking to .langdef instead?».

My (hopefully well-grounded) assumptions are:

• A Language Designer session is sure to be capable of testing everything ranging from language definition to syntax highlighting in Live Test. All the information needed in this segment of the general workflow is provided by a .langdef file, if generated.
• There’s no «.langdef» for Ast Nodes: they must take the shape of .NET classes, i.e. they must be either generated or hand-written.
• A grammar must be an hand-written .NET class.

Assuming/hoping the aforementioned statements are true, I’m wondering whether the following workflow is anything near to being realistic:

1. User (say U) defines a language in Language Designer to the point of successful testing in Live Test;
2. Neither langdef nor .g. files are generated by U.
3. U defines Ast Nodes in the Designer and then lets the Designer generate their classes.
4. U codes a grammar. Since there is no class such as SimpleTokenId, as shown in var @addition = new Terminal(SimpleTokenId.Addition, "Addition") , Token Ids are manually input in the Lexer pane ( here ) and used in the grammar, for example: var @addition = new Terminal(3011, "Addition"), where 3011 is an arbitrary Token Id.
5. This grammar uses the generated classes for Ast Nodes.
6. U writes and builds a very basic assembly (let’s say a Class Library project in Visual Studio) containing the grammar class and, of course, the Ast Nodes classes the grammar uses.

With these things in place, would it be possible for U to load this assembly in the LL Parser Debugger so that the language’s grammar is tested? Would it, and yet I’m missing one or more steps?

Thank you very much for your attention.

Hi Richard,

Yes your assumptions are correct.  I think that might work but I would highly suggest letting the Language Designer output code-behind files for you if you plan on going to the grammar step, and having it output a token ID class.  That's always easier to manage later than having to remember random ID values that could potentially get changed, and is a best practice.

The LL Parser Debugger will look in the assembly you point it to for a class that implements ILLParser.  If it finds one, it will create a new SyntaxLanguage behind the scenes with an instance of that parser class as the only language service on it.  Then it assigns that language to a document and attaches the debugger to run.  So to answer your question, as long as you have an ILLParser in an assembly, I believe the LL Parser Debugger will work.

Thank you very much for your help!