Release Date for version 4.0 ?

Since we get this question daily, we made this post a sticky so that we can provide some status updates over the coming weeks to where v4.0 sits. If you have v4.0 questions, post them here.

Please note that we do offer annual subscriptions, meaning you can purchase a v3.1 license with subscription now and you will be sure to get v4.0 for free when it comes out.

Ok, onto the current status. We are currently, and have been for several weeks, exclusively been working on v4.0 development. We are building a complete framework for advanced programmers to do some sophisticated parsing of languages. In addition to that, v4.0 has knocked out a huge chunk of the accumulated TODO list items from previous versions.

While implementing code for v4.0 language enhancements, we have been prototyping a C# language add-on. The add-on is an encapsulated language meaning that you can simply set it to your document, set up another couple classes that work with the language, and you instantly have IntelliPrompt with very little code on your end. This has taken weeks of development alone, but in creating a complex language add-on like this, it has helped us develop and refine our parsing framework. Until the public beta is released, details on this will not be released other than various hints that have been posted in the forums. We have IntelliPrompt working for C# but there are some things with it that still need tweaking.

Most of the major core new features (from our TODO list) have been implemented but we might still want to tackle a few more. We also still need to get working again a couple v3.1 features that were broken when writing the v4.0 code.

In response to requests for more documentation/samples, this past week we started on adding some quickstart samples. These samples cover various feature areas (indicators, line colors, some other new features, etc.) and make it easy to learn how to use those features since the code is in a small form and you don't need to search through the mammoth MainForm to figure out how things were done.

We have not started on documentation for v4.0 yet since the object model has been WIP up until recently. Once some more tweaks are done and the code is made a little more stable, we will begin on new documentation.

Since v4.0 adds a lot of great new core features as well as a framework for building advanced languages, we feel that a lot of documentation is needed for the advanced languages area. Therefore we will not release a beta until we have some documentation prepared that goes over key concepts.

We have also created a language called Simple which uses a very basic C-like language syntax and only supports several keywords, operators, and functions/function calls. The full source code is in the sample project and it demonstates making a programmatic lexical parser, building a semantic grammar with a complete AST object model, IntelliPrompt implementation, showing the object model in a document outline TreeView and more. This is an excellent sample to get you started with an advanced language.

There will be a public beta once things are a little more refined and some documentation is complete. We already have some customers using an early private beta and they all love what they see so far.

So that's where things currently stand. Again we are working on this non-stop to get it out the door. We'll continue to post here every once and a while until the release so you are advised of the status.

Is the new parser static enough to where we could start writing our own languages yet?

Also, is the new semantic parser using antlr or a variant of it? We already have an antlr implementation for our language so I was curious if we could possibly use that under the cover or if we would need to redo our language definition?

Thanks,

Ashton Hobbs

v4.0 update... we've been spending the past several days refining the C# IntelliPrompt. The quick info now pulls in XML documentation for all types and members, even the tricky XML doc ID's like with generics, arrays, and byref parameters.

The code SyntaxLanguage class has been refined have provides a great base where you can put all your programmatic code related to a language. While one feature of v4.0 is allowing the writing of programmatic lexical parsers, we have not forgotten about "dynamic" (v3.1 style) language definitions. Now you can specify a custom language class so that when you load your XML language definition it could be loaded into an instance of HtmlSyntaxLanguage class, for example, which would inherit DynamicSyntaxLanguage but could also implement code for doing source code formatting, intellisense, etc.

A major goal of v4.0 is to allow you to encapsulate all your language related code in the SyntaxLanguage class so that when you assign it to a Document, everything just works without any other SyntaxEditor event handlers needed. This has been achieved and it's working great. For our advanced C# language, you set which assemblies to use as external references, set the language to the Document, and then everything including indenting, intellisense, etc. works with no extra code needed.

The past few days we have been tweaking various code areas and knocking out TODO notes throughout the code.

More on new features... SyntaxEditor v4.0 features a complete parser generator framework. It uses a LL(k) algorithm to take your grammar and build an AST node tree from it. While we don't use ANTLR, we have a custom syntax which is extremely robust, probably moreso than any other LL(k) parser generator we've seen. The syntax for the grammar is and has been pretty stable for a while now. We even have a Grammar Designer in the sample project that highlights the grammar for you to help identify where matches are occurring and where user code blocks are. As mentioned in the previous post, we have a "Simple" language example which implements a semantic grammar using our parser generator.

Some other areas of languages might still be tweaked a little but they are fairly stable at this point in terms of changes. Ashton, since you've worked with us before, if you'd like to be a part of the private beta, send over an e-mail.

Yesterday we enhanced our parser generator even more. Now in the grammar file you can have a section like this: This tells the parser generator to automatically build a base AstNode class for your language, an AstVistitor class that implements the Visitor pattern, an enum that has all the node types in it, and a concrete AST node class for each AstNode tag defined. So in the above definition, it will make an abstract AST node class named Statement which inherits from the generated AstNode base class. It will also make an AST node class named AssignmentStatement which inherits Statement. AssignmentStatement will have two properties, both of which are child AST nodes. Note that the Identifier and Expression AST node type definitions are not included in the above code block.

The parser generator builds all of these classes with full XML documentation and has them implement all the core features that an AST node should implement to work with the SyntaxEditor v4.0 language framework.

This is really neat stuff because it will be a huge timesaver for those customers implementing their own advanced language. With a tiny bit of work, you can auto-generate an entire set of AST node classes. Then you can take those generated results and tweak them to suit your needs.

Quote:
Please note that we do offer annual subscriptions, meaning you can purchase a v3.1 license with subscription now and you will be sure to get v4.0 for free when it comes out.

Sure, for an extra $214.00 assuming it's less than a year off. But assuming pricing is similar to v3.1 and that v4.1 is at least a year off, you would be better off purchasing v3.1 now and upgrading to v4.0 when it comes out, right?

In my project, SyntaxEditor is clearly the right way to go and I'm chomping at the bit to make the purchase. However, I feel like purchasing it now would just be throwing money away since version 4.0 will be a large improvement over an already very impressive product.

Have you considered doing something like Microsoft does in the month or two before a new OS is released and offer upgrades to the next version for free or the price of S&H?

Thanks and look forward to using your product,
Jared

There's only several code changes left that we need to resolve before a public beta is ready. Once we get those completed, we'll start working on documentation and then we can release the public beta.

We still plan to implement more enhancements to 4.0 after the release of the public beta and during the beta period, and possibly start on the VB.NET parser. The nice thing is that the work we did on the C# parser uses a common reflection database, which can eventually be used with VB.NET. And the .NET AST object model we've made for C# (although it might need a couple tweaks here and there) can also be used with VB.NET. So really, implementing VB.NET IntelliPrompt will mostly come down using our Grammar Designer to build a semantic parser using our new parser generator. That semantic parser will reuse the AST node classes we already implemented for C#. Then we need to do some code to do VB.NET context determination, which is used for populating IntelliPrompt.

We just completed work on our enhanced highlighting styles. Wait until you see all the options available... It's really cool. Now for a single highlighting style (like for a keyword for example), you can say you want it in a different font family, font size, put a border around it with any line style (solid, dotted, dashed, etc.), do strike-through with any line style, or do underline with any line style. And of course span indicators fully support all these options so you no longer have to draw them yourselves. It's great stuff, especially for apps that do text comparison between two documents.

We've begun working on documentation on all the new features. We also are going to work on reorganizing and enhancing the existing general feature documentation. Once the documentation is mostly complete, we should be ready for the release of a public beta.

Documentation is progressing really well. We're probably doubling the amount of documentation that was included in previous releases. We also are updating and reorganizing existing documentation to make it much easier to find information on the many features found in SyntaxEditor.

Hi

I while back you were talking about a full C# parser that you were thinking of selling separately from Syntax Editor. Can you confirm the details of this, i.e. will this be source code or just a 'plug-in' pre-compiled component?

While the component will undoubtedly be useful for a lot of people, the source code would be extremely helpful even for those of us who don't want C#. What this would do is show the other end of the spectrum from your 'quick start' examples, i.e. whats involved in implementing a complex parser properly (I'm assuming this has a significant amount of code with some interesting problems that needed to be solved).

Also, any ideas of pricing yet (Syntax Editor upgrade and C# parser)?

Thanks
Russell Mason

Hi Russell,

These are our plans... we will finally be offering source code for the core SyntaxEditor product via a Blueprint license like our other products. The C# parser (and eventually a VB.NET parser) will be bundled and separately as a ".NET Languages Add-on" product to help keep the price of the core SyntaxEditor product down. We plan on making other language add-ons as well and packaging multiple languages in each.

Each add-on will sell in two variations: with and without source code. The add-on will be pretty cheap and will be an Enterprise license, meaning that as long as you have the proper number of code SyntaxEditor licenses, all you need is one add-on license and anyone in your organization can use it. It also will include a year of free upgrades, like an annual subscription, at no extra cost.

So as you mentioned, we make it possible for you to have the full add-on source at a reasonable price and if you do choose to get it, we hope that you might contribute back any ideas/code to improve it for future releases. The benefit of contributing some of your own code is that if we include it, then you don't have to re-merge it with every new release of the add-on product.

While there is a "Simple" language add-on with full source included in the sample project in v4.0 to help you get started, the C# implementation in the .NET Languages Add-on is MUCH more complex. I believe at last count, the C# source files for the .NET Languages Add-on had over 37,000 lines in the code files.

As for pricing, the core SyntaxEditor prices will be going up a little bit. As mentioned the add-on will be very cheap considering we spent months of work building the entire infrastructure (one of the main reasons v4.0 took so long), and it is an Enterprise/subscription license. We're thinking +/- $300 for the binary version and +/- $700 for the binary/source version.

We are pleased to announce the new feature summary for v4.0. This is a very high level summary so in reality there is much more to the new version but this gives a general idea of what's implemented for the upcoming public beta:

Major New v4.0 Features

• A complete parser generator framework for defining semantic parsers.
• A semantic parser service which can perform semantic parsing in a separate worker thread.
• An AST node class framework for building abstract syntax trees for code documents.
• Reworked the design of syntax languages so that they are much more abstracted and can totally encapsulate all the code for handling IntelliPrompt, smart indent, code formatting, etc.
• Virtual space past the end of lines.
• Virtual space past the end of the document (virtual lines).
• Block selection and related features such as controlling the types of selections that are able to be made.
• Read-only regions of text within an editable document.
• Text statistics that scan text and give statistics such as word, sentence, character counts, readability scores and can display the results in a Form. Languages implementations can create custom statistics such as comment coverage.
• IntelliPrompt smart tag features.
• Improved parameter info features to aid in selected parameter tracking and context switching.
• Ability to suspend lexical parsing when a batch of document modifications are made.
• Ability to disable lexical and/or semantic parsing completely, which is useful for working with extremely large files.
• A Document.PreTextChanging event that lets you cancel a document modification before it occurs.
• Various whitespace-trimming methods and options to trim trailing whitespace on paste or text replacement.
• Designer features for the Document.Language property, allowing a language to be chosen if there is one available in the component tray or a dynamic language to be loaded from a file.
• Support for non-editable code snippet declarations and methods so that functions can be used to update fields (enables dynamically building case statements for a switch on an enumeration, etc.).
• Changed code snippets to only load snippet header information by default for saving on memory. The full snippet data is loaded on demand when the code snippet is activated.
• Automated line modification marking and enhanced how it tracks modified text ranges.
• An improved object model for span indicators that uses layers.
• Detailed hit-testing for anywhere in the editor.
• Custom EditorView buttons that can be placed on any side of a scrollbar.
• A new renderer interface that lets you override how SyntaxEditor is painted, alter the colors used for the default drawing scheme, or perform custom draw.
• Multiple font families/sizes in the same document.
• Highlighting style enhancements for semi-transparent backgrounds, borders, strike-outs, underlines, and more.
• Ability to modify the caret style and width for both insert and overwrite modes.
• Over 40 new built-in commands.
• An enormous amount of object model enhancements.
• Dramatically reduced memory usage.
• Sample project enhanced with a Grammar Designer and many QuickStarts.
• More than double the number of core documentation topics.
• Full source for a sample language add-on in the C# sample project.
• An advanced C# language add-on that has automated IntelliPrompt, smart indent, XML comment auto-complete, and much more. (Sold separately)

Minor New v4.0 Features

• Ability for bracket highlighting brackets to be bold.
• Code snippet field modified event.
• Option for current line highlighting to extend to all margins.
• Option to cut/copy text with HTML formatting.
• Built-in Find/Replace and Go to Line dialogs.
• Unified all mouse click and hover events into two more robust events.
• Drop-shadows option for all IntelliPrompt popups.
• IntelliPrompt popup ability to show images and handle link clicks.
• Carriage return-only line terminated text is supported.
• Custom document line border colors.
• Ability to create IntelliPromptMemberListItem subclasses and measure/draw the items yourself. This allows for you to draw things like muliple columns, multiple images, etc.
• Total page count calculated for Print dialog.
• Regular expression engine support for zero-width look-ahead and look-behind.
• Regular expression find support for zero-width matches such as $.
• Regular expression replace substitution $0 works like $&.
• When clicking on the cross-section of a four-way view split, the splits may be resized in both directions.
• Many other minor features.

Just curious: Do you think you can deliver the 4.0 beta this week? Sorry for nagging about that but I am on a very tight schedule and need the parser desperately.
And thanks for the feature list.

We are certainly trying to do so. We have all the maintenance releases for our other controls (including v3.1) ready to go with a new Shared/WinUICore library update that is needed for v4.0. The v4.0 documentation is more or less ready. We just are working on fixing several things we discovered while documenting. Then it will be ready.

Hi,

Very much looking forward to trying this new build. Will there be a link to download from of do we need to contact someone to be able to test?

For me, I'm still using 2.5 - can this version be upgraded from?

There will probably be a posting in this forum for it. Upgrade pricing will apply for any previous version, even 1.0. There are numerous breaking changes in the v4.0 release but all of them are documented in detail in the Release History and more general information about breaking change areas are in a long help topic in the documentation.

Great - thanks! Can't wait to take a look

We're going to try and get the v4.0 beta out sometime today, hopefully in the next several hours.