Posted 12 years ago by Wesner Moise
Version: 4.0.0257
Platform: .NET 2.0
Environment: Windows Vista (32-bit)
Avatar
When do you plan to support language syntax of C# 3.0 and VB 9.0 in the .NET Addon?

Also, I noticed in the blueprint source some todo comments about missing support for arrays of rank > 1 and multilevel pointers in the AST implementation. Can I assume that the todos are incorrect and that the whole language is supported.

The todo comments are in AssemblyDomTypeReference.cs around line 92, (and reprinted below)

// Get array/pointer info
if ((type.IsArray) || (type.IsPointer))
{
arrayPointerInfo = new DomArrayPointerInfo();
if (type.IsArray)
{
// TODO: This doesn't handle multiple levels of array ranks
arrayPointerInfo.ArrayRanks = new int[] {type.GetArrayRank()};

// 5/10/2007 - Have to do a special check because nested type arrays aren't reflected correct in Type (http://www.actiprosoftware.com/Support/Forums/ViewForumTopic.aspx?ForumTopicID=2337#8632)
if (type.FullName != null)
{
int nestedClassDelimiterIndex = type.FullName.LastIndexOf('+');
if ((nestedClassDelimiterIndex != -1) && (type.DeclaringType == null))
declaringTypeFullName = type.FullName.Substring(0, nestedClassDelimiterIndex);
}
}
if (type.IsPointer)
{
// TODO: This doesn't handle more than one level of pointer
arrayPointerInfo.PointerLevel = 1;
}
}
}

Comments (15)

Posted 12 years ago by Actipro Software Support - Cleveland, OH, USA
Avatar
We're currently looking into the grammars for both languages to determine the impact of the changes. It is likely that we will start on .NET 3.5 syntax as one of the next projects we begin, but I don't have a timeframe for you at this time.

Per your comments, the TODOs need to be examined further. That type is used to load type references from assemblies. There seems to be a limitation in the .NET framework's Type class where it doesn't appear to have a way to programmatically retrieve multiple dimensions of arrays and pointers. For instance, it does return array info like int[,,] but doesn't seem to have a way to get info for int[,,][,]. Similar for pointers.

If you have any insight in the meantime, feel free to email us or reply here. Thanks!


Actipro Software Support

Posted 12 years ago by Wesner Moise
Avatar
For arrays, doesn't type.GetArrayRank() work for getting the rank of the array.

For jagged arrays, type.GetElementType() should work by returning the underlying type, which would be another array type.

For pointers, doesn't type.GetElementType() work recursively for getting the inner pointer type.
Posted 12 years ago by Actipro Software Support - Cleveland, OH, USA
Avatar
Thanks Wesner, GetElementType did the trick and the change will be in the upcoming maintenance release.


Actipro Software Support

Posted 12 years ago by Wesner Moise
Avatar
How's Orcas support coming or do you recommend that I try extending the grammar myself?
Posted 12 years ago by Actipro Software Support - Cleveland, OH, USA
Avatar
Hi Wesner,

We've completed all .NET 3.5 C#/VB grammar updates including query expressions except for one major issue.

The big problem is new VB's XML literal syntax and how the lexer parses it. It's pretty difficult to explain but the general problem is that XML literal syntax is very similar to VB attribute syntax. The solution is to really rewrite and make major improvements to our lexical parser so that somehow it ties in more closely with the semantic parser. This way it knows when it can parse an expression (XML literal) vs. an attribute. Right now as you know the lexer is not that smart and does everything based on text pattern recognition.

We can't parse XML literals in the lexer as one big token (one workaround idea) because there is no way to know whether the < starts an XML literal or an attribute. If it is a tag start, it needs to encompass other tags. But since there are a couple cases where a thing that looks like a start tag could be a VB attribute specification or an XML literal, that is ruled out due to ambiguity.

The other option we've worked on was parsing tokens separately like we do now. Then having the semantic parser "skip" over keywords, etc. if it sees it's in an XML literal. That works great except for one issue. Look at this code sample:
Dim n = <bool title='adasd'/>
That is a valid XML tag but if we parse tokens separately, when the lexer encounters the ' character, it will start a comment. And then the whole rest of the document thinks it's in an XML tag from the semantic parsing since it isn't properly closed (the close is comment text now). So the gist of this is, if they don't have any ' characters in the XML literals, everything parses fine, although the highlighting isn't right in the XML literal but at least there are no semantic errors. If they do add a ' in the XML literal then everything breaks down and semantic errors will occur all over.

We're not sure what to do at this point with what we have since all the rest of it is ready to go but the XML literal thing is a blocking issue.

I'd love to hear your thoughts on the issue.


Actipro Software Support

Posted 12 years ago by Wesner Moise
Avatar
An XML attribute string value always contains an equal sign preceeding the quote possibly including whitespace. It's not legal VB syntax for a comment to follow an equal sign except inside XML.
Posted 12 years ago by Actipro Software Support - Cleveland, OH, USA
Avatar
Unfortunately I don't believe that is correct. XML literals are treated as expressions and as such can appear anywhere that an expression can. We have a version of the official specs from Microsoft on XML literals. Here is one sample out of that which shows other scenarios:
Imports <xmlns:a = "http://SomeNamespace">
Dim book = <a:book>
  <title>TCP/IP Illustrated</title>
  <author>
    <last>Stevens</last>
    <first>W.</first>
  </author>
</a:book>

Dim firstName = book...<first>(0)
Dim b = firstName.Ancestors(GetXmlNamespace(a) + "book")(0)

Console.WriteLine(b.<title>.Value)
'Output: TCP/IP Illustrated
And here is an example of using XML right within a method call:
Me.MySub(<xmltag></xmltag>)


Actipro Software Support

Posted 12 years ago by Wesner Moise
Avatar
Some clarification:

By XML attribute string value, I am referring to the right hand side of an attribute assignment expression.

Me.MySub(<xmltag attr='attribute string value'></xmltag>)

What I was saying was that apostrophe could signify two things
(1) Comment
(2) Beginning of an attribute string value

In case 1, an equal sign can not legally appear before a comment. An expression or statement cannot terminate in an equal sign, so an underscore character is required to continue analysis to the next line.

In case 2, an equal sign is always required before the first apostrophe, except for intervening whitespace and underscore characters.

In case 1,
Dim x = 'Comment

In case 2, an equal sign always appears before an attribute value.

<xmltag attr = 'Attribute String Value'
Posted 12 years ago by Actipro Software Support - Cleveland, OH, USA
Avatar
Ahh thank you for the clarification. That info would help with that scenario. I should also note that the REM keyword (which act like ') and pre-processor directives cause similar issues to what I'm describing.

Let me give a more advanced sample so you can see deeper issues:
Dim xml = <roottag>
It's valid text but here the lexer thinks it's a comment</roottag>
Again the main issue here is that the lexer currently doesn't know when it is in an XML literal block or not. It's just scanning tokens like:
1) Dim
2) xml
3) =
4) <
5) roottag
6) >
...

And we can't just scan the entire XML literal as one token since there are ambiguous scenarios with VB attributes, because the lexer doesn't use semantic information to know where attributes and XML literals are allowed in the code, etc.


Actipro Software Support

Posted 12 years ago by Wesner Moise
Avatar
> Again the main issue here is that the lexer currently doesn't know when it is in an XML literal block or not. It's just scanning tokens like:

This is really surprising. I assume that you would actually have a different lexical scope for XML literals just as you do for XML documentation comments and other stuff.
Posted 12 years ago by Actipro Software Support - Cleveland, OH, USA
Avatar
Ideally yes, that would be the case but the thing that prevents it is the ambiguity between XML literals and VB attributes. When the lexer encounters a <, it doesn't know which type it is. I've been through tons of scenarios and can find workarounds for many however a major one is this one:

This is an attribute:
<Flags>Enum MyEnum
  MyValue
End Enum
This is an XML literal:
Dim xml = <Flags>Enum MyEnum
  MyValue
End Enum</Flags>
See how the <Flags> syntax is the same in both samples? I know we can check the character before an see it's a = meaning it must be an XML literal in the second case. However the exception to being able to do that is in a scenario where the item in question follows a '(' character, as in a parameter list.

Attributes can be applied to Parameter (see the VB grammar spec), thereby making it impossible to know whether <Flags is one type or the other when following a '('.

I was thinking that statement terminators before may be a problem like here:
Dim z = <z/>
z.Add("Text")
<z/>.Add("Text")
Since the <z/> at the start of the last line may not know what type it is either, but luckily that syntax is not permitted when I try to compile it in VS2008.

So do you have any input on the following the '(' issue? If you do, we may be able to have a breakthrough and get this working.


Actipro Software Support

Posted 12 years ago by Wesner Moise
Avatar
Ok, XML literals in VB is complicated.

Fortunately, my priority is C#. When are you releasing the version that has the new syntax... I can't wait... Please do it soon.
Posted 12 years ago by Actipro Software Support - Cleveland, OH, USA
Avatar
So no ideas on the VB portion? :)

We're working on getting some major WPF product updates out the door and web site updated right now. The updated C#/VB syntax could have possibly been released before this except for the VB issue.

We're still going to think about it more and once the WPF updates are out, will have to come to a decision on what to do. So probably near the end of the month or early January.


Actipro Software Support

Posted 12 years ago by Wesner Moise
Avatar
I have version 4.0.262 source code but I didn't see the changes for C# 3.0. It doesn't look like you release it.
Posted 12 years ago by Actipro Software Support - Cleveland, OH, USA
Avatar
Correct, it's in a separate work in progress project.


Actipro Software Support

The latest build of this product (v2018.1 build 0341) was released 8 months ago, which was after the last post in this thread.

Add Comment

Please log in to a validated account to post comments.