Quantcast
Channel: Gardens Point LEX
Viewing all 71 articles
Browse latest View live
↧

Released: Gardens Point LEX version 1.2.2 (Aug 25, 2014)

August 25, 2014, 9:23 pm
$
0
0
The main distribution is a zip file. This contains the binary executable, documentation, source code and the examples.

Changes

There are minimal changes from v1.2.1. Some code has been cleaned up, including moving some variable declarations to avoid unused variable warning with some scanner configurations.

The copyright document includes a clarification that all output produced by GPLEX is the property of the user.

Source Code

The distribution contains a complete VS2012 project for the application. However the latest code revisions are always available via the source code tab of this page.

Documentation

Documentation is included in the zip archive, but is also available separately from this download page.
The main documentation is a single file. It now includes hyperlinked figure references, table of contents and index.
The documentation also describes the examples in the archive.
↧
Search

New Post: Returning Tokens Value

August 28, 2014, 8:52 am
$
0
0
Hi,

I'm sorry to ask such a dumb question but i can't find a way to return the token associated value from the scanner to the parser.

I'm used to JFLEX-like languages where you can return token like this:
return Symbol( sym.token, value);
Can you please telle me how i can do it?
↧
↧

New Post: Returning Tokens Value

August 29, 2014, 4:15 am
$
0
0
Hi NLK
The question is not dumb. The whole idea of the tools is to be used by people
who are experts in their application area but not necessarily in parsing
theory.

There is about 150+ lines of documentation for gplex and gppg, so I will point
you at the areas that you need to read to help with this particular problem.
The page numbers refer to the current versions of the doco, 1.2.2 for gplex,
1.5.2 for gppg.

lex scanners and yacc parsers play nicely together by sharing two types: a
semantic action type %YYSTYPE which is the type of the yylval variable, and
the location %YYLTYPE which is the type of the yylloc variable. The default
types for these two variables are int and LexLocation.

For gplex/gppg there is an abstract scanner type
AbstractScanner<TValue,TSpan>
which is the base type of the scanner produced by gplex. The *.y file may
declare the %YYSTYPE and %YYLTYPE types and gppg will declare an instance
of the abstract base class with these type arguments.
The scanner Abstract Scanner interface is given in figure-2 on page 8 of gppg.pdf.
Setting the yylval and yylloc values are discussed in section 4.4 of
gplex.pdf, starting at page 32.

The important thing to know about the semantic action type is that it is used
to pass values from the scanner to the parser, but also as the type referred
to by the $x variables in the semantic actions of the parser production rules.
So it is possible to choose a type that is the result type of the computations
of the parser. This is a typical use case with tree-building parsers which
will be passed leaf nodes from the scanner, and then build (sub-)trees by the
actions of the parser as particular structures are recognized. See the
tree-building example in the testfiles directory of the gppg distribution for
an example of this.

There are various patterns of use:
(1) a scanner interface where only the identity of the token matters, with the
parser grabbing any information it needs from the scanner's yytext variable if
needed. gplex's own scanner and parser work like this.
(2) an action type that in original lex would have been a union type. Since C#
does not have a union construct gppg uses a struct type with one field of each
required type. Gppg has this structure.

Section 2.4.7 and 2.4.8 of gppg.pdf explains how this works. In the case of
the union construct in gppg.y the only field that is used by the scanner is
the integer iVal field, all the others are fields used by the parser
productions.

For the scanner, the semantic actions must name the field of yylval to which
they are assigning (see lines 158 and 159 of gppg.lex for an example).

However in the parser, the declaration of the types of the non-terminal symbols
(see lines 42-46 of gppg.y) allows gppg to generate the field access code
itself from the production declarations. Thus, looking at the semantic action
at line 198 of gppg.y
$1.Add($3);
will use the known type of $1 (TokenList) and $3 (TokenDecl) to compute that
$1 must access the TokenList field of ValueType object at position-1 on the
value stack, while "$3" expands to access the tokenInfo field of the ValueType
object at position-3 on the value stack. Of course all of this is done by the
parser generator, so you don't need to know this.

Hope this helps. Let me know if you have more question.

Cheers
John
↧

New Post: Tuned for Performance?

September 24, 2014, 4:46 am
$
0
0
Has the code that is generated by GPLEX been tuned for performance?

I’m asking because I noted that of the four overloads of SetSource, the SetSource(string, int) is the fastest, slightly faster than the two having a stream parameter, and the SetSource(IList<string>) is unacceptable slow with large files (50 MB) (because of LineBuffer.findIndex).

Thanks!
↧

New Post: Tuned for Performance?

September 28, 2014, 4:44 am
$
0
0
Hi Martin
You are quite right. The performance of the buffering with a list of strings as source is pretty bad. Not much can be done about this as indexing into the character position of the list as though it were a stream is really inefficient.

However, thanks for the comment. I should at least warn users about the performance in this case. And, come to think of it, if there is no choice but to use an IList then it would be possible to speed up LineBuffer.findIndex by constructing an index table on the fly, as lines are read. I guess programs only want to index backwards into lines that have been already read?

Hmm, might have a look at that next time I get a spare hour.
John
↧
↧

Created Unassigned: caseInsensitive flag [13321]

November 14, 2014, 4:08 pm
$
0
0
The flag caseInsensitive not working anymore.

&gtExec Command="$(GplexTool) /caseInsensitive &&amp;quot;/out:...

Please reintroduce the flag caseInsensitive.
↧

Commented Unassigned: caseInsensitive flag [13321]

November 17, 2014, 3:25 pm
$
0
0
The flag caseInsensitive not working anymore.

&gtExec Command="$(GplexTool) /caseInsensitive &&amp;quot;/out:...

Please reintroduce the flag caseInsensitive.
Comments: ** Comment from web user: k_john_gough **

Ok, sounds bad. I will check it out, and get back to users.

↧

Created Unassigned: WinRT compatibility && I can't build it [13328]

November 22, 2014, 8:13 am
$
0
0
I want to use GPLEX with a WinRT (portable class library) application, however, it's not at all compatible.

Serialization, Reflection, Encoding, Files, Console and other things differ.

There are issues in both the boiler plate code and the generated code.

Do you have in mind to Make it Portable Class Library compatible?

I'd be prepared (in principle) to tackle this myself, but the donwloaded zip file appears to be missing parser.cs, Scanner.cs and GplexBuffers.cs ) so I can't build it.

Iain
↧
Search

Commented Unassigned: WinRT compatibility && I can't build it [13328]

November 30, 2014, 4:06 am
$
0
0
I want to use GPLEX with a WinRT (portable class library) application, however, it's not at all compatible.

Serialization, Reflection, Encoding, Files, Console and other things differ.

There are issues in both the boiler plate code and the generated code.

Do you have in mind to Make it Portable Class Library compatible?

I'd be prepared (in principle) to tackle this myself, but the donwloaded zip file appears to be missing parser.cs, Scanner.cs and GplexBuffers.cs ) so I can't build it.

Iain
Comments: ** Comment from web user: IainDowns **

For what it's worth, I've discovered that you have to run the batch file GenerateAll to make the cs files.

To get this create RT compatible code I've done the following:-

**In the target project, created a SerialiazableAttribute class which does nothing (the Exceptions contain a Serializable attribute)
**In the Gplex SpecFiles removed the BufferException constructor with serialization int GplexBuffers. WinRT doesn't support this - DataContracts are the new way, apparently.

**In the target project Created a class to mimic Console

public class Console
{
private static Console _console ;

public static Console Out
{
get { if (_console == null) { _console = new Console();}
return _console;
}
}

public void Write(string format, params object[] p)
{
System.Diagnostics.Debug.WriteLine(format, p);
}

public static void WriteLine(string format, params object[] p)
{
System.Diagnostics.Debug.WriteLine(format, p);
}
}

** in the lex file use the nofiles option AND unicode. In the target project define NOFILES to stop the file reading stuff being compiled. THis leaves me with a scanner that requires a string input, but bypasses the swathes of changes to support lack of code pages and the changes to the file reading bits (async and so on)

** Rework the way GetMaxParseToken is defined (in gplexx.frame) to avoid reflection

private static int GetMaxParseToken() {
##-->translate $Tokens t;
##-->translate if ($Tokens.TryParse("maxParseToken", false, out t))
{
return (int) t;
}
else
{
return int.MaxValue;
}
}


I think that's it. GPLex won't run under WinRT and the generated code has some restrictions, but it's got me going. I will leave any more sophisticated fixes to someone who understands how it works.

Sadly, the scanner generated still doesn't correctly tokenise CSS2.1 which was my intent, but that's hardly the fault of GPLex!

↧
↧

Commented Unassigned: caseInsensitive flag [13321]

November 30, 2014, 10:51 pm
$
0
0
The flag caseInsensitive not working anymore.

&gtExec Command="$(GplexTool) /caseInsensitive &&amp;quot;/out:...

Please reintroduce the flag caseInsensitive.
Comments: ** Comment from web user: k_john_gough **

Can't find any problem with case insensitive scanners.
See attached test file.

↧

New Post: What does gplex really do?

February 11, 2015, 9:19 am
$
0
0
Hi,
I got the same situation as Macias, but I couldn't find the example on the download page mentioned there "I have put up a working example on the download page that creates a list of scanner result objects.".
Could you provide a link?
Thanks.
↧

Created Unassigned: Argument out of range exception [13615]

June 4, 2015, 12:32 am
$
0
0
This simple test case with cyrillic symbols fails:

```
%namespace LexScanner
%option verbose noparser

%%
абракадабра {}
%%
```

I've double checked file encoding with Notepad++ and Vim. However this test case gives no error:

```
%namespace LexScanner
%option verbose noparser

%%
л {}
%%
```
I'm using this line in cmd to generate lexer:
>\> gplex ./Lexer.lex
↧

Commented Unassigned: Parsing a backslash [12206]

July 30, 2015, 8:24 am
$
0
0
I'm building a lexer to parse Zinc and MiniZinc files. In these files the wedge (/\) and vee (\/) are represented by slashes and backslashes. The lexer however reports in the case of \/ this as an "OTHER" and "OPDIV" character. I tried solving this problem by replacing \\ by the hexadecimal code of a backslash, but this method failed to give the correct result. I can't find anything in the manual describing this problem. Is it a bug or dit I do something wrong?
Comments: ** Comment from web user: talweiss1982 **

I'm having the same issue when using unicode mode.
It seems GPLEX is ignoring '\'
If you have found a solution for this problem please email me at tal@ayende.com

↧
↧

New Post: Using the ErrorHandler class

August 24, 2015, 1:51 am
$
0
0
Hi,

I am working on a Project to check a NC-Code for its grammar.
Therefor I use a scanner- and a Parser-class created by the gplex- and gppg-engine.
Until now I succeeded to run these two classes together with my own main program.

Now I want to have some information about occurred errors in the Input-File.
Therefor I think I can use the ErrorHandler-class, which is provided in the project-folder from the download package.

After including this class in my project and declaring all the neccessary namespaces there are still some undefined parameter called "LexSpan", "TooManyErrorsException" and "StringUtilities".

I don't know what I have to do now and hope for some help.

Thanks, Alex
↧

New Post: Using the ErrorHandler class

August 24, 2015, 7:16 am
$
0
0
After reviewing the documentation i came to the issue, that i can't use the "ErrorHandler" from the download package.

I need to build my own "ErrorHandler"?
If it is so, where can I get some information about how to do that?

Thanks, Alex
↧
Search

Created Unassigned: [€] as regular expression crashes gplex [13731]

October 3, 2015, 1:21 pm
$
0
0
Adding the euro sign € to a character set literal (i.e., [€]) crashes gplex, with error:

Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index

Unhandled Exception: System.ArgumentOutOfRangeException: Index was out of range.
Must be non-negative and less than the size of the collection.
Parameter name: index
at System.Collections.BitArray.Set(Int32 index, Boolean value)
at System.Collections.BitArray.set_Item(Int32 index, Boolean value)
at QUT.Gplex.Automaton.NFSA.NState.AddClsTrans(Leaf leaf, NState nxt)
at QUT.Gplex.Automaton.NFSA.NfsaInstance.MakePath(RegExTree tree, NState startState, NState endState)
at QUT.Gplex.Automaton.NFSA.Build(AAST ast)
at QUT.Gplex.Automaton.TaskState.Process(String fileArg)
at QUT.Gplex.Program.Main(String[] args)
C:\Mischa\vhdl_gppg\ConsoleApplication1\ConsoleApplication2_vhdl\vhdllexer.lex(1,1):error 3:Index was out of range. Must be non-negative and less than the size
of the collection.
Parameter name: index

(Oddly, adding the euro sign between quotes seems to work just fine, as a workaround.)
Just my 2 cts. Keep up the good works!
↧

New Post: How implement Continous Line

June 12, 2016, 6:51 pm
$
0
0
Hi,

How implement continous line?
source code Example

DEFINE APP_WRONG_SUPERCLASS_LOC ;

"Sorry -- this application must instantiate "+ CHR(13)+ ;
            "an object that descends from "+APP_SUPERCLASS+"."
In this case semicolon is continuation character !

Please help
Thanks!
↧
↧

Created Unassigned: " X:" breaks recognition of XY [14013]

November 14, 2016, 5:46 am
$
0
0
Dear John,

thanks for providing gplex, which is of great help to me. I get unexpected behaviour when parsing data containing positional info. The real .lex file is quite large and my token is XYZWPR, but I could break the issue down to case 2 breaking the recognition of case 1.

case1 XY
//case2 [ ]X:
case2 " X:"

The input file being some testfile containing the string "XYZWPR". When case 2 is inactive (commented out), case 1 is recognized, when case 2 is active, there is no recognition of either pattern. It does not matter if case 2 is "[ ]X:" or " X:". The issue can be avoided by not checking for the trailing blank (i.e. if case 2 is "X:", there's recognition as expected), or in my real use case by checking case1 for YZWPR instead of XYZWPR but I found it worth mentioning. I get this behaviour on Linux / Mono as well as on VisualStudio2015/Win7. The sample lex file is attached.

best regards
Arnd
↧

New Post: Why is sealed?

November 26, 2016, 5:20 am
$
0
0
Hy everybody!
What the reason for the generated scanner class is marked as "sealed"? Where I can see an special options for manage this?
public sealed partial class MyScanner : ScanBase
I want to use same scanner for two own parsers, but first should be generated for "usual" grammar (where EOL does not affect the rules), and other - where EOL used in the grammar.
For this I hoped override the yylex() method for skip any EOL tokens (for first case).
    public class MyScannerWithSkipEOL : MyScanner
    {
        public override int yylex()
        {
            int token = base.yylex();
            while (token == (int)Token.EOL)
                token = base.yylex();
            return token;
        }
    }
But generated MyScanner is sealed
↧

Patch Uploaded: #18527

November 27, 2016, 11:59 pm
$
0
0

kep4uk has uploaded a patch.

Description:
Hi, I hope that you will accept my patch about discussion 659726.
I added the $modifier key for manage generated scanner class.
Regards, Nikolay

↧
Viewing all 71 articles
Browse latest View live
Search
<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>