Strand 500 Series Parser/Lexer/Grammar

[peternewman]

Bit of a shot in the dark, and I guess I'm mostly looking at people like Richard, Gareth and probably a few other likely suspects, but has anyone ever written a parser or lexer for the 500 series command line syntax, or written anything about the grammar? Equally are there references anywhere of all valid commands or will I need to dig through the manual?

 

I need to parse some basic 500 series commands and output the relevant MIDI data to trigger them, so while I realise I could just knock something up, I thought the future potential for a full parser/lexer could be quite interesting.

 

If the above was all gobbledygook (and you want to know more), have a look at Parsing and Lexical analysis on Wikipedia.

[Dmills]

Not I, but the thought has occured for a couple of projects.

I have a horrible feeling that there may be quite a few special cases where the syntax is not actually a valid LALR grammar, picking this apart to avoid shift/reduce conflicts may be interesting.

 

Please let me know if you find anything useful.

 

Regards, Dan.

[blackbird]

On a very similar note ... does anyone have any recommendations for parser generators?

 

I need one that will work with .Net and not force me into Lexing as well (input is natively tokenised). Have been looking in the direction of ANTLR, but am slowly resigning myself to writing a parser by hand ... unfortunately my grammar is not LALR(1)!!

 

Oh, and trying to find parser examples which aren't just a calculator, or iterative programming languages is bloody impossible!

[peternewman]

I must admit I'm rather rusty on it all Dan. I did a course on it as part of my degree, which I found quite interesting, but I've not really touched it since, so it needs a bit more investigation. For the thing I'm currently looking at, just lexing and converting the tokens to the MIDI may well do the job and hence avoid such issues, but like you there are projects where full parsing could be interesting.

 

There seems to be a fair selection of options listed on Wikipedia blackbird, apart from that a more computing/programming based forum may give you more success.

[Dmills]

Non LALR grammar will complicate matters as most parser generators that I am aware of tend to assume it. Mind you I date from when lex and yacc were what you used so the world may well have moved on.

There are also interesting approaches that do not use FSA in the sense that most parsers do, see for example Vaughan Pratts 1973 paper at http://portal.acm.org/citation.cfm?id=512931 for one example.

 

Regards, Dan (Just how many of us have some CS know how anyway?).

[Germaine]

I'm not well versed in this field but have you had a look at controlling the strand with rs232 as opposed to involving midi?

[peternewman]

TBH I wasn't even aware you could use RS232 for control. For my project I really need to parse the command lines anyway, basically someone has already written what commands they want executed, so I need to tokenize it, and then output the tokens to the 500, via either RS232/MIDI or whatever. Although I also have a list of the relevant MIDI commands, which is what is making me lean in that direction a bit.

[dbuckley]

On a very similar note ... does anyone have any recommendations for parser generators?

 

Oh yes, I know of a truly great parser generater, which has proven excellent at generating parsers for small control languages, like a frontend to the Peavey Digitool I wrote, which is very similar to what you want - syntax in to RS232 out; unfortunately the tool's author parsed away a couple of years back and so the product has sadly gone from the marketplace. To give you its name would just frustrate you as you'd fall in love with the trial, still downloadable, and then be stuffed.

 

RIP Jerry... "The problem with parsing is that almost all explanations of it just add to the confusion."

[peternewman]

unfortunately the tool's author parsed away a couple of years back

Intentional pun or Freudian slip (my bold)?

 

Well I started having a bit of a go at this finally the other day, in JavaCC. I've so far got the beginnings of a grammar in place for channel selection and level setting, including a bit of code to handle single to double digit conversion, but nothing too exciting yet.

 

So far all it actually does is prints out what you put in, but having parsed it and by displaying the tokens. Anyway its a start:

 

5 thru 10 + 17 thru 20 - 7 + 25 @ 5
Parsed:  5  thru  10  +  17  thru  20  -  7  +  25  at  50

 

I have put it on hold a little, because I realised the priority is actually going to be parsing ASCII Light Cue files, as exported from Strand (among others), anyway see my other topic for that.

[dbuckley]

unfortunately the tool's author parsed away a couple of years back

Intentional pun or Freudian slip (my bold)?

Absolutely intentional. Jerry had a perchant for terrible parsing-related puns, and I'm sure he'd fully approve of that one :)

 

Given its christmas / new year, and the alteraive to an hours keyboading is continuing to sort out paperwork dating back to the 90s I though I'd have a crack at your quoted command:

 

c:\br_parz>parz 5 thru 10 + 17 thru 20 - 7 + 25 @ 5
Set channel(s): 5 6 8 9 10 17 18 19 20 25 to level 5

c:\br_parz>parz 3 +4 +5 +6 +7 + 17 thru 25 - 19 thru 22 @ half
Set channel(s): 3 4 5 6 7 17 18 23 24 25 to level 50

 

And the code that does it, to illustrate to power of using a parser generator rather than coding the thing by hand. Examining the grammer section, in particular the 'statements' area, its easy to see how this could be expanded for many other types of commands.

 

/*
;************************************************************************
;* Setup
;************************************************************************
*/

[
test file mask = "*.txt"
parser name = "br_parz"
disregard white space
case sensitive = OFF
lexeme {integer, eol}
distinguish lexemes
line numbers
pointer input
pointer type = unsigned char *
]


/*
;************************************************************************
;* Grammer
;************************************************************************
*/

grammer $
-> statements?..., eof

statements
-> [statement], eol

statement
-> !setter = 1;, command

command
-> Channel Specification, [Additional Channel Specifications...], "@", level:n = {
	SetSelectedChannels(n);
}


Additional Channel Specifications
-> "+", !setter = 1;, Channel Specification

-> "-", !setter = 0;, Channel Specification


Channel Specification
-> integer:from, "thru", integer:to = {
	int I;
	for (I = from; I <= to; I++)
		selectedchannels[I] = setter;
}

-> integer:I = {
	selectedchannels[I] = setter;
}


(int) level
-> "off"   = 0;
-> "on"	= 100;
-> "full"  = 100;
-> "max"   = 100;
-> "half"  = 50;
-> integer

/*
;************************************************************************
;* Lexology
;************************************************************************
*/

noteol = ~(eof + '\n')
eof = 0
all letters = 1..126 - '"'
digit = '0-9'

eol
->'\n'
->'\r'

white space
-> ' ' + '\t'

(int) integer
-> digit:d								=d - '0';
-> integer:n, digit:d					=10*n + d - '0';



/*
;=========================================================================
;= Embedded C
;=========================================================================
*/
{
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#define MAX_CHANS 512
int selectedchannels[MAX_CHANS+1];		/* we never use zero! */
int setter;

void
ClearAllSelections(void)
{
int I;
for (I = 1; I <= MAX_CHANS; I++)
	selectedchannels[I] = 0;
}


void
SetSelectedChannels(int level)
{
int I;
printf("Set channel(s): ");
for (I = 1; I <= MAX_CHANS; I++)
	if (selectedchannels[I])
		printf("%d ", I);
printf("to level %d\n", level);

}

void
main(int argc, char *argv[])
{
char inp[256], *pi;
int I;

if (argc < 2){
	printf(" Usage is %s followed by control tokens\n");
	exit(1);
}

*inp = '\0';
for (I = 1; I < argc; I++){
	strcat(inp, argv[I]);
	strcat(inp, " ");
}
strcat(inp, "\n");

PCB.pointer = (unsigned char *)inp;
br_parz();
}



/*
;************************************************************************
;* end of embedded C
;************************************************************************
*/
}

 

Ok, I've used some beginner C code techniques, but thats for illustration rather than for production quality code :P And for some reason lower case 'I' variables have come out upper case...

[obsoperator]

Yes, it's a clean-looking job. But doesn't the code, in its basic design [if (selectedchannels)], make it impossible to command channels to level zero?

[peternewman]

Nope, "SetSelectedChannels(int level)" uses the level to set the level, zero or otherwise. The selectedchannels array just has flags of 0 or 1 for each channel as to whether its level needs setting or not.

[nick123]

I generally use Bison (YACC) for non-trivial parsers, and did so recently for this exact task in a tracking console GUI that I've recently started.

 

When I did the CLI part I used a small Bison generated parser running in its own thread that's fed tokens from the main thread; this decoupled the parser, which blocks on a call to the lexer function yylex(), so it could receive tokens asynchronously from the main thread and from whatever source produced them. As the parser works through the grammar it sends events back via the GUI events mechanism triggering behaviour as it goes. So if the channels display is active, after you type "1 THRU 5 +", the channels 1 to 5 will show as selected, then "7@" would also highlight 7, and finally "5 ENTER" will show the channels captured and set to 50%; of course you can also use the mouse to select, wheel to change levels etc. Error recovery isn't as great or flexible with Bison as it could be and there are some types of grammar that Bison can't handle that other generators can, it's also lalr(1), but it's powerful and flexible, easy to use with a notation similar to BNF, and great for this type of thing. So far the grammar hasn't been a problem.

 

I'd probably never have started this and got back into my passion for lighting had my G/F not bought me an enttec pro as an inspired xmas present a couple of years ago or so. She'd thought it's what I needed to control the garden, but I immediately saw it as the way to go for replacing the X10 that I'd installed in my house that proved no good for scene setting, and the DMX solution for H/A turned out great. Now I'm getting back into stage lighting, writing a tracking console GUI around my original H/A DMX engine seemed a fun idea. In the end I never got to do the garden lighting before we moved, but I did put in several segments of computer controlled irrigation, built an RF link between the house and garden for that, and tied it into the H/A system running on a little slimfit PC in the loft; so now I can control the garden remotely from where we are now (I do still own the other garden!). That was also my first attempt at hand soldering 0402's (not sure why I picked such small ones) and reflow soldering in a toaster oven as I wanted to use one of the FTDI USB chips in the transmitter, and that worked out well. Definitely all fun stuff these type of projects