StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>After looking at how languages like Python and SML/NJ handle their REPLs, I got a nice one working in my interpreter. Instead of having the prompt/echo logic in the outermost parser driver loop, I put it in the innermost lexer input routine. Actions in the parser and lexer set flags that control the prompting by input routine.</p> <p>I'm using a reentrant scanner, so <code>yyextra</code> contains the state passed between the layers of the interpreter. It looks roughly like this:</p> <pre><code>typedef struct Interpreter { char* ps1; // prompt to start statement char* ps2; // prompt to continue statement char* echo; // result of last statement to display BOOL eof; // set by the EOF action in the parser char* error; // set by the error action in the parser BOOL completeLine // managed by yyread BOOL atStart; // true before scanner sees printable chars on line // ... and various other fields needed by the interpreter } Interpreter; </code></pre> <p>The lexer input routine:</p> <pre><code>size_t yyread(FILE* file, char* buf, size_t max, Interpreter* interpreter) { // Interactive input is signaled by yyin==NULL. if (file == NULL) { if (interpreter->completeLine) { if (interpreter->atStart && interpreter->echo != NULL) { fputs(interpreter->echo, stdout); fputs("\n", stdout); free(interpreter->echo); interpreter->echo = NULL; } fputs(interpreter->atStart ? interpreter->ps1 : interpreter->ps2, stdout); fflush(stdout); } char ibuf[max+1]; // fgets needs an extra byte for \0 size_t len = 0; if (fgets(ibuf, max+1, stdin)) { len = strlen(ibuf); memcpy(buf, ibuf, len); // Show the prompt next time if we've read a full line. interpreter->completeLine = (ibuf[len-1] == '\n'); } else if (ferror(stdin)) { // TODO: propagate error value } return len; } else { // not interactive size_t len = fread(buf, 1, max, file); if (len == 0 && ferror(file)) { // TODO: propagate error value } return len; } } </code></pre> <p>The top level interpreter loop becomes:</p> <pre><code>while (!interpreter->eof) { interpreter->atStart = YES; int status = yyparse(interpreter); if (status) { if (interpreter->error) report_error(interpreter); } else { exec_statement(interpreter); if (interactive) interpreter->echo = result_string(interpreter); } } </code></pre> <p>The Flex file gets these new definitions:</p> <pre><code>%option extra-type="Interpreter*" #define YY_INPUT(buf, result, max_size) result = yyread(yyin, buf, max_size, yyextra) #define YY_USER_ACTION if (!isspace(*yytext)) { yyextra->atStart = NO; } </code></pre> <p>The <code>YY_USER_ACTION</code> handles the tricky interplay between tokens in the language grammar and lines of input. My language is like C and ML in that a special character (';') is required to end a statement. In the input stream, that character can either be followed by a newline character to signal end-of-line, or it can be followed by characters that are part of a new statement. The input routine needs to show the main prompt if the only characters scanned since the last end-of-statement are newlines or other whitespace; otherwise it should show the continuation prompt.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload