Note that there are some explanatory texts on larger screens.

plurals
  1. PORepresenting statement-terminating newlines in a grammar?
    primarykey
    data
    text
    <p>A lot of programming languages have statements terminated by line-endings. Usually, though, line endings are allowed in the middle of a statement if the parser can't make sense of the line; for example, </p> <pre><code>a = 3 + 4 </code></pre> <p>...will be parsed in Ruby and Python* as the statement <code>a = 3+4</code>, since <code>a = 3+</code> doesn't make any sense. In other words, the newline is ignored since it leads to a parsing error.</p> <p>My question is: how can I simply/elegantly accomplish that same behavior with a tokenizer and parser? I'm using Lemon as a parser generator, if it makes any difference (though I'm also tagging this question as yacc since I'm sure the solution applies equally to both programs).</p> <p>Here's how I'm doing it now: allow a statement terminator to occur optionally in any case where there wouldn't be syntactic ambiguity. In other words, something like</p> <pre><code>expression ::= identifier PLUS identifier statement_terminator. expression ::= identifier PLUS statement_terminator identifier statement_terminator. </code></pre> <p>... in other words, it's ok to use a newline after the plus because that won't have any effect on the ambiguity of the grammar. My worry is that this would balloon the size of the grammar and I have a lot of opportunities to miss cases or introduce subtle bugs in the grammar. Is there an easier way to do this? </p> <p>EDIT*: Actually, that code example won't work for Python. Python does in fact ignore the newline if you pass in something like this, though:</p> <pre><code>print (1, 2, 3) </code></pre>
    singulars
    1. This table or related slice is empty.
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
 

Querying!

 
Guidance

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload