Databases
StackOverflow2013
Postsdbo
POHow do I implement a two-pass scanner using Flex?
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POHow do I implement a two-pass scanner using Flex?
    text
    copied!<p>As a pet-project, I'd like to attempt to implement a basic language of my own design that can be used as a web-scripting language. It's trivial to run a C++ program as an Apache CGI, so the real work lies in how to parse an input file containing non-code (HTML/CSS markup) and server-side code.</p> <p>In my undergrad compiler course, we used <a href="http://www.gnu.org/software/flex/" rel="nofollow noreferrer">Flex</a> and <a href="http://www.gnu.org/software/bison/" rel="nofollow noreferrer">Bison</a> to generate a scanner and a parser for a simple language. We were given a copy of the grammar and wrote a parser that translated the simple language to a simple assembly for a virtual machine. The flex scanner tokenizes the input, and passes the tokens to the Bison parser.</p> <p>The difference between that and what I'd like to do is that like PHP, this language could have plain HTML markup and the scripting language interspersed like the following:</p> <pre><code>&lt;p&gt;Hello, &lt;? echo "World ?&gt; &lt;/p&gt; </code></pre> <p>Am I incorrect in assuming that it would be efficient to parse the input file as follows:</p> <ol> <li>Scan input until a script start tag is found (' <li>Second scanner tokenizes the server-side script section of the input file (from the open tag: '') and passes the token to the parser, which has no need to know about the markup in the file.</li> <li>Control is returned to the first scanner that continues this general pattern.</li> </ol> <p>Basically, the first scanner only differentiates between Markup (which is returned directly to the browser unmodified) and code, which is passed to the second scanner, which in turn tokenizes the code and passes the tokens to the parser. </p> <p>If this is <em>not</em> a solid design pattern, how do languages such as PHP handle scanning input and parsing code efficiently?</p>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload