Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>If there's a good lexer/parser library for Java, you could define a lexer like the following pseudo-lexer code:</p> <pre><code>Delimiter: , Item: ([^,"]+) | ("[^,"]+") Data: Item Delimiter Data | Item </code></pre> <p>How lexers work is that it starts at the top level token definition (in this case Data) and attempts to form tokens out of the string until it cannot or until the string is all gone. So in the case of your string the following would happen:</p> <ul> <li>I want to make Data out of 200,6, "California, USA".</li> <li>I can make Data out of an Item, a Delimiter and Data. </li> <li>I looked - 200 is an Item and then , is a Delimiter so I can tokenize that and keep going.</li> <li>I want to make data out of 6, "California, USA"</li> <li>I can make Data out of an Item, a Delimiter and Data. </li> <li>I looked - 6 is an Item and then , is a Delimiter so I can tokenize that and keep going.</li> <li>I want to make data out of "California, USA"</li> <li>I can make Data out of an Item, a Delimiter and Data. </li> <li>I looked - "California, USA" is an Item, but I see no Delimiter after it, so let's try something else.</li> <li>I can make Data out of an Item.</li> <li>I looked - "California, USA" is an item, so I can tokenize that and keep going.</li> <li>The string is empty. I'm done. Here's your tokens.</li> </ul> <p>(I learned about how lexers work from the guide to PLY, a Python lexer/parser: <a href="http://www.dabeaz.com/ply/ply.html" rel="nofollow">http://www.dabeaz.com/ply/ply.html</a> )</p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
 

Querying!

 
Guidance

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload