StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>I'm no ANTLR expert, but here's a possible grammar:</p> <pre><code>grammar Str; parse : ((Space)* statement (Space)* ';')+ (Space)* EOF ; statement : print | assignment ; print : 'print' '(' (Identifier | stringLiteral) ')' ; assignment : Identifier (Space)* '=' (Space)* stringLiteral ; stringLiteral : '"' (Identifier | EscapeSequence | NormalChar | Space | Interpolation)* '"' ; Interpolation : '${' Identifier '}' ; Identifier : ('a'..'z' | 'A'..'Z' | '_') ('a'..'z' | 'A'..'Z' | '_' | '0'..'9')* ; EscapeSequence : '\\' SpecialChar ; SpecialChar : '"' | '\\' | '$' ; Space : (' ' | '\t' | '\r' | '\n') ; NormalChar : ~SpecialChar ; </code></pre> <p>As you notice, there are a couple of <code>(Space)*</code>-es inside the example grammar. This is because the <code>stringLiteral</code> is a <em>parser-rule</em> instead of a <em>lexer-rule</em>. Therefor, when tokenizing the source file, the lexer cannot know if a white space is part of a string literal, or is just a space inside the source file that can be ignored.</p> <p>I tested the example with a little Java class and all worked as expected:</p> <pre><code>/* the same grammar, but now with a bit of Java code in it */ grammar Str; @parser::header { package antlrdemo; import java.util.HashMap; } @lexer::header { package antlrdemo; } @parser::members { HashMap<String, String> vars = new HashMap<String, String>(); } parse : ((Space)* statement (Space)* ';')+ (Space)* EOF ; statement : print | assignment ; print : 'print' '(' ( id=Identifier {System.out.println("> "+vars.get($id.text));} | st=stringLiteral {System.out.println("> "+$st.value);} ) ')' ; assignment : id=Identifier (Space)* '=' (Space)* st=stringLiteral {vars.put($id.text, $st.value);} ; stringLiteral returns [String value] : '"' {StringBuilder b = new StringBuilder();} ( id=Identifier {b.append($id.text);} | es=EscapeSequence {b.append($es.text);} | ch=(NormalChar | Space) {b.append($ch.text);} | in=Interpolation {b.append(vars.get($in.text.substring(2, $in.text.length()-1)));} )* '"' {$value = b.toString();} ; Interpolation : '${' i=Identifier '}' ; Identifier : ('a'..'z' | 'A'..'Z' | '_') ('a'..'z' | 'A'..'Z' | '_' | '0'..'9')* ; EscapeSequence : '\\' SpecialChar ; SpecialChar : '"' | '\\' | '$' ; Space : (' ' | '\t' | '\r' | '\n') ; NormalChar : ~SpecialChar ; </code></pre> <p>And a class with a main method to test it all:</p> <pre><code>package antlrdemo; import org.antlr.runtime.*; public class ANTLRDemo { public static void main(String[] args) throws RecognitionException { String source = "name = \"Bob\"; \n"+ "msg = \"Hello ${name}\"; \n"+ "print(msg); \n"+ "print(\"Bye \\${for} now!\"); "; ANTLRStringStream in = new ANTLRStringStream(source); StrLexer lexer = new StrLexer(in); CommonTokenStream tokens = new CommonTokenStream(lexer); StrParser parser = new StrParser(tokens); parser.parse(); } } </code></pre> <p>which produces the following output:</p> <pre><code>> Hello Bob > Bye \${for} now! </code></pre> <p>Again, I am no expert, but this (at least) gives you <em>a</em> way to solve it.</p> <p>HTH.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload