Databases
StackOverflow2013
Postsdbo
POstring parsing optimization : ruby
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POstring parsing optimization : ruby
    text
    copied!<p>I am working on a parser that is currently way too slow for my needs (like 40x slower than I would like) and would like advice on methods to increase my speed. I have tried and am currently using a <a href="https://gist.github.com/1511123" rel="nofollow">custom regex parser</a>, aswell as a custom parser using <a href="http://corelib.rubyonrails.org/classes/StringScanner.html" rel="nofollow">strscanner class</a>. Ive heard a lot of positive comments on <a href="http://www.rubyinside.com/treetop-powerful-but-easy-ruby-parser-library-701.html" rel="nofollow">treetop</a>, and have considered trying to combine the regex into one huge regex that would cover all matches, but would like to get some feedback w/ experience before I rewrite my parser yet again.</p> <p>The basic rules of the strings that I am parsing are:</p> <ul> <li>3 segments (BoL operators, message, EoL operators)</li> <li>~6 BoL operators BoL operators can be in any order</li> <li>2 EoL operators EoL operators can be in any order</li> <li>Quantity of any specific operator can be 0, 1, or >1 but only 1 is used rest are removed and discarded </li> <li>Operators in the 'message' section of the string are not captured / removed</li> <li>Whitespaces is allowed before &amp; after operators but not required </li> <li>Some BoL operators can have whitespace in the setting</li> </ul> <p>My current Regex parser works by running the string through a loop that checks for BoL or EoL operators 1 at a time and cutting them out, ending the loop when there are no more operators of the given type as so...</p> <pre><code>loop{ if input =~ /^\s+/ then input.gsub!(/^\s+/,'') end if input =~ /reges for operator_a/ #sets sets operator_a input.gsub!(/regex for operator_a)/, '') elsif input =~ /regex for operator_b/ sets operator_b input.gsub!(/regex for operator_b/,'') elsif input =~ /regex for operator_c/ sets operator_c etc .. etc .. etc.. else break end } </code></pre> <p>The question I have, What would be the best way to optimize this code? Treetop, another library/gem that I have not found yet, combining the loops into one huge regex, something else?</p> <p>Please restrict all answers and input to the Ruby language, I know that it is not the 'best' tool for this job, it is the language that I use.</p> <p>More specific grammer / examples if that helps. This is for parsing communication commands sent to a game by users, so far the only commands are say, and whisper. The begenning of line operators accepted are ::{target}, :{adverb}, ={verb}, and #{direction of}. The end of line operators are {emoticon (aka. :D :( :)}, which sets adverb if not already set, and end of line puncutation which sets verb if not already set. the character ' is an alias for say, and sayto is an alias for say:: examples :</p> <blockquote> <p>':happy::my sword=as# my helm Bol command operators work.</p> </blockquote> <pre><code>{:action=&gt;:say, :adverb=&gt;"happily", :verb=&gt;"ask", :direction=&gt;"my helm", :message=&gt;"Bol command operators work."} </code></pre> <blockquote> <p>say yep say works</p> </blockquote> <pre><code>{:action=&gt;:say, :message=&gt;" yep say works"} </code></pre> <blockquote> <p>sayto my sword yep sayto works as do EoL operators!:)</p> </blockquote> <pre><code>{:action=&gt;:say, :target=&gt;"my sword", :adverb=&gt;"happily", :verb=&gt;"say", :message=&gt;"yep sayto works as do EoL operators!"} </code></pre> <blockquote> <p>whisper::my friend : happy Bol command operators work with whisper.</p> </blockquote> <pre><code>{:action=&gt;:whisper, :target=&gt;"my friend", :adverb=&gt;"happily", :message=&gt;"Bol command operators work with whisper."} </code></pre> <blockquote> <p>whisp:happy::tinkerbell and they work in a different order.</p> </blockquote> <pre><code>{:action=&gt;:whisper, :adverb=&gt;"happily", :target=&gt;"tinkerbell", :message=&gt;"and they work in a different order."} </code></pre> <blockquote> <p>':bash=exclaim::hammer BoL operators work in this order too.</p> </blockquote> <pre><code>{:action=&gt;:say, :adverb=&gt;"bashfully", :verb=&gt;"exclaim", :target=&gt;"hammer", :message=&gt;"BoL operators work in this order too."} </code></pre> <blockquote> <p>sayto bells =say :sad #wontwork Bol > Eol and directed !work with directional? :)</p> </blockquote> <pre><code>{:action=&gt;:say, :verb=&gt;"say", :adverb=&gt;"sadly", :direction=&gt;"wontwork", :message=&gt;"Bol &gt; Eol and directed !work with directional?"} </code></pre> <blockquote> <p>'all EoL removed closest to end used and reinserted. !!??!?....... :) ? :(</p> </blockquote> <pre><code>{:action=&gt;:say, :adverb=&gt;"sadly", :verb=&gt;"ask", :message=&gt;"all EoL removed closest to end used and reinserted?"} </code></pre>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload