StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POcode throws std::bad_alloc, not enough memory or can it be a bug?
text
Body
copied!<p>I am parsing using a pretty large grammar (1.1 GB, it's data-oriented parsing). The parser I use (bitpar) is said to be optimized for highly ambiguous grammars. I'm getting this error:</p> <pre><code>1terminate called after throwing an instance of 'std::bad_alloc' what(): St9bad_alloc dotest.sh: line 11: 16686 Aborted bitpar -p -b 1 -s top -u unknownwordsm -w pos.dfsa /tmp/gsyntax.pcfg /tmp/gsyntax.lex arbobanko.test arbobanko.results </code></pre> <p>Is there hope? Does it mean that it has ran out of memory? It uses about 15 GB before it crashes. The machine I'm using has 32 GB of RAM, plus swap as well. It crashes before outputting a single parse tree; I think it crashes after reading the grammar, during an attempt to construct a chart parse for the first sentence.</p> <p>The parser is an efficient CYK chart parser using bit vector representations; I presume it is already pretty memory efficient. If it really requires too much memory I could sample from the grammar rules, but this will decrease parse accuracy of course.</p> <p>I think the problem is probably that I have a very large number of non-terminals, I should probably try to look for a different parser (any suggestions?)</p> <p>UPDATE: for posterity's sake, I found the problem a long time ago. The grammar was way too big due to a bug, so the parser couldn't handle it with the available memory. With the correct grammar (which is an order of magnitude smaller) it works fine.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload