StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>In both the first and the final examples the original sequence passed to <code>split-with</code> is retained while being realized in full in memory; hence the OOME. The way this happens is indirect; what is retained directly is <code>t</code>, while the original sequence is being held onto by <code>t</code>, a lazy seq, <em>in its unrealized state</em>.</p> <p>The way <code>t</code> causes the original sequence to be held is as follows. Prior to being realized, <code>t</code> is a <code>LazySeq</code> object storing a thunk which may be called upon at some point to realize <code>t</code>; this thunk needs to store a pointer to the original sequence argument to <code>split-with</code> before it is realized to pass it on to <code>take-while</code> -- see the implementation of <code>split-with</code>. Once <code>t</code> is realized, the thunk becomes eligible for GC (the field which holds it in the <code>LazySeq</code> object is set to <code>null</code>) at <code>t</code> no longer holds the head of the huge input seq.</p> <p>The input seq itself is being realized in full by <code>(count d)</code>, which needs to realize <code>d</code>, and thus the original input seq.</p> <p>Moving on to why <code>t</code> is being retained:</p> <p>In the first case, this is because <code>(count d)</code> gets evaluated before <code>(count t)</code>. Since Clojure evaluates these expressions left to right, the local <code>t</code> needs to hang around for the second call to count, and since it happens to hold on to a huge seq (as explained above), that leads to the OOME.</p> <p>The final example where only <code>(count d)</code> is returned should ideally not hold on to <code>t</code>; the reason that is not the case is somewhat subtle and best explained by referring to the second example.</p> <p>The second example happens to work fine, because after <code>(count t)</code> is evaluated, <code>t</code> is no longer needed. The Clojure compiler notices this and uses a clever trick to have the local reset to <code>nil</code> simultaneously with the <code>count</code> call being made. The crucial piece of Java code does something like <code>f(t, t=null)</code>, so that the current value of <code>t</code> is passed to the appropriate function, but the local is cleared before control is handed over to <code>f</code>, since this happens as a side effect of the expression <code>t=null</code> which is an argument to <code>f</code>; clearly here Java's left-to-right semantics are key to this working.</p> <p>Back to the final example, this doesn't work, because <code>t</code> is not actually used anywhere and unused locals are not handled by the locals clearing process. (The clearing happens at the point of last use; in absence of such a point in the program, there is no clearing.)</p> <p>As for <code>count</code> realizing lazy sequences: it must do that, as there is no general way of predicting the length of a lazy seq without realizing it.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload