StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>I had the same problem and did not want to write any Java code. I managed to solve this with STX via Joost.</p> <p>As per <a href="http://stx.sourceforge.net/documents/spec-stx-20070427.html#external-filters" rel="nofollow">spec</a>:</p> <blockquote> <p>an STX process may split a large XML document into smaller fragments, pass each of these fragments to an external filter (for example an XSLT processor), and combine the results into a large XML result document.</p> </blockquote> <p>That's exactly what I needed. Largest example of XML file I have is 1.5 GB, and I had an XSLT template to process it. When using Saxon free edition, it consumed over <strong>3GB</strong> memory when processing. With Joost it took <strong>less than 90MB</strong>.</p> <p>My XML file contains a large list of products and each of them has a complex XML structure. So I did not want to re-implement my XSLT in STX, but wanted just to split processing per products, while using the same XSLT for each product.</p> <p>Here are code details, hope it will be helpful for somebody.</p> <p>Original XSLT file (it was the first XSLT I implemented, so sorry for bad usage of for-each statements):</p> <pre><code><?xml version="1.0" encoding="UTF-8"?> <xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:fn="http://www.w3.org/2005/xpath-functions"> <xsl:template match="/"> <xsl:for-each select="Products/Product">  </xsl:for-each> </xsl:template> </xsl:stylesheet> </code></pre> <p>I converted it into following STX:</p> <pre><code><?xml version="1.0" encoding="UTF-8"?> <stx:transform version="1.0" output-method="text" output-encoding="UTF-8" xmlns:stx="http://stx.sourceforge.net/2002/ns"> <stx:buffer name="xslt-product"> <xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:fn="http://www.w3.org/2005/xpath-functions"> <xsl:template match="Product">  </xsl:template> </xsl:stylesheet> </stx:buffer> <stx:template match="/"> <stx:process-children /> </stx:template> <stx:template match="Product"> <stx:process-self filter-method="http://www.w3.org/1999/XSL/Transform" filter-src="buffer(xslt-product)" /> </stx:template> </stx:transform> </code></pre> <p>When running Joost I still had to add Saxon libraries, as I use functions in my XSLT, so I needed XSLT 2.0 support. In the end the command to run transformation was like this:</p> <pre><code>java -Djavax.xml.transform.TransformerFactory=net.sf.saxon.TransformerFactoryImpl -cp joost.jar:commons-discovery-0.5.jar:commons-logging-1.1.1.jar:saxon9he.jar net.sf.joost.Main my-source.xml my-convert.stx </code></pre> <p>The bottom line is that now I can run the transformation on low-memory servers without having implemented any Java code or re-implement original XSLT rules!</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload