Databases
StackOverflow2013
Postsdbo
POWhat is an alternative to using DOM XML parser for large XML Documents for multiple find operations?
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POWhat is an alternative to using DOM XML parser for large XML Documents for multiple find operations?
    primarykey
    data
    text
    <p>I am storing data for ranking users in XML documents - one row per user - containing a 36 char key, score, rank, and username as attributes.</p> <p><code>&lt;?xml version=\"1.0\" encoding=\"UTF-8\"?&gt;</code><br> <code>&lt;!DOCTYPE Ranks [&lt;!ELEMENT Rank ANY &gt;&lt;!ATTLIST Rank id ID #IMPLIED&gt;]&gt;</code><br> <code>&lt;Ranks&gt;</code><br> <code>..&lt;Rank id="&lt;userKey&gt;" score="36.0" name="John Doe" rank=15&gt;&lt;/Rank&gt;..</code><br> <code>&lt;/Ranks&gt;</code></p> <p>There are several such documents which are parsed on request using a DOM parser and kept in memory until the file is updated. This happens from within a HttpServlet which is backing a widget. Every time the widget is loaded it calls the servlet with a get request which then requires one of the documents to be queried. The queries on the documents require the following operations:</p> <ul> <li>Look up - finding a particular ID</li> <li>Iterate through each Rank element and get the id attribute</li> </ul> <p>In my test environment the number of users is &lt;100 and everything works well. However we are soon supposed to be delivering to a system with 200K+ users. I have serious concerns about the scalability of my approach - i.e. OutOfMemoryException! <p>I'm stuck for ideas for an implementation which balances performance and memory usage. While DOM is good for find operations it may choke because of the large size. I don't know much about StAX, but from what I have read it seems that it might solve the memory issue but could really slow down the queries as I will have to effectively iterate through the document to find the element of interest (Is that correct?). <p>Questions:</p> <ul> <li>Is it possible to use StAX for multiple find (like getElementById) operations on large documents quick enough to serve an HttpRequest?</li> <li>What is the maximum file size that a DOM Parser can handle? </li> <li>Is it possible to estimate how much memory per user would be used for an XML document with the above structure?</li> </ul> <p>Thanks</p> <p>Edit: I am not allowed to use databases. </p> <p>Edit: Would it be better/neater to use a custom formatted file instead and use Regular expressions to search the file for the required entry?</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USJayAgl
    1. USJayAgl
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. CO[This project](http://xmltk.sourceforge.net/) might be of interest. I read about it a while ago but never tried it out myself. (I ended up writing my own specialized XML stream processor for .NET.)
      singulars
      1. POWhat is an alternative to using DOM XML parser for large XML Documents for multiple find operations?
      1. USharpo
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload