Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>In case anyone else here runs into this problem I want to document what myself and a coworker were able to figure out in order to increase speed. First off a note or two about the data:</p> <ul> <li>There were a large number of users they accounted for roughly 30% of the nodes </li> <li>There were also a large number of hashtags as people will tend to hash just about anything</li> <li>Both of these had to be guaranteed unique</li> </ul> <p>Now that that's out of the way on to the optimizations. First and formost you need to ensure that your insert loop completes each time a node is inserted. There were no real examples of this for us to look at so intially the code looked like this (pseudo code)</p> <pre><code>Transaction begin While(record.next()){ parse record create unique user create unique hashtag create comment insert into graph } Transaction success Transaction finish </code></pre> <p>While this worked ok and finished relatively quickly for small datasets it didn't scale well. So we took a look at the purpose of each function and refactored the code to look like the following:</p> <pre><code>While(record.next()){ Transaction begin parse record create unique user create unique hashtag create comment insert into graph Transaction success Transaction finish } </code></pre> <p>This greatly sped things up but it wasn't enough for my co-worker. So he found that Lucene indexes could be created on node attributes and that we could reference those in the Unique Node factory. This gave us another significant speed boost. So much so that we could insert 1,000,000 nodes in ~10 seconds without resorting to using the batch loader. Thanks to everyone for their help.</p>
    singulars
    1. This table or related slice is empty.
    1. POInserting large number of nodes into Neo4J
      singulars
      1. PTQuestion
    1. PTAnswer
    1. This table or related slice is empty.
    1. USChris Maness
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. POInserting large number of nodes into Neo4J
      singulars
      1. PTQuestion
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTAcceptedByOriginator
    2. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    1. CO"So he found that Lucene indexes could be created on node attributes and that we could reference those in the Unique Node factory." Could you please explain that a bit further? I have an index with an indexed node attribute, e.g. 'name' and use that for Unique Node factory. Is that what you meant?
      singulars
      1. PO
      1. USjoewhite86
    2. COYes, please! I'd also like to know what this magic incantation for increasing neo4j insertion speed was! :-)
      singulars
      1. PO
      1. USStewart
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload