Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>I recommend using a <code>BufferedOutputStream</code> wrapping a <code>FileOutputStream</code>. I do not believe you will see <em>any</em> performance improvement by mucking with <code>ByteBuffer</code> and <code>FileChannel</code>, and that you'll be left with a lot of hard-to-maintain code if you go that route.</p> <p>The reasoning is quite simple: regardless of the approach you take, the steps involved are the same:</p> <ol> <li>Generate bytes. You don't say how you plan to do this, and it could introduce an additional level of temporary buffering into the equation. But regardless, the Java data has to be turned into bytes.</li> <li>Accumulate bytes into a buffer. You want to buffer your data before writing it, so that you're not making lots of small writes. That's a given. But where that buffer lives is immaterial.</li> <li>Move bytes from Java heap to C heap, across JNI barrier. Writing a file is a native operation, and it doesn't read directly from the Java heap. So whether you buffer on the Java heap and then move the buffered bytes, or buffer in a direct <code>ByteBuffer</code> (and yes, you want a <em>direct</em> buffer), you're still moving the bytes. You will make more JNI calls with the <code>ByteBuffer</code>, but that's a marginal cost.</li> <li>Invoke <code>fwrite</code>, a kernel call that copies bytes from the C heap into a kernel-maintained disk buffer.</li> <li>Write the kernel buffer to disk. This will outweigh all the other steps combined, because <em>disks are slow</em>.</li> </ol> <p>There may be a few microseconds gained or lost depending on exactly how you implement these steps, but you can't change the basic steps.</p> <p>The <code>FileChannel</code> does give you the option to call <code>force()</code>, to ensure that step #5 actually happens. This is likely to actually <em>decrease</em> your overall performance, as the underlying <code>fsync</code> call will not return until the bytes are written. And if you really want to do it, you can always get the channel from the underlying stream.</p> <p>Bottom line: I'm willing to bet that you're actually IO-bound, and there's no cure for that save better hardware.</p>
    singulars
    1. This table or related slice is empty.
    1. POHow to implement a buffered / batched FileChannel in Java?
      singulars
      1. PTQuestion
    1. PTAnswer
    1. This table or related slice is empty.
    1. USparsifal
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    1. COAre both FileOutputStream and FileChannel non-blocking? I guess so... So the only advantage of FileChannel is that it can take a ByteBuffer directly. And it has force() which we don't care. But as people have mentioned before, FileChannel is faster then FileOutputStream: http://stackoverflow.com/questions/1605332/java-nio-filechannel-versus-fileoutputstream-performance-usefulness
      singulars
      1. PO
      1. USchrisapotek
    2. COI take all such posts with an extremely large grain of salt. *Stu Thompson* gave a lot of good pointers on how to tune an IO-heavy application, but his benchmark results may (will) have very little to do with your application.
      singulars
      1. PO
      1. USparsifal
    3. COI think the point i forgot to emphasize is that i will *also need to read* at random points of the big file. So some kind of swapping will be needed.
      singulars
      1. PO
      1. USchrisapotek
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload