StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PORandom long IO pauses when writing data periodically
text
Body
copied!<p>I'm writing an Android application. As far as I know, most SD cards in devices have ~4Mb/s write speed. In the worst case, my application needs to create and write ~0.5Mb to a new file about 5 times a second. Generally, IO is very fast when doing this but, after creating and writing to several files in a short period of time, I experience a sudden long wait time of ~2 seconds.</p> <p>Here are some stats from my application (note that only one IO operation is ever running at a time):</p> <pre><code>E = Time IO operation began S = Number of Mbs written WT = Time taken to write to file E S WT 14.546 ~0.41 MB 0.02s 15.061 ~0.40 MB 0.019s 15.600 ~0.42 MB 0.073s 16.054 ~0.41 MB 0.02s 16.538 ~0.36 MB 0.019s 17.007 ~0.33 MB 0.018s 17.475 ~0.32 MB 0.017s 18.030 ~0.38 MB 0.07s 19.991 ~0.38 MB 1.542s <-- 20.124 ~0.34 MB 0.018s 20.233 ~0.25 MB 0.015s 20.390 ~0.38 MB 0.021s 20.624 ~0.36 MB 0.08s 20.858 ~0.37 MB 0.018s 21.304 ~0.32 MB 0.018s 21.796 ~0.33 MB 0.017s 22.257 ~0.35 MB 0.02s 22.780 ~0.37 MB 0.07s 24.366 ~0.27 MB 1.178s <-- 24.522 ~0.40 MB 0.021s 24.648 ~0.34 MB 0.019s 24.866 ~0.38 MB 0.018s 25.319 ~0.29 MB 0.07s 25.850 ~0.45 MB 0.021s 26.288 ~0.39 MB 0.018s 26.796 ~0.43 MB 0.035s 27.249 ~0.33 MB 0.069s 27.671 ~0.41 MB 0.018s 30.054 ~0.44 MB 1.874s <-- </code></pre> <p>Notice that most of the IO operations finish in <0.05s but, after about 8 write operations where ~4Mb of data has been written in total, the next IO operation takes ~1.5s. Why is this happening? This pattern happens fairly consistently.</p> <p>My files are being written with the following code (I'm not flushing or syncing):</p> <pre><code> RandomAccessFile fos = new RandomAccessFile(filename, "rw"); FileChannel outChannel = fos.getChannel(); byteBuffer.rewind(); outChannel.write(byteBuffer); fos.close(); </code></pre> <p>I'm no expert in file IO but I know the fast IO operation times indicate the data hasn't actually been written to disk yet and the big pauses are probably buffers being flushed. How do I avoid the pauses? It seems as if my IO operations are small and infrequent enough that I should be able to avoid this pausing.</p> <p>For some context: I'm using the command pattern with mementos to implement undo/redo in a context where there are many destructive operations the user can perform. I cannot store the mementos in memory because I don't have enough memory so I write the mementos to disk instead. This works fantastic except for the above pausing problem as it means the UI is forced to wait for the IO to finish and 1.5s of lag is too much to hide from the user. I know there are other was to implement undo/redo but every approach has flaws.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload