StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POHow can I profile file I/O?
text
Body
copied!<p>Our build is annoyingly slow. It's a Java system built with <a href="http://en.wikipedia.org/wiki/Apache_Ant" rel="nofollow noreferrer">Ant</a>, and I'm running mine on Windows XP. Depending on the hardware, it can take between 5 to 15 minutes to complete. </p> <p>Watching overall performance metrics on the machine, as well as correlating hardware differences with build times, indicates that the process is I/O bound. It also shows that the process does a lot more reading than writing.</p> <p>However, I haven't found a good way to determine <em>which</em> files are being read or written, and how many times. My suspicion is that with our many subprojects and subsequent invocations of the compiler, the build is re-reading the same commonly used libraries many times.</p> <p>What are some profiling tools that will tell me what a given process is doing with which files? Free is nice, but not essential.</p> <hr> <p>Using <a href="https://stackoverflow.com/questions/493177/how-can-i-profile-file-i-o/493201#493201">Process Monitor, as suggested by Jon Skeet,</a> I was able to confirm my suspicion: almost all of the disk activity was reading and re-reading of libraries, with the JDK's copies of "rt.jar" and other libraries at the top of the list. I can't make a RAM disk large enough to hold all the libraries I used, but mounting the "hottest" libraries on a RAM disk cut build times by about 40%; clearly, Windows file system caching isn't doing a good enough job, even though I've told Windows to optimize for that. </p> <p>One interesting thing I noticed is that the typical 'read' operation on a <a href="http://en.wikipedia.org/wiki/JAR_%28file_format%29" rel="nofollow noreferrer">JAR</a> file is just a few dozen bytes; usually there are two or three of these, followed by a skip several kilobytes further on in the file. It appeared to be ill-suited to bulk reads.</p> <p>I'm going to do more testing with <em>all</em> of my third-party libraries on a flash drive, and see what effect that has.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload