StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POjava ioexception error=24 too many files open
text
Body
copied!<p>I'm writing a genetic algorithm that needs to read/write lots of files. The fitness test for the GA is invoking a program called <code>gradif</code>, which takes a file as input and produces a file as output.</p> <p>Everything is working except when I make the population size and/or the total number of generations of the genetic algorithm too large. Then, after so many generations, I start getting this: <code>java.io.FileNotFoundException: testfiles/GradifOut29 (Too many open files)</code>. (I get it repeatedly for many different files, the index <code>29</code> was just the one that came up first last time I ran it). It's strange because I'm not getting the error after the first or second generation, but after a significant amount of generations, which would suggest that each generation opens up more files that it doesn't close. But as far as I can tell I'm closing all of the files.</p> <p>The way the code is set up is the <code>main()</code> function is in the <code>Population</code> class, and the <code>Population</code> class contains an array of <code>Individuals</code>. Here's my code:</p> <p>Initial creation of input files (they're random access so that I could reuse the same file across multiple generations)</p> <pre><code>files = new RandomAccessFile[popSize]; for(int i=0; i<popSize; i++){ files[i] = new RandomAccessFile("testfiles/GradifIn"+i, "rw"); } </code></pre> <p>At the end of the entire program:</p> <pre><code>for(int i=0; i<individuals.length; i++){ files[i].close(); } </code></pre> <p>Inside the <code>Individual</code>'s fitness test:</p> <pre><code>FileInputStream fin = new FileInputStream("testfiles/GradifIn"+index); FileOutputStream fout = new FileOutputStream("testfiles/GradifOut"+index); Process process = Runtime.getRuntime().exec ("./gradif"); OutputStream stdin = process.getOutputStream(); InputStream stdout = process.getInputStream(); </code></pre> <p>Then, later....</p> <pre><code>try{ fin.close(); fout.close(); stdin.close(); stdout.close(); process.getErrorStream().close(); }catch (IOException ioe){ ioe.printStackTrace(); } </code></pre> <p>Then, afterwards, I append an 'END' to the files to make parsing them easier.</p> <pre><code>FileWriter writer = new FileWriter("testfiles/GradifOut"+index, true); writer.write("END"); try{ writer.close(); }catch(IOException ioe){ ioe.printStackTrace(); } </code></pre> <p>My redirection of stdin and stdout for <code>gradif</code> are from <a href="https://stackoverflow.com/a/11336368/1390487">this answer</a>. I tried using the <code>try{close()}catch{}</code> syntax to see if there was a problem with closing any of the files (there wasn't), and I got that from <a href="https://stackoverflow.com/a/4289528/1390487">this answer</a>.</p> <p>It should also be noted that the <code>Individual</code>s' fitness tests run concurrently.</p> <p>UPDATE: I've actually been able to narrow it down to the <code>exec()</code> call. In my most recent run, I first ran in to trouble at generation 733 (with a population size of 100). Why are the earlier generations fine? I don't understand why, if there's no leaking, the algorithm should be able to pass earlier generations but fail on later generations. And if there is leaking, then where is it coming from?</p> <p>UPDATE2: In trying to figure out what's going on here, I would like to be able to see (preferably in real-time) how many files the JVM has open at any given point. Is there an easy way to do that?</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload