StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POWhy is Ruby's popen3 crashing because "Too many files open"?
text
Body
copied!<p>I am using Popen3 to run some Perl scrips and then dump their output into a text file. Once in the text file, I search for the result of the Perl script. I get the error after running for about 40 minutes, which is about 220 files.</p> <pre><code>ruby/1.8/open3.rb:49:in `pipe': Too many open files (Errno::EMFILE) from /ruby/1.8/open3.rb:49:in `popen3' from ./RunAtfs.rb:9 from ./RunAtfs.rb:8:in `glob' from ./RunAtfs.rb:8 </code></pre> <p>The script is below.</p> <pre><code>require 'logger' require 'open3' atfFolder = ARGV[0] testResult = ARGV[1] res = "result.txt" open('result.txt', 'w') { } Dir.glob(atfFolder+'/*.pl') do |atfTest| Open3.popen3("atf.pl -c run-config.pl -t #{atfTest}") do |i, o, e, t| while line = e.gets $testFile = testResult + line[/^[0-9]+$/].to_s + "testOutput.txt" log = Logger.new($testFile) log.info(line) end log.close end lastLine = `tail +1 #{$testFile}` file = File.open(res, 'a') if(lastLine.include? "(PASSED)") file.puts("Test #{atfTest} --> Passed") file.close File.delete($testFile) else file.puts("Test #{atfTest} --> Failed!") file.close end end </code></pre> <p>This script is processing 4900 Perl files so I don't know if that is just too many files for <code>popen3</code> or I am not using it correctly.</p> <p>Thanks for helping me!</p> <p>I refactored my script after some very helpful pointers! The code is working great!</p> <pre><code>require 'open3' atf_folder, test_result = ARGV[0, 2] File.open('result.txt', 'w') do |file| end Dir.glob("#{ atf_folder }/*.pl") do |atf_test| test_file = atf_test[/\/\w+.\./][1..-2].to_s + ".txt" comp_test_path = test_result + test_file File.open(comp_test_path, 'w') do |file| end Open3.popen3("atf.pl -c run-config.pl -t #{ atf_test }") do |i, o, e, t| while line = e.gets File.open(comp_test_path, 'a') do |file| file.puts(line) end end end last_line = `tail +1 #{comp_test_path}` File.open('result.txt', 'a') do |file| output_str = if (last_line.include? "(PASSED)") File.delete(comp_test_path) "Passed" else "Failed!" end file.puts "Test #{ atf_test } --> #{ output_str }" end end </code></pre>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload