StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PORuby readpartial and read_nonblock not throwing EOFError
text
Body
copied!<p>I'm trying to understand and recreate a simple preforking server along the lines of unicorn where the server on start forks 4 processes which all wait (to accept) on the controlling socket.</p> <p>The controlling socket <code>@control_socket</code> binds to 9799 and spawns 4 workers which wait to accept a connection. The work done on each worker is as follows</p> <pre> <code> def spawn_child fork do $STDOUT.puts "Forking child #{Process.pid}" loop do @client = @control_socket.accept loop do request = gets if request respond(@inner_app.call(request)) else $STDOUT.puts("No Request") @client.close end end end end end </pre> <p></code></p> <p>I've used a very simple rack app which simply returns a string with the status code 200 and a Content-Type of text/html.</p> <p>The problem i face is that my server works as it should when i read incoming requests (by hitting the url at "<a href="http://localhost:9799" rel="noreferrer">http://localhost:9799</a>") using a <code>gets</code> instead of something like <code>read</code> or <code>read_partial</code> or <code>read_nonblock</code>. When I use non blocking reads it never seems to throw the EOFError, which according to my understanding means it does not receive the <code>EOF</code> state.</p> <p>This causes the read <code>loop</code> to not complete. Here is the code snippet which does this bit of work.</p> <pre> <code> # Reads a file using IO.read_nonblock # Returns end of file when using get but doesn't seem to return # while using read_nonblock or readpartial # The fact that the method is named gets is just bad naming, please ignore def gets buffer = "" i =0 loop do puts "loop #{i}" i += 1 begin buffer << @client.read_nonblock(READ_CHUNK) puts "buffer is #{buffer}" rescue Errno::EAGAIN => e puts "#{e.message}" puts "#{e.backtrace}" IO.select([@client]) retry rescue EOFError $STDOUT.puts "-" * 50 puts "request data is #{buffer}" $STDOUT.puts "-" * 50 break end end puts "returning buffer" buffer end </code> </pre> <p>However the code works perfectly if the I use a simple <code>gets</code> instead of <code>read</code> or <code>read_nonblock</code> or if replace the <code>IO.select([@client])</code> with a <code>break</code>.</p> <p>Here is when the code works and returns the response. The reason I intend to use read_nonblock is unicorn uses an equivalent using the <a href="http://bogomips.org/kgio.git/tree/ext/kgio/read_write.c" rel="noreferrer">kgio</a> library which implements a non_blocking read.</p> <pre> <code> def gets @client.gets end </code> </pre> <p>The entire code is pasted next.</p> <pre> require 'socket' require 'builder' require 'rack' require 'pry' module Server class Prefork # line break CRLF = "\r\n" # number of workers process to fork CONCURRENCY = 4 # size of each non_blocking read READ_CHUNK = 1024 $STDOUT = STDOUT $STDOUT.sync # creates a control socket which listens to port 9799 def initialize(port = 21) @control_socket = TCPServer.new(9799) puts "Starting server..." trap(:INT) { exit } end # Reads a file using IO.read_nonblock # Returns end of file when using get but doesn't seem to return # while using read_nonblock or readpartial def gets buffer = "" i =0 loop do puts "loop #{i}" i += 1 begin buffer << @client.read_nonblock(READ_CHUNK) puts "buffer is #{buffer}" rescue Errno::EAGAIN => e puts "#{e.message}" puts "#{e.backtrace}" IO.select([@client]) retry rescue EOFError $STDOUT.puts "-" * 50 puts "request data is #{buffer}" $STDOUT.puts "-" * 50 break end end puts "returning buffer" buffer end # responds with the data and closes the connection def respond(data) puts "request 2 Data is #{data.inspect}" status, headers, body = data puts "message is #{body}" buffer = "HTTP/1.1 #{status}\r\n" \ "Date: #{Time.now.utc}\r\n" \ "Status: #{status}\r\n" \ "Connection: close\r\n" headers.each {|key, value| buffer << "#{key}: #{value}\r\n"} @client.write(buffer << CRLF) body.each {|chunk| @client.write(chunk)} ensure $STDOUT.puts "*" * 50 $STDOUT.puts "Closing..." @client.respond_to?(:close) and @client.close end # The main method which triggers the creation of workers processes # The workers processes all wait to accept the socket on the same # control socket allowing the kernel to do the load balancing. # # Working with a dummy rack app which returns a simple text message # hence the config.ru file read. def run # copied from unicorn-4.2.1 # refer unicorn.rb and lib/unicorn/http_server.rb raw_data = File.read("config.ru") app = "::Rack::Builder.new {\n#{raw_data}\n}.to_app" @inner_app = eval(app, TOPLEVEL_BINDING) child_pids = [] CONCURRENCY.times do child_pids << spawn_child end trap(:INT) { child_pids.each do |cpid| begin Process.kill(:INT, cpid) rescue Errno::ESRCH end end exit } loop do pid = Process.wait puts "Process quit unexpectedly #{pid}" child_pids.delete(pid) child_pids << spawn_child end end # This is where the real work is done. def spawn_child fork do $STDOUT.puts "Forking child #{Process.pid}" loop do @client = @control_socket.accept loop do request = gets if request respond(@inner_app.call(request)) else $STDOUT.puts("No Request") @client.close end end end end end end end p = Server::Prefork.new(9799) p.run </pre> <p>Could somebody explain to me why the reads fail with 'read_partial' or 'read_nonblock' or 'read'. I would really appreciate some help on this. </p> <p>Thanks.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload