StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POHaving problems running a long blocking function in a thread in Tornado
text
Body
copied!<p>I am very new to Tornado. Was just seeing how to handle a request which blocks in Tornado. I run the blocking code in a separate thread. However the main thread still blocks till the threaded function finishes. I am not using gen.coroutine here, but have tried that and the result is the same</p> <pre><code>counter = 0 def run_async(func): @wraps(func) def function_in_a_thread(*args, **kwargs): func_t = Thread(target=func, args=args, kwargs=kwargs) func_t.start() return function_in_a_thread def long_blocking_function(index, sleep_time, callback): print "Entering run counter:%s" % (index,) time.sleep(sleep_time) print "Exiting run counter:%s" % (index,) callback('keyy' + index) class FooHandler(tornado.web.RequestHandler): @web.asynchronous def get(self): global counter counter += 1 current_counter = str(counter) print "ABOUT to spawn thread for counter:%s" % (current_counter,) long_blocking_function( index=current_counter, sleep_time=5, callback=self.done_waiting) print "DONE with the long function" def done_waiting(self, response): self.write("Whatever %s " % (response,)) self.finish() class Application(tornado.web.Application): def __init__(self): handlers = [(r"/foo", FooHandler), ] settings = dict( debug=True, ) tornado.web.Application.__init__(self, handlers, **settings) def main(): application = Application() application.listen(8888) tornado.ioloop.IOLoop.instance().start() if __name__ == "__main__": main() </code></pre> <p>When I issue back to back requests the FooHandler blocks and does not recieve any requests till the long_blocking_function finishes. So I end up seeing something like</p> <pre><code>ABOUT to spawn thread for counter:1 Entering run counter:1 Exiting run counter:1 DONE with the long function ABOUT to spawn thread for counter:2 Entering run counter:2 Exiting run counter:2 DONE with the long function ABOUT to spawn thread for counter:3 Entering run counter:3 Exiting run counter:3 DONE with the long function </code></pre> <p>I was expecting something along these lines(as I am issuing multiple requests before the first call to long_blocking_function finishes) but am only seeing trace similar to above</p> <pre><code>ABOUT to spawn thread for counter:1 DONE with the long function ABOUT to spawn thread for counter:2 DONE with the long function ABOUT to spawn thread for counter:3 DONE with the long function ABOUT to spawn thread for counter:4 DONE with the long function </code></pre> <p>I have looked at <a href="https://stackoverflow.com/questions/13051591/tornado-blocking-asynchronous-requests">Tornado blocking asynchronous requests</a> and tried both the solutions. But both of them are blocking when I run them with back to back requests to the same handler. Can somebody figure out what I am doing wrong? I know tornado doesn't do well with multithreading, but I should be able to run a new thread from it, in a non-blocking way.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload