Databases
StackOverflow2013
Postsdbo
POTwisted deferred vs blocking in web services
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POTwisted deferred vs blocking in web services
    primarykey
    data
    text
    <p>I'm struggling to produce the same behavior in web service code that uses Deferred objects as in code that does not. My objective is to write a decorator that will delegate processing of any method (which is decoupled from Twisted) to the Twisted thread pool, so that the reactor is not blocked, without changing any of that method's semantics.</p> <p>When an instance of class echo below is exposed as a web service, this code:</p> <pre><code>from twisted.web import server, resource from twisted.internet import defer, threads from cgi import escape from itertools import count class echo(resource.Resource): isLeaf = True def errback(self, failure): return failure def callback1(self, request, value): #raise ValueError # E1 lines = ['&lt;html&gt;&lt;body&gt;\n', '&lt;p&gt;Page view #%s in this session&lt;/p&gt;\n' % (value,), '&lt;/body&gt;&lt;/html&gt;\n'] return ''.join(lines) def callback2(self, request, encoding): def execute(message): #raise ValueError # E2 request.write(message.encode(encoding)) #raise ValueError # E3 request.finish() #raise ValueError # E4 return server.NOT_DONE_YET return execute def render_GET(self, request): content_type, encoding = 'text/html', 'UTF-8' request.setHeader('Content-Type', '%s; charset=%s' % tuple(map(str, (content_type, encoding)))) s = request.getSession() if not hasattr(s, 'counter'): s.counter = count(1) d = threads.deferToThread(self.callback1, request, s.counter.next()) d.addCallback(self.callback2(request, encoding)) d.addErrback(self.errback) #raise ValueError # E5 return server.NOT_DONE_YET </code></pre> <p>will display an HTML document to the browser when all the raise statements are commented out, and display a nicely formatted stack trace (which Twisted does for me) when the raise statement labelled "E5" is included. That is what I want. Likewise, if I do not use Deferred objects at all and place all the behavior from callback1 and callback2 within render_GET(), an exception raised anywhere within render_GET will produce the desired stack trace.</p> <p>I am trying to write code that will respond to the browser immediately, not cause resource leaks within Twisted, and cause the browser stack trace to also be displayed in the cases where any of the raise statements "E1" to "E3" is included in the deferred code--though of course I understand that the stack traces themselves will be different. (The "E4" case I don't care about as much.) After reading the Twisted documentation and other questions on this site I am unsure how to achieve this. I would have thought that adding an errback should facilitate this, but evidently not. There must be something about Deferred objects and the twisted.web stack that I'm not understanding.</p> <p>The effects on logging I document here may be affected by my use of the PythonLoggingObserver to bridge Twisted logging to the standard logging module.</p> <p>When "E1" is included, the browser waits until the reactor is shut down, at which point the ValueError exception with stack trace is logged and the browser receives an empty document.</p> <p>When "E2" is included, the ValueError exception with stack trace is logged immediately, but the browser waits until the reactor shuts down at which point it receives an empty document.</p> <p>When "E3" is included, the ValueError exception with stack trace is logged immediately, the browser waits until the reactor shuts down, and at that point receives the intended document.</p> <p>When raise statement "E4" is included, the intended document is returned to the browser immediately, and the ValueError exception with stack trace is logged immediately. (Is there any possibility of a resource leak in this case?)</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. This table or related slice is empty.
    1. USwberry
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POTwisted deferred vs blocking in web services
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POTwisted deferred vs blocking in web services
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POTwisted deferred vs blocking in web services
      1. This table or related slice is empty.
      1. VTUpMod
    1. COYour `errback` method doesn't do anything. It is simply acting as a pass-thru. Returning the `failure` to the deferred will just cause the next `errback` to be called, or if its the end of the chain the failure will sit in the deferred till it gets garbage collected, then the original exception will be raised, with the original stack trace. If you wanted the trace earlier, then have you `errback` re raise the original exception and **not** return the `failure`.
      singulars
      1. POTwisted deferred vs blocking in web services
      1. USthomas
    2. COIf I use an errback function that simply raises ValueError again the results are unchanged from the problem statement. The Twisted documentation seems to be telling me that raising and returning a Failure instance are equivalent in the callback/errback chain. It seems necessary for the errback function to directly call the request.processingFailed() method to get a stack trace to the browser.
      singulars
      1. POTwisted deferred vs blocking in web services
      1. USwberry
    3. COYes sorry. If you look at my answer below you can see that I have corrected my mistake. What I meant is that you can print out the exception trace to stdout or to the log module. I didn't understand that you wanted to print the trace to the browser. In that case I will modify my answer to do what you want.
      singulars
      1. POTwisted deferred vs blocking in web services
      1. USthomas
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload