Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>How about using twisted instead? You can reach for much simpler application structure. You can send all 400 requests from the django process at once and wait for all of them to finish. This works simultaneously because twisted sets the sockets into non-blocking mode and only reads the data when its available. </p> <p>I had a similar problem a while ago and I've developed a nice bridge between twisted and django. I'm running it in production environment for almost a year now. You can find it here: <a href="https://github.com/kowalski/featdjango/" rel="nofollow">https://github.com/kowalski/featdjango/</a>. In simple words it has the main application thread running the main twisted reactor loop and the django view results is delegated to a thread. It use a special threadpool, which exposes methods to interact with reactor and use its asynchronous capabilities.</p> <p>If you use it, your code would look like this: </p> <pre><code>from twisted.internet import defer from twisted.web.client import getPage import threading def get_reports(self, urls, *args, **kw): ct = threading.current_thread() defers = list() for url in urls: # here the Deferred is created which will fire when # the call is complete d = ct.call_async(getPage, args=[url] + args, kwargs=kw) # here we keep it for reference defers.append(d) # here we create a Deferred which will fire when all the # consiting Deferreds are completed deferred_list = defer.DeferredList(defers, consumeErrors=True) # here we tell the current thread to wait until we are done results = ct.wait_for_defer(deferred_list) # the results is a list of the form (C{bool} success flag, result) # below unpack it reports = list() for success, result in results: if success: reports.append(result) else: # here handle the failure, or just ignore pass return reports </code></pre> <p>This still is something you can optimize a lot. Here, every call to getPage() would create a separate TCP connection and close it when its done. This is as optimal as it can be, providing that each of your 400 requests is sent to a different host. If this is not a case, you can use a http connection pool, which uses persistent connections and http pipelineing. You instantiate it like this:</p> <pre><code>from feat.web import httpclient pool = httpclient.ConnectionPool(host, port, maximum_connections=3) </code></pre> <p>Than a single request is perform like this (this goes instead the getPage() call):</p> <pre><code>d = ct.call_async(pool.request, args=(method, path, headers, body)) </code></pre>
    singulars
    1. This table or related slice is empty.
    1. POTweaking celery for high performance
      singulars
      1. PTQuestion
    1. PTAnswer
    1. This table or related slice is empty.
    1. USMarek Kowalski
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. COThis looks interesting, but celery is already used in my system, so I would prefer using it. If I won't find a solution I will look into it.
      singulars
      1. PO
      1. USlev
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload