Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>It seems sensible to multi-thread or multi-process to some degree to accomplish this. The degree to which you multi-thread or multi-process is the question.</p> <p>1) You could dump the polling system entirely and use a thread/process per connection. That thread can then stall as long as it wants working on the processing for that connection. You'd then have to decide on creating/killing a thread/process each time (probably easiest) or having a pool of threads/processes (probably fastest).</p> <p>2) You could have a thread/process for the networky bits and hand off the processing to one other thread. This is less parallel, but it does mean you can at least keep handling network connections whilst you're chopping through the list of work. This gives you control of what processing is being handled at least. It would be easy to prioritise incoming connections this way, whereas option 1 might not.</p> <p>3) (sort of possible 1 &amp; 2) You could use asynchronous I/O to multiplex your connections. You still to handle the processing in the same way as 1 &amp; 2 above.</p> <p>You also have the question of threads vs processes. Threads are probably quicker to get going but it's more difficult to ensure data integrity. Processes are going to be more resilient but require more interfacing between them.</p> <p>You also have to decide on a way to pass data between the threads/processes. This is less of an issue for option 1 as you only have to pass off the connection to the thread. Option 2 may (depending on what your data is) be more of a problem. You could use a message queue for passing the messages about but if you have a lot of data to send shared memory is more appropriate. Shared memory is a pain to engineer for processes but easy with threads (as all threads share the same memory space).</p> <p>There are performance issues as you get to this scale too. It's worth investigating performance characteristics for these things. The differences to how calls like select and poll scale is significant when you're dealing with a lot of connections.</p> <p>Without knowledge of what data is being sent and received it's hard to give solid recommendations.</p> <p>Incidentally, this isn't a new problem. <a href="http://www.kegel.com/" rel="nofollow">Dan Kegel</a> had a <a href="http://www.kegel.com/c10k.html" rel="nofollow">good article</a> about it a few years back. It's now out-of-date, but the overview is still good. You should research the current state of the art for the concepts he discusses though.</p>
    singulars
    1. This table or related slice is empty.
    1. POCompute data from multiple clients simultaneously
      singulars
      1. PTQuestion
    1. PTAnswer
    1. USJoe
    1. USJoe
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    1. COFirstly, thanks for your answer @Joe, it features some interesting and instructive ideas. About the answer: The data that is being sent is a single jpeg image (it's binary representation) on every connection (not much more than 100kB). I've been searching about the option of creating a thread per connection and, although the most appealing of the three for me, but I have my doubts about it's implementation given such a considerable scale. It would be really inspiring if there were some example of someone that has used this approach successfully in a similar case, but I didn't found any.
      singulars
      1. PO
      1. USStr1101
    2. COI also found promising option 3, but I have no experience with it and the examples I found in the web show very different methods (coroutines, state machines, green threads, etc...) and I don't really know in which one I should focus. (p.s.:C10k is a very good article, but it is hard to find some clear updates to the ideas it presents)
      singulars
      1. PO
      1. USStr1101
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload