Databases
StackOverflow2013
Postsdbo
POHow to design a program for many connections?
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POHow to design a program for many connections?
    primarykey
    data
    text
    <p>I've been self-teaching myself programming(java..mostly from books and alot from reading questions here) and my programs work but I'm not sure they are as fast as they could be. I started profiling my applications and a big trend in all my programs seems to be connecting/pulling/sending data to other services(queue servers, database servers,etc. typically 40%+ of the program is spend on connection related tasks.).</p> <p>Is there a design pattern or a structure people follow to get rid of this problem? If I have a simple problem that pulls data from a database, does something to the result(for example, result+1) and sends it back, then how can I structure the program so most of the time is spend on the processing of data itself? I understand the polling of data is probally more complex then I work I have to do, but what if I'm doing this at a large scale and need to do many simple tasks from a database/queue server.</p> <p>This has been bothering me for a while, so I learned how to use threads and created a poll of threads that stay ready to concurrently send data(so my program isn't waiting for acknowledgments) but I can't seem to figure out how to improve the inflow of data(say I query 50 data items a time, as the program works on the data I still can't poll new data faster than the cpu can process it). I've also made sure I don't reopen connections/channels on each request but still it takes a long time.</p> <p>I'm sure this is a common problem and would appreciate if anyone could point me to any resources where I can learn how to design my programs(preferably java) so such a big chunk of it isn't spend on connection/service api's to get data. I'm looking for general pointers and any examples would be nice. Let me know if the question is too general and I can easily recreate my problem with a code sample.</p> <p>Thanks and hopefully I explained this correctly.</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. This table or related slice is empty.
    1. USLostsoul
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. COAre you using any ORM/JPA framework, connection pooling, caches, ...?
      singulars
      1. POHow to design a program for many connections?
      1. USmrab
    2. COYour question is essentially "how do I effectively deal with high latency?" and that is an open question in the industry right now; there is no slam dunk obviously correct solution. Determining what set of abstractions are needed to be embedded in languages and frameworks to help developers solve these problems is quite difficult. We are attempting to do so in C# and VB by making a form of coroutines a first-class language feature, but that is certainly not the only possible solution.
      singulars
      1. POHow to design a program for many connections?
      1. USEric Lippert
    3. COIt is also possible that the client is simply more powerful than the database and it will not be able to feed you data faster than you can process it without tuning on that side.
      singulars
      1. POHow to design a program for many connections?
      1. USAffe
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload