Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>After some time of brain breaking, I figured out my own mistakes... I want to put this new knowledge, so... here I go</p> <p>I made a very <strong>big mistake</strong> by declaring the Connection objet as a Static object in my code... so obviously, despite I created a new Connection for each new data object I created, every transaction went through a single, static, connection.</p> <p>With that first issue corrected, I went back to the design table, and realized that my process was: </p> <ol> <li>Read an Id from an input table</li> <li>Take a block of data related to the Id read in step 1, stored in other input tables</li> <li>Crunch numbers: Read the related input tables and process the data stored in them</li> <li>Save the results in one or more output tables</li> <li>Repeat the process while I have pending Ids in the input table</li> </ol> <p>Just by using a dedicated connection for input reading and a dedicated connection for output writing, the performance of my program increased... but I needed a lot more!</p> <p>My original approach for steps 3 and 4 was to save into the output each one of the results as soon as I had them... But I found a better approach: </p> <ul> <li>Read the input data</li> <li>Crunch the numbers, and put the results in a bunch of queues (one for each output table)</li> <li>A separated thread is checking every second if there's data in any of the queues. If there's data in the queues, write it to the tables.</li> </ul> <p>So, by dividing input and output tasks using different connections, and by redirecting the core process output to a queue, and by using a dedicated thread for output storage tasks, I finally achieved what I wanted: Multithreaded DML execution!</p> <hr> <p>I know there are better approaches to this particular problem, but this one works quite fine.</p> <p>So... if anyone is stuck with a problem like this... I hope this helps.</p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
 

Querying!

 
Guidance

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload