Databases
StackOverflow2013
Postsdbo
POUsing ExecutorService to process jobs in parallel
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POUsing ExecutorService to process jobs in parallel
    primarykey
    data
    text
    <p>I am writing a java program that needs to process a lot of URLs.<br> Each URLs will run the following jobs IN ORDER: download, analyze, compress</p> <p>Instead of having one single thread to do all the jobs at once per URL, I want each job to have a fixed amount of threads, so that all the jobs will have threads running concurrently at any given time. </p> <p>For example, the download job will have multiple threads to fetch and download URLs, as soon as one of the URL is downloaded, it will pass it on to a thread in analyze job and as soon as it completes, it will then pass on to a thread in compress job, etc.</p> <p>I am thinking of using the CompletionService in java, since it returns a result as soon as its finished, but I am not sure how it works, so far my code looks like this:</p> <pre><code>ExecutorService executor = Executors.newFixedThreadPool(3); CompletionService&lt;DownloadedItem&gt; completionService = new ExecutorCompletionService&lt;DownloadedItem&gt;(executor); //while list has URL do { executor.submit(new DownloadJob(list.getNextURL());//submit to queue for download //} //while there is URL left do { Future&lt;DownloadedItem&gt; downloadedItem = executor.take();//take the result as soon as it finish //what to do here?? //} </code></pre> <p>My question is how do I move the downloaded item to the analyze job and do the work there without waiting for all the download jobs to complete? I am thinking of creating a CompletionService for each job, is that a viable method? If not, is there a better alternative way to solve this problem? Please provide examples.</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USNiroshan
    1. USuser1695270
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    3. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POUsing ExecutorService to process jobs in parallel
      1. USLanguagesNamedAfterCofee
      1. VTFavorite
    2. VO
      singulars
      1. POUsing ExecutorService to process jobs in parallel
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POUsing ExecutorService to process jobs in parallel
      1. This table or related slice is empty.
      1. VTUpMod
    1. This table or related slice is empty.
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload