Databases
StackOverflow2013
Postsdbo
POStrange behaviour of ExecutorService
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POStrange behaviour of ExecutorService
    primarykey
    data
    text
    <p>I have 5000 similar Callable tasks to be executed in 8 threads of ExecutorService created by Executors.newFixedThreadPool(8). Each task goes to a database to retrieve a lot of data to process. </p> <p>Everything works fine 99% time, BUT sometimes I see a very strange execution log messages in the log file, when DB is slow or stuck (don't ask why) and 8 currently running tasks are stalled and not finished yet in all 8 threads, ExecutorService starts submitting more tasks to execute one by one!</p> <p>So the log shows that at some point ExecutorService goes crazy and starts calling call() method of Callable of more and more tasks in the waiting queue without waiting the previous tasks to complete. More and more tasks send requests to DB which finally brings DB to it's knees and Java heap memory becomes exhausted.</p> <p>It looks like something strange is happening inside ExecutorService or my understanding of the situation is wrong. Has anyone seen anything like that? </p> <p>My brain stack is overflown</p> <p>p.s. the is the quote from the Java API:</p> <blockquote> <p>Executors.newFixedThreadPool(int nThreads)</p> <p>Creates a thread pool that reuses a fixed number of threads operating off a shared unbounded queue. At any point, at most nThreads threads will be active processing tasks. If additional tasks are submitted when all threads are active, they will wait in the queue until a thread is available. <strong>If any thread terminates due to a failure during execution prior to shutdown, a new one will take its place if needed to execute subsequent tasks</strong>.</p> </blockquote> <p>Could this actually happen that my tasks cause thread to die and ExecutorService creates more threads and submits new 8 tasks to them and they die and ExecutorService creates 8 more threads and submits more 8 tasks?</p> <p>p.s.s.: the whole operation inside call() of Callable is surrounded with try catch so if any exception is happening inside my operation the exception will be captured and logged. None of this is happening. The call is called and just never returns, while next tasks are beign called one by one and never returning and never finishing and never throwing any exceptions. </p> <p>I suspect my tasks cause threads in thread pool to die. How is it possible to imitate?</p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. USkachanov
    1. USkachanov
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. COI expect you're going to need to post example code. That shouldn't happen, so it sounds like there's something else going on.
      singulars
      1. POStrange behaviour of ExecutorService
      1. USTim
    2. COUnfortunately real example is a trade secret and imitation example will not reproduce the issue
      singulars
      1. POStrange behaviour of ExecutorService
      1. USkachanov
    3. COAre you sure the tasks are actually stalled, or are the database connections just hanging around? Maybe it's just a case where your code in the Callable isn't cleaning up properly in the event of an error.
      singulars
      1. POStrange behaviour of ExecutorService
      1. USDaniel Pryden
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload