Databases
StackOverflow2013
Postsdbo
POMemory settings with thousands of threads
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POMemory settings with thousands of threads
    primarykey
    data
    text
    <p>I'm playing around with the JVM (Oracle 1.7 64 bit) on a Linux box (AMD 6 Core, 16 GB RAM) to see how the number of threads in an application affects performance. I'm hoping to measure at which point context switching degrades performance.</p> <p>I have created a little application that creates a thread execution pool:</p> <pre><code>Executors.newFixedThreadPool(numThreads) </code></pre> <p>I adjust <code>numThreads</code> everytime I run the program, to see the effect it has.</p> <p>I then submit <code>numThread</code> jobs (instances of <code>java.util.concurrent.Callable</code>) to the pool. Each one increments an <code>AtomicInteger</code>, does some work (creates an array of random integers and shuffles it), and then sleeps a while. The idea is to simulate a web service call. Finally, the job resubmits itself to the pool, so that I always have <code>numThreads</code> jobs working.</p> <p>I am measuring the throughput, as in the number of jobs that are processed per minute.</p> <p>With several thousand threads, I can process up to 400,000 jobs a minute. Above 8000 threads, the results start to vary a lot, suggesting that context switching is becoming a problem. But I can continue to increase the number of threads to 30,000 and still get higher throughput (between 420,000 and 570,000 jobs per minute).</p> <p>Now the question: I get a <code>java.lang.OutOfMemoryError: Unable to create new native thread</code> with more than about 31,000 jobs. I have tried setting <code>-Xmx6000M</code> which doesn't help. I tried playing with <code>-Xss</code> but that doesn't help either.</p> <p>I've read that <code>ulimit</code> can be useful, but increasing with <code>ulimit -u 64000</code> didn't change anything.</p> <p>For info:</p> <pre><code>[root@apollo ant]# ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 127557 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 1024 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 1024 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited </code></pre> <p>So the question #1: What do I have to do to be able to create a bigger thread pool?</p> <p>Question #2: At what stage should I expect to see context switching really reducing throughput and causing the process to grind to a halt?</p> <hr> <p>Here are some results, after I modified it to do a little more processing (as was suggested) and started recording average response times (as was also suggested).</p> <pre><code>// ( (n_cores x t_request) / (t_request - t_wait) ) + 1 // 300 ms wait, 10ms work, roughly 310ms per job =&gt; ideal response time, 310ms // ideal num threads = 1860 / 10 + 1 = 187 threads // // results: // // 100 =&gt; 19,000 thruput, 312ms response, cpu &lt; 50% // 150 =&gt; 28,500 thruput, 314ms response, cpu 50% // 180 =&gt; 34,000 thruput, 318ms response, cpu 60% // 190 =&gt; 35,800 thruput, 317ms response, cpu 65% // 200 =&gt; 37,800 thruput, 319ms response, cpu 70% // 230 =&gt; 42,900 thruput, 321ms response, cpu 80% // 270 =&gt; 50,000 thruput, 324ms response, cpu 80% // 350 =&gt; 64,000 thruput, 329ms response, cpu 90% // 400 =&gt; 72,000 thruput, 335ms response, cpu &gt;90% // 500 =&gt; 87,500 thruput, 343ms response, cpu &gt;95% // 700 =&gt; 100,000 thruput, 430ms response, cpu &gt;99% // 1000 =&gt; 100,000 thruput, 600ms response, cpu &gt;99% // 2000 =&gt; 105,000 thruput, 1100ms response, cpu &gt;99% // 5000 =&gt; 131,000 thruput, 1600ms response, cpu &gt;99% // 10000 =&gt; 131,000 thruput, 2700ms response, cpu &gt;99%, 16GB Virtual size // 20000 =&gt; 140,000 thruput, 4000ms response, cpu &gt;99%, 27GB Virtual size // 30000 =&gt; 133,000 thruput, 2800ms response, cpu &gt;99%, 37GB Virtual size // 40000 =&gt; - thruput, -ms response, cpu &gt;99%, &gt;39GB Virtual size =&gt; java.lang.OutOfMemoryError: unable to create new native thread </code></pre> <p>I interpret them as:</p> <p>1) Even though the application is sleeping for 96.7% of the time, that still leaves lots of thread switching to be done 2) Context switching is measurable, and is shown in the response time.</p> <p>What is interesting here is that When tuning an app, you'd might choose the acceptable response time, say 400ms, and increase number of threads until you get that response time, which in this case would let the app process around 95 thousand requests a minute.</p> <p>Often people say that the ideal number of threads is near the number of cores. In apps that have wait time (blocked threads, say waiting for a database or web service to respond), the calculation needs to consider that (see my equation above). But even that theoretical ideal isn't an actual ideal, when you look at the results or when you tune to a specific response time.</p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. USAnt Kutschera
    1. USAnt Kutschera
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POMemory settings with thousands of threads
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POMemory settings with thousands of threads
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POMemory settings with thousands of threads
      1. This table or related slice is empty.
      1. VTUpMod
    1. COOut of curiousity, have you tried seeing what the performance looks like with only as many threads as you have cores? (Minus a couple, maybe?) It depends a LOT on what the actual work being performed is, but if it's not I/O bound, you may be surprised...
      singulars
      1. POMemory settings with thousands of threads
      1. UST.J. Crowder
    2. COFor question 2, you shouldn't max out throughput. You can stop adding more thread when the increase in number of threads results in more increase the response time than the increase in utilization/throughput.
      singulars
      1. POMemory settings with thousands of threads
      1. USnhahtdh
    3. COTo get maximum performance, the number of threads is roughly ( (n_cores x t_request) / (t_request - t_wait) ) + 1 where n_cores is the number of cores, t_request the time taken to process and wait for the web service response, and t_wait is the time taken waiting for the web service response. So the optimum is indeed quite close to the number of cores, but since I am simulating waiting for a web service response, its higher than just the number of cores. The point is though, that I want to show that lots of threads causes a problem, and I don't seem to be able to do that :-)
      singulars
      1. POMemory settings with thousands of threads
      1. USAnt Kutschera
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload