Databases
StackOverflow2013
Postsdbo
POboost::thread and starting or not starting large numbers of threads at the same time
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POboost::thread and starting or not starting large numbers of threads at the same time
    primarykey
    data
    text
    <p>In researching my question on this forum, I have found my same basic question about boost::thread asked a number of times before, but none of the answers seem to answer my specific needs. The thrust of the question from the way I'm thinking about it is how to instantiate a number of threads without actually launching any of them until you are ready. The answer always seems to be in the vein of why not just wait until you are ready to launch each thread before instantiating it. Indeed, it appears to me that's the way boost::thread is designed to work. That is, it appears that instantiating a boost::thread object seems to want also to launch the thread, and it's not clear to me if these to functions can really be separated.</p> <p>I'll try to explain a little more about my application than some of the other questioners have explained. I need to launch at least dozens if not hundreds or even thousands of threads. Each thread will run the exact same function, and the threads will be differentiated from each other by a single and unique argument that is passed to the function as a part of each thread. Each thread will use large amounts of shared memory that will be read only and that does not need to be synchronized. Each thread also will use a substantial amount of local memory that will be allocated dynamically and which will be released prior to the completion of the thread. The local memory also does not need to be synchronized. Indeed, I'm thinking I may be able to get by without any synchronization between the threads at all except joins to await the completion of each thread.</p> <p>The program will run for dozens or even hundreds or possibly thousands of hours, so I want to use all the processor cores on my machine, but I don't want to initiate more threads at the same time than the number of processor cores that I have available. The problem is that if I were to initiate too many threads at the same time, they would quickly exhaust my machine's memory because of the local memory used by each thread. But if I can manage to initiate only as many threads as I have processor cores, then the memory situation will be fine.</p> <p>So if I have n cores, I want to launch n threads and wait until one of them is done. When one of then is done, I want to launch one more, etc. until all the work is done. I have looked at thread_group and join_all. On the thread_group itself, I can't figure out how to wait until one of the first n threads is completed before instantiating the (n+1)st one, etc. And join_all is not what I really need because I just want to wait until less than n threads are still running, and it shouldn't matter which one finishes first. Whenever the number of active threads becomes less than n, I want to launch a new one. I'm quite sure that add_thread() is going to be involved in this somehow or other. I'm also quite sure that the boost::thread size() function is going to be involved somehow or other. But I can't really see how to put all the pieces together. And the problem always seems to come down to not quite being able to understand how to instantiate a thread only when I'm ready. And for example, all the boost::thread examples I have been able to find seem to have "statically named" thread objects, viz., "boost::thread MyWorkerFunction" etc. that get instantiated as they are encountered in the code. Well, I could have thousands of threads for the same MyWorkerFunction and I only want to dispatch n of the threads at the same time, and I don't understand how to do that.</p> <p>Thanks, Jerry</p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. USuser2738430
    1. USuser2738430
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. COmay be you should read about the keyword "thread pool"?
      singulars
      1. POboost::thread and starting or not starting large numbers of threads at the same time
      1. USoguzalb
    2. COhttp://www.boost.org/doc/libs/1_54_0/doc/html/thread/thread_management.html#thread.thread_management.threadgroup
      singulars
      1. POboost::thread and starting or not starting large numbers of threads at the same time
      1. USChris Desjardins
    3. COIf examples you have seen are all like that: 'boost::thread examples I have been able to find seem to get instantiated as they are encountered in the code', that is somewhat surprising, and saddening. Continually creating/terminating/destroying threads is expensive, slow and error-prone. If some functionality is required many times during an app run, I dont see much point in all that misery - use a threadpool or dedicated app-lifetime threads that get signaled to run and, after performing some task/action, loop around to wait on a signal again.
      singulars
      1. POboost::thread and starting or not starting large numbers of threads at the same time
      1. USMartin James
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload