Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>The copying activity of cudaMemcpyAsync (as well as kernel activity) can be overlapped with <em>any</em> host code. Furthermore, data copy to and from the device (via cudaMemcpyAsync) can be overlapped with kernel activity. All 3 activities: host activity, data copy activity, and kernel activity, can be done asynchronously to each other, and can overlap each other. </p> <p>As you have seen and demonstrated, host activity and data copy or kernel activity can be overlapped with each other in a relatively straightforward fashion: kernel launches return immediately to the host, as does cudaMemcpyAsync. However, to get best overlap opportunities between data copy and kernel activity, it's necessary to use some additional concepts. For best overlap opportunities, we need:</p> <ol> <li>Host memory buffers that are pinned, e.g. via cudaHostAlloc()</li> <li>Usage of cuda streams to separate various types of activity (data copy and kernel computation)</li> <li>Usage of cudaMemcpyAsync (instead of cudaMemcpy)</li> </ol> <p>Naturally your work also needs to be broken up in a separable way. This normally means that if your kernel is performing a specific function, you may need multiple invocations of this kernel so that each invocation can be working on a separate piece of data. This allows us to copy data block B to the device while the first kernel invocation is working on data block A, for example. In so doing we have the opportunity to overlap the copy of data block B with the kernel processing of data block A.</p> <p>The main differences with cudaMemcpyAsync (as compared to cudaMemcpy) are that:</p> <ol> <li>It can be issued in any stream (it takes a stream parameter)</li> <li>Normally, it returns control to the host <em>immediately</em> (just like a kernel call does) rather than waiting for the data copy to be completed.</li> </ol> <p>Item 1 is a necessary feature so that data copy can be overlapped with kernel computation. Item 2 is a necessary feature so that data copy can be overlapped with host activity.</p> <p>Although the concepts of copy/compute overlap are pretty straightforward, in practice the implementation requires some work. For additional references, please refer to:</p> <ol> <li><a href="http://docs.nvidia.com/cuda/cuda-c-best-practices-guide/index.html#asynchronous-transfers-and-overlapping-transfers-with-computation" rel="noreferrer">Overlap copy/compute section</a> of the CUDA best practices guide.</li> <li>Sample code showing a <a href="http://docs.nvidia.com/cuda/cuda-samples/index.html#simple-multi-copy-and-compute" rel="noreferrer">basic implementation of copy/compute overlap</a>.</li> <li>Sample code showing a full <a href="http://docs.nvidia.com/cuda/cuda-samples/index.html#concurrent-kernels" rel="noreferrer">multi/concurrent kernel copy/compute overlap scenario</a>.</li> </ol> <p>Note that some of the above discussion is predicated on having a compute capability 2.0 or greater device (e.g. concurrent kernels). Also, different devices may have one or 2 copy engines, meaning simultaneous copy <em>to the device</em> and copy <em>from the device</em> is only possible on certain devices. </p>
    singulars
    1. This table or related slice is empty.
    1. POAbout cudaMemcpyAsync Function
      singulars
      1. PTQuestion
    1. PTAnswer
    1. This table or related slice is empty.
    1. USRobert Crovella
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. POAbout cudaMemcpyAsync Function
      singulars
      1. PTQuestion
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTAcceptedByOriginator
    1. COThanks Robert. Your answer is that I wanted. And in my program, I'm also using streams. In my case, my code repeats these steps almost 1000times so I'm using 1000 streams. But I don't know in my case using 1000 streams is best way or not. Is it better to use more and more streams?
      singulars
      1. PO
      1. USUmbrella
    2. COIf you haven't run into trouble yet, then I guess you haven't hit [the stream limit](http://stackoverflow.com/questions/3565793/max-number-of-streams-in-cuda). But normally it's only necessary to create 3 or at most 4 streams, to arrange for copy and compute overlap. You simply re-use the same stream IDs over and over again, as you are processing through your data in chunks. If you are using a stream ID and then retiring it, there's no problem with your approach. But you're only going to get concurrency from around 16 simultaneous activities or less, not thousands.
      singulars
      1. PO
      1. USRobert Crovella
    3. COCan I get a web site link or a name of textbook which is described above your explanation? Because I'm writting a paper so I need it for reference. Anyway your advice is very helpful to my work. Thank you very very much.
      singulars
      1. PO
      1. USUmbrella
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload