Databases
StackOverflow2013
Postsdbo
POParallel I/O SSD vs HDD surprising results
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POParallel I/O SSD vs HDD surprising results
    text
    copied!<p>I have a very strange situation happening with some of my tests regarding paralell I/O. Here is the situation.. I have multiple threads opening a file handler to the same file and reading from multiple locations of the file (evenly spaced intervals) a finite number of bytes and dumping that into an array. All is done with boost threads. Now, I assume with an HDD that should be slower due to the random access seeking. This is why my tests are in fact targeted towards SSD. Turns out I almost do not get any speedup when reading the same file from a solid state disk compared to a HDD. Wonder what the problem might be? Does that seem very surprising just to me / I am also posting my code below to see what I am exactly doing: </p> <pre><code> void readFunctor(std::string pathToFile, size_t filePos, BYTE* buffer, size_t buffPos, size_t dataLn, boost::barrier&amp; barier) { FILE* pFile; pFile = fopen(pathToFile.c_str(), "rb"); fseek(pFile, filePos, SEEK_SET); fread(buffer, sizeof(BYTE), dataLn, pFile); fclose(pFile); barier.wait(); } void joinAllThreads(std::vector&lt;boost::shared_ptr&lt;boost::thread&gt; &gt; &amp;threads) { for (std::vector&lt;boost::shared_ptr&lt;boost::thread&gt; &gt;::iterator it = threads.begin(); it != threads.end(); ++it) { (*it).get()-&gt;join(); } } void readDataInParallel(BYTE* buffer, std::string pathToFile, size_t lenOfData, size_t numThreads) { std::vector&lt;boost::shared_ptr&lt;boost::thread&gt; &gt; threads; boost::barrier barier(numThreads); size_t dataPerThread = lenOfData / numThreads; for (int var = 0; var &lt; numThreads; ++var) { size_t filePos = var * dataPerThread; size_t bufferPos = var * dataPerThread; size_t dataLenForCurrentThread = dataPerThread; if (var == numThreads - 1) { dataLenForCurrentThread = dataLenForCurrentThread + (lenOfData % numThreads); } boost::shared_ptr&lt;boost::thread&gt; thread( new boost::thread(readFunctor, pathToFile, filePos, buffer, bufferPos, dataLenForCurrentThread, boost::ref(barier))); threads.push_back(thread); } joinAllThreads(threads); } </code></pre> <p>Now.. in my main file I pretty much have..:</p> <pre><code> int start_s = clock(); size_t sizeOfData = 2032221073; boost::shared_ptr&lt;BYTE&gt; buffer((BYTE*) malloc(sizeOfData)); readDataInParallel(buffer.get(), "/home/zahari/Desktop/kernels_big.dat", sizeOfData, 4); clock_t stop_s = clock(); printf("%f %f\n", ((double) start_s / (CLOCKS_PER_SEC)) * 1000, (stop_s / double(CLOCKS_PER_SEC)) * 1000); </code></pre> <p>Surprisingly, when reading from SSD, I do not get any speedup compared to HDD? Why might that be? </p>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload