Databases
StackOverflow2013
Postsdbo
POstd::cin really slow
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POstd::cin really slow
    primarykey
    data
    text
    <p>So I was trying to write myself a command for a linux pipeline. Think of it as a replica of gnu 'cat' or 'sed', that takes input from stdin, does some processing and writes to stdout.</p> <p>I originally wrote an AWK script but wanted more performance so I used the following c++ code:</p> <pre><code>std::string crtLine; crtLine.reserve(1000); while (true) { std::getline(std::cin, crtLine); if (!std::cin) // failbit (EOF immediately found) or badbit (I/O error) break; std::cout &lt;&lt; crtLine &lt;&lt; "\n"; } </code></pre> <p>This is exactly what cat (without any parameters does). As it turns out, this program is about as slow as its awk counterpart, and nowhere near as fast as cat.</p> <p>Testing on a 1GB file:</p> <pre><code>$time cat 'file' | cat | wc -l real 0m0.771s $time cat 'file' | filter-range.sh | wc -l real 0m44.267s </code></pre> <p>Instead of getline(istream, string) I tried cin.getline(buffer, size) but no improvements. This is embarassing, is it a buffering issue? I also tried fetching 100KB at a time instead of just one line, no help! Any ideas?</p> <p>EDIT: What you folks say makes sense, BUT the culprit is not string building/copying and neither is scanning for newlines. (And neither is the size of the buffer). Take a look at these 2 programs:</p> <pre><code>char buf[200]; while (fgets(buf, 200, stdin)) std::cout &lt;&lt; buf; $time cat 'file' | ./FilterRange &gt; /dev/null real 0m3.276s char buf[200]; while (std::cin.getline(buf, 200)) std::cout &lt;&lt; buf &lt;&lt; "\n"; $time cat 'file' | ./FilterRange &gt; /dev/null real 0m55.031s </code></pre> <p>Neither of them manipulate strings and both of them do newline scanning, however one is 17 times slower than the other. They differ only by the use of cin. I think we can safely conclude that cin screws up the timing.</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. UShaelix
    1. UShaelix
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POstd::cin really slow
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POstd::cin really slow
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POstd::cin really slow
      1. USNathan Adams
      1. VTFavorite
    1. COWhat else is in `filter-range.sh`? Why don't you invoke your C++ program directly? Also, the typical pattern for that loop is `while(std::getline(std::cin, crtLine)) { std::cout << crtLine << "\n"; }`, but changing that shouldn't affect your question.
      singulars
      1. POstd::cin really slow
      1. USRobᵩ
    2. COIf you are looking for performance, you should try C-style I/O functions instead of cin/cout ;)
      singulars
      1. POstd::cin really slow
      1. USLihO
    3. CODid you compile with optimizations? -O2 or -O3? That probably won't shave 44 seconds off, but if your worried about timing it should definitely be done.
      singulars
      1. POstd::cin really slow
      1. USSaulBack
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload