Databases
StackOverflow2013
Postsdbo
POI think STL is causing my application triple its memory usage
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POI think STL is causing my application triple its memory usage
    primarykey
    data
    text
    <p>I am inputting a 200mb file in my application and due to a very strange reason the memory usage of my application is more than 600mb. I have tried vector and deque, as well as std::string and char * with no avail. I need the memory usage of my application to be almost the same as the file I am reading, any suggestions would be extremely helpful. Is there a bug that causes so much memory consumption? Could you pinpoint the problem or should I rewrite the whole thing?</p> <p>Windows Vista SP1 x64, Microsoft Visual Studio 2008 SP1, 32Bit Release Version, Intel CPU</p> <p>The whole application until now:</p> <pre><code>#include &lt;string&gt; #include &lt;vector&gt; #include &lt;iostream&gt; #include &lt;iomanip&gt; #include &lt;fstream&gt; #include &lt;sstream&gt; #include &lt;iterator&gt; #include &lt;algorithm&gt; #include &lt;time.h&gt; static unsigned int getFileSize (const char *filename) { std::ifstream fs; fs.open (filename, std::ios::binary); fs.seekg(0, std::ios::beg); const std::ios::pos_type start_pos = fs.tellg(); fs.seekg(0, std::ios::end); const std::ios::pos_type end_pos = fs.tellg(); const unsigned int ret_filesize (static_cast&lt;unsigned int&gt;(end_pos - start_pos)); fs.close(); return ret_filesize; } void str2Vec (std::string &amp;str, std::vector&lt;std::string&gt; &amp;vec) { int newlineLastIndex(0); for (int loopVar01 = str.size(); loopVar01 &gt; 0; loopVar01--) { if (str[loopVar01]=='\n') { newlineLastIndex = loopVar01; break; } } int remainder(str.size()-newlineLastIndex); std::vector&lt;int&gt; indexVec; indexVec.push_back(0); for (unsigned int lpVar02 = 0; lpVar02 &lt; (str.size()-remainder); lpVar02++) { if (str[lpVar02] == '\n') { indexVec.push_back(lpVar02); } } int memSize(0); for (int lpVar03 = 0; lpVar03 &lt; (indexVec.size()-1); lpVar03++) { memSize = indexVec[(lpVar03+1)] - indexVec[lpVar03]; std::string tempStr (memSize,'0'); memcpy(&amp;tempStr[0],&amp;str[indexVec[lpVar03]],memSize); vec.push_back(tempStr); } } void readFile(const std::string &amp;fileName, std::vector&lt;std::string&gt; &amp;vec) { static unsigned int fileSize = getFileSize(fileName.c_str()); static std::ifstream fileStream; fileStream.open (fileName.c_str(),std::ios::binary); fileStream.clear(); fileStream.seekg (0, std::ios::beg); const int chunks(1000); int singleChunk(fileSize/chunks); int remainder = fileSize - (singleChunk * chunks); std::string fileStr (singleChunk, '0'); int fileIndex(0); for (int lpVar01 = 0; lpVar01 &lt; chunks; lpVar01++) { fileStream.read(&amp;fileStr[0], singleChunk); str2Vec(fileStr, vec); } std::string remainderStr(remainder, '0'); fileStream.read(&amp;remainderStr[0], remainder); str2Vec(fileStr, vec); } int main (int argc, char *argv[]) { std::vector&lt;std::string&gt; vec; std::string inFile(argv[1]); readFile(inFile, vec); } </code></pre>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USBradleyDotNET
    1. USSMeyers
    plurals
    1. PL
      singulars
      1. LTLinked
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    3. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POI think STL is causing my application triple its memory usage
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POI think STL is causing my application triple its memory usage
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POI think STL is causing my application triple its memory usage
      1. USfinitud
      1. VTFavorite
    1. COWhich STL are you using? On which machine?
      singulars
      1. POI think STL is causing my application triple its memory usage
      1. USEdouard A.
    2. COWhat's the memory usage with a very very small file?
      singulars
      1. POI think STL is causing my application triple its memory usage
      1. USDJClayworth
    3. COJust so you know, you don't *need* to seperate call open with fstreams, you can just do: std::ifstream file("whatever", std::ios::binary); also, when the ifstream object is destructed, it will auto-close too. So usually you don't need to explicitly close either.
      singulars
      1. POI think STL is causing my application triple its memory usage
      1. USEvan Teran
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload