Databases
StackOverflow2013
Postsdbo
POC - Unable to free allocated memory
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POC - Unable to free allocated memory
    primarykey
    data
    text
    <p>I have a problem with an application I'm currently developing. In this program I have to read huge amounts (billions) of data from text files and manage them consequently, but since it's a two students project, the reading part will be developed by my mate. For testing reason I wrote a small procedures that generates pseudo-random structures to replace what my mate will do.</p> <p>The problem is the following: a big amount of the generated data (due to redundancy) can be discarded in order to free its memory. But even invoking the free() function the memory usage keeps growing. So I tried to develop a debug application that simply generates a chunk of data and immediately frees it. And repeats that for thousands of times. <strong>Well, I can't grasp the reason, but the memory allocated to the process grows to ~1.8 GB ram and then crashes.</strong> Why? The strangest thing, that makes me thing there's a lot I'm not understanding well, is that when the process crashes the malloc does NOT return a NULL pointer, because the process always crashes when readCycles == 6008 and bypasses the NULL check.</p> <p>I already read other related topics here on StackOverflow and I understood why free() doesn't reduce the memory allocated to my process. That's fine. But why the memory usage keeps growing? Shouldn't the malloc allocate previously freed memory instead of constantly requesting new one?</p> <p>This is the most relevant part of my code:</p> <pre><code>#define NREAD 1000 #define READCYCLES 10000 #define N_ALPHA_ILLUMINA 7 #define N_ALPHA_SOLID 5 #define SEQLEN 76 typedef struct{ char* leftDNA; char* leftQuality; unsigned long int leftRow; char* rightDNA; char* rightQuality; unsigned long int rightRow; } MatePair; unsigned long int readCycles = 0; MatePair* readStream(MatePair* inputStream, short* eof, unsigned long int* inputSize){ double r; unsigned long int i, j; unsigned long int leftRow; int alphabet[] = {'A', 'C', 'G', 'T', 'N'}; inputStream = (MatePair*) malloc (sizeof(MatePair) * (NREAD + 1)); printf("%d\n", readCycles); if (inputStream == NULL){ (*eof) = 1; return; } for (i = 0; i &lt; NREAD; i++){ leftRow = readCycles * NREAD + i; inputStream[i].leftDNA = (char*) malloc (SEQLEN); inputStream[i].rightDNA = (char*) malloc (SEQLEN); inputStream[i].leftQuality = (char*) malloc (SEQLEN); inputStream[i].rightQuality = (char*) malloc (SEQLEN); for (j = 0; j &lt; SEQLEN; j++){ r = rand() / (RAND_MAX + 1); inputStream[i].leftDNA[j] = alphabet[(int)(r * 5)]; inputStream[i].rightDNA[j] = alphabet[(int)(r * 5)]; inputStream[i].leftQuality[j] = (char) 64 + (int)(r * 60); inputStream[i].rightQuality[j] = (char) 64 + (int)(r * 60); } inputStream[i].leftDNA[SEQLEN - 1] = '\0'; inputStream[i].rightDNA[SEQLEN - 1] = '\0'; inputStream[i].leftQuality[SEQLEN - 1] = '\0'; inputStream[i].rightQuality[SEQLEN - 1] = '\0'; inputStream[i].leftRow = leftRow; inputStream[i].rightRow = leftRow; } inputStream[i].leftRow = -1; readCycles++; (*inputSize) = NREAD; (*eof) = readCycles &gt; READCYCLES; return inputStream; } int main(int argc, char* argv[]){ short eof = 0; unsigned long int inputSize = 0; MatePair* inputStream = NULL; while (!eof){ inputStream = readStream(inputStream, &amp;eof, &amp;inputSize); free(inputStream); inputStream = NULL; } return 0; } </code></pre> <p>I forgot to mention that, but before posting here, instead of calling free(inputStream), I tried invoking freeMemory(inputStream). Not sure if it's the correct way of doing it, though.</p> <pre><code>void freeMemory(MatePair* memblock){ for ( ; memblock-&gt;leftRow != 1; memblock++){ free(memblock -&gt; leftDNA); free(memblock -&gt; leftQuality); free(memblock -&gt; rightDNA); free(memblock -&gt; rightQuality); } } </code></pre>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USAlex
    1. USAlex
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. COTry adding checks for all your `malloc` calls and run the program with valgrind. Also not your problem, but `r = rand() / (RAND_MAX + 1)` is always going to be `0` unless you add a cast to `double` before dividing.
      singulars
      1. POC - Unable to free allocated memory
      1. USuser786653
    2. COYou have several memory leaks here, where are the memory allocated for inputstream[i].leftDNA etc released.
      singulars
      1. POC - Unable to free allocated memory
      1. USharald
    3. COYes I already noticed the (double) thing, forget to modify that yet because I was more worried with the memory leak thing :)
      singulars
      1. POC - Unable to free allocated memory
      1. USAlex
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload