Databases
StackOverflow2013
Postsdbo
POWhy is MD5Sum so fast
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POWhy is MD5Sum so fast
    primarykey
    data
    text
    <p>I've been studying hashing in C/C++ and tried to replicate the md5sum command in Linux. After analysing the source code, it seems that md5sum relies on the md5 library's md5_stream. I've approximated the md5_stream function from the md5.h library into the code below, and it runs in ~13-14 seconds. I've tried to call the md5_stream function directly and got ~13-14 seconds. The md5sum runs in 4 seconds. What have the GNU people done to get the speed out of the code?</p> <p>The md5.h/md5.c code is available in the CoreUtils source code.</p> <pre><code>#include &lt;QtCore/QCoreApplication&gt; #include &lt;QtCore/QDebug&gt; #include &lt;iostream&gt; #include &lt;iomanip&gt; #include &lt;fstream&gt; #include "md5.h" #define BLOCKSIZE 32784 int main() { FILE *fpinput, *fpoutput; if ((fpinput = fopen("/dev/sdb", "rb")) == 0) { throw std::runtime_error("input file doesn't exist"); } struct md5_ctx ctx; size_t sum; char *buffer = (char*)malloc (BLOCKSIZE + 72); unsigned char *resblock = (unsigned char*)malloc (16); if (!buffer) return 1; md5_init_ctx (&amp;ctx); size_t n; sum = 0; while (!ferror(fpinput) &amp;&amp; !feof(fpinput)) { n = fread (buffer + sum, 1, BLOCKSIZE - sum, fpinput); if (n == 0){ break; } sum += n; if (sum == BLOCKSIZE) { md5_process_block (buffer, BLOCKSIZE, &amp;ctx); sum = 0; } } if (n == 0 &amp;&amp; ferror (fpinput)) { free (buffer); return 1; } /* Process any remaining bytes. */ if (sum &gt; 0){ md5_process_bytes (buffer, sum, &amp;ctx); } /* Construct result in desired memory. */ md5_finish_ctx (&amp;ctx, resblock); free (buffer); for (int x = 0; x &lt; 16; ++x){ std::cout &lt;&lt; std::setfill('0') &lt;&lt; std::setw(2) &lt;&lt; std::hex &lt;&lt; static_cast&lt;uint16_t&gt;(resblock[x]); std::cout &lt;&lt; " "; } std::cout &lt;&lt; std::endl; free(resblock); return 0; } </code></pre> <p>EDIT: Was a default mkspec problem in Fedora 19 64-bit.</p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. USOlivierLi
    1. USMagikWorx
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. VO
      singulars
      1. POWhy is MD5Sum so fast
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POWhy is MD5Sum so fast
      1. This table or related slice is empty.
      1. VTUpMod
    1. COWhen you build your code, are you building in debug mode? What optimization levels are you using?
      singulars
      1. POWhy is MD5Sum so fast
      1. USTim
    2. COIt's gnuutils... the source is available and you can look at it yourself.
      singulars
      1. POWhy is MD5Sum so fast
      1. USMarc B
    3. COTim : Currently I'm building in Qt. By default the Qt build environment adds the -O2 build flag. I compile and test in Release mode, so debugging libraries and hooks should not be interfering. MarcB : I obviously know that it's GNUUtils if I'm looking at the md5sum source code. I am looking for insight in how or where they optimized their build.
      singulars
      1. POWhy is MD5Sum so fast
      1. USMagikWorx
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload