StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
primarykey
Id
13593041
data
AcceptedAnswerId
0
AnswerCount
0
ClosedDate
CommentCount
9
CommunityOwnedDate
CreationDate
2012-11-27T21:05:57.050
FavoriteCount
0
LastActivityDate
2013-02-18T18:43:26.720
LastEditDate
2013-02-18T18:43:26.720
LastEditorUserId
586784
OwnerUserId
586784
ParentId
12290639
PostTypeId
2
Score
28
ViewCount
0
LastEditorDisplayName
text
Body
Summary (notable points): <ul> <li>Two packages benchmarked (one with 78 compilation units, one with 301 compilation units)</li> <li>Traditional Compiling (Multi Unit Compilation) resulted in a 7% faster application (in the 78 unit package); no change in application runtime in the 301 unit package.</li> <li>Both Traditional Compiling and Header-only benchmarks used the same amount of memory when running (in both packages).</li> <li>Header-only Compiling (Single Unit Compilation) resulted in an executable size that was 10% smaller in the 301 unit package (only 1% smaller in the 78 unit package).</li> <li>Traditional Compiling used about a third of the memory to build over both packages.</li> <li>Traditional Compiling took three times as long to compile (on the first compilation) and took only 4% of the time on recompile (as header-only has to recompile the all sources).</li> <li>Traditional Compiling took longer to link on both the first compilation and subsequent compilations.</li> </ul> Box2D benchmark, data: <a href="https://gist.github.com/4155964#file_box2d_data_gcc.csv" rel="noreferrer">box2d_data_gcc.csv</a> Botan benchmark, data: <a href="https://gist.github.com/4155964#file_botan_data_gcc.csv" rel="noreferrer">botan_data_gcc.csv</a> Box2D SUMMARY (78 Units) <img src="https://i.stack.imgur.com/nv3Ek.png" alt="enter image description here"> Botan SUMMARY (301 Units) <img src="https://i.stack.imgur.com/hr8oc.png" alt="enter image description here"> NICE CHARTS: Box2D executable size: <img src="https://i.stack.imgur.com/YOeGa.png" alt="Box2D executable size"> Box2D compile/link/build/run time: <img src="https://i.stack.imgur.com/OCoJd.png" alt="Box2D compile/link/build/run time"> Box2D compile/link/build/run max memory usage: <img src="https://i.stack.imgur.com/LzMXz.png" alt="Box2D compile/link/build/run max memory usage"> Botan executable size: <img src="https://i.stack.imgur.com/6Bqvx.png" alt="Botan executable size"> Botan compile/link/build/run time: <img src="https://i.stack.imgur.com/yad3Z.png" alt="Botan compile/link/build/run time"> Botan compile/link/build/run max memory usage: <img src="https://i.stack.imgur.com/YQ48a.png" alt="Botan compile/link/build/run max memory usage"> <hr> <h2>Benchmark Details</h2> TL;DR <hr> The projects tested, <a href="http://box2d.org/" rel="noreferrer">Box2D</a> and <a href="http://botan.randombit.net/" rel="noreferrer">Botan</a> were chosen because they are potentially computationally expensive, contain a good number of units, and actually had few or no errors compiling as a single unit. Many other projects were attempted but were consuming too much time to "fix" into compiling as one unit. The memory footprint is measured by polling the memory footprint at regular intervals and using the maximum, and thus might not be fully accurate. Also, this benchmark does not do automatic header dependency generation (to detect header changes). In a project using a different build system, this may add time to all benchmarks. There are 3 compilers in the benchmark, each with 5 configurations. Compilers: <ul> <li>gcc</li> <li>icc</li> <li>clang</li> </ul> Compiler configurations: <ul> <li>Default - default compiler options</li> <li>Optimized native - <code>-O3 -march=native</code></li> <li>Size optimized - <code>-Os</code></li> <li>LTO/IPO native - <code>-O3 -flto -march=native</code> with clang and gcc, <code>-O3 -ipo -march=native</code> with icpc/icc</li> <li>Zero optimization - <code>-Os</code></li> </ul> I think these each can have different bearings on the comparisons between single-unit and multi-unit builds. I included LTO/IPO so we might see how the "proper" way to achieve single-unit-effectiveness compares. Explanation of csv fields: <ul> <li><code>Test Name</code> - name of the benchmark. Examples: <code>Botan, Box2D</code>.</li> <li>Test Configuration - name a particular configuration of this test (special cxx flags etc.). Usually the same as <code>Test Name</code>.</li> <li><code>Compiler</code> - name of the compiler used. Examples: <code>gcc,icc,clang</code>.</li> <li><code>Compiler Configuration</code> - name of a configuration of compiler options used. Example: <code>gcc opt native</code></li> <li><code>Compiler Version String</code> - first line of output of compiler version from the compiler itself. Example: <code>g++ --version</code> produces <code>g++ (GCC) 4.6.1</code> on my system.</li> <li><code>Header only</code> - a value of <code>True</code> if this test case was built as a single unit, <code>False</code> if it was built as a multi-unit project.</li> <li><code>Units</code> - number of units in the test case, even if it is built as a single unit.</li> <li><code>Compile Time,Link Time,Build Time,Run Time</code> - as it sounds.</li> <li><code>Re-compile Time AVG,Re-compile Time MAX,Re-link Time AVG,Re-link Time MAX,Re-build Time AVG,Re-build Time MAX</code> - the times across rebuilding the project after touching a single file. Each unit is touched, and for each, the project is rebuilt. The maximum times, and average times are recorded in these fields.</li> <li><code>Compile Memory,Link Memory,Build Memory,Run Memory,Executable Size</code> - as they sound.</li> </ul> To reproduce the benchmarks: <ul> <li>The bullwork is <a href="https://gist.github.com/realazthat/4155964#file-run-py" rel="noreferrer">run.py</a>.</li> <li>Requires <a href="http://code.google.com/p/psutil/" rel="noreferrer">psutil</a> (for memory footprint measurements).</li> <li>Requires GNUMake.</li> <li>As it is, requires gcc, clang, icc/icpc in the path. Can be modified to remove any of these of course.</li> <li>Each benchmark should have a data-file that lists the units of that benchmarks. <a href="https://gist.github.com/4155964#file-run-py" rel="noreferrer">run.py</a> will then create two test cases, one with each unit compiled separately, and one with each unit compiled together. Example: <a href="https://gist.github.com/realazthat/4155964#file-box2d-data.data" rel="noreferrer">box2d.data</a>. The file format is defined as a json string, containing a dictionary with the following keys <ul> <li><code>"units"</code> - a list of <code>c/cpp/cc</code> files that make up the units of this project</li> <li><code>"executable"</code> - A name of the executable to be compiled.</li> <li><code>"link_libs"</code> - A space separated list of installed libraries to link to.</li> <li><code>"include_directores"</code> - A list of directories to include in the project.</li> <li><code>"command"</code> - optional. special command to execute to run the benchmark. For example, <code>"command": "botan_test --benchmark"</code></li> </ul></li> <li>Not all C++ projects can this be easily done with; there must be no conflicts/ambiguities in the single unit.</li> <li>To add a project to the test cases, modify the list <code>test_base_cases</code> in <a href="https://gist.github.com/4155964#file_run.py" rel="noreferrer">run.py</a> with the information for the project, including the data file name.</li> <li>If everything runs well, the output file <code>data.csv</code> should contain the benchmark results.</li> </ul> To produce the bar charts: <ul> <li>You should start with a data.csv file produced by the benchmark.</li> <li>Get <a href="https://gist.github.com/realazthat/4155964#file-chart-py" rel="noreferrer">chart.py</a>. Requires <a href="http://matplotlib.org/" rel="noreferrer">matplotlib</a>.</li> <li>Adjust the <code>fields</code> list to decide which graphs to produce.</li> <li>Run <code>python chart.py data.csv</code>.</li> <li>A file, <code>test.png</code> should now contain the result.</li> </ul> <h3>Box2D</h3> <ul> <li>Box2D was used from <a href="http://code.google.com/p/box2d/source/checkout" rel="noreferrer">svn as is</a>, revision 251.</li> <li>The benchmark was taken from <a href="https://github.com/joelgwebber/bench2d/blob/master/c/Bench2d.cpp" rel="noreferrer">here</a>, modified <a href="https://gist.github.com/4155964#file_box2d_bench1.cpp" rel="noreferrer">here</a> and might not be representative of a good Box2D benchmark, and it might not use enough of Box2D to do this compiler benchmark justice.</li> <li>The box2d.data file was manually written, by finding all the .cpp units.</li> </ul> <h3>Botan</h3> <ul> <li>Using <a href="http://botan.randombit.net/download.html" rel="noreferrer">Botan-1.10.3</a>.</li> <li>Data file: <a href="https://gist.github.com/4155964#file_botan_bench.data" rel="noreferrer">botan_bench.data</a>.</li> <li>First ran <code>./configure.py --disable-asm --with-openssl --enable-modules=asn1,benchmark,block,cms,engine,entropy,filters,hash,kdf,mac,bigint,ec_gfp,mp_generic,numbertheory,mutex,rng,ssl,stream,cvc</code>, this generates the header files and Makefile.</li> <li>I disabled assembly, because assembly might intefere with optimizations that can occure when the function boundaries do not block optimization. However, this is conjecture and might be totally wrong.</li> <li>Then ran commands like <code>grep -o "\./src.*cpp" Makefile</code> and <code>grep -o "\./checks.*" Makefile</code> to obtain the .cpp units and put them into <a href="https://gist.github.com/4155964#file_botan_bench.data" rel="noreferrer">botan_bench.data</a> file.</li> <li>Modified <code>/checks/checks.cpp</code> to not call the x509 unit tests, and removed x509 check, because of conflict between Botan typedef and openssl.</li> <li>The benchmark included in the Botan source was used.</li> </ul> <h2>System specs:</h2> <ul> <li>OpenSuse 11.4, 32-bit</li> <li>4GB RAM</li> <li><code>Intel(R) Core(TM) i7 CPU Q 720 @ 1.60GHz</code></li> </ul>
Tags
Title
singulars
PostAcceptedAnswerId
1. This table or related slice is empty.
PostParentId
1. POQuantifiable metrics (benchmarks) on the usage of header-only c++ libraries
 singulars
 PostTypePostTypeId
 PTQuestion
PostTypePostTypeId
1. PTAnswer
UserLastEditorUserId
1. USRealz Slaw
UserOwnerUserId
1. USRealz Slaw
plurals
PostLinksPostIdRelatedPostId
1. This table or related slice is empty.
PostLinksRelatedPostIdPostId
1. This table or related slice is empty.
PostsAcceptedAnswerId
1. POQuantifiable metrics (benchmarks) on the usage of header-only c++ libraries
 singulars
 PostTypePostTypeId
 PTQuestion
PostsParentIdCreationDate
1. This table or related slice is empty.
VotesPostIdCreationDate
1. VO
 singulars
 PostPostId
 PO
 UserUserId
 This table or related slice is empty.
 VoteTypeVoteTypeId
 VTUpMod
2. VO
 singulars
 PostPostId
 PO
 UserUserId
 This table or related slice is empty.
 VoteTypeVoteTypeId
 VTUpMod
3. VO
 singulars
 PostPostId
 PO
 UserUserId
 This table or related slice is empty.
 VoteTypeVoteTypeId
 VTAcceptedByOriginator
CommentsPostId

Querying!

Guidance

A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.