StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<h1>Profile your code</h1> <p>From the Python wiki: </p> <blockquote> <p>The first step to speeding up your program is learning where the bottlenecks lie. It hardly makes sense to optimize code that is never executed or that already runs fast Before you change any of it, you should profile it. </p> </blockquote> <p>This page should have most of what you will need (Read the rest too): <a href="http://wiki.python.org/moin/PythonSpeed/PerformanceTips#Profiling_Code" rel="nofollow">http://wiki.python.org/moin/PythonSpeed/PerformanceTips#Profiling_Code</a></p> <h2>Try to use more of the native Python style</h2> <p>Without having seen the profiler results, I bet that it would speed up if you got rid of the nested for-loops; all of the talks about Python coding style from PyCon 2013 say that list comprehensions and other techniques are faster. Try this talk: </p> <p><a href="http://pyvideo.org/video/1758/loop-like-a-native-while-for-iterators-genera" rel="nofollow">http://pyvideo.org/video/1758/loop-like-a-native-while-for-iterators-genera</a> </p> <p>And do a search for a talk entitled, "Transforming Code into Beautiful, Idiomatic Python". </p> <h1>Or, use Pandas (designed for dataset work)</h1> <p>Also, you might be able to speed up the calculation by using the built-in functionality of the Python module <code>pandas</code>. It will allow you to load the entire input file at once and perform counts and sums, and other reduction-type operations on your large data-set easily. </p> <p>The book <strong>Python for Data Analysis</strong> (written by the creator of pandas) has an example of doing basic statistics on several, real datasets. You would probably get your money's worth from just the examples alone. I purchased it and love it. </p> <p>I wish you good luck. </p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload