StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
primarykey
Id
7776848
data
AcceptedAnswerId
0
AnswerCount
0
ClosedDate
CommentCount
1
CommunityOwnedDate
CreationDate
2011-10-15T09:01:54.220
FavoriteCount
0
LastActivityDate
2011-10-15T16:19:43.977
LastEditDate
2011-10-15T16:19:43.977
LastEditorUserId
943436
OwnerUserId
943436
ParentId
7776293
PostTypeId
2
Score
14
ViewCount
0
LastEditorDisplayName
text
Body
I agree with @aix, <code>multiprocessing</code> is definitely the way to go. Regardless you will be i/o bound -- you can only read so fast, no matter how many parallel processes you have running. But there can easily be some speedup. Consider the following (input/ is a directory that contains several .txt files from Project Gutenberg). <pre><code>import os.path from multiprocessing import Pool import sys import time def process_file(name): ''' Process one file: count number of lines and words ''' linecount=0 wordcount=0 with open(name, 'r') as inp: for line in inp: linecount+=1 wordcount+=len(line.split(' ')) return name, linecount, wordcount def process_files_parallel(arg, dirname, names): ''' Process each file in parallel via Poll.map() ''' pool=Pool() results=pool.map(process_file, [os.path.join(dirname, name) for name in names]) def process_files(arg, dirname, names): ''' Process each file in via map() ''' results=map(process_file, [os.path.join(dirname, name) for name in names]) if __name__ == '__main__': start=time.time() os.path.walk('input/', process_files, None) print "process_files()", time.time()-start start=time.time() os.path.walk('input/', process_files_parallel, None) print "process_files_parallel()", time.time()-start </code></pre> When I run this on my dual core machine there is a noticeable (but not 2x) speedup: <pre><code>$ python process_files.py process_files() 1.71218085289 process_files_parallel() 1.28905105591 </code></pre> If the files are small enough to fit in memory, and you have lots of processing to be done that isn't i/o bound, then you should see even better improvement.
Tags
Title
singulars
PostAcceptedAnswerId
1. This table or related slice is empty.
PostParentId
1. PORead txt file with multi-threaded in python
 singulars
 PostTypePostTypeId
 PTQuestion
PostTypePostTypeId
1. PTAnswer
UserLastEditorUserId
1. USAustin Marshall
UserOwnerUserId
1. USAustin Marshall
plurals
PostLinksPostIdRelatedPostId
1. This table or related slice is empty.
PostLinksRelatedPostIdPostId
1. This table or related slice is empty.
PostsAcceptedAnswerId
1. PORead txt file with multi-threaded in python
 singulars
 PostTypePostTypeId
 PTQuestion
PostsParentIdCreationDate
1. This table or related slice is empty.
VotesPostIdCreationDate
1. VO
 singulars
 PostPostId
 PO
 UserUserId
 This table or related slice is empty.
 VoteTypeVoteTypeId
 VTUpMod
2. VO
 singulars
 PostPostId
 PO
 UserUserId
 This table or related slice is empty.
 VoteTypeVoteTypeId
 VTAcceptedByOriginator
3. VO
 singulars
 PostPostId
 PO
 UserUserId
 This table or related slice is empty.
 VoteTypeVoteTypeId
 VTUpMod
CommentsPostId
1. This table or related slice is empty.

Querying!

Guidance

A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.