StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
primarykey
Id
20819362
data
AcceptedAnswerId
0
AnswerCount
0
ClosedDate
CommentCount
0
CommunityOwnedDate
CreationDate
2013-12-28T21:04:14.153
FavoriteCount
0
LastActivityDate
2013-12-28T21:04:14.153
LastEditDate
2017-05-23T12:03:44.403
LastEditorUserId
-1
OwnerUserId
794276
ParentId
20818522
PostTypeId
2
Score
0
ViewCount
0
LastEditorDisplayName
text
Body
Just as a quick check on the number of file handlers you have / are used try this (unix): <pre><code>cat /proc/sys/fs/file-nr </code></pre> That will give you (number of allocated file handlers) - (number of allocated but unused file handlers) - (maximum number of file handlers) --- see <a href="https://stackoverflow.com/questions/10369309/opendir-too-many-open-files">here</a>. It is possible to change the limits in sysctl.conf (on linux - see <a href="https://stackoverflow.com/questions/10369309/opendir-too-many-open-files">link</a> above) - but this is probably not a good idea resource management-wise and, therefore, isn't really scalable. And, yes, things start to get slower as more handlers get used for opening each file (as they don't get closed until after the shell execution stops/ends) and eventually it fails when there are no more handlers available. One potential solution could incorporate <a href="http://www.scipy.org/" rel="nofollow noreferrer">Python/SciPy/Pandas</a> and a simple database. There's great documentation and large community support. An example that is closely related to your post is <a href="https://stackoverflow.com/questions/11174367/averaging-data-from-multiple-data-files-in-python-with-pandas?rq=1">here</a>. A small post on interfacing Pandas and a database linked <a href="https://stackoverflow.com/questions/10065051/python-pandas-and-databases-like-mysql">here</a>. I haven't tried this but I'll give it a shot: For the database you could use something like the pandas <code>io.sql</code> module to create a useful representation of each dat file (maybe using the A# header as an identifier for each table). Then the data can be manipulated by any number of methods e.g. <a href="https://stackoverflow.com/questions/11174367/averaging-data-from-multiple-data-files-in-python-with-pandas?rq=1">glued</a>. This does not retain the <code>./merge.sh data* > dataComb.dat</code> functionality that you ask for but a simple python command line script could potentially handle all the steps to get data in and processed as you wish. I think it would be quite the learning curve but it could pay off in scalability/flexibility for the future.
Tags
Title
singulars
PostAcceptedAnswerId
1. This table or related slice is empty.
PostParentId
1. POcombine/average multiple data files
 singulars
 PostTypePostTypeId
 PTQuestion
PostTypePostTypeId
1. PTAnswer
UserLastEditorUserId
1. USCommunity
UserOwnerUserId
1. UShmcmurray
plurals
PostLinksPostIdRelatedPostId
1. This table or related slice is empty.
PostLinksRelatedPostIdPostId
1. This table or related slice is empty.
PostsAcceptedAnswerId
1. This table or related slice is empty.
PostsParentIdCreationDate
1. This table or related slice is empty.
VotesPostIdCreationDate
1. This table or related slice is empty.
CommentsPostId
1. This table or related slice is empty.

Querying!

Guidance

A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.