StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POShuffling a large text file without/with group order maintained
primarykey
Id
19690954
data
AcceptedAnswerId
19692412
AnswerCount
2
ClosedDate
CommentCount
1
CommunityOwnedDate
CreationDate
2013-10-30T18:25:06.270
FavoriteCount
1
LastActivityDate
2013-10-31T21:46:47.810
LastEditDate
2013-10-31T21:36:46.720
LastEditorUserId
402480
OwnerUserId
402480
ParentId
0
PostTypeId
1
Score
1
ViewCount
214
LastEditorDisplayName
text
Body
Instead of making a script, it there a one liner to shuffle a large tab separated text file, based on the unique elements in the first column. That means, for each unique element in the first column, number of rows will be equal and be specified by the user. There are two output possibilities, maintaining the row order or randomized row order. Input : <pre><code>chr1 3003204 3003454 * 37 + chr1 3003235 3003485 * 37 + chr1 3003148 3003152 * 37 - chr1 3003461 3003711 * 37 + chr11 71863609 71863647 * 37 + chr11 71864025 71864275 * 37 + chr11 71864058 71864308 * 37 - chr11 71864534 71864784 * 37 + chrY 90828920 90829170 * 23 - chrY 90829096 90829346 * 23 + chrY 90828924 90829174 * 23 - chrY 90828925 90829175 * 23 - </code></pre> Output (1 row per category - defined by the user) Output1 (randomized - row order will change) : <pre><code>chr1 3003235 3003485 * 37 + chr11 71863609 71863647 * 37 + chrY 90828925 90829175 * 23 - </code></pre> Output1 (randomized - row order will be maintained) : <pre><code>chr1 3003204 3003454 * 37 + chr11 71863609 71863647 * 37 + chrY 90828920 90829170 * 23 - </code></pre> I tried using <code>sort -u</code> with <code>cut</code> on first column to fetch unique elements and then running a combination of <code>grep</code> and <code>head</code> for each element to generate the output file, which can be randomized using <code>shuf</code>, there might be a better solution as the file can be huge > 50 Million lines. Cheers
Tags
<linux><sorting><text><random><shuffle>
Title
Shuffling a large text file without/with group order maintained
singulars
PostAcceptedAnswerId
1. PO
 singulars
 PostTypePostTypeId
 PTAnswer
PostParentId
1. This table or related slice is empty.
PostTypePostTypeId
1. PTQuestion
UserLastEditorUserId
1. USSukhdeep Singh
UserOwnerUserId
1. USSukhdeep Singh
plurals
PostLinksPostIdRelatedPostId
1. This table or related slice is empty.
PostLinksRelatedPostIdPostId
1. This table or related slice is empty.
PostsAcceptedAnswerId
1. This table or related slice is empty.
PostsParentIdCreationDate
1. PO
 singulars
 PostTypePostTypeId
 PTAnswer
2. PO
 singulars
 PostTypePostTypeId
 PTAnswer
VotesPostIdCreationDate
1. VO
 singulars
 PostPostId
 POShuffling a large text file without/with group order maintained
 UserUserId
 This table or related slice is empty.
 VoteTypeVoteTypeId
 VTUpMod
2. VO
 singulars
 PostPostId
 POShuffling a large text file without/with group order maintained
 UserUserId
 USjkshah
 VoteTypeVoteTypeId
 VTFavorite
CommentsPostId
1. CO+1 for interesting question.
 singulars
 PostPostId
 POShuffling a large text file without/with group order maintained
 UserUserId
 USjkshah

Querying!

Guidance

A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.