StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
primarykey
Id
5449583
data
AcceptedAnswerId
0
AnswerCount
0
ClosedDate
CommentCount
1
CommunityOwnedDate
CreationDate
2011-03-27T13:44:19.280
FavoriteCount
0
LastActivityDate
2011-03-27T13:44:19.280
LastEditDate
LastEditorUserId
0
OwnerUserId
211158
ParentId
5419657
PostTypeId
2
Score
8
ViewCount
0
LastEditorDisplayName
text
Body
For chunked input processing I would use the <a href="http://hackage.haskell.org/package/enumerator">enumerator</a> package. <pre><code>import Data.Enumerator import Data.Enumerator.Binary (enumFile) </code></pre> We use bytestrings <pre><code>import Data.ByteString as BS </code></pre> and IO <pre><code>import Control.Monad.Trans (liftIO) import Control.Monad (mapM_) import System (getArgs) </code></pre> Your main function could look like following: <pre><code>main = do (filepath:_) <- getArgs let destination run_ $ enumFile filepath $$ writeFile (filepath ++ ".cpy") </code></pre> enumFile reads 4096 bytes per chunk and passes these to writeFile, which writes it down. enumWrite is defined as: <pre><code>enumWrite :: FilePath -> Iteratee BS.ByteString IO () enumWrite filepath = do liftIO (BS.writeFile filepath BS.empty) -- ensure the destination is empty continue step where step (Chunks xs) = do liftIO (mapM_ (BS.appendFile filepath) xs) continue step step EOF = yield () EOF </code></pre> As you can see, the step function takes chunks of bytestrings and appends them to the destination file. These chunks have the type Stream BS.Bytestring, where Stream is defined as: <pre><code>data Stream a = Chunks [a] | EOF </code></pre> On an EOF step terminates, yielding (). To have a much more elaborate read on this I personally recommend Michael Snoymans <a href="http://www.yesodweb.com/book/enumerator">tutorial</a> <h2>The numbers</h2> <pre><code>$ time ./TestCopy 5MB ./TestCopy 5MB 2,91s user 0,32s system 96% cpu 3,356 total $ time ./TestCopy2 5MB ./TestCopy2 5MB 0,04s user 0,03s system 93% cpu 0,075 total </code></pre> That's quite an improvement. Now in order to implement your fold you probably want to write an Enumeratee, which is used to transform a input stream. Fortunately there is already a map function defined in the enumerator package, which can be modified for your need, i.e. it can be modified to carry over state. <h2>On the construction of the intermediate result</h2> You construct wordsList in reverse order and reverse it afterwards. I think <a href="http://hackage.haskell.org/packages/archive/dlist/latest/doc/html/Data-DList.html">difference lists</a> do a better job, because appends take only O(1) time due to the fact that appending is only a function composition. I'm not sure whether they takes more space though. Here's a rough sketch of difference lists: <pre><code>type DList a = [a] -> [a] emptyList :: DList a emptyList = id snoc :: DList a -> a -> DList a snoc dlist a = dlist . (a:) toList :: DList a -> [a] toList dlist = dlist [] </code></pre> This answer is probably not needed anymore, but I added it for completeness.
Tags
Title
singulars
PostAcceptedAnswerId
1. This table or related slice is empty.
PostParentId
1. POIO over big files in haskell: Performance issue
 singulars
 PostTypePostTypeId
 PTQuestion
PostTypePostTypeId
1. PTAnswer
UserLastEditorUserId
1. This table or related slice is empty.
UserOwnerUserId
1. USLong
plurals
PostLinksPostIdRelatedPostId
1. This table or related slice is empty.
PostLinksRelatedPostIdPostId
1. This table or related slice is empty.
PostsAcceptedAnswerId
1. This table or related slice is empty.
PostsParentIdCreationDate
1. This table or related slice is empty.
VotesPostIdCreationDate
1. VO
 singulars
 PostPostId
 PO
 UserUserId
 This table or related slice is empty.
 VoteTypeVoteTypeId
 VTUpMod
2. VO
 singulars
 PostPostId
 PO
 UserUserId
 This table or related slice is empty.
 VoteTypeVoteTypeId
 VTUpMod
3. VO
 singulars
 PostPostId
 PO
 UserUserId
 This table or related slice is empty.
 VoteTypeVoteTypeId
 VTUpMod
CommentsPostId
1. This table or related slice is empty.

Querying!

Guidance

A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.