StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
primarykey
Id
14818799
data
AcceptedAnswerId
0
AnswerCount
0
ClosedDate
CommentCount
0
CommunityOwnedDate
CreationDate
2013-02-11T18:47:18.493
FavoriteCount
0
LastActivityDate
2013-02-11T22:12:38.147
LastEditDate
2013-02-11T22:12:38.147
LastEditorUserId
90801
OwnerUserId
90801
ParentId
14818033
PostTypeId
2
Score
9
ViewCount
0
LastEditorDisplayName
text
Body
When using pmap I have found that fairly large chunks are required to overcome the switching and future overhead try a chunk size of 10,000 for an opperation as fast as <code>+</code>. The potential gains are bounded by the overhead of generating the chunks. This results in an optimal value that balances the available cores and the time required to make the chunks. In this case with <code>+</code> as the workload I was unable to make this faster than the single threaded option. If you're interested in doing this without pmap and potentially using fork/join check out the new(ish) <a href="http://clojure.com/blog/2012/05/08/reducers-a-library-and-model-for-collection-processing.html">reducers library</a> The OOM situation comes from the first test realizing the lazy sequence from <code>(range n)</code> which is then retained so it can be passed to the second sequence. If I make the + function much slower by defining a <code>slow+</code> function and use that the diference between single thread, pmap over chunks, and reducers w/ forkJoin become visable: <pre><code>user> *clojure-version* {:major 1, :minor 5, :incremental 0, :qualifier "RC15"} (require '[clojure.core.reducers :as r]) (def workers (+ 2 (.. Runtime getRuntime availableProcessors))) (defn slow+ ([] 0) ([x] x) ([x y] (reduce + (range 100000)) (+ x y))) (defn run-test [] (let [n 8000] (time (reduce slow+ (range n))) (time (reduce slow+ (pmap #(reduce slow+ %) (partition-all (* workers 100) (range n))))) (time (r/fold slow+ (vec (range n)))))) user> (run-test) "Elapsed time: 28655.951241 msecs" ; one thread "Elapsed time: 6975.488591 msecs" ; pmap over chunks "Elapsed time: 8170.254426 msecs" ; using reducer </code></pre>
Tags
Title
singulars
PostAcceptedAnswerId
1. This table or related slice is empty.
PostParentId
1. POHow can I compute the sum of a large list of numbers in parallel using Clojure
 singulars
 PostTypePostTypeId
 PTQuestion
PostTypePostTypeId
1. PTAnswer
UserLastEditorUserId
1. USArthur Ulfeldt
UserOwnerUserId
1. USArthur Ulfeldt
plurals
PostLinksPostIdRelatedPostId
1. This table or related slice is empty.
PostLinksRelatedPostIdPostId
1. This table or related slice is empty.
PostsAcceptedAnswerId
1. POHow can I compute the sum of a large list of numbers in parallel using Clojure
 singulars
 PostTypePostTypeId
 PTQuestion
PostsParentIdCreationDate
1. This table or related slice is empty.
VotesPostIdCreationDate
1. VO
 singulars
 PostPostId
 PO
 UserUserId
 This table or related slice is empty.
 VoteTypeVoteTypeId
 VTUpMod
2. VO
 singulars
 PostPostId
 PO
 UserUserId
 This table or related slice is empty.
 VoteTypeVoteTypeId
 VTUpMod
3. VO
 singulars
 PostPostId
 PO
 UserUserId
 This table or related slice is empty.
 VoteTypeVoteTypeId
 VTUpMod
CommentsPostId
1. This table or related slice is empty.

Querying!

Guidance

A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.