Databases
StackOverflow2013
Postsdbo
PODifferent results for N>1 goroutines (on N>1 Cpu:s). Why?
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PODifferent results for N>1 goroutines (on N>1 Cpu:s). Why?
    primarykey
    data
    text
    <p>I have a test program that gives different results when executing more than one goroutine on more than one Cpu (Goroutines = Cpus). The "test" is about syncing goroutines using channels, and the program itself counts occurences of chars in strings. It produces consistent results on one Cpu / one goroutine. </p> <p>See code example on playground (Note: Run on local machine to execute on multi core, and watch the resulting numbers vary): <a href="http://play.golang.org/p/PT5jeCKgBv" rel="nofollow">http://play.golang.org/p/PT5jeCKgBv</a> .</p> <p><strong>Code summary</strong>: The program counts occurences of 4 different chars (A,T, G,C) in (DNA) strings.</p> <p><strong>Problem</strong>: Result (n occurences of chars) varies when executed on multiple Cpu's (goroutines). Why? </p> <p><strong>Description</strong>:</p> <ol> <li>A goroutine spawns work (<strong>SpawnWork</strong>) as strings to <strong>Workers</strong>. Sets up artificial string input data (hardcoded strings are copied n times).</li> <li>Goroutine <strong>Workers</strong> (Worker) are created equalling the numbers of Cpu's. </li> <li><strong>Workers</strong> checks each char in string and counts A,T's and sends the sum into a channel, and G,C counts to another channel. </li> <li><strong>SpawnWork</strong> closes workstring channel as to control Workers (which consumes strings using range, which quits when the input channel is closed by <strong>SpawnWork</strong>).</li> <li>When <strong>Workers</strong> has consumed its ranges (of chars) it sends a quit signal on the quit channel (quit &lt;- true). These "pulses" will occure Cpu number of times ( Cpu count = goroutines count). </li> <li><strong>Main</strong> (select) loop will quit when it has received Cpu-count number of quit signals.</li> <li><strong>Main</strong> func prints a summary of occurences of Chars (A,T's, G,C's).</li> </ol> <p><strong>Simplified code</strong>:</p> <p><em><strong>1.</strong> "Worker" (goroutines) counting chars in lines:</em></p> <pre><code>func Worker(inCh chan *[]byte, resA chan&lt;- *int, resB chan&lt;- *int, quit chan bool) { //for p_ch := range inCh { for { p_ch, ok := &lt;-inCh // similar to range if ok { ch := *p_ch for i := 0; i &lt; len(ch); i++ { if ch[i] == 'A' || ch[i] == 'T' { // Count A:s and T:s at++ } else if ch[i] == 'G' || ch[i] == 'C' { // Count G:s and C:s gc++ } } resA &lt;- &amp;at // Send line results on separate channels resB &lt;- &amp;gc // Send line results on separate channels } else { quit &lt;- true // Indicate that we're all done break } } } </code></pre> <p><em><strong>2.</strong> Spawn work (strings) to workers:</em></p> <pre><code>func SpawnWork(inStr chan&lt;- *[]byte, quit chan bool) { // Artificial input data StringData := "NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN\n" + "NTGAGAAATATGCTTTCTACTTTTTTGTTTAATTTGAACTTGAAAACAAAACACACACAA\n" + "... etc\n" + // ... for scanner.Scan() { s := scanner.Bytes() if len(s) == 0 || s[0] == '&gt;' { continue } else { i++ inStr &lt;- &amp;s } } close(inStr) // Indicate (to Workers) that there's no more strings coming. } </code></pre> <p><em><strong>3.</strong> Main routine:</em></p> <pre><code>func main() { // Count Cpus, and count down in final select clause CpuCnt := runtime.NumCPU() runtime.GOMAXPROCS(CpuCnt) // Make channels resChA := make(chan *int) resChB := make(chan *int) quit := make(chan bool) inStr := make(chan *[]byte) // Set up Workers ( n = Cpu ) for i := 0; i &lt; CpuCnt; i++ { go Worker(inStr, resChA, resChB, quit) } // Send lines to Workers go SpawnWork(inStr, quit) // Count the number of "A","T" &amp; "G","C" per line // (comes in here as ints per row, on separate channels (at and gt)) for { select { case tmp_at := &lt;-resChA: tmp_gc := &lt;-resChB // Ch A and B go in pairs anyway A += *tmp_at // sum of A's and T's B += *tmp_gc // sum of G's and C's case &lt;-quit: // Each goroutine sends "quit" signals when it's done. Since // the number of goroutines equals the Cpu counter, we count // down each time a goroutine tells us it's done (quit at 0): CpuCnt-- if CpuCnt == 0 { // When all goroutines are done then we're done. goto out } } } out: // Print report to screen } </code></pre> <p>Why does this code count consistently only when executed on a singel cpu/goroutine? That is, the channels doesn't seem to sync, or the main loop quits forcefully before all goroutines are done? Scratching head.</p> <p>(Again: See/run the full code at the playground: <a href="http://play.golang.org/p/PT5jeCKgBv" rel="nofollow">http://play.golang.org/p/PT5jeCKgBv</a> )</p> <p>// Rolf Lampa</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USRIL
    1. USRIL
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. PODifferent results for N>1 goroutines (on N>1 Cpu:s). Why?
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. PODifferent results for N>1 goroutines (on N>1 Cpu:s). Why?
      1. This table or related slice is empty.
      1. VTDownMod
    3. VO
      singulars
      1. PODifferent results for N>1 goroutines (on N>1 Cpu:s). Why?
      1. This table or related slice is empty.
      1. VTUpMod
    1. COHave you tried running it with `go run -race`? http://golang.org/doc/articles/race_detector.html
      singulars
      1. PODifferent results for N>1 goroutines (on N>1 Cpu:s). Why?
      1. USMatrixFrog
    2. COI looked through the code and didn't see any problems so I tried `-race` myself. It found a few problems but one of them is that the `at` variable at the beginnning of `Worker` is exposed to `main` because you do `tmp_at := <-resChA`. Maybe if you pass the numbers themselves through the channels (instead of pointers to the numbers) then it won't be racy?
      singulars
      1. PODifferent results for N>1 goroutines (on N>1 Cpu:s). Why?
      1. USMatrixFrog
    3. CO@MatrixFog: I actually had forgot about the -race option. Doh! However, removing the pointers does not do away with the races, that really puzzles me. I'll keep trying to narrowing it down.
      singulars
      1. PODifferent results for N>1 goroutines (on N>1 Cpu:s). Why?
      1. USRIL
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload