Databases
StackOverflow2013
Postsdbo
POScalacheck, unit testing multi-process algorithms
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POScalacheck, unit testing multi-process algorithms
    primarykey
    data
    text
    <p>I am currently developing a Scala framework for multi-process, parallel algorithms using MPJ-Express (i.e. a Java MPI-variant).</p> <p>MPJ-Express, and basically all MPI-variants, work by starting many processes with the same program. Since I have no control over the processes at runtime (my program does not spawn processes), I can't use standard unit testing frameworks due to the following reasons:</p> <ol> <li><p>Not all processes will own a local copy of the results after execution (ideally, the results should be collected at an arbitrary root-process)</p></li> <li><p>Lack of control over standard-output. It is not easily possible to get output only from a single process.</p></li> <li>Lack of flow control. All processes must enter the same test at the same time to allow single-program multiple data algorithms.</li> </ol> <p>Number 2 is mainly the problem, as 3 might work as expected and 1 can be fixed with an extra communication operation. Does anyone have practical experience, or know of better unit testing strategies for multi-process algorithms?</p> <p><strong>EDIT</strong></p> <p>Right now, I seem to be having luck with the following code using <em>scalacheck</em>:</p> <pre><code>package it.vigtig.thesis.collection.scalacheck import java.io.OutputStream import java.io.PrintStream import org.scalacheck.Prop.forAll import org.scalacheck.Properties import it.vigtig.thesis.env.DistEnv.globalRank import it.vigtig.thesis.env.DistEnv.parallelize object CollectionCheck { def main(args: Array[String]) { parallelize(args) { if (globalRank &gt; 0) { Console.setOut(new PrintStream(new OutputStream() { def write(b: Int) { //nop } })) } ST.main(Array()) } } } object ST extends Properties("String") { // def println(a: String*) = gprintln(a) property("startsWith") = forAll((a: String, b: String) =&gt; (a + b).startsWith(a)) property("concatenate") = forAll((a: String, b: String) =&gt; (a + b).length &gt; a.length &amp;&amp; (a + b).length &gt; b.length) property("substring") = forAll((a: String, b: String, c: String) =&gt; (a + b + c).substring(a.length, a.length + b.length) == b) } </code></pre> <p>The above code reroutes the scala-println method to a nop-operation for all other processes than p=0. I should be able to run the test-suites with parallel-methods, allowing only the root process to verify the results. The above yields the following output:</p> <pre><code>MPJ Express (0.38) is started in the multicore configuration rank-0: 0.212437745 time taken for initialize + String.startsWith: OK, passed 100 tests. ! String.concatenate: Falsified after 0 passed tests. &gt; ARG_0: &gt; ARG_1: + String.substring: OK, passed 100 tests. </code></pre>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USFelix
    1. USFelix
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POScalacheck, unit testing multi-process algorithms
      1. This table or related slice is empty.
      1. VTUpMod
    1. COI am getting deadlocks whenever I call MPI-methods inside a property. Using a simple mpi-barrier works perfectly just outside the properties, and I can verify that all processes are calling the method, but I get into a deadlock whenever MPI-calls are used inside a property-body. Any help would be very welcome :)
      singulars
      1. POScalacheck, unit testing multi-process algorithms
      1. USFelix
    2. COPerforming standard I/O in one process only inside a conditional statement based on its rank is the _de facto_ standard way to do I/O in MPI programming.
      singulars
      1. POScalacheck, unit testing multi-process algorithms
      1. USHristo Iliev
    3. COhttps://github.com/rickynils/scalacheck/issues/37 Apparently, it was due to a kind of bug in scalacheck. I might close this topic :/
      singulars
      1. POScalacheck, unit testing multi-process algorithms
      1. USFelix
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload