Databases
StackOverflow2013
Postsdbo
POJava ExecutorService - scaling
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POJava ExecutorService - scaling
    text
    copied!<p>I am trying to write a program in Java using <code>ExecutorService</code> and its function <code>invokeAll</code>. My question is: does the <code>invokeAll</code> function solve the tasks simultaneously? I mean, if I have two processors, there will be two workers at the same time? Because aI can't make it scale correctly. It takes the same time to complete the problem if I give <code>newFixedThreadPool(2)</code> or 1.</p> <pre><code>List&lt;Future&lt;PartialSolution&gt;&gt; list = new ArrayList&lt;Future&lt;PartialSolution&gt;&gt;(); Collection&lt;Callable&lt;PartialSolution&gt;&gt; tasks = new ArrayList&lt;Callable&lt;PartialSolution&gt;&gt;(); for(PartialSolution ps : wp) { tasks.add(new Map(ps, keyWords)); } list = executor.invokeAll(tasks); </code></pre> <p><code>Map</code> is a class that implements <code>Callable</code> and <code>wp</code> is a vector of Partial Solutions, a class that holds some information in different times.</p> <p>Why doesn't it scale? What could be the problem?</p> <p>This is the code for PartialSolution:</p> <pre><code>import java.util.HashMap; import java.util.Vector; public class PartialSolution { public String fileName;//the name of a file public int b, e;//the index of begin and end of the fragment from the file public String info;//the fragment public HashMap&lt;String, Word&gt; hm;//here i retain the informations public HashMap&lt;String, Vector&lt;Word&gt;&gt; hmt;//this i use for the final reduce public PartialSolution(String name, int b, int e, String i, boolean ok) { this.fileName = name; this.b = b; this.e = e; this.info = i; hm = new HashMap&lt;String, Word&gt;(); if(ok == true) { hmt = new HashMap&lt;String, Vector&lt;Word&gt;&gt;(); } else { hmt = null; } } } </code></pre> <p>An this is the code for Map:</p> <pre><code>public class Map implements Callable&lt;PartialSolution&gt; { private PartialSolution ps; private Vector&lt;String&gt; keyWords; public Map(PartialSolution p, Vector&lt;String&gt; kw) { this.ps = p; this.keyWords = kw; } @Override public PartialSolution call() throws Exception { String[] st = this.ps.info.split("\\n"); for(int j = 0 ; j &lt; st.length ; j++) { for(int i = 0 ; i &lt; keyWords.size() ; i++) { if(keyWords.elementAt(i).charAt(0) != '\'') { int k = 0; int index = 0; int count = 0; while((index = st[j].indexOf(keyWords.elementAt(i), k)) != -1) { k = index + keyWords.elementAt(i).length(); count++; } if(count != 0) { Word wr = this.ps.hm.get(keyWords.elementAt(i)); if(wr != null) { Word nw = new Word(ps.fileName); nw.nrap = wr.nrap + count; nw.lines = wr.lines; int grep = count; while(grep &gt; 0) { nw.lines.addElement(ps.b + j); grep--; } this.ps.hm.put(keyWords.elementAt(i), nw); } else { Word nw = new Word(ps.fileName); nw.nrap = count; int grep = count; while(grep &gt; 0) { nw.lines.addElement(ps.b + j); grep--; } this.ps.hm.put(keyWords.elementAt(i), nw); } } } else { String regex = keyWords.elementAt(i).substring(1, keyWords.elementAt(i).length() - 1); StringBuffer sb = new StringBuffer(regex); regex = sb.toString(); Pattern pt = Pattern.compile(regex); Matcher m = pt.matcher(st[j]); int count = 0; while(m.find()) { count++; } if(count != 0) { Word wr = this.ps.hm.get(keyWords.elementAt(i)); if(wr != null) { Word nw = new Word(this.ps.fileName); nw.nrap = wr.nrap + count; nw.lines = wr.lines; int grep = count; while(grep &gt; 0) { nw.lines.addElement(ps.b + j); grep--; } this.ps.hm.put(keyWords.elementAt(i), nw); } else { Word nw = new Word(this.ps.fileName); nw.nrap = count; int grep = count; while(grep &gt; 0) { nw.lines.addElement(ps.b + j); grep--; } this.ps.hm.put(keyWords.elementAt(i), nw); } } } } } this.ps.info = null; return this.ps; } } </code></pre> <p>So in Map i take every line from the fragment and search for every expression the number of appearances and i save also the number of line. After i process all the fragment, in the same PartialSolution i save the informations in a hash map and return the new PartialSolution. In the next step i combine the PartialSolutions with the same fileName and introduce them in a Callable class Reduce, who is the same as map, the difference is that it makes other operations, but returns also a PartialSolution.</p> <p>This is the code to run the Map tasks:</p> <pre><code>List&lt;Future&lt;PartialSolution&gt;&gt; list = new ArrayList&lt;Future&lt;PartialSolution&gt;&gt;(); Collection&lt;Callable&lt;PartialSolution&gt;&gt; tasks = new ArrayList&lt;Callable&lt;PartialSolution&gt;&gt;(); for(PartialSolution ps : wp) { tasks.add(new Map(ps, keyWords)); } list = executor.invokeAll(tasks); </code></pre> <p>In task i create task of type Map and in list i obtain them. I don't know how to read the JVM thread dump. I hope it's good enough what informations i gave you. I work in NetBeans 7.0.1 if that helps.</p> <p>Thank you, Alex</p>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload