StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POMultithreaded search operation
text
Body
copied!<p>I have a method that takes an array of queries, and I need to run them against different search engine Web API's, such as Google's or Yahoo's. In order to parallelize the process, a thread is spawned for each query, which are then <a href="http://download.oracle.com/javase/6/docs/api/java/lang/Thread.html#join%28%29" rel="nofollow noreferrer"><code>join</code></a>ed at the end, since my application can only continue <em>after</em> I have the results of <em>every</em> query. I currently have something along these lines:</p> <pre><code>public abstract class class Query extends Thread { private String query; public abstract Result[] querySearchEngine(); @Override public void run() { Result[] results = querySearchEngine(query); Querier.addResults(results); } } public class GoogleQuery extends Query { public Result querySearchEngine(String query) { // access google rest API } } public class Querier { /* Every class that implements Query fills this array */ private static ArrayList<Result> aggregatedResults; public static void addResults(Result[]) { // add to aggregatedResults } public static Result[] queryAll(Query[] queries) { /* for each thread, start it, to aggregate results */ for (Query query : queries) { query.start(); } for (Query query : queries) { query.join(); } return aggregatedResults; } } </code></pre> <p>Recently, I have found that there's a <em>new</em> API in Java for doing concurrent jobs. Namely, the <code>Callable</code> interface, <code>FutureTask</code> and <code>ExecutorService</code>. I was wondering if this new API is the one that should be used, and if they are more efficient than the traditional ones, <code>Runnable</code> and <code>Thread</code>.</p> <p>After studying this new API, I came up with the following code (simplified version):</p> <pre><code> public abstract class Query implements Callable<Result[]> { private final String query; // gets set in the constructor public abstract Result[] querySearchEngine(); @Override public Result[] call() { return querySearchEngine(query); } } public class Querier { private ArrayList<Result> aggregatedResults; public Result[] queryAll(Query[] queries) { List<Future<Result[]>> futures = new ArrayList<Future<Result[]>>(queries.length); final ExecutorService service = Executors.newFixedThreadPool(queries.length); for (Query query : queries) { futures.add(service.submit(query)); } for (Future<Result[]> future : futures) { aggregatedResults.add(future.get()); // get() is somewhat similar to join? } return aggregatedResults; } } </code></pre> <p>I'm new to this concurrency API, and I'd like to know if there's something that can be <strong>improved</strong> in the above code, and if it's better than the first option (using <code>Thread</code>). There are some classes which I didn't explore, such as <code>FutureTask</code>, et cetera. I'd love to hear any advice on that as well.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload