Databases
StackOverflow2013
Postsdbo
POspark python API java_gateway socket connection error
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POspark python API java_gateway socket connection error
    text
    copied!<p>My cluster is <code>Spark-0.7.2</code> + <code>Mesos-0.9</code>. I wrote a spark program in python, and it runs well in local mode. But some errors occurred when I ran it on mesos. Here is the error info:</p> <pre><code>13/09/30 15:40:13 INFO TaskSetManager: Finished TID 13 in 242 ms (progress: 2/3) 13/09/30 15:40:13 INFO DAGScheduler: Completed ResultTask(4, 1) send Exception in thread "DAGScheduler" spark.SparkException: EOF reached before Python server acknowledged at spark.api.python.PythonAccumulatorParam.addInPlace(PythonRDD.scala:303) at spark.api.python.PythonAccumulatorParam.addInPlace(PythonRDD.scala:278) at spark.Accumulable.$plus$plus$eq(Accumulators.scala:52) at spark.Accumulators$$anonfun$add$2.apply(Accumulators.scala:235) at spark.Accumulators$$anonfun$add$2.apply(Accumulators.scala:233) at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:93) at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:93) at scala.collection.Iterator$class.foreach(Iterator.scala:660) at scala.collection.mutable.HashTable$$anon$1.foreach(HashTable.scala:157) at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:190) at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:43) at scala.collection.mutable.HashMap.foreach(HashMap.scala:93) at spark.Accumulators$.add(Accumulators.scala:233) at spark.scheduler.DAGScheduler.handleTaskCompletion(DAGScheduler.scala:494) at spark.scheduler.DAGScheduler.processEvent(DAGScheduler.scala:300) at spark.scheduler.DAGScheduler.spark$scheduler$DAGScheduler$$run(DAGScheduler.scala:364) at spark.scheduler.DAGScheduler$$anon$1.run(DAGScheduler.scala:107) 13/09/30 15:40:13 INFO TaskSetManager: Finished TID 12 in 407 ms (progress: 3/3) </code></pre> <p>This isn't happening every time. It seems the socket connection is not statable. Can someone suggest how to fix this problem?</p>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload