Databases
StackOverflow2013
Postsdbo
POMulti-threading. Exception in thread
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POMulti-threading. Exception in thread
    primarykey
    data
    text
    <p>I'm trying to understand with an <a href="http://www.ibm.com/developerworks/aix/library/au-threadingpython/index.html" rel="nofollow">example</a>. Here's the code:</p> <pre><code>import Queue import threading import urllib2 import time from BeautifulSoup import BeautifulSoup hosts = ["http://yahoo.com", "http://google.com", "http://amazon.com", "http://ibm.com", "http://apple.com"] queue = Queue.Queue() out_queue = Queue.Queue() class ThreadUrl(threading.Thread): """Threaded Url Grab""" def __init__(self, queue, out_queue): threading.Thread.__init__(self) self.queue = queue self.out_queue = out_queue def run(self): while True: #grabs host from queue host = self.queue.get() #grabs urls of hosts and then grabs chunk of webpage url = urllib2.urlopen(host) chunk = url.read() #place chunk into out queue self.out_queue.put(chunk) #signals to queue job is done self.queue.task_done() class DatamineThread(threading.Thread): """Threaded Url Grab""" def __init__(self, out_queue): threading.Thread.__init__(self) self.out_queue = out_queue def run(self): while True: #grabs host from queue chunk = self.out_queue.get() #parse the chunk soup = BeautifulSoup(chunk) print soup.findAll(['title']) #signals to queue job is done self.out_queue.task_done() start = time.time() def main(): #spawn a pool of threads, and pass them queue instance for i in range(5): t = ThreadUrl(queue, out_queue) t.setDaemon(True) t.start() #populate queue with data for host in hosts: queue.put(host) for i in range(5): dt = DatamineThread(out_queue) dt.setDaemon(True) dt.start() #wait on the queue until everything has been processed queue.join() out_queue.join() main() print "Elapsed Time: %s" % (time.time() - start) </code></pre> <p>Sometimes I get this error here:</p> <p>Exception in thread Thread-10 (most likely raised during interpreter shutdown)</p> <p>Explain please what caused it.</p> <p><strong>Update by another author:</strong></p> <p>Here's the full exception I see in similar code:</p> <pre><code>Exception in thread Thread-1 (most likely raised during interpreter shutdown): Traceback (most recent call last): File "/usr/local/lib/python2.7/threading.py", line 552, in __bootstrap_inner File "/usr/local/lib/python2.7/threading.py", line 505, in run File "mine.py", line 86, in run File "/usr/local/lib/python2.7/Queue.py", line 168, in get File "/usr/local/lib/python2.7/threading.py", line 237, in wait &lt;type 'exceptions.TypeError'&gt;: 'NoneType' object is not callable </code></pre>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. USRy4an Brase
    1. USglebus
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POMulti-threading. Exception in thread
      1. USunutbu
      1. VTFavorite
    2. VO
      singulars
      1. POMulti-threading. Exception in thread
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POMulti-threading. Exception in thread
      1. This table or related slice is empty.
      1. VTUpMod
    1. COSome ideas how people might be more willing to help you: 1. Explain what you have done to diagnose the problem and why your efforts failed. 2. Try to reduce the code to the minimum necessary for the error to occur. 3. Give the full traceback of the error.
      singulars
      1. POMulti-threading. Exception in thread
      1. USSven Marnach
    2. COI see the same error in similar code. I'll add my full exception above.
      singulars
      1. POMulti-threading. Exception in thread
      1. USRy4an Brase
    3. COI find this error occasionally when I am using the multiprocessing.Pool class. It seems to be from the daemon threads it creates still being operational when the interpreter starts deleting things during shutdown. A solution which, if it has not solved it, it has lessened the frequency below my observation threshold is to make a call to pool.terminate() or similar when you are done with the multi-processing.
      singulars
      1. POMulti-threading. Exception in thread
      1. USJames Matta
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload