Databases
StackOverflow2013
Postsdbo
POHow to prevent blocking forever on urlopen
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POHow to prevent blocking forever on urlopen
    primarykey
    data
    text
    <p>I'm running into a problem in that <code>urllib2.urlopen</code>/<code>requests.post</code> is very occasionally blocking forever on <code>socket.recv</code> and never returning.</p> <p>I'm trying to find out why this is happening and address that problem, but in the mean time I wondered if there was a way from preventing it blocking forever?</p> <p>I already know about the <code>timeout</code> optional argument for <code>urllib2.urlopen</code> and <code>socket.setdefaulttimeout</code> but unfortunately for my use case a timeout isn't a solution as I'm uploading files with POST any timeout value I use would risk interrupting a normal file upload.</p> <p>I've also seen some solutions using signals, but this will have the same problem as using timeouts for me (and is also out the question because I'm not doing this from the main thread).</p> <p>Is it possible to timeout only if no data has been sent/received through the socket for a certain amount of time perhaps? Or maybe there's some way I can use select / poll to prevent the deadlock / blocking that I'm experiencing?</p> <p>If there is a solution using select / poll, how would I go about incorporating this into <code>urllib2.urlopen</code>/<code>requests.post</code>?</p> <hr> <p>I also had the idea that if I could send file data through a write type of interface, so I'd control iterating over the file and sending chunks at a time I could probably have enough control to avoid the stalls. I'm not sure how to achieve it though so I asked the question: <a href="https://stackoverflow.com/questions/15230595/upload-a-file-with-a-file-write-interface">Upload a file with a file.write interface</a></p> <p><strong>UPDATE</strong> It seems I've always had a misconception of the meaning of <code>timeout</code> in python, it seems it is actually an idle timeout or read/write timeout (probably the first time I've <a href="http://markmail.org/message/joyaeqco47v3u2o2" rel="nofollow noreferrer">disagreed with Guido</a>). I always thought it was the max amount of time the response should return in - thank you @tomasz for pointing this out!!</p> <p>But after adding timeout parameters (tested with both <code>urllib2</code> and <code>requests</code>) I've come across some really odd and subtle scenarios, possibly mac specific, where the timeout doesn't work correctly which I'm getting more and more inclined to believe is a bug. I'm going to continue to investigate and find out exactly what the issue is. Again thank you tomasz for your help with this!</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USCommunity
    1. USGP89
    plurals
    1. PL
      singulars
      1. LTLinked
    2. PL
      singulars
      1. LTLinked
    3. PL
      singulars
      1. LTLinked
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    3. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POHow to prevent blocking forever on urlopen
      1. USGP89
      1. VTFavorite
    2. VO
      singulars
      1. POHow to prevent blocking forever on urlopen
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POHow to prevent blocking forever on urlopen
      1. USGP89
      1. VTBountyStart
    1. COFirst of all - *why* is it blocking forever?
      singulars
      1. POHow to prevent blocking forever on urlopen
      1. USCode Painters
    2. CO@CodePainters I don't know - Ideally I would solve the actual issue, and I'm going to continue to try, but it could be a server side issue (that I dont control) so until I can find the cause I'd like to put in place something as a fallback so the uploads never freeze for eternity and release that as a hotfix in the meantime.
      singulars
      1. POHow to prevent blocking forever on urlopen
      1. USGP89
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload