Databases
StackOverflow2013
Postsdbo
POPython tell when an ftp transfer sits on completion
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POPython tell when an ftp transfer sits on completion
    text
    copied!<p>I have to download some files from an FTP server. Seems prosaic enough. However, the way this server behaves is if the file is very large, the connection will just hang when the download ostensibly completes.</p> <p>How can I handle this gracefully using ftplib in python?</p> <p>Sample python code:</p> <pre><code>from ftplib import FTP ... ftp = FTP(host) ftp.login(login, passwd) files=ftp.nlst() ftp.set_debuglevel(2) for fname in files: ret_status = ftp.retrbinary('RETR ' + fname, open(fname, 'wb').write) </code></pre> <p>debug output from the above:</p> <pre><code>*cmd* 'TYPE I' *put* 'TYPE I\r\n' *get* '200 Type set to I.\r\n' *resp* '200 Type set to I.' *cmd* 'PASV' *put* 'PASV\r\n' *get* '227 Entering Passive Mode (0,0,0,0,10,52).\r\n' *resp* '227 Entering Passive Mode (0,0,0,0,10,52).' *cmd* 'RETR some_file' *put* 'RETR some_file\r\n' *get* '125 Data connection already open; Transfer starting.\r\n' *resp* '125 Data connection already open; Transfer starting.' [just sits there indefinitely] </code></pre> <p>This is what it looks like when I attempt the same download using curl -v:</p> <pre><code>* About to connect() to some_server port 21 (#0) * Trying some_ip... connected * Connected to some_server (some_ip) port 21 (#0) &lt; 220 Microsoft FTP Service &gt; USER some_user &lt; 331 Password required for some_user. &gt; PASS some_password &lt; 230 User some_user logged in. &gt; PWD &lt; 257 "/some_dir" is current directory. * Entry path is '/some_dir' &gt; EPSV * Connect data stream passively &lt; 500 'EPSV': command not understood * disabling EPSV usage &gt; PASV &lt; 227 Entering Passive Mode (0,0,0,0,11,116). * Trying some_ip... connected * Connecting to some_ip (some_ip) port 2932 &gt; TYPE I &lt; 200 Type set to I. &gt; SIZE some_file &lt; 213 229376897 &gt; RETR some_file &lt; 125 Data connection already open; Transfer starting. * Maxdownload = -1 * Getting file with size: 229376897 { [data not shown] % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 218M 100 218M 0 0 182k 0 0:20:28 0:20:28 --:--:-- 0* FTP response timeout * control connection looks dead 100 218M 100 218M 0 0 182k 0 0:20:29 0:20:29 --:--:-- 0* Connection #0 to host some_server left intact curl: (28) FTP response timeout * Closing connection #0 </code></pre> <p>wget output is kind of interesting as well, it notices the connection is dead, then attempts to re-download the file which only confirms that it is already finished:</p> <pre><code>--2009-07-09 11:32:23-- ftp://some_server/some_file =&gt; `some_file' Resolving some_server... 0.0.0.0 Connecting to some_server|0.0.0.0|:21... connected. Logging in as some_user ... Logged in! ==&gt; SYST ... done. ==&gt; PWD ... done. ==&gt; TYPE I ... done. ==&gt; CWD not needed. ==&gt; SIZE some_file ... 229376897 ==&gt; PASV ... done. ==&gt; RETR some_file ... done. Length: 229376897 (219M) 100%[==========================================================&gt;] 229,376,897 387K/s in 18m 54s 2009-07-09 11:51:17 (198 KB/s) - Control connection closed. Retrying. --2009-07-09 12:06:18-- ftp://some_server/some_file (try: 2) =&gt; `some_file' Connecting to some_server|0.0.0.0|:21... connected. Logging in as some_user ... Logged in! ==&gt; SYST ... done. ==&gt; PWD ... done. ==&gt; TYPE I ... done. ==&gt; CWD not needed. ==&gt; SIZE some_file ... 229376897 ==&gt; PASV ... done. ==&gt; REST 229376897 ... done. ==&gt; RETR some_file ... done. Length: 229376897 (219M), 0 (0) remaining 100%[+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++] 229,376,897 --.-K/s in 0s 2009-07-09 12:06:18 (0.00 B/s) - `some_file' saved [229376897] </code></pre>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload