StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>I might have just found the problem and the solution, but unfortunately I had to replace Python's <code>zipfile</code> module with a hacked one of my own (called <code>myzipfile</code> here).</p> <pre><code>$ diff -u ~/run/lib/python2.7/zipfile.py myzipfile.py --- /home/msabramo/run/lib/python2.7/zipfile.py 2010-12-22 17:02:34.000000000 -0800 +++ myzipfile.py 2011-04-11 11:51:59.000000000 -0700 @@ -5,6 +5,7 @@ import binascii, cStringIO, stat import io import re +import copy try: import zlib # We may need its compression method @@ -877,7 +878,7 @@ # Only open a new file for instances where we were not # given a file object in the constructor if self._filePassed: - zef_file = self.fp + zef_file = copy.copy(self.fp) else: zef_file = open(self.filename, 'rb') </code></pre> <p>The problem in the standard <code>zipfile</code> module is that when passed a file object (not a filename), it uses that same passed-in file object for every call to the <code>open</code> method. This means that <code>tell</code> and <code>seek</code> are getting called on the same file and so trying to open multiple files within the zip file is causing the file position to be shared and so multiple <code>open</code> calls result in them stepping all over each other. In contrast, when passed a filename, <code>open</code> opens a new file object. My solution is for the case when a file object is passed in, instead of using that file object directly, I create a copy of it.</p> <p>This change to <code>zipfile</code> fixes the problems I was seeing:</p> <pre><code>$ ./test_zip_file.py ~/data.zip StringIO 1 Processed 250 items. $ ./test_zip_file.py ~/data.zip BytesIO 1 Processed 250 items. $ ./test_zip_file.py ~/data.zip direct 1 Processed 250 items. </code></pre> <p>but I don't know if it has other negative impacts on <code>zipfile</code>...</p> <p><strong>EDIT:</strong> I just found a mention of this in the Python docs that I had somehow overlooked before. At <a href="http://docs.python.org/library/zipfile.html#zipfile.ZipFile.open" rel="noreferrer">http://docs.python.org/library/zipfile.html#zipfile.ZipFile.open</a>, it says: </p> <blockquote> <p><strong>Note:</strong> If the ZipFile was created by passing in a file-like object as the first argument to the constructor, then the object returned by <code>open()</code> shares the ZipFile’s file pointer. Under these circumstances, the object returned by <code>open()</code> should not be used after any additional operations are performed on the ZipFile object. If the ZipFile was created by passing in a string (the filename) as the first argument to the constructor, then <code>open()</code> will create a new file object that will be held by the ZipExtFile, allowing it to operate independently of the ZipFile.</p> </blockquote>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload