Databases
StackOverflow2013
Postsdbo
POHow to recover a broken python "cPickle" dump?
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POHow to recover a broken python "cPickle" dump?
    primarykey
    data
    text
    <p>I am using <code>rss2email</code> for converting a number of RSS feeds into mail for easier consumption. That is, I <em>was</em> using it because it broke in a horrible way today: On every run, it only gives me this backtrace:</p> <pre><code>Traceback (most recent call last): File "/usr/share/rss2email/rss2email.py", line 740, in &lt;module&gt; elif action == "list": list() File "/usr/share/rss2email/rss2email.py", line 681, in list feeds, feedfileObject = load(lock=0) File "/usr/share/rss2email/rss2email.py", line 422, in load feeds = pickle.load(feedfileObject) TypeError: ("'str' object is not callable", 'sxOYAAuyzSx0WqN3BVPjE+6pgPU', ((2009, 3, 19, 1, 19, 31, 3, 78, 0), {})) </code></pre> <p>The only helpful fact that I have been able to construct from this backtrace is that the file <code>~/.rss2email/feeds.dat</code> in which <code>rss2email</code> keeps all its configuration and runtime state is somehow broken. Apparently, <code>rss2email</code> reads its state and dumps it back using <code>cPickle</code> on every run.</p> <p>I have even found the line containing that <code>'sxOYAAuyzSx0WqN3BVPjE+6pgPU'</code>string mentioned above in the giant (>12MB) <code>feeds.dat</code> file. To my untrained eye, the dump does not appear to be truncated or otherwise damaged.</p> <p>What approaches could I try in order to reconstruct the file?</p> <p>The Python version is 2.5.4 on a Debian/unstable system.</p> <p><strong>EDIT</strong></p> <p>Peter Gibson and J.F. Sebastian have suggested directly loading from the pickle file and I had tried that before. Apparently, a <code>Feed</code> class that is defined in <code>rss2email.py</code> is needed, so here's my script:</p> <pre><code>#!/usr/bin/python import sys # import pickle import cPickle as pickle sys.path.insert(0,"/usr/share/rss2email") from rss2email import Feed feedfile = open("feeds.dat", 'rb') feeds = pickle.load(feedfile) </code></pre> <p>The "plain" pickle variant produces the following traceback:</p> <pre><code>Traceback (most recent call last): File "./r2e-rescue.py", line 8, in &lt;module&gt; feeds = pickle.load(feedfile) File "/usr/lib/python2.5/pickle.py", line 1370, in load return Unpickler(file).load() File "/usr/lib/python2.5/pickle.py", line 858, in load dispatch[key](self) File "/usr/lib/python2.5/pickle.py", line 1133, in load_reduce value = func(*args) TypeError: 'str' object is not callable </code></pre> <p>The <code>cPickle</code> variant produces essentially the same thing as calling <code>r2e</code> itself:</p> <pre><code>Traceback (most recent call last): File "./r2e-rescue.py", line 10, in &lt;module&gt; feeds = pickle.load(feedfile) TypeError: ("'str' object is not callable", 'sxOYAAuyzSx0WqN3BVPjE+6pgPU', ((2009, 3, 19, 1, 19, 31, 3, 78, 0), {})) </code></pre> <p><strong>EDIT 2</strong></p> <p>Following J.F. Sebastian's suggestion around putting "printf debugging" into <code>Feed.__setstate__</code> into my test script, these are the last few lines before Python bails out.</p> <pre><code> u'http:/com/news.ars/post/20080924-everyone-declares-victory-in-smutfree-wireless-broadband-test.html': u'http:/com/news.ars/post/20080924-everyone-declares-victory-in-smutfree-wireless-broadband-test.html'}, 'to': None, 'url': 'http://arstechnica.com/'} Traceback (most recent call last): File "./r2e-rescue.py", line 23, in ? feeds = pickle.load(feedfile) TypeError: ("'str' object is not callable", 'sxOYAAuyzSx0WqN3BVPjE+6pgPU', ((2009, 3, 19, 1, 19, 31, 3, 78, 0), {})) </code></pre> <p>The same thing happens on a Debian/etch box using python 2.4.4-2.</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USbrian d foy
    1. UShillu
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    3. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POHow to recover a broken python "cPickle" dump?
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POHow to recover a broken python "cPickle" dump?
      1. This table or related slice is empty.
      1. VTUpMod
    1. COI've updated my answer.
      singulars
      1. POHow to recover a broken python "cPickle" dump?
      1. USjfs
    2. COThe pickle format is quite simple. I remember I'd parsed it by hand when solving http://pythonchallenge.com puzzles (I was a beginner in Python and hadn't recognized the pickle format at the time). Therefore as last resort you can just extract the necessary data from feeds.dat by hand
      singulars
      1. POHow to recover a broken python "cPickle" dump?
      1. USjfs
    3. COHere "by hand" stands for a simple regex-based python script
      singulars
      1. POHow to recover a broken python "cPickle" dump?
      1. USjfs
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload