StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POpython conflicts in two external packages
text
Body
copied!<p>I am writing code to combine functions from the python rawdog RSS reader library and the BeautifulSoup webscraping library. There is a conflict somewhere in the innards that I am trying to overcome. </p> <p>I can replicate the problem with this simplified code:</p> <pre><code> import sys, gzip def scrape(filename): contents = gzip.open(filename,'rb').read() contents = contents.decode('utf-8','replace') import BeautifulSoup as BS print 'before rawdog: ', len(BS.BeautifulSoup(contents)) # prints 4, correct answer from rawdoglib import rawdog as rd print 'after rawdog: ', len(BS.BeautifulSoup(contents)) # prints 3, incorrect answer </code></pre> <p>It does not matter what order or where I do the imports, the import of rawdog always causes the BS.BeautifulSoup() method to return the wrong response. I don't actually need rawdog anymore by the time I get to needing BeautifulSoup, so I've tried removing the package at that point, but BS is still broken. Fixes I have tried that have not worked:</p> <ul> <li>I noticed that the rawdog code does its own import of BeautifulSoup. So I tried removing <code>import BeautifulSoup</code> from the rawdog code and re-installing rawdog</li> <li>removing the rawdog modules before importing BeautifulSoup: <ul> <li><code>for x in filter(lambda y: y.startswith('rawdog'), sys.modules.keys()): del sys.modules[x]</code></li> </ul></li> <li>importing more specific classes/methods from rawdog, e.g <code>from rawdoglib.rawdog import FeedState</code></li> <li>give the problem method a new name, before and after importing rawdog: <code>from BeautifulSoup import BeautifulSoup as BS</code></li> <li><code>from __future__ import absolute_import</code></li> </ul> <p>No luck, I always get len(BeautifulSoup(contents)) == 3 if rawdog was ever imported into the namespace. Both packages are complex enough that I haven't been able to figure out exactly what the problem overlap is, and I'm not sure what tools to use to try to figure that out, other than searching through dir(BeautifulSoup) and dir(rawdog), where I haven't found good clues. </p> <p>Updates, responding to answers: I omitted that the problem does not occur with every input file, which is crucial, sorry. The offending files are quite large so I don't think I can post them here. I will try to figure out the crucial difference between the good and bad files and post it. Thanks for the debugging help so far. </p> <p>Further debugging! I have identified this block in the input text as problematic:</p> <pre><code> function SwitchMenu(obj){ if(document.getElementById){ var el = document.getElementById(obj); var ar = document.getElementById("masterdiv").getElementsByTagName("span"); //DynamicDrive.com change if(el.style.display != "block"){ //DynamicDrive.com change for (var i=0; i<ar.length; i++){ if (ar[i].className=="submenu") //DynamicDrive.com change ar[i].style.display = "none"; } el.style.display = "block"; }else{ el.style.display = "none"; } } </code></pre> <p>}</p> <p>If I comment out this block, then I get the correct parse through BeautifulSoup with or without the rawdog import. With the block, rawdog + BeautifulSoup is faulty. So should I just search my input for a block like this, or is there a better workaround?</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload