Databases
StackOverflow2013
Postsdbo
POBeautifulSoup and Amazon.co.uk
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POBeautifulSoup and Amazon.co.uk
    primarykey
    data
    text
    <p>I am trying to parse amazon to compile a list of prices, as part of a bigger project relating to statistics. However, I am stumped. I was wondering If anyone can review my code and tell me where I went wrong?</p> <pre><code>#!/usr/bin/python # -*- coding: utf-8 -*- import mechanize from bs4 import BeautifulSoup URL_00 = "http://www.amazon.co.uk/Call-Duty-Black-Ops-PS3/dp/B007WPF7FE/ref=sr_1_2?ie=UTF8&amp;qid=1352117194&amp;sr=8-2" bro = mechanize.Browser() resp = bro.open(URL_00) html = resp.get_data() soup_00 = BeautifulSoup(html) price = soup_00.find('b', {'class':'priceLarge'}) print price #this should return at the very least the text enclosed in a tag </code></pre> <p>According to the screenshot, what I wrote above should work, shouldn't it?</p> <p><img src="https://i.stack.imgur.com/SXRUt.png" alt="http://i.imgur.com/bPVe1.png (cannot post an image as a newbie..)"></p> <p>Well all I get in the print out is "[]", if I change the line before last to this:</p> <pre><code> price = soup_00.find('b', {'class':'priceLarge'}).contents[0].string </code></pre> <p>or</p> <pre><code>price = soup_00.find('b', {'class':'priceLarge'}).text </code></pre> <p>I get a "noneType" error.</p> <p>I am quite confused as to why this is happening. The page encoding in the URL on chrome says UTF8, to which my script is adjusted in line #2. I have changed it to ISO (as per inner HTML of the page) but this makes zero difference, so I am positive encoding is not the issue here.</p> <p>Also, don't know if this is relevant at all, but my system locale on linux being UTF-8 should not cause a problem should it?</p> <p>Any ideas would be welcome.</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. This table or related slice is empty.
    1. USNopeNopeNope
    plurals
    1. PL
      singulars
      1. LTLinked
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POBeautifulSoup and Amazon.co.uk
      1. This table or related slice is empty.
      1. VTUpMod
    1. COAnd you have confirmed that Mechanize is given the exact same HTML as your browser? Do *not* assume that Amazon will send the exact same response to different user agents.
      singulars
      1. POBeautifulSoup and Amazon.co.uk
      1. USMartijn Pieters
    2. COJust so you don't think I ignored the comment =) I have added a user agent to mechanize to mach mine. And a downloaded version of the page from my browser is equal to the one mechanize pulls, so this should not be an issue...
      singulars
      1. POBeautifulSoup and Amazon.co.uk
      1. USNopeNopeNope
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload