Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>You say:</p> <blockquote> <p>The character that causes the problem is: å, </p> </blockquote> <p>How do you know that? What are you viewing your text with?</p> <p>So you can't publish the URL and your API key; what about reading the data, writing it to a file (in binary mode), and publishing that? </p> <p>When you open that file in your web browser, what encoding does it detect?</p> <p>At the very least, do this</p> <pre><code>data.decode('utf8') # where data is what you get from ur.read() </code></pre> <p>This will produce an exception that will tell you the byte offset of the non-UTF-8 stuff.</p> <p>Then do this:</p> <p><code>print repr(data[offset-10:offset+60])</code></p> <p>and show us the results.</p> <p>Assuming the encoding is actually <code>cp1252</code> and decoding the bytes in the lxml error message:</p> <pre><code>&gt;&gt;&gt; guff = "\xEA\x76\x65\x73" &gt;&gt;&gt; from unicodedata import name &gt;&gt;&gt; [name(c) for c in guff.decode('1252')] ['LATIN SMALL LETTER E WITH CIRCUMFLEX', 'LATIN SMALL LETTER V', 'LATIN SMALL LE TTER E', 'LATIN SMALL LETTER S'] &gt;&gt;&gt; </code></pre> <p>So are you seeing e-circumflex followed by <code>ves</code>, or a-ring followed by <code>ves</code>, or a-ring followed by something else?</p> <p>Does the data start with an XML declaration like <code>&lt;?xml version="1.0" encoding="UTF-8"?&gt;</code>? If not, what does it start with?</p> <p>Clues for encoding guessing/confirmation: What language is the text written in? What country?</p> <p><strong>UPDATE</strong> based on further information supplied.</p> <p>Based on the snippet that you showed in the vicinity of the error, the movie title is "La science des rêves" (the science of dreams). </p> <p>Funny how PHP gags on "F***ing Åmål" but Python chokes on French dreams. Are you sure that you did the same query?</p> <p>You should have told us it was IMDB up front, you would have got your answer much sooner. </p> <p><strong>SOLUTION</strong> before you pass <code>data</code> to the <code>lxml</code> parser, do this:</p> <pre><code>data = data.replace('encoding="UTF-8"', 'encoding="iso-8859-1"') </code></pre> <p>That's based on the encoding that they declare on their website, but that may be a lie too. In that case, try <code>cp1252</code> instead. It's definitely <strong>not iso-8859-2</strong>.</p>
    singulars
    1. This table or related slice is empty.
    1. POIgnore encoding errors in Python (iterparse)?
      singulars
      1. PTQuestion
    1. PTAnswer
    1. USJohn Machin
    1. USJohn Machin
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. POIgnore encoding errors in Python (iterparse)?
      singulars
      1. PTQuestion
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTAcceptedByOriginator
    1. COThe data from the API shouldn't be published, otherwise I would have. I added more info to my question. Thanks for your extensive answer =)
      singulars
      1. PO
      1. USMartti Laine
    2. COYour solution worked. Thanks a ton! PS. the data is not from IMDB, it's from a database of a subtitle provider.
      singulars
      1. PO
      1. USMartti Laine
    3. CO@MarttiLaine: Indirectly then; they have IMDB serial numbers e.g. `<imdb>0354899</imdb>` in their data.
      singulars
      1. PO
      1. USJohn Machin
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload