Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>You should indeed be nervous. The whole idea that you might have a mixture of bytes and text in some data structure is horrifying. It violates the fundamental principle of working with string data: decode at input time, work exclusively in unicode, encode at output time.</p> <p>Update in response to comment:</p> <p>You are about to output some sort of HTTP request. This needs to be prepared as a byte string. The fact that urllib.urlencode is not capable of properly preparing that byte string if there are unicode characters with ordinal >= 128 in your dict is indeed unfortunate. If you have a mixture of byte strings and unicode strings in your dict, you need to be careful. Let's examine just what urlencode() does:</p> <pre><code>&gt;&gt;&gt; import urllib &gt;&gt;&gt; tests = ['\x80', '\xe2\x82\xac', 1, '1', u'1', u'\x80', u'\u20ac'] &gt;&gt;&gt; for test in tests: ... print repr(test), repr(urllib.urlencode({'a':test})) ... '\x80' 'a=%80' '\xe2\x82\xac' 'a=%E2%82%AC' 1 'a=1' '1' 'a=1' u'1' 'a=1' u'\x80' Traceback (most recent call last): File "&lt;stdin&gt;", line 2, in &lt;module&gt; File "C:\python27\lib\urllib.py", line 1282, in urlencode v = quote_plus(str(v)) UnicodeEncodeError: 'ascii' codec can't encode character u'\x80' in position 0: ordinal not in range(128) </code></pre> <p>The last two tests demonstrate the problem with urlencode(). Now let's look at the str tests.</p> <p>If you insist on having a mixture, then you should at the very least ensure that the str objects are encoded in UTF-8.</p> <p>'\x80' is suspicious -- it is not the result of any_valid_unicode_string.encode('utf8').<br> '\xe2\x82\xac' is OK; it's the result of u'\u20ac'.encode('utf8').<br> '1' is OK -- all ASCII characters are OK on input to urlencode(), which will percent-encode such as '%' if necessary.</p> <p>Here's a suggested converter function. It doesn't mutate the input dict as well as returning it (as yours does); it returns a new dict. It forces an exception if a value is a str object but is not a valid UTF-8 string. By the way, your concern about it not handling nested objects is a little misdirected -- your code works only with dicts, and the concept of nested dicts doesn't really fly.</p> <pre><code>def encoded_dict(in_dict): out_dict = {} for k, v in in_dict.iteritems(): if isinstance(v, unicode): v = v.encode('utf8') elif isinstance(v, str): # Must be encoded in UTF-8 v.decode('utf8') out_dict[k] = v return out_dict </code></pre> <p>and here's the output, using the same tests in reverse order (because the nasty one is at the front this time):</p> <pre><code>&gt;&gt;&gt; for test in tests[::-1]: ... print repr(test), repr(urllib.urlencode(encoded_dict({'a':test}))) ... u'\u20ac' 'a=%E2%82%AC' u'\x80' 'a=%C2%80' u'1' 'a=1' '1' 'a=1' 1 'a=1' '\xe2\x82\xac' 'a=%E2%82%AC' '\x80' Traceback (most recent call last): File "&lt;stdin&gt;", line 2, in &lt;module&gt; File "&lt;stdin&gt;", line 8, in encoded_dict File "C:\python27\lib\encodings\utf_8.py", line 16, in decode return codecs.utf_8_decode(input, errors, True) UnicodeDecodeError: 'utf8' codec can't decode byte 0x80 in position 0: invalid start byte &gt;&gt;&gt; </code></pre> <p>Does that help?</p>
    singulars
    1. This table or related slice is empty.
    1. POurllib.urlencode doesn't like unicode values: how about this workaround?
      singulars
      1. PTQuestion
    1. PTAnswer
    1. USJohn Machin
    1. USJohn Machin
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. POurllib.urlencode doesn't like unicode values: how about this workaround?
      singulars
      1. PTQuestion
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTAcceptedByOriginator
    3. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    1. This table or related slice is empty.
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload