Databases
StackOverflow2013
Postsdbo
POSQL Server (SQLCMD), Python and encoding issue when using non ascii chars
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POSQL Server (SQLCMD), Python and encoding issue when using non ascii chars
    primarykey
    data
    text
    <p>i'm facing an encoding issue with my python code, when asking data that are in SQL Server 2005.</p> <p>(because i was <a href="https://stackoverflow.com/questions/7914108/error-unable-to-find-vcvarsall-bat-when-installing-pymssql-2-0-0b1-what-am-i">unable to compile PyMSSQL-2.0.0b1</a>) i'm using this <a href="http://code.activestate.com/recipes/144183-python-database-interface-for-ms-sql-server/" rel="nofollow noreferrer">piece of code</a> and i am able to do some select but now i stick with the issue that i do not know what SQLCMD is output-ting to me :(</p> <p>(i had to work with European language contained in table, so i had to face other encodings with accent and so on)</p> <p>for example : <br/></p> <ul> <li>when i read it (select) from the Ms SQLServer Management Studio i have this country name : 'Ceská republika' (note the first a is with acute on it)<br/></li> <li>when using it from SQLCMD from command line (Powershell in Windows 7), it is still ok, i can see the "Cesk'a with acute'"<br/></li> <li><p>now when using Python with the os.popen trick from the <a href="http://code.activestate.com/recipes/144183-python-database-interface-for-ms-sql-server/" rel="nofollow noreferrer">recipe</a>, that is with this connection string :<br/></p> <p>sqlcmd -U adminname -P password -S servername -d dbname /w 8192 -u</p></li> </ul> <p>i get this string : 'Cesk\xa0 republika'</p> <p>notice the \xa0 that i do know what encoding it is, and how i can pass from this \xa0 to {a with acute}...</p> <p>if i test from Python, and unicode i should have this one '\xe1'</p> <pre><code>&gt;&gt;&gt; unicode('Cesk\xa0 republika') Traceback (most recent call last): File "&lt;pyshell#13&gt;", line 1, in &lt;module&gt; unicode('Cesk\xa0 republika') UnicodeDecodeError: 'ascii' codec can't decode byte 0xa0 in position 4: ordinal not in range(128) &gt;&gt;&gt; unicode_a_with_acute = u'\N{LATIN SMALL LETTER A WITH ACUTE}' &gt;&gt;&gt; unicode_a_with_acute u'\xe1' &gt;&gt;&gt; print unicode_a_with_acute á &gt;&gt;&gt; print unicode_a_with_acute.encode('cp1252') á &gt;&gt;&gt; unicode_a_with_acute.encode('cp1252') '\xe1' &gt;&gt;&gt; print 'Cesk\xa0 republika'.decode('cp1252') Cesk  republika &gt;&gt;&gt; print 'Cesk\xa0 republika'.decode('utf8') Traceback (most recent call last): File "&lt;pyshell#21&gt;", line 1, in &lt;module&gt; print 'Cesk\xa0 republika'.decode('utf8') File "C:\Python27\lib\encodings\utf_8.py", line 16, in decode return codecs.utf_8_decode(input, errors, True) UnicodeDecodeError: 'utf8' codec can't decode byte 0xa0 in position 4: invalid start byte </code></pre> <p>so what SQLCMD is giving to me? How should i force it and/or os.popen and others to be sure that i have understandable utf8 for Python?</p> <p>(notice, i have tried both with and without the -u ending on the os.popen cmd for SQLCMD and that should stand for asking to SQLCMD to answer in unicode, with no effect, also i have tried to feed it with a "select" python string encoded in utf8 with no more success :</p> <pre><code> sqlstr = unicode('select * from table_pays where country_code="CZ"') cu = c.cursor lst = cu.execute(sqlstr) rows = cu.fetchall() for x in rows: print x ( 'CZ ', 'Cesk\xa0 republika ') </code></pre> <p>)</p> <p>another point : from what i googl-ed, about "sqlcmd.exe", there are also these parameters that could may be help : </p> <pre><code>[ -f &lt; codepage &gt; | i: &lt; codepage &gt; [ &lt; , o: &lt; codepage &gt; ] ] </code></pre> <p>but i was unable to specify the right one, i do not know what are the possible values, BTW using (or not using) the :</p> <pre><code>[ -u unicode output] </code></pre> <p>dit not help me also...</p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. USCommunity
    1. USlctv31
    plurals
    1. PL
      singulars
      1. LTLinked
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POSQL Server (SQLCMD), Python and encoding issue when using non ascii chars
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POSQL Server (SQLCMD), Python and encoding issue when using non ascii chars
      1. USChristophe Debove
      1. VTBountyStart
    3. VO
      singulars
      1. POSQL Server (SQLCMD), Python and encoding issue when using non ascii chars
      1. This table or related slice is empty.
      1. VTUpMod
    1. This table or related slice is empty.
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload