Databases
StackOverflow2013
Postsdbo
PORead Unicode characters from command-line arguments in Python 2.x on Windows
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PORead Unicode characters from command-line arguments in Python 2.x on Windows
    text
    copied!<p>I want my Python script to be able to read Unicode command line arguments in Windows. But it appears that sys.argv is a string encoded in some local encoding, rather than Unicode. How can I read the command line in full Unicode?</p> <p>Example code: <code>argv.py</code></p> <pre><code>import sys first_arg = sys.argv[1] print first_arg print type(first_arg) print first_arg.encode("hex") print open(first_arg) </code></pre> <p>On my PC set up for Japanese code page, I get:</p> <pre><code>C:\temp&gt;argv.py "PC・ソフト申請書08.09.24.doc" PC・ソフト申請書08.09.24.doc &lt;type 'str'&gt; 50438145835c83748367905c90bf8f9130382e30392e32342e646f63 &lt;open file 'PC・ソフト申請書08.09.24.doc', mode 'r' at 0x00917D90&gt; </code></pre> <p>That's Shift-JIS encoded I believe, and it "works" for that filename. But it breaks for filenames with characters that aren't in the Shift-JIS character set—the final "open" call fails:</p> <pre><code>C:\temp&gt;argv.py Jörgen.txt Jorgen.txt &lt;type 'str'&gt; 4a6f7267656e2e747874 Traceback (most recent call last): File "C:\temp\argv.py", line 7, in &lt;module&gt; print open(first_arg) IOError: [Errno 2] No such file or directory: 'Jorgen.txt' </code></pre> <p>Note—I'm talking about Python 2.x, not Python 3.0. I've found that Python 3.0 gives <code>sys.argv</code> as proper Unicode. But it's a bit early yet to transition to Python 3.0 (due to lack of 3rd party library support).</p> <p><strong>Update:</strong></p> <p>A few answers have said I should decode according to whatever the <code>sys.argv</code> is encoded in. The problem with that is that it's not full Unicode, so some characters are not representable.</p> <p>Here's the use case that gives me grief: I have <a href="https://stackoverflow.com/q/142844/60075">enabled drag-and-drop of files onto .py files in Windows Explorer</a>. I have file names with all sorts of characters, including some not in the system default code page. My Python script doesn't get the right Unicode filenames passed to it via sys.argv in all cases, when the characters aren't representable in the current code page encoding.</p> <p>There is certainly some Windows API to read the command line with full Unicode (and Python 3.0 does it). I assume the Python 2.x interpreter is not using it.</p>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload