Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>Thanks for the replies, it works!</p> <p>And since the source files are in mixed formats, I added a list of source formats to be tried in sequence (<code>sourceFormats</code>), and on <code>UnicodeDecodeError</code> I try the next format:</p> <pre><code>from __future__ import with_statement import os import sys import codecs from chardet.universaldetector import UniversalDetector targetFormat = 'utf-8' outputDir = 'converted' detector = UniversalDetector() def get_encoding_type(current_file): detector.reset() for line in file(current_file): detector.feed(line) if detector.done: break detector.close() return detector.result['encoding'] def convertFileBestGuess(filename): sourceFormats = ['ascii', 'iso-8859-1'] for format in sourceFormats: try: with codecs.open(fileName, 'rU', format) as sourceFile: writeConversion(sourceFile) print('Done.') return except UnicodeDecodeError: pass def convertFileWithDetection(fileName): print("Converting '" + fileName + "'...") format=get_encoding_type(fileName) try: with codecs.open(fileName, 'rU', format) as sourceFile: writeConversion(sourceFile) print('Done.') return except UnicodeDecodeError: pass print("Error: failed to convert '" + fileName + "'.") def writeConversion(file): with codecs.open(outputDir + '/' + fileName, 'w', targetFormat) as targetFile: for line in file: targetFile.write(line) # Off topic: get the file list and call convertFile on each file # ... </code></pre> <p>(EDIT by Rudro Badhon: this incorporates the original try multiple formats until you don't get an exception as well as an alternate approach that uses chardet.universaldetector)</p>
    singulars
    1. This table or related slice is empty.
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. VO
      singulars
      1. This table or related slice is empty.
    2. VO
      singulars
      1. This table or related slice is empty.
    3. VO
      singulars
      1. This table or related slice is empty.
 

Querying!

 
Guidance

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload