Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>Looking at the <a href="http://jamesabbottdd.com/examples/testfile.csv">file in question</a>:</p> <pre class="lang-none prettyprint-override"><code> $ curl -s http://jamesabbottdd.com/examples/testfile.csv | xxd | head -n3 0000000: fffe 4300 6100 6d00 7000 6100 6900 6700 ..C.a.m.p.a.i.g. 0000010: 6e00 0900 4300 7500 7200 7200 6500 6e00 n...C.u.r.r.e.n. 0000020: 6300 7900 0900 4200 7500 6400 6700 6500 c.y...B.u.d.g.e. </code></pre> <p>The <a href="http://en.wikipedia.org/wiki/Byte_order_mark">byte order mark<code>ffee</code> at the start</a> suggests the file encoding is little endian UTF-16, and the <code>00</code> bytes at every other position back this up.</p> <p>This would suggest that you should be able to do this:</p> <pre class="lang-ruby prettyprint-override"><code>CSV.foreach('./testfile.csv', :encoding =&gt; 'utf-16le') do |row| ... </code></pre> <p>However that gives me <code>invalid byte sequence in UTF-16LE (ArgumentError)</code> coming from <a href="https://github.com/ruby/ruby/blob/v2_0_0_353/lib/csv.rb#L1985">inside the CSV library</a>. I <em>think</em> this is due to <a href="http://ruby-doc.org/core-2.0.0/IO.html#method-i-gets">IO#gets</a> only returning a single byte for some reason when faced with the BOM when <a href="https://github.com/ruby/ruby/blob/v2_0_0_353/lib/csv.rb#L1978">called in CSV</a>, resulting in the invalid UTF-16.</p> <p>You can get CSV to strip of the BOM, by using <code>bom|utf-16-le</code> as the encoding:</p> <pre><code>CSV.foreach('./testfile.csv', :encoding =&gt; 'bom|utf-16le') do |row| ... </code></pre> <p>You might prefer to convert the string to a more familiar encoding instead, in which case you could do:</p> <pre><code>CSV.foreach('./testfile.csv', :encoding =&gt; 'utf-16le:utf-8') do |row| ... </code></pre> <p>Both of these appear to work okay.</p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. VO
      singulars
      1. This table or related slice is empty.
    2. VO
      singulars
      1. This table or related slice is empty.
    3. VO
      singulars
      1. This table or related slice is empty.
 

Querying!

 
Guidance

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload