Databases
StackOverflow2013
Postsdbo
POUTF-8 encoding in a rails model
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POUTF-8 encoding in a rails model
    text
    copied!<p>I have a MySQL database, set to use UTF-8.</p> <p>In my database.yml, the database is set to utf8.</p> <p>I am doing some HTML scraping and inserting into the MySQL database.</p> <p>If I retrieve the HTML from the database in PHP, it correctly encodes all characters and produces fine input:</p> <pre><code>// code $result = mysql_query("SELECT raw_html FROM pages WHERE id = 1"); echo mysql_result($result,0); // output Hawaiʻi. </code></pre> <p>And the output looks great. However, in rails, I get strange characters:</p> <pre><code>// code in the controller @page = Page.find(params[:id]) // code in the view &lt;%= @page.raw_html %&gt; // output HawaiÊ»i </code></pre> <p>Is there somewhere else I need to force UTF-8? I've tried using the iconv library to no avail (unless I'm using it wrong).</p> <p>UPDATE: I've reproduced the same problem when using the console. So:</p> <pre><code>Page.find(2).raw_html[91..94] "Ê»" </code></pre> <p>The problem also occurs under the console (script/console) if that sheds any more light on the issue.</p> <p>UPDATE 2: Okay, on further investigation I've realized I was doing something dumb. But it didn't fix it.</p> <p>While the table was set to UTF8, the column was not. I've changed the column to be 'utf8_general_ci'. However (and this makes me think I'm screwing something basic up), this actually produces the correct result:</p> <pre><code>@raw_html = Iconv.conv('LATIN1','UTF-8',@page.raw_html[0..10000]) </code></pre> <p>That comes out lovely. Unfortunately, if I run the whole page through, I get:</p> <pre><code>Iconv::IllegalSequence in PagesController#show "€²18″N&lt;"... </code></pre> <p>So there's some other funky stuff going on in there. Could it be that I still have it 'latin' encoded, even though I've explicitly set both the table and the column to UTF-8 (and repopulated the HTML) ? I'm currently using the mysql2 gem as well, per Jeffrey's suggestion.</p> <p>UPDATE 3: To clarify, I'm getting console errors as well. This is the command:</p> <pre><code>Page.find(2).raw_html[91..94] </code></pre> <p>And this is the response:</p> <pre><code>"Ê»" </code></pre>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload