Databases
StackOverflow2013
Postsdbo
POImporting XML file in Rails app, UTF-16 encoding prοblem
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POImporting XML file in Rails app, UTF-16 encoding prοblem
    text
    copied!<p>I'm trying to import an XML file via a web page in a Ruby on Rails application, the code ruby view code is as follows (I've removed HTML layout tags to make reading the code easier)</p> <pre><code>&lt;% form_for( :fmfile, :url =&gt; '/fmfiles', :html =&gt; { :method =&gt; :post, :name =&gt; 'Form_Import_DDR', :enctype =&gt; 'multipart/form-data' } ) do |f| %&gt; &lt;%= f.file_field :document, :accept =&gt; 'text/xml', :name =&gt; 'fmfile_document' %&gt; &lt;%= submit_tag 'Import DDR' %&gt; &lt;% end %&gt; </code></pre> <p>Results in the following HTML form</p> <pre><code>&lt;form action="/fmfiles" enctype="multipart/form-data" method="post" name="Form_Import_DDR"&gt;&lt;div style="margin:0;padding:0"&gt;&lt;input name="authenticity_token" type="hidden" value="3da97372885564a4587774e7e31aaf77119aec62" /&gt; &lt;input accept="text/xml" id="fmfile_document" name="fmfile_document" size="30" type="file" /&gt; &lt;input name="commit" type="submit" value="Import DDR" /&gt; &lt;/form&gt; </code></pre> <p>The Form_Import_DDR method in the 'fmfiles_controller' is the code that does the hard work of reading the XML document in using REXML. The code is as follows</p> <pre><code>@fmfile = Fmfile.new @fmfile.user_id = current_user.id @fmfile.file_group_id = 1 @fmfile.name = params[:fmfile_document].original_filename respond_to do |format| if @fmfile.save require 'rexml/document' doc = REXML::Document.new(params[:fmfile_document].read) doc.root.elements['File'].elements['BaseTableCatalog'].each_element('BaseTable') do |n| @base_table = BaseTable.new @base_table.base_table_create(@fmfile.user_id, @fmfile.id, n) end </code></pre> <p>And it carries on reading all the different XML elements in.</p> <p>I'm using Rails 2.1.0 and Mongrel 1.1.5 in Development environment on Mac OS X 10.5.4, site DB and browser on same machine.</p> <p>My question is this. This whole process works fine when reading an XML document with character encoding UTF-8 but fails when the XML file is UTF-16, does anyone know why this is happening and how it can be stopped?</p> <p>I have included the error output from the debugger console below, it takes about 5 minutes to get this output and the browser times out before the following output with the 'Failed to open page'</p> <pre><code>Processing FmfilesController#create (for 127.0.0.1 at 2008-09-15 16:50:56) [POST] Session ID: BAh7CDoMdXNlcl9pZGkGOgxjc3JmX2lkIiVmM2I3YWU2YWI4ODU2NjI0NDM2 NTFmMDE1OGY1OWQxNSIKZmxhc2hJQzonQWN0aW9uQ29udHJvbGxlcjo6Rmxh c2g6OkZsYXNoSGFzaHsABjoKQHVzZWR7AA==--dd9f588a68ed628ab398dd1a967eedcd09e505e0 Parameters: {"commit"=&gt;"Import DDR", "authenticity_token"=&gt;"3da97372885564a4587774e7e31aaf77119aec62", "action"=&gt;"create", "fmfile_document"=&gt;#&lt;File:/var/folders/LU/LU50A0vNHA07S4rxDAOk4E+++TI/-Tmp-/CGI.3001.1&gt;, "controller"=&gt;"fmfiles"} [4;36;1mUser Load (0.000350)[0m [0;1mSELECT * FROM "users" WHERE (id = 1) LIMIT 1[0m [4;35;1mFmfile Create (0.000483)[0m [0mINSERT INTO "fmfiles" ("name", "file_group_id", "updated_at", "report_created_at", "report_link", "report_version", "option_on_open_account_name", "user_id", "option_default_custom_menu_set", "option_on_close_script", "path", "report_type", "option_on_open_layout", "option_on_open_script", "created_at") VALUES('TheTest_fp7 2.xml', 1, '2008-09-15 15:50:56', NULL, NULL, NULL, NULL, 1, NULL, NULL, NULL, NULL, NULL, NULL, '2008-09-15 15:50:56')[0m REXML::ParseException (#&lt;Iconv::InvalidCharacter: "਼䙍偒数 (followed by a few thousand similar looking chinese characters) 䙍偒数潲琾", ["\n"]&gt; /Library/Ruby/Site/1.8/rexml/encodings/ICONV.rb:7:in `conv' /Library/Ruby/Site/1.8/rexml/encodings/ICONV.rb:7:in `decode' /Library/Ruby/Site/1.8/rexml/source.rb:50:in `encoding=' /Library/Ruby/Site/1.8/rexml/parsers/baseparser.rb:210:in `pull' /Library/Ruby/Site/1.8/rexml/parsers/treeparser.rb:21:in `parse' /Library/Ruby/Site/1.8/rexml/document.rb:190:in `build' /Library/Ruby/Site/1.8/rexml/document.rb:45:in `initialize' </code></pre>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload