Databases
StackOverflow2013
Postsdbo
POHow to deal with invalid byte sequence for user input via Rack middleware?
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POHow to deal with invalid byte sequence for user input via Rack middleware?
    text
    copied!<p>So some of the users in our system are copying and pasting text into my app. In my logs I will occasionally notice this:</p> <pre><code>ArgumentError (invalid byte sequence in US-ASCII): vendor/bundle/ruby/1.9.1/gems/activesupport-2.3.14/lib/active_support/core_ext/object/blank.rb:68:in `=~' vendor/bundle/ruby/1.9.1/gems/activesupport-2.3.14/lib/active_support/core_ext/object/blank.rb:68:in `!~' vendor/bundle/ruby/1.9.1/gems/activesupport-2.3.14/lib/active_support/core_ext/object/blank.rb:68:in `blank?' vendor/bundle/ruby/1.9.1/gems/actionpack-2.3.14/lib/action_controller/response.rb:202:in `nonempty_ok_response?' vendor/bundle/ruby/1.9.1/gems/actionpack-2.3.14/lib/action_controller/response.rb:188:in `handle_conditional_get!' vendor/bundle/ruby/1.9.1/gems/actionpack-2.3.14/lib/action_controller/response.rb:141:in `prepare!' vendor/bundle/ruby/1.9.1/gems/actionpack-2.3.14/lib/action_controller/base.rb:540:in `send_response' vendor/bundle/ruby/1.9.1/gems/actionpack-2.3.14/lib/action_controller/base.rb:534:in `process' vendor/bundle/ruby/1.9.1/gems/actionpack-2.3.14/lib/action_controller/filters.rb:606:in `process_with_filters' vendor/bundle/ruby/1.9.1/gems/actionpack-2.3.14/lib/action_controller/base.rb:391:in `process' vendor/bundle/ruby/1.9.1/gems/actionpack-2.3.14/lib/action_controller/base.rb:386:in `call' vendor/bundle/ruby/1.9.1/gems/actionpack-2.3.14/lib/action_controller/routing/route_set.rb:438:in `call' </code></pre> <p>And here are the request details. Notice the invalid characters.</p> <pre><code>Parameters: {"attendee"=&gt;{"segment"=&gt;"Middle Market \xE2\x80\x93 West Region"}} </code></pre> <p>The problem is that the character encodings are off. My application is setup for UTF-8, and I believe they are sending ASCII characters. I need a way to filter this so that I can avoid the above error.</p> <p>Notice that the error is being raised before it even gets to my application. Therefore, I'm assuming the best solution is Rack middleware that filters user input to ensure its the proper encoding.</p> <p>Any idea on the best way to accomplish this? I'm 100% fine if invalid characters get replaced with a question mark. Or a response is sent back saying your input has invalid characters. Right now they are just getting a generic error, and on ajax requests they get nothing.</p> <p>Thanks.</p>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload