Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>There are <em>hundreds</em> of control characters in unicode. If you are sanitizing data from the web or some other source that might contain non-ascii characters, you will need Python's <a href="http://docs.python.org/2/library/unicodedata.html">unicodedata module</a>. The <code>unicodedata.category(…)</code> function returns the <a href="http://www.unicode.org/reports/tr44/#GC_Values_Table">unicode category code</a> (e.g., control character, whitespace, letter, etc.) of any character. For control characters, the category always starts with "C".</p> <p>This snippet removes all control characters from a string.</p> <pre class="lang-py prettyprint-override"><code>import unicodedata def remove_control_characters(s): return "".join(ch for ch in s if unicodedata.category(ch)[0]!="C") </code></pre> <hr> <p>Examples of <a href="http://www.unicode.org/reports/tr44/#GC_Values_Table">unicode categories</a>:</p> <pre><code>&gt;&gt;&gt; from unicodedata import category &gt;&gt;&gt; category('\r') # carriage return --&gt; Cc : control character 'Cc' &gt;&gt;&gt; category('\0') # null character ---&gt; Cc : control character 'Cc' &gt;&gt;&gt; category('\t') # tab --------------&gt; Cc : control character 'Cc' &gt;&gt;&gt; category(' ') # space ------------&gt; Zs : separator, space 'Zs' &gt;&gt;&gt; category(u'\u200A') # hair space -------&gt; Zs : separator, space 'Zs' &gt;&gt;&gt; category(u'\u200b') # zero width space -&gt; Cf : control character, formatting 'Cf' &gt;&gt;&gt; category('A') # letter "A" -------&gt; Lu : letter, uppercase 'Lu' &gt;&gt;&gt; category(u'\u4e21') # 両 ---------------&gt; Lo : letter, other 'Lo' &gt;&gt;&gt; category(',') # comma -----------&gt; Po : punctuation 'Po' &gt;&gt;&gt; </code></pre>
    singulars
    1. This table or related slice is empty.
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. VO
      singulars
      1. This table or related slice is empty.
    2. VO
      singulars
      1. This table or related slice is empty.
    3. VO
      singulars
      1. This table or related slice is empty.
 

Querying!

 
Guidance

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload