Databases
StackOverflow2013
Postsdbo
PODetect Unicode Character Range in PHP
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PODetect Unicode Character Range in PHP
    primarykey
    data
    text
    <p>Evening,</p> <p>Does anyone have an idea what the quickest way to detect the Unicode range of a string is in PHP? I thought there would be something to do this in PHP, but I can't find anything. Ideally what I want is a function that will say, 100% of 'John Jones' is Latin OR 'Jones језик' is 50% Latin and 50% Cyrillic.</p> <p>You could do it with something like the below in ReEx:</p> <pre><code>strA = 'John Jones'; $strB = 'Српски језик'; $strC = 'Հայաստանի Հանրապետություն'; preg_match( '~[\p{Cyrillic}\p{Common}]+~u', $strB, $res ); </code></pre> <p>But this will require checking against each range, which does not seem a good idea. Alternatively, you could get the unicode value of each character and check which range it is in. But I'd imagine someone has already made something like this.</p> <p><strong>EDIT</strong></p> <p>To give a little more idea on why this may be useful, as pointed out in the comments, some people sometimes mix the visually identical Latin and Cyrillic characters. e.g. this is a search for Croatia with a Cyrillic 'С' and the rest in Latin:</p> <p><a href="https://www.google.am/search?q=%22%D0%A1roatia%22&amp;aq=f&amp;oq=%22%D0%A1roatia%22" rel="nofollow">https://www.google.am/search?q=%22%D0%A1roatia%22&amp;aq=f&amp;oq=%22%D0%A1roatia%22</a></p> <p>Search again with full-Latin and you will get about 100,000,000 results instead of 20,000. In such cases it would be desirable to replace characters as is appropriate in the context of the text. A good example of where such detection is useful is people who use Cyrillic letter to bypass profanity filters.</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USKohjah Breese
    1. USKohjah Breese
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. PODetect Unicode Character Range in PHP
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. PODetect Unicode Character Range in PHP
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. PODetect Unicode Character Range in PHP
      1. This table or related slice is empty.
      1. VTUpMod
    1. COwhat problem are you trying to solve that makes you think you need to known which percentage of characters is from which unicode script block? for instance, knowing that 80% of the characters are from Latin and 20% from Latin Extended A won't tell you if it's Romanian, or Vietnamese.
      singulars
      1. PODetect Unicode Character Range in PHP
      1. USMike 'Pomax' Kamermans
    2. COHi, I've got a table of human names. Some of these are in various non-Latin characters and some are mixed. I need to detect which are which, as many alphabets, such as Cyrillic can be transliterated, but others, such as Arabic can't, or at least not meaningfully. Further in the case of people from Eastern Europe, Russia etc. they sometimes use Cyrillic characters that look exactly the same as Latin ones or vice-versa, leading to mixed string. I'm converting all names to Latin, and I need the above data, so I can know what to do to convert each name to Latin.
      singulars
      1. PODetect Unicode Character Range in PHP
      1. USKohjah Breese
    3. COWhy? just store UTF8 without converting to latin. It's 2013, virtually all common technologies support it.
      singulars
      1. PODetect Unicode Character Range in PHP
      1. USMike 'Pomax' Kamermans
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload