Databases
StackOverflow2013
Postsdbo
POWhy does modern Perl avoid UTF-8 by default?
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POWhy does modern Perl avoid UTF-8 by default?
    primarykey
    data
    text
    <p>I wonder why most modern solutions built using Perl don't enable <a href="http://en.wikipedia.org/wiki/UTF-8" rel="noreferrer">UTF-8</a> by default.</p> <p>I understand there are many legacy problems for core Perl scripts, where it may break things. But, from my point of view, in the 21<sup>st</sup> century, big new projects (or projects with a big perspective) should make their software UTF-8 proof from scratch. Still I don't see it happening. For example, <a href="http://en.wikipedia.org/wiki/Moose_%28Perl%29" rel="noreferrer">Moose</a> enables strict and warnings, but not <a href="http://en.wikipedia.org/wiki/Unicode" rel="noreferrer">Unicode</a>. <a href="http://search.cpan.org/~chromatic/Modern-Perl-1.03/lib/Modern/Perl.pm" rel="noreferrer">Modern::Perl</a> reduces boilerplate too, but no UTF-8 handling.</p> <p>Why? Are there some reasons to avoid UTF-8 in modern Perl projects in the year 2011?</p> <hr> <p>Commenting @tchrist got too long, so I'm adding it here.</p> <p>It seems that I did not make myself clear. Let me try to add some things.</p> <p><strong>tchrist</strong> and I see situation pretty similarly, but our conclusions are completely in opposite ends. I agree, the situation with Unicode is complicated, but this is why we (Perl users and coders) need some layer (or pragma) which makes UTF-8 handling as easy as it must be nowadays.</p> <p><strong>tchrist</strong> pointed to many aspects to cover, I will read and think about them for days or even weeks. Still, this is not my point. <strong>tchrist</strong> tries to prove that there is not one single way "to enable UTF-8". I have not so much knowledge to argue with that. So, I stick to live examples.</p> <p>I played around with <a href="http://en.wikipedia.org/wiki/Rakudo_Perl" rel="noreferrer">Rakudo</a> and UTF-8 was just there <strong>as I needed</strong>. I didn't have any problems, it just worked. Maybe there are some limitation somewhere deeper, but at start, all I tested worked as I expected.</p> <p>Shouldn't that be a goal in modern Perl&nbsp;5 too? I stress it more: I'm not suggesting UTF-8 as the default character set for core Perl, I suggest the possibility to trigger it <strong>with a snap</strong> for those who develop <strong>new</strong> projects.</p> <p>Another example, but with a more negative tone. Frameworks should make development easier. Some years ago, I tried web frameworks, but just threw them away because "enabling UTF-8" was so obscure. I did not find how and where to hook Unicode support. It was so time-consuming that I found it easier to go the old way. Now I saw here there was a bounty to deal with the same problem with <a href="http://en.wikipedia.org/wiki/Mason_%28Perl%29" rel="noreferrer">Mason</a> 2: <em><a href="https://stackoverflow.com/questions/5858596/how-to-make-mason2-utf8-clean">How to make Mason2 UTF-8 clean?</a></em>. So, it is pretty new framework, but using it with UTF-8 needs deep knowledge of its internals. It is like a big red sign: STOP, don't use me!</p> <p>I really like Perl. But dealing with Unicode is painful. I still find myself running against walls. Some way <strong>tchrist</strong> is right and answers my questions: new projects don't attract UTF-8 because it is too complicated in Perl&nbsp;5.</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USJens
    1. USw.k
    plurals
    1. PL
      singulars
      1. LTLinked
    2. PL
      singulars
      1. LTLinked
    3. PL
      singulars
      1. LTLinked
    1. PL
      singulars
      1. LTLinked
    2. PL
      singulars
      1. LTLinked
    3. PL
      singulars
      1. LTLinked
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    3. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POWhy does modern Perl avoid UTF-8 by default?
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POWhy does modern Perl avoid UTF-8 by default?
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POWhy does modern Perl avoid UTF-8 by default?
      1. This table or related slice is empty.
      1. VTUpMod
    1. COHi Folks - there's a few flags raised here on these comments. What I've done is taken a snapshot of the comments here and dropped them into this chat room where you can carry on this discussion: http://chat.stackoverflow.com/rooms/846/why-does-modern-perl-avoid-utf-8-by-default
      singulars
      1. POWhy does modern Perl avoid UTF-8 by default?
      1. USKev
    2. COI'm sorry but I agree with @tchrist -- UTF-8 is extremely hard. There's no framework or tool that just "flips a switch" and then handles it correctly. It's something you have to think about directly when designing your application -- not something any kind of framework or language can handle for you. If rakudo just happened to work for you, you were not adventurous enough with your test cases -- as it will take several of the examples in @tchrist's answer and butcher then.
      singulars
      1. POWhy does modern Perl avoid UTF-8 by default?
      1. USBilly ONeal
    3. COWhat exactly are you hoping Moose or Modern::Perl will do? Magically make randomly-encoded character data in files and databases into valid data again?
      singulars
      1. POWhy does modern Perl avoid UTF-8 by default?
      1. USjrockway
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload