International Letters

[Shawazi]

I am trying to write some regex where I can accept all letters, including international ones with dots and lines over them. I can't seem to get it right because \w does A-Za-z0-9, but still does not include other characters. Any ideas?

[NogDog]

[\w\xC0-\xD6\xD8-\xF6\xF8-\xFF]

[Weedpacket]

For me:

echo preg_match('/\w/', 'â')

returns 1; a match.

You might want to have a look at using [man]setlocale[/man](LC_CTYPE, $something) to change the current locale to something that regards accented characters as "letters". Right now [man]setlocale[/man](LC_CTYPE, 0) would return something like 'C' or 'POSIX' and in those \w works as you describe (plus '_').

[Shawazi]

$new_url = 'Āċēŝê';
$new_url = ereg_replace("[\w\xC0-\xD6\xD8-\xF6\xF8-\xFF]", "_", $new_url);

For some reason it prints out an underscore followed by one of those question boxes in a square for unknown characters. So basically there are five characters I'm testing with and it shows:

????_?

[NogDog]

What character set are you using for the pages involved? I would recommend using UTF-8. This should either be set in the page headers or in each page's head section via a meta tag:

<meta http-equiv='Content-Type' content='text/html; [b]charset=UTF-8[/b]'>

While any form elements on the page should conform to that specified character set by default, you can specify it explicitly in the form tag via the "accept-charset" attribute:

<form action="foobar.php" method="post" [b]accept-charset="UTF-8"[/b]>

If you need to support multi-byte character sets, then you will probably need to delve into the use of the [man]mbstring[/man] functions and settings.

[Shawazi]

My site currently displays the characters correctly. If I look in the text of the page I see "Āċēŝê" perfectly. However I use some SEO links so something named "Dave's Project" would link to www.mysite.com/daves_project/5.

I am looking for a simple solution so that "Davê's Project" could turn into Dav_s_Project, taking out all international letters from the URL to make them underscores or erased all together. After running ereg_replace, it replaces the letters with what I ask plus the question marks. It looks like the mbstring thing is something I should look into but it does not make much sense to me overall.

[NogDog]

Maybe instead of replacing characters you could use [man]urlencode/man?

$name = "Davê's Project";
$link = "http://www.mysite.com/" . urlencode($name) . "/5";

[Shawazi]

I may resort to using urlencode. I am trying to avoid using hex in the URL though. Would mbstring be able to help run ereg_replace without messing up the conversion?

[NogDog]

Unfortunately (or fortunately?) I've not had occasion to use mbstring yet, so you now probably know at least as much about it as I do. 😉

[Shawazi]

Okay percent signs in URL or seeing what mbstring does flips a coin

[Shawazi]

Well doing

$new_url = urlencode($new_url);

doesn't change the code at all! I don't get it, but it keeps the international characters the same and of course the browser then changes them to % codes in the address bar. Meanwhile using an htaccess rule like

[\w\xC0-\xD6\xD8-\xF6\xF8-\xFF\%_\(\)-]

is still not working.

[NogDog]

From this page, you might want to try what he suggests about using the "RewriteMap" in your .htaccess.

[Shawazi]

It's interesting but the part with the letters I do not need to escape since I am not using it for $1. I am just using it for SEO, where a number that comes after is what's used for $1