[RESOLVED] Stuck on an urldecode issue...

vaaaska

I can't seem to solve this one...I've worked this forwards and backwards and have even tried used every user submitted function in the urlencode() and urldecode() pages in the Manual.

So, I have this urlencoded (somehow) string which comes from Yahoo Japan:

1) %BF%CB%C0%B8%BE%B5%B0%EC

Which equals this in the real world:

2) ????

I need to decode 1 and return 2 in any kind of format that a utf-8 page will display.

Perhaps Yahoo has some kind of proprietary urlencoding schema (but I can't find any record of one)? Anybody have any ideas how I might sort this one out?

Much thanks...

etully

Just a wild shot in the dark here: I am guessing that those codes (BF, CB, etc) are simply high acsii characters that your display device (web browser, text editor, etc) doesn't know how to display.

The only solution I can think of is to build your own conversion table and use str_replace (instead of urldecode) to change the characters. For example, you might say, "CB" is "Copyright Symbol" and "BF" is "Star". That way, you might not actually see the copyright symbol or the star but you will at least know what was there.

I realize that this is a weak solution. It's the best I've got.

If you wanted to go further with that weak solution, you could replace all instances of "%CB" with "<img src='star.gif'>" or something like that. This way, you would at least be able to see something.

Hopefully someone has a better solution than mine.

(I don't think Yahoo is using any proprietary coding scheme, by the way. Is this in the referer variable or are they delivering you the string in some text document?)

vaaaska

From the referring url (the whole thing). I do pull it out and use the actual variable (p). I'm trying to tackle this part...and then work on the variable before I save it to the table so it comes up right.

I'll look into the ascill conversion idea...thanks. 😉

etully

Since it's coming from the referer url, then I can guarantee that it's not a Yahoo proprietary code because it doesn't even come from Yahoo... it comes from the user's browser.

sneakyimp

Those are probably double-byte characters that don't exist in western languages. The ability to display them might require that a user have double-byte language support (like for chinese, japanese, etc.) installed on their operating system. I believe that when you install windows in English that these are turned off by default.

The basic idea is that chinese and japanese have way more characters than a single byte can support and therefore need 2 bytes to display them. If your target audience is asian, you should be ok. otherwise, you may need to rely on a plug-in or render images or something to display this stuff.

I'm not certain about this. If you don't have the double byte fonts/language support installed, you might try getting it and see if those chars display differently in your browser.

EDIT: If they are in fact urlencoded versions of double-byte chars, I'm wondering how your server running PHP might deal with them. I think PEAR supports some routines to obtain and check for japanese language support.

vaaaska

Yeah, I'm pretty sure they are double-byte chars. It's a Japanese server (in Japan even) but this doesn't really help much. 😉

Google serves it up fine everytime. Yahoo encodes these up about 90% of the time...

Ah yes...they are Chinese characters actually. When it comes time to decode them it doesn't have anywhere to go. A map is necessary, but I have yet to find a mapping of all the Chinese chars that could make this possible. I think I'm going to have to replace the chars with some kind of spaceholder or whatever until I find one.

sneakyimp

or maybe you can get chinese support on your server machine? i'm thinking that urlencode or urldecode without chinese support isn't going to do a THING. I know very little about php language support but I'm guessing that php or possibly your server OS too might need a module to support chinese.

maybe this is helpful?
http://us2.php.net/manual/he/ref.mbstring.php

if i view that page, i get a lot of ??? chars so i'm guessing my windows machine here has no chinese support.

BUT, i know that PHP can render images with text. if your server and php both support multibyte characters and you install some multibyte fonts, you might be able to generate image of the multibyte chars for folks without that kind of language support.

which brings me to the notion that if your users don't have that support installed, what use is this yahoo info to them?

vaaaska

Well, it's not a solution, but it is possible that the cliend CAN see these doublt-byte characters. Honestly, I had not even thought about that - and I should ask.

The thing in question is just their statistics package and I don't even think they have actually used it yet. 😉

sneakyimp

this might be helpful...

install east asian languages on windows xp:
http://newton.uor.edu/Departments&Programs/AsianStudiesDept/Language/asianlanguageinstallation_XP.html

i'm sure macs and linux also have ways to support east asian languages.

vaaaska

Macs already do. But dealing with the encoding of double-byte chars might be a bit much.

Actually, I've already spent too much time on an issue that is really not an issue. If it works it's fine and if it doesn't I'll just to a preg_replace and strip it out of there.