Special characters problem in fgetcsv

computerzworld

Hello,
I am trying to import a csv having special characters available in it. For example I am having name like xÿz then its converting ÿ to junk character �. And due to that value is not enterd in database and its skipping the characters after this junk character. My db collation is utf8_general_ci So in database only x is getting entered. So is there anyway to preserve special characters and store them as it is? Please help me. Thanks in adavance.

johanafm

And what's the character encoding of the file?

http://se2.php.net/manual/en/function.mb-detect-encoding.php
http://se2.php.net/manual/en/function.iconv.php

NogDog

Of possible help: http://www.charles-reace.com/blog/2008/10/03/utf8-in-php-and-mysql/

computerzworld

I have changed my characterset to utf-8 by using

header('Content-Type: text/html; charset=utf-8');

but in the page its displaying junk character instead of original character. Also my database collation is utf8_general_ci as I have said before.

johanafm

I repeat my question: And what's the character encoding of the file (or stream or whatever you get your csv from)?

computerzworld

Hi thanks for reply....
Actually I am having this value getting stored in 2D array. So I tried to check its character encoding like mb_detect_encoding($arr[1][1]) but it returned me null value instead of its encoding.

computerzworld

actually its BOM(Byte order mark) generated in my csv. So is there anyway to remove it from my string?

computerzworld

also as soon as i have changed character encoding of file using save as from Western (ISO-8859-15) to utf-8 it was imported successfully so is there anyway so that I can change character encoding of file before importing it?

johanafm

mb_detect_encoding is not supposed to return null. It should either return the encoding, or false. So if you really did get null, something else is wrong.
If you don't specify the detect order yourself, you'd have to know what the value of mb_detect_order is.

Iirc, BOM is allowed but pointless for UTF-8, and at the very least not recommended due to problems that may arise from its presence. Notepad and some other windows editors uses it by default.
If BOM is the only issue, you could check for its presence and strip it before storing the rest in the DB.

As for encoding

$str = file_get_contents("8859-15.txt");
$str = iconv('ISO-8859-15', 'UTF-8', $str);
//then save to db.

I'd leave the file in its original format.

computerzworld

Thanks once again for reply but what I am doing is opening a file using fopen & then reading its contents using fgetcsv so is there anyway to convert its encoding in between?

johanafm

One option is to use the above and then str_getcsv()

computerzworld

But is there anyway to change the character encoding of file using php? I have tried using iconv & mb_convert_encoding but it didn't worked.

johanafm

file_get_contents
iconv
file_put_contents
fgetcsv