fopen stripping characters out?

jwwtaker

I'm having a hair-pulling issue that i've never seen before. When i nano this file i can see the characters in there in ascii format. they are the ord(1) character as a delimiter between the fields. I can see them just fine regular viewing, but as soon as I do either a file_get_contents or an fopen and view the content, those characters are stripped out! I've tried everything, any ideas on why these characters would disappear??

Help is greatly appreciated, thanks!

Content of the file i'm trying to parse.

#export_date^{Agenre_id^{Aparent_id^{Aname^B}}}
#primaryKey:genre_id^B
1267581199941^{A1^{A^{AUnknown^B}}}
1267581199941^{A2^{A34^{ABlues^B}}}

After doing an fopen with fgets, the top line looks like this
#export_dategenre_idcollection_idis_primary

bpat1434

This character in nano "^A" (if it's anything like ViM) is really the key sequence:

ctl + v followed by ctl + shift + A

So that gives the "visible" output of "^A" when in actuality it's some other character (unicode or something).

I would be willing to bet if you replaced those "characters" with something from the ascii table (maybe a vertical pipe, or a specific sequence of characters like |"|) then it should work. You could then explode on the sequence or single character as you normally would.

jwwtaker

this is documented by the source as having the delimiter being ord(1). when I do a strpos on the string that fgets gets, it's not there. I can't replace since it cant find it anymore after opening the file with PHP.

bpat1434

No, I meant replace it using ViM or nano.

Weedpacket

What are you looking for? "^A" or chr(1)?

halojoy

jwwtaker;10945434 wrote:
this is documented by the source as having the delimiter being ord(1).

when I do a strpos on the string that fgets gets, it's not there.

I can't replace since it cant find it anymore after opening the file with PHP.

documented by the source as having ...

Why dont you give a link to the source of your script.
Instead of we have to make guessings and try to reconstruct it.
Thanks 🙂

To me it looks as a flatfile database concept.
- Where 'unit separator' (US) is '^A'
- and 'record separator' (RS) is '^B'
- The first 2 lines, starting with one '#'
tell:
- line1: table fields (columns) names: export_date, genre_id, parent_id, name
- line2: constraints, in this case set: primaryKey:genre_id
export_date is one UNIX TIMESTAMP [man]time[/man]

Following lines, without '#' are, records, rows in table
all separated with the record separator '^B' to mark end-of-row:

1267581199941  1  (null)  Unknown^B
1267581199941  2  34  Blues^B

If the above is so, as I get it,
then it is not very difficult to [man]explode[/man] while testing for leading #
and using '^A' and '^B' for this explode
and so fetch all the data into one array.

jwwtaker

i'm sorry the separator characters are ord(1). The scripts are behind a login on apples server. they are daily updated files that i need to import to keep an up to date itunes database. I cant use nano to replace because its automated daily and these files need to be imported by a script into my database. My issue is as you said halojoy that the flat database import would be great, however as soon as I do the fopen, the ^A characters disappear and give the line I put in the end of my original comment.

#export_dategenre_idcollection_idis_primary

So I have no way of distinguishing between the fields...

laserlight

Try supplying FILE_BINARY as the second argument to [man]file/man or [man]file_get_contents/man.

EDIT:
Oh wait, that is the default setting anyway.

EDIT #2:
I decided to test on PHP 5.3.1, and sorry, but I cannot duplicate your problem. I tested with this script:

<?php
$content = file_get_contents('test.txt');
echo strpos($content, chr(1));

test.txt contains:

this is a field^Athis is another field^A

where ^A denotes the character with ordinal value of 1.

The output of the script was 15, which is exactly the position of the first delimiter.

Weedpacket

Just to ease confusion about whether it's caret-capital A, or ctrl-v ctrl-A, or ord(1), or chr(1) or the character formerly known as "^A", let's refer to it by its proper name, SOH. Just a thought.

ord(1) is the number 49.

I don't use nano, but in the text editor I do use SOH would show up as a single character, usually rendered as a "no glyph defined" filler.

laserlight

Weedpacket wrote:
Just to ease confusion about whether it's caret-capital A, or ctrl-v ctrl-A, or ord(1), or chr(1) or the character formerly known as "^A", let's refer to it by its proper name, SOH. Just a thought.

So, what is SOH? 🙂 But yeah, I agree that SOH is much more concise than "character with ordinal value/number of 1" or "start of heading character in ASCII".

Weedpacket wrote:
ord(1) is the number 49.

I have a hunch that jwwtaker translated the ord(1) notation that he/she used to denote SOH into an actual call of ord(1) instead of chr(1).

jwwtaker

i've tried both b and t when doing fopen. the funny thing is if i read the contents of the file into file_get_contents, it also loses the characters. and yes it is SOH according to the documentation. The file just simply does not have those characters anymore after doing either fopen or file_get_contents.

laserlight

Post the smallest and simplest PHP script that demonstrates the problem. For reference, look at the script that I posted in post #8.

By the way, did you test with my script using my test data? What was your output?

bradgrafelman

laserlight;10945489 wrote:
So, what is SOH? 🙂

Start Of Heading

http://asciitable.com/

laserlight

bradgrafelman wrote:
Start Of Heading

http://asciitable.com/

Right. That misses my point that it is really about agreeing on a concise notation:

laserlight wrote:
But yeah, I agree that SOH is much more concise than "character with ordinal value/number of 1" or "start of heading character in ASCII".