[RESOLVED] fsockopen() help

nino1 · May 5, 2009

My host has disabled fopen() and curl() so the only way to access the files is through fsockopen() at this time. I had a code that did what I wanted it to do before but now I'm stuck.

The purpose of the code was to fetch quotes from Yahoo finance in csv format. This is the code I had:

$fp = fopen ("http://finance.yahoo.com/d/quotes.csv?s=^DJI&f=sl1d1t1c1ohgv&e=.csv","r");
$data = fgetcsv ($fp, 1000, ",");
echo $data[1];
fclose($fp);

I tried to do it with fsockopen() without any luck. I don't know how to clean up the code so that only the quote is displayed. Any help would be appreciated.
This is the fsockopen() code that I tried:

$fp = fsockopen("download.finance.yahoo.com", 80, $errno, $errstr, 30);
if (!$fp) {
echo "$errstr ($errno)<br />\n";
} else {
$out = "GET /d/quotes.csv?s=^DJI&f=sl1d1t1c1ohgv&e=.csv HTTP/1.1\r\n";
$out .= "Host: download.finance.yahoo.com\r\n";
$out .= "Content-Type: text/html\r\n";
$out .= "Connection: Close\r\n\r\n";
fwrite($fp, $out);
while (!feof($fp)) {
$data = fgetcsv ($fp, 1000, ",");
echo $data[0];
}
fclose($fp);
}

sneakyimp · May 5, 2009

does the code fetch anything? I'm not clear on what isn't working.

nino1 · May 7, 2009

It does. However, I don't know how to clean up the headers and other useless text so that only the quote is available. Is there any other way that would be similar to fopen() yet better than fsockopen()?

sneakyimp · May 7, 2009

I guess there are two ways to go about this. One is to use some string parsing to clean up what you get from your fsockopen code in which case you would need to post some of it.

The other possibility is to seek out some other PHP function that doesn't depend on curl or fopen to function. Unfortunately, HTTPRequest requires curl. You might be able to use a package in the PEAR HTTP packages that works.

nino1 · May 9, 2009

The problem I have with fsockopen() is that if the server I'm getting data is a bit slow, the quote returns blank (because my web page finishes loading before the page opened with fsockopen() does). With fopen() there was no such problem.

With my code using fsockopen() this is returned:

HTTP/1.1 200 OKDate: Sat09 May 2009 00:36:22 GMTP3P: policyref="http://p3p.yahoo.com/w3c/p3p.xml"CP="CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA POL HEA PRE GOV"Cache-Control: privateConnection: closeTransfer-Encoding: chunkedContent-Type: application/octet-stream3b USDJPY=X98.4855/8/20095:01pmN/AN/AN/AN/AN/A0

Out of that mess, I just need the bolded part (USDJPY=X98.485) and out of that I need to be able to have a space between USDJPY=X and 98.485.

sneakyimp · May 9, 2009

I don't know what you mean by 'the quote returns blank'.

i'm also not sure what you mean when yo say your web page finishes before the fsockopen doeds. as far as I know the socket-related functions you are using should block -- meaning one line must complete it's work before the next line gets executed. if you are having this load in an iframe or something, you might want to use some javascript tricks to have it say 'loading quote'.

Part of the problem with your output is the way that you are fetching data from the socket. If you change your read loop like this, you should get some data that is a bit more structured:

$data = array();
while (!feof($fp)) {
  $data[] = fgetcsv ($fp, 1000, ",");
}
fclose($fp);
var_dump($data);

nino1 · May 11, 2009

By 'quote returns blank' I mean that the code displays just a blank space instead of whatever it is that it's supposed to display, as if the code didn't finish executing.

The code you posted outputs

array(13) { [0]=> array(1) { [0]=> string(15) "HTTP/1.1 200 OK" } [1]=> array(2) { [0]=> string(9) "Date: Mon" [1]=> string(24) "11 May 2009 23:53:31 GMT" } [2]=> array(2) { [0]=> string(49) "P3P: policyref="http://p3p.yahoo.com/w3c/p3p.xml"" [1]=> string(158) "CP="CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA POL HEA PRE GOV"" } [3]=> array(1) { [0]=> string(22) "Cache-Control: private" } [4]=> array(1) { [0]=> string(17) "Connection: close" } [5]=> array(1) { [0]=> string(26) "Transfer-Encoding: chunked" } [6]=> array(1) { [0]=> string(38) "Content-Type: application/octet-stream" } [7]=> array(1) { [0]=> string(0) "" } [8]=> array(1) { [0]=> string(7) "3b " } [9]=> array(9) { [0]=> string(8) "USDJPY=X" [1]=> string(5) "97.61" [2]=> string(9) "5/11/2009" [3]=> string(6) "7:53pm" [4]=> string(3) "N/A" [5]=> string(3) "N/A" [6]=> string(3) "N/A" [7]=> string(3) "N/A" [8]=> string(3) "N/A" } [10]=> array(1) { [0]=> string(0) "" } [11]=> array(1) { [0]=> string(1) "0" } [12]=> array(1) { [0]=> string(0) "" } }

I just need the bolded part. How do I get it?

sneakyimp · May 12, 2009

OK so you need to paste the SOURCE of the output rather than what appears directly in the browser -- the reason being that when the browser parses the source, it ignores line breaks, replacing them with spaces. I can't make heads or tails of the output without line breaks.

As you may have noticed, $data is a nested array...meaning an array containing arrays. If we are lucky, the structure will consistent so we can just grab the value without checking much.

bpat1434 · May 12, 2009

I don't get all that data back... I get something a little less helpful.... (thank you yahoo for sending a header line for the csv file...)....

HTTP/1.1 200 OK
Date: Tue, 12 May 2009 01:26:34 GMT
P3P: policyref="http://p3p.yahoo.com/w3c/p3p.xml", CP="CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA POL HEA PRE GOV"
Cache-Control: private
Connection: close
Transfer-Encoding: chunked
Content-Type: application/octet-stream

4f     

"^DJI",8418.77,"5/11/2009","4:02pm",-155.88,8569.23,8569.23,8410.33,332633152

0

Now, the trick here is to remember that all headers will be separated by a single line break (\r\n). The body is separated by a double line-break (\r\n\r\n). So using that, we can just read the entire string back from Yahoo. Then just grab the content you need ( a simple regex will suffice ) and then work your way down from there parsing the CSV.

Unfortunately at this time [man]str_getcsv/man isn't around, so we're forced to use regular expressions.

Below is a copy of what I came up with to quickly parse it out. Just remember, you'll have to test on your own since our outputs differ slightly.

<?php

$fp = fsockopen('download.finance.yahoo.com', 80, $errno, $errstr, 30);
if(!$fp)
{
        echo 'Encountered Error (#'.$errno.'): '.$errstr;
        exit;
}

$out = 'GET /d/quotes.csv?s=^DJI&f=sl1d1t1c1ohgv&e=.csv HTTP/1.1'."\r\n";
$out .= 'Host: download.finance.yahoo.com'."\r\n";
$out .= 'Content-Type: text/html'."\r\n";
$out .= 'Connection: Close'."\r\n\r\n";

fwrite($fp, $out);

$data = '';
while(!feof($fp))
{
        $data .= fgets($fp, 8192);
}

echo '<pre>'.htmlspecialchars($data).'</pre>';

preg_match("~\r\n\r\n(.*)$~iUs", $data, $matches);

$lines = explode("\n", $matches[1]);

echo '<pre>'.print_r($lines, 1).'</pre>';

preg_match_all('~(?:\'|")?(.*)(?:\\1)?,+~iUs', $lines[1], $matches);                                                                                                                                                                     

echo '<pre>'.print_r($matches, 1).'</pre>';

echo '<ul>';
foreach($matches[1] as $match)
{
        echo '<li>'.$match.'</li>';
}
echo '</ul>';

That gives the following output:

HTTP/1.1 200 OK
Date: Tue, 12 May 2009 01:26:34 GMT
P3P: policyref="http://p3p.yahoo.com/w3c/p3p.xml", CP="CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA POL HEA PRE GOV"
Cache-Control: private
Connection: close
Transfer-Encoding: chunked
Content-Type: application/octet-stream

4f     

"^DJI",8418.77,"5/11/2009","4:02pm",-155.88,8569.23,8569.23,8410.33,332633152




Array
(
    [0] => 4f     

    [1] => "^DJI",8418.77,"5/11/2009","4:02pm",-155.88,8569.23,8569.23,8410.33,332633152
    [2] => 
    [3] => 0
    [4] => 
)

Array
(
    [0] => Array
        (
            [0] => "^DJI",
            [1] => 8418.77,
            [2] => "5/11/2009",
            [3] => "4:02pm",
            [4] => -155.88,
            [5] => 8569.23,
            [6] => 8569.23,
            [7] => 8410.33,
        )

[1] => Array
    (
        [0] => "^DJI"
        [1] => 8418.77
        [2] => "5/11/2009"
        [3] => "4:02pm"
        [4] => -155.88
        [5] => 8569.23
        [6] => 8569.23
        [7] => 8410.33
    )

)

* "^DJI"
* 8418.77
* "5/11/2009"
* "4:02pm"
* -155.88
* 8569.23
* 8569.23
* 8410.33

nino1 · May 12, 2009

Thanks for the help so far, guys.
I'm still a bit unclear about something. How do I get certain data out of an array? Out of this array, for example (as posted in the example above)?

[1] => Array
(
[0] => "^DJI"
[1] => 8418.77
[2] => "5/11/2009"
[3] => "4:02pm"
[4] => -155.88
[5] => 8569.23
[6] => 8569.23
[7] => 8410.33
)

)

Out of that array I only need the [1] part.

Edit: Nevermind, I can just use it as a regular array and do

echo $matches[1][1];

to get the results. I'll go test this out before I mark this resolved. I might need more help, who knows. Thanks a lot!

[RESOLVED] fsockopen() help

Nnino1

Ssneakyimp

Nnino1

Ssneakyimp

Nnino1

Ssneakyimp

Nnino1

Ssneakyimp

bpat1434

Nnino1