Http Request

Benedetto

i write some line of code to implement a program that retrieve html page. Its works

well for many web pages, but one says "object moved". i try with IE and its work, so why my script didn't work?
i use snoopy class to perform screen scraping.
Anyone can help me?

P.S. Sorry for my english

dewen

try to send a http request header and see what's the response

for example:
[$] telnet www.sitetocheck.com 80

HEAD / HTTP/1.1
Host: mysite.com
Connection: close
Accept-Encoding: gzip
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/vnd.ms-excel, application/msword, application/x-shockwave-flash, /
Accept-Language: en-us
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322)

Benedetto

i try with "telnet www.meridiana.it 80" but
nothing happens... now i try to access with a web site www.web-sniffer.net that send HTTP request.
its don't works, so what are the problem?
if you can try to fix the problem, the site that i would to grab page is www.meridiana.it

Thanks

dewen

I guess their server is hosting more than one domains, so in order to make it find the right domain: www.meridiana.it

you need to put www.meridiana.it in host as:

telnet www.meridiana.it 80

GET / HTTP/1.1
Host: www.meridiana.it
Connection: close
Accept-Encoding: gzip
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/vnd.ms-excel, application/msword, application/x-shockwave-flash, /
Accept-Language: en-us
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)

I tried and it worked this way

Benedetto

I have found the reason why my screen scraping don't work... www.meridiana.it redirect to another server so when I used fsockopen its return 302 object moved...
Now I try with cURL function with option FollowRedirect=1 and its work!!!
Thanks you for your ideas!!