are we aloud to talk about parsing on here??

0o0o0

just askin.. I found a site that you can parse basically all the pages (if you wanted).. except one!

I dont really want it to parse my question is HOW DID HE DO THIS?? so it cant be parsed.

i want to use this technique ( whatever it is ) it for my own site..

goto it in IE....

http://www.homefinder.ca/agents/55410-latif-merali

see what comes up.... then hit refresh.. it says "You are not authorized to access this page."

how is this done??

if I go in firefox.. and do it.. refresh makes it come up everytime.. but in IE its has that error on refresh.

Lemme know thanks!

laserlight

0o0o0 wrote:
just askin.. I found a site that you can parse basically all the pages (if you wanted).. except one!

I dont really want it to parse my question is HOW DID HE DO THIS?? so it cant be parsed.

What do you mean by "cant be parsed"? From what I see you can load the page using your browser, hence the clientside output can be parsed.

0o0o0 wrote:
i want to use this technique ( whatever it is ) it for my own site..

see what comes up.... then hit refresh.. it says "You are not authorized to access this page."

Looks like a bug to me: why would anyone want to prevent a refresh of that page?

0o0o0

ok thanks.

laserlight

0o0o0 wrote:
so HOW or WHAT makes this site un parsable because it would be great to know for future.

It is not "unparsable". They are probably doing things like checking user agent headers to guess if it is a legitimate user or a script that is accessing the page. These things can be defeated, otherwise legitimate users will not be able to view the page either.

0o0o0

hmm.. before it auto redirected.. to index..

so.. tried...

include 'simple_html_dom.php';
$url="http://www.homefinder.ca/agents/55410-latif-merali";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 2);
curl_setopt($ch, CURLOPT_TIMEOUT, 60);
curl_setopt ($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.53 Safari/525.19");
$exec = curl_exec ($ch);
$html = str_get_html($exec);

echo "$html";

gets me..

You are being redirected. a little progress closer.. lol

I seriously want to know how this was done. added code?? in the link?? hidden??

laserlight

0o0o0 wrote:
I seriously want to know how this was done. added code?? in the link?? hidden??

It could be done on the server side and/or the client side.

0o0o0

reads cookie files?

laserlight

Tell you what, let's approach this from a different angle: what are you trying to protect against, and why? Maybe we can suggest what approach to take.

If you want to probe further into what homefinder.ca is doing, go ahead. I have no inclination to investigate as the data is not interesting to me, and if it were, I would be negotiating an API with them instead of trying to figure out how to bypass whatever mechanisms they have in place to prevent me from getting at their data from a script.

0o0o0

ah its just something to do.. I can call mls for data.. but it was interesting that they have this limitiation only on the agent pages.. and not the actual listing pages themselves. Wanted to know how it was done, and use it myself on my own projects. thanks for the replies/messages!