How to simulate link click

gshank

I'm doing a PHP fopen on a URL link in order to read the contents of the page and "scrape" off the information I need. However, as part of the page, they have a link for the next 20 results as follows:
<a href="javascript:__doPostBack('MegaGrid$ctl24$ctl1','')">Next 20</a>
Is there a way in my PHP code to simulate clicking that link AND getting the results back. I can't just reopen the URL because the URL doesn't change (that I can see) after clicking the link. I've tried several things that didn't work and just can't find anything that does what I need.

Weedpacket

You'd have to look either (a) examine the code for __doPostBack() to see what it does, or look at the request that it sends to the header when you run it, then reproduce that.

gshank

The page has <form name="Form1" method="post" action="list.aspx" and the __doPostBack sets some form vars and calls form.submit() but all this occurs on the server as a result of me clicking the link right? It seems to me, if I could replicate whatever the protocol is that tells the server I've clicked the link, it might work. I'm wondering if I could make use of the http classes/methods (http://us.php.net/manual/en/ref.http.php) to do what I want.

dougal85

I thought __doPostBack looked like .NET, this often posts back to the same page. It's how .NET tries to give a state to web pages.

When you click the link you execute the javascript inside the browser. Therefore, you can't do it that easily. JavaScript all runs on the clients machine rather than the server...

There is no particular protocol as such. When you click a lick the browser looks to see whats attached with it, in this case its a function call to the javascript - so the browser runs this script. If its a link to another page then the browser simply moves to the URL.

I don't really think this is going to be a trivial task, as I assume you want it to be automatic and robust enough to work on the majority of pages.

gshank

I agree that it's not trivial. I don't really care about it being robust. It just needs to be a one-time thing where it pulls all information off the site programatically and into a database instead of doing it manually because there are a ton of listings but they only display 20 at a time.