Can spider script log in to password protected site?

recurzion

Hello,

I am a programmer by background, and have even made several minor PHP scripts, but I am by no means a PHP expert.

My company has need of a PHP script that can access a site that requires a user to log in, parse some of the data from the password protected web page, and store it in a local DB.

The programmer working on this project claims that it is not possible to do this in PHP, but I find this very hard to believe. I've been doing a lot of searching on this forum, as well as using google, but I can't seem to find what I need.

Can someone point me in the right direction?

Is it possible to have a PHP script log in to a remote site? (For example, a spider / script that logs a particular user in to a phpBB forum and retreives his Private Messages -- this is not what we need to do, just an example of the functionality)
If so, how? (Just pointing me to the proper functions to use, and I can take it from there)

Thanks in advance!

Sincerely,

Joe

Weedpacket

A bit vague, but it certainly sounds feasible. Basically, what you're wanting is to write a web client that will automatically do the login stuff, request the resources you want, and then go digging through the results.

All the web client stuff (basically, the login, and requesting the pages) can be done with the cURL extension - for simplicity - or through sockets (starting with fsockopen) if you want do do http protocol yourself.

drawmack

how does the log in on the site work?

if it is with a form using the get method then it is very simple as you can do it with just appending the appropriate name value pairs to the url.

if it is a form using get then it becomes a bit trickier. But there is still a pretty simple solution.

Step 1: have php generate a page that contains the form you want and the values you want

Step 2: have javascript submit the form to the place that wants the info.

Step 3: you're in.

of course both of these rely on you being in a web browser environment.

If you're not in a web browser environment then you may wish to look into fsockopen.