web crawler PHP

[ktsirig]

Hello,
I am supposed to construct a page that searches in specific websites to extract information, like those sites from where you can rent a car for example. There is a form in the site where the user selects some fields (for instance departure and drop-off date), then the data are submitted to the other page that searches 2-3 sites and finds which cars are available on those dates.
I wanted to ask if there are ready scripts to do that, if not, some hints on how to start.
I am familiar with PHP forms and data extraction from mysql databases, but when you extract data from other sites, I have no clue how I can begin and deal with it...

[bogu]

curl - to reach the desire page (ex. There are pages that can be reach only after authentication, or u need to submit some search forms for your desired content, etc.) ...
String Functions - to parse the html content and grab the content u want ...

[MarkR]

Screen-scraping is a generally flawed system as your code is at the mercy of the design of a foreign site.

Unless they create the pages especially for you with a legal agreement/contract in place, they have no motivation to keep the structure the same.

You can of course use fopen() to fetch pages, use DOMDocument::parseHTML to parse HTML, then traverse the DOM to get what you want, but it is highly unlikely that it will continue working indefinitely.

Your best bet is to get an agreement with the companies for a machine-readable interface that won't change without warning.

Mark