Need Help - PHP simple program

naed11

Please help:

I need some advice on what would be the best way to write this program.

I need it to find a specific url. Locate and open 25 specific links on the page. Go to the 25 pages and record 5 pieces of information from each page. Then collect data and convert into a XML document.

program should hit the URL, find the URL of each links on the page, follow each URL to pull the specific data on that page, and then convert the data into XML format.

Any ideas or thoughts?

thanks

dalecosp

Yes; go ahead and get your degree and then apply with Google ;-)

naed11

funny kinda. not really. Any ideas anyone?

traq

My first thought (judging by how you asked the question) was that you probably aren't familiar enough with PHP, or programming in general, to write your "program" without being shown, step-by-step, how to do it.

Do you have anything "so far"? Any code? Any plans for how you might approach the task? If you have nothing, then you're basically asking someone to "do it for you."

My second thought was that this might be a school assignment. If that's the case, I would be even less inclined to "do it for you." On the other hand, if you care to demonstrate that you can make a serious attempt, I might enjoy helping you through it if specific problems come up.

You might start by studying the cURL and DOM (and/or SimpleXML) extensions.

naed11

My approach plans are creating a crawlBot to find and index the main page and links. Using CURL to put each of the pages into a variable and manipulate them from there. parsing it either as plain text using php string functions or as something else.

naed11

so when i have the required data, i am familiar with a PHP snippet to convert the objects to XML.

Weedpacket

Do you already know how to program?

If you do, it should be pretty straightforward (check the PHP manual for what facilities it provides for reading files, parsing and searching web pages, and generating XML).

naed11

Yes. I use php in small functions with html daily. Its just piecing these all together collaboratively. could i write it as a dreamweaver .php

Weedpacket

naed11 wrote:
Yes. I use php in small functions with html daily.

That's not "knowing how to program". Could you explain exactly what you want to do to a total moron (someone who'd have to ask "what does 'piece' mean?")?

johanafm

It's very easy to accomplish what you want.

Retrieve the first document
Create a DOM Document
Grab the URLs you need, preferably using XPath.
Retrieve the pages using those URLs
Create DOM Documents.
Retrieve the data you need
Create an XML document, preferably using DOM.
Save it

... provided the crawled documents doesn't change in such a way as to screw with your code. Crawling is usually the worst way of doing something. If you have legal rights to get this data there there will be some kind of API present to retrieve this data in a fashion that is likely to never* change in 100 cases out of slightly more** than 100.

"never" means pretty much never"
** "slightly more" means so slightly that your calculator wouldn't show it using all its decimal points.