[RESOLVED] Is there a way to bypass ActiveX while screen scraping?

[Webspeeder]

Hey all.

I'm trying to write PHP scripts to automate a manual task of logging into a website and submitting forms.

I'm able to get to the screens I need but when I "submit" the data to the page, I'm not getting any data in return and the data is not being posted to the sites DB. The site runs ASP and i believe the roadblock is the use of ActiveX by the site.

Here is the scenario.

If I'm in a non-IE browser, the site doesn't work, it displays a message about needing to install ActiveX.

While running from a PHP script, I'm able to fetch the screens that simply display HTML elements.

I have the program name (this program is used to save data to a D😎 and form variables that are submitted to that program. The method used to call this program is GET. When I manually construct the URL and paste it into the URL of an IE browser, the data is sent/saved to the system DB.

If I use that exact same query string via a script, I get nothing in response and the data is not sent/saved to the system.

For the sake of testing, I ran some scripts against imdb.com and am able to call/parse/recall pages fine.

My questions is this. Is there any type of option that allows the bypass of ActiveX? Or a way to send it what it needs to execute properly?

I've looked at http://us2.php.net/curl_setopt but didn't see anything relating to ActiveX. I also did a search on this site for ActiveX but with all the threads (and vague names) it's difficult to find anything pertaining to my situation.

Any help would be greatly appreciated.

...

[sneakyimp]

I'm not sure how to solve your problem, but I think it's worth pointing out that you are assuming that your script's failure to load the page is because the page requires activeX. This may or may not be the case. The page could depend on session data. Does your script handle cookies? Try looking into the cookie-related info on [man]curl_setopt[/man].

[Webspeeder]

sneakyimp wrote:

I'm not sure how to solve your problem, but I think it's worth pointing out that you are assuming that your script's failure to load the page is because the page requires activeX. This may or may not be the case. The page could depend on session data. Does your script handle cookies? Try looking into the cookie-related info on [man]curl_setopt[/man].

Yeah, I figured it out. I was handling cookies, the issue was that I had snippets of what I needed to do in separate scripts. When I combined them together, I was able to post data.

But thank you for your response.

How do you mark a thread as resolved?

[sneakyimp]

AT the top of this topic, there's a 'thread tools' menu. It's on there.