[Resolved] Spiders

erikwebb

Are PHP files subject to people downloading your files using spiders just as much as HTML files?

TheIceman5

downloading? didnt think you could download a html page? webbrowser parses it??

superwormy

PHP simply generates HTML ( or other stuff too if you want to be technical about it ) so yes, anyone could download the resulting HTML.

They could not, however, download your actual PHP code.

erikwebb

Have you guys ever used a spider? It goes through an entire server and downloads every single file. This is what I'm wondering about, I know the principles of how PHP works.

Weedpacket

Er, no it doesn't. A spider is just an http client like any other. It follows links. There's nothing to stop a spider from guessing URLs and requesting them; a spider might pick an IP address and hit port 80 with a request for index.html, for example, but that's nothing any client couldn't do. It's no more able to "go through an entire server and download every single file" than Netscape 4.

scoppc

Originally posted by erikwebb
Have you guys ever used a spider? It goes through an entire server and downloads every single file. This is what I'm wondering about, I know the principles of how PHP works.

It will be a disaster if it's happen! Web Spidey === Web Client, just that they are automatic and fast. Lots of purpose to this, anyway, just read the details here!

erikwebb

Basically, I think I'm just voicing my opinion more than asking a question. The type of spiders I'm speaking of can download entire sites (I have done it myself, but by no means for monetary gain). This threatens all the hard work any of us have put into a project. I support open-source just as much as the next guy, but I feel if someone asks to view my code or I willingly display it, then it's one thing, but if they clearly take it with poor intent, that's just crap.

superwormy

erikwebb you have no idea what you're talking about.

It is absolutely impossible ( unless you have a SERIOUSLY misconfigured server ) to download the source files to a website without some sort of access to those files FROM THE FILESYSTEM, and NOT THROUGH HTTP ( ie, NOT THROUGH THE WEBSERVER )

Yes, it IS very possible to download the resulting HTML generated, but you WILL NOT get PHP / ASP / Java / Perl code with it.

I urge you to either think about what you're saying before you type it, or do some frickin' research and figure out what the heck you're talking about.

I don't mean to be harsh... but seriously, you're just way off base, and a whole bunch of us has told you that now.

And if you've done it yourself, I want to see a link / I want an expample of how you did this.

erikwebb

I have done it myself, I have the www.webstyleguide.com web site on my computer, and no, I didn't download the files one-by-one. This is probably not the best example, I've never used this myself, but I just searched for "spider" at Download.com and found this: http://www.softbytelabs.com/BlackWidow/index.html . They use the term "site ripping."

superwormy

We UNDERSTAND that 'site-ripping' is possible, HOWEVER, 'site-ripping' like this DOES NOT GRAB PHP / ASP / JAVA / PERL / CGI / ANY OTHER WEB LANGUAGE SOURCE FILES!

IT ONLY GRABS THE RESULTS OF ANY HTML THAT THOSE WEB LANGUAGES HAVE CREATED!

By the time a user-agent or Spider recieves the page, it IS NO DIFFERENT FROM A REGULAR HTML FILE, so if you develop an application, THERE IS NO POSSIBLE WAY, SHORT OF A SERIOUS MIS-CONFIGURATION OF YOUR WEBSERVER OR SERIOUS SECURITY BREACH, THAT ANYBODY WILL HAVE ACCESS TO YOUR ACTUAL CODE. THIS APPLIES TO PHP / ASP / JAVA / PERL / CGI / PYTHON AND ALL OTHER SERVER-SIDE SCRIPTING LANGUAGES.

Yes though, they WILL be able to download any HTML source files your application generates. And no, theres no way to stop that, short of disconnecting your webserver and not allowing regular user-agents like Internet Explorer to see it as well. And in that case... why would you be developing web apps?

erikwebb

You seriously need to relax, you're getting way too worked up about this. But anyways, sorry, I must have misread your last post because I missed the part about it downloading the "generated" HTML. But no one else explained it as you did. I just misread your post, I apologize. I certainly do feel like a idiot now. 😃

superwormy

LOL no problem, just typing in CAPS there to make sure I got my point across :-)

No offense intended, I've misread posts before as well :-P