robots.txt va htacess

[kbc1]

Hi All

Quick question

If I use htacess ot redirect all HTP requests to my index.php file, is there any need for a robots.txt file?

Surely a htaccess would force the robot to follow the htaccess rules?

Any thoughts are greatly appreciated.

kbc1

[sneakyimp]

If I recall correctly, robots.txt is used to either exclude bots from accessing certain files or to tell the bot where to find a sitemap. .htaccess is something entirely different. It tells apache how to serve up pages. for the most part, bots and users should get the same behavior from an htaccess file. users don't give a hoot about the robots.txt. robots care a lot about robots.txt because it tells them which files to ignore and where it might find a sitemap.

a sitemap can be useful for getting pages into a search engine that are not somehow linked from your home page.

[dagon]

a robot will follow any instructions in htacces, its using http after all. it really depends what your using robot.txt for, you still may want to exclude some urls and or bots even if all traffic is going through one page.