monaro
06-14-2004, 05:50 AM
If you havnt set up your robot.txt file now could be as good a time as any to do so.
# robots.txt for http // www example . com
User-agent: *
Disallow: /cyberworld/map/ # This is an infinite virtual URL space
Disallow: /tmp/ # these will soon disappear
Disallow: /foo.html
I shall post some more useful information soon. I know we all know these, I am just helping out the new faces here at Xnations.
Have a great day!
# robots.txt for http // www example . com
User-agent: *
Disallow: /cyberworld/map/ # This is an infinite virtual URL space
Disallow: /tmp/ # these will soon disappear
Disallow: /foo.html
I shall post some more useful information soon. I know we all know these, I am just helping out the new faces here at Xnations.
Have a great day!