Hi All
It seems that my sites generally have the following directory structure:
everything including brochure and store section pages fall under the /actalog/ directory. Only index.html is outside the actalog/ directoyr..is that how it should be?
Also the site map only includes the store sections and does not include the brochure sections..I would like to include the Brochure sections..for spiderability..how do I do that?
Thirdly looking at my stats, I see that the bots for some reason seem to like indexing the /cgi-bin/ folder...why is that..and is that good? At the moment I have nothing in Robot.txt file...
It seems that my sites generally have the following directory structure:
everything including brochure and store section pages fall under the /actalog/ directory. Only index.html is outside the actalog/ directoyr..is that how it should be?
Also the site map only includes the store sections and does not include the brochure sections..I would like to include the Brochure sections..for spiderability..how do I do that?
Thirdly looking at my stats, I see that the bots for some reason seem to like indexing the /cgi-bin/ folder...why is that..and is that good? At the moment I have nothing in Robot.txt file...
Comment