links through the cgi-bin. workaround?

[pinbrook]

Just had a thought about cgi-links, re nav, BS, NP and sitemap

How does Big G see a hard coded link to a page and a cgi link to the same page? or does it matter?

I was specifically thinking about temp cgi-links from BS and NP then disapearing when you change BS and NP.

Would this have an impact if you have multple products on one page, or is this another time were one product per page wins over.

Is it better to have nav as cgi and sitemap as static, is it better to have all one type on a site...

[gabrielcrowe]

doh.

can open, worms everywhere.

[RuralWeb]

can open, worms everywhere

It may well turn out to be the case. The "problem" has always been there eg sites tha use cgi nav and site maps BUt the change is that google is getting much better at following the cgi links and as a result duplicate pages are being created.

One of two things needs to happen - either make the links non cgi or make them in a format that google cannot follow. If you use also bought and related items then you could eventually make two copies of your website

[leehack]

Am i missing something obvious on this problem, if you only use the cgi-bin on the marketing lists, just disallow the cgi-bin in robots.txt. It can't duplicate anything then. No matter how good or bad Google is at using the cgi-bin nowadays, it's still better not to use it if possible.

You can have a site that only functions through the cgi-bin on things not SE important anyway.

[RuralWeb]

just disallow the cgi-bin in robots.txt

Yep - easy done

[pinbrook]

Am i missing something obvious on this problem

My musings were centred on having both types of links. Thus is it better to concentrate efforts on one type of link only.

Thus if you have BS etc, and cgi-nav and cgi sitemap you are fine

BUT would you have a problem if you had BS, cgi-nav and hard coded links in sitemap?

If you have a mixture, then yes you could disallow robots access to cgi bin, but you would need to be careful to ensure theres something on the site spiderable.

Its just an issue I hadn't really considered before now.

[RuralWeb]

Its just an issue I hadn't really considered before now

Its only been happening over the last few months in the sites I have been watching but it is getting to be more of a problem.

[pinbrook]

i used Norman's code to create a pretty sitemap and by accident it created cgi links (I think designer works the same as business) so now all my links in SW are cgi.

But xml sitemap is not.... Oh Gawd

[RuralWeb]

Just been looking at this one with a site that uses best sellers etc and the bad news is that google is seeing duplicate pages for products that are shown in the best sellers. The good news is that it is indexing very few of them - but that does not solve the problem.

[dave_finlayson]

So is the thinking definately that if you use only static links (except for marketing) it is best to stop Google indexing the cgi-bin?

[wjcampbe]

So is the thinking definately that if you use only static links (except for marketing) it is best to stop Google indexing the cgi-bin?

I always do FWIW

[dave_finlayson]

Thanks Bill, certainly going to give it a try as we use both best sellers and also bought so probably have a horrendous amount of duplicate pages!

[RuralWeb]

gabriels original idea is the better one ie getting the best sellers to use static links.

[dave_finlayson]

Yes that I would love but don't think there will be a workable hack and can't imagine Actinic changing it anytime soon!

[RuralWeb]

Yes that I would love but don't think there will be a workable hack

There is