cgi-bin links solution

[gabrielcrowe]

i start a new thread, but really, its all about this thread:

http://community.actinic.com/showthread.php?t=31572

i'm a bad lad. bumping a thread in such a cruel way. arent i?

[TraceyHand]

actually, Gabe, when you posted it on the other thread I was thinking it should have had a new thread of it's own.

So there...I give you permission

[gabrielcrowe]

hehe, thanks.

so, what do we all think?

[george]

Just what exactly are you trying to avoid re the cgibinthingymajig Gabriel?

[gabrielcrowe]

good question, actually.

cgi-bin links are bad for search engines IMHO.

you see, in your site, there are static files, and those files contain data.

well, there are also cgi-bin links. these links fetch the pages that are static. so, they are duplicates. with different urls.

not good.

i want to ensure a cleaner site, by removing as much of the cgi-bin interaction as possible. i really do believe that perl is too old. its bin folder is a relic of years gone by.

now in the robots.txt, it'd be wise to stop google and other searching beasties from looking through the cgi-bin. that should stop them spidering duplicates.

but there are wider implications for the technology i'm working on. for example, how about a search results page, where all the results have static links?

this could be used in many ways, but i'm really only getting started with the idea.

[jont]

cgi-bin links are bad for search engines IMHO.

I have held this belief until about 5 mins ago. I prevented Google accessing the cgi-bin and had all links on the site as static and also with SPP.

For a product we import exclusively into the UK we languished on page 6 of the naturals despite a lot of content. I removed the block to cgi-bin and now we return as number 1 (removed Friday and checked again today) with the cgi-bin link being the first item

It is a never changing landscape with Google and proves you can not sit back and assume what was the case 6 months back holds true today.

Thought I would post as may be worth others looking into what is happening with their own restricted cgi-bin.

[RuralWeb]

Things have changed as jont says and now G etc can read some cgi links far better than it used to.

[gabrielcrowe]

very interesting stuff.

and i'll be sure to keep a watch out on it. Its true that a cgi-bin link can be as popular as a standard link now. I still stand by the notion that having the same file contents shown by a standard link and also by a cgi link is bad duplication.

If you never had gci links in the first place, your high googlyness for that link would not have happened.

possibly.

[gabrielcrowe]

this problem in Actinic has been solved and is working on some very large and demanding sites:

http://community.actinic.com/showthread.php?t=32892

However, wether it will help you SEO wise is actually debatable.

[completerookie]

Jont

It is a never changing landscape with Google and proves you can not sit back and assume what was the case 6 months back holds true today.

is it a never changing landscape or an ever changing landscape
looks like you have dyslexic fingers