Bing not indexing site

Hi Karen,

You've not enabled redirects - so Google and Bing are seeing your site listed 4 times, with duplicate content, and that'll be another reason why your site is dropping in G.

Make sure your hosting company set these up correctly (www, non-www, http, https) and then update your Google and Bing Webmaster services to correctly identify the sites. This really is imperative and should have been done even when you were an HTTP only site.

You don't have a robots.txt at / or /acatalog - it's most likely you've got something misconfigured in Bing Webmaster tools. Maybe your host can assist (could there be something in your htaccess file blocking everything EXCEPT the root and CGI-BIN)?

Did you come across this help post? https://www.bing.com/webmaster/help/...index-2141dfab

Justin

Follow up... it may not be the answer, but the positioning of elements in your HTML is incorrect on some templates.

In our testing, cgi-bin templates are correctly positioned in the HTML head. On /acatalog pages (those not being indexed by Bing), they're within the HTML body.

Maybe a closing tag is missing somewhere in one of your templates?

Presume from this that fixing this problem may fix your Bing? (Plus fix the problem and tell Bing Webmaster tools and it should speed up the reindexing).

If you need help for this issue and can't resolve in house, please contact your Partner or our own services team would be able to assist.

Thanks,
Justin

Hi Karen,

Your results in Bing are the oddest thing I've come across.

From your results in Bing Tools it's clear that Bing have crawled your site and sees the 8,000 or so pages in the /acatalog folder but just won't include them in the search index. The only pages it will show in the index are a few in the /cgi-bin folder.

I've looked but there's no sign of a robots.txt file that could be causing any problems. Also Google indexes the pages without any problems.

I've looked at several other V14 sites on Bing and none of them have the same sort of problem as you're having, which suggests the problem isn't sellerdeck or V14

The only things I can think of checking are:

1. When you used the 'fetch as bing' tool and it fetched it OK. Did you scroll down the page to see the page had been fetched correctly?

2. I wonder if somehow Bing has got itself into some kind of a duplicate content 'Loop'. ie the first page you have in the Bing Serps is the cookie error page and if they see that for the majority of the pages they're looking at then that will look like duplicate content and as a result they might be refusing to index the rest of your pages.

It's a bit of a stretch, and a little counter intuitive, but I can't help wondering if it's worth creating a robots.txt file to block bing from the cgi-bin. If done correctly, then at worst Bing won't index any of your pages but there's a chance it might force it to ignore the cgi-bin pages and to look properly at your pages in the /acatalog folder.

Mike

The odd thing is if I run one of your pages through an HTML validator it says the same thing about the body tag being used when it's already open and a couple of other similar errors but I can't see any sign of an earlier body tag.

https://validator.w3.org/nu/?doc=htt...aphics_45.html

There are quite a few errors being thrown up so it's probably a good idea to track them down and fix them. I wouldn't worry too much about the warnings but fixing the errors is probably a good idea.

Although having just validated a page from another V14 site that is properly indexed in Bing it shows exactly the same errors about Head and Body tags so I'm not really convinced that this is the problem.

Mike

Originally Posted by jtaylor

Hi Karen,

You've not enabled redirects - so Google and Bing are seeing your site listed 4 times, with duplicate content, and that'll be another reason why your site is dropping in G.

You don't have a robots.txt at / or /acatalog - it's most likely you've got something misconfigured in Bing Webmaster tools. Maybe your host can assist (could there be something in your htaccess file blocking everything EXCEPT the root and CGI-BIN)?

Did you come across this help post? https://www.bing.com/webmaster/help/...index-2141dfab

Justin

Hi Justin. It is not dropping in Google as such. It is just Bing. I contacted Bing a while ago and they can't see why I get the results I do in their search engine, they are as baffled as I am.

I don't have an .htaccess file in place as far as I know.

I haven't done anything in Bing webmasters as far as I can see. What could I have done there that would be so detrimental? I did read through that link some time ago and there is nothing there.

I shall set up redirects, you're right, should have done that some time ago.

I'm with 1and1, they wont assist with anything.

Originally Posted by jtaylor

Follow up... it may not be the answer, but the positioning of elements in your HTML is incorrect on some templates.

In our testing, cgi-bin templates are correctly positioned in the HTML head. On /acatalog pages (those not being indexed by Bing), they're within the HTML body.

Maybe a closing tag is missing somewhere in one of your templates?

Presume from this that fixing this problem may fix your Bing? (Plus fix the problem and tell Bing Webmaster tools and it should speed up the reindexing).

If you need help for this issue and can't resolve in house, please contact your Partner or our own services team would be able to assist.

Thanks,
Justin

Oh really? How odd. I can't see a missing closing tag. What is it that is appearing in the body? My partner is me so I stand alone in this I did ask support, as I have cover, but they want to charge extra as they see it as SEO, which I guess it technically is, although it is a fault within Sellerdeck by the looks of it. I haven't really changed anything that wold account for that though so I'm baffled.

Originally Posted by Mike Hughes

Hi Karen,

Your results in Bing are the oddest thing I've come across.

From your results in Bing Tools it's clear that Bing have crawled your site and sees the 8,000 or so pages in the /acatalog folder but just won't include them in the search index. The only pages it will show in the index are a few in the /cgi-bin folder.

I've looked but there's no sign of a robots.txt file that could be causing any problems. Also Google indexes the pages without any problems.

I've looked at several other V14 sites on Bing and none of them have the same sort of problem as you're having, which suggests the problem isn't sellerdeck or V14

The only things I can think of checking are:

1. When you used the 'fetch as bing' tool and it fetched it OK. Did you scroll down the page to see the page had been fetched correctly?

2. I wonder if somehow Bing has got itself into some kind of a duplicate content 'Loop'. ie the first page you have in the Bing Serps is the cookie error page and if they see that for the majority of the pages they're looking at then that will look like duplicate content and as a result they might be refusing to index the rest of your pages.

It's a bit of a stretch, and a little counter intuitive, but I can't help wondering if it's worth creating a robots.txt file to block bing from the cgi-bin. If done correctly, then at worst Bing won't index any of your pages but there's a chance it might force it to ignore the cgi-bin pages and to look properly at your pages in the /acatalog folder.

Mike

Hi Mike, yes it fetched OK as far as I could see. No reports of duplicate contents in the webmasters console, and Bing says that they can't see any problems.

Originally Posted by Mike Hughes

The odd thing is if I run one of your pages through an HTML validator it says the same thing about the body tag being used when it's already open and a couple of other similar errors but I can't see any sign of an earlier body tag.

https://validator.w3.org/nu/?doc=htt...aphics_45.html

There are quite a few errors being thrown up so it's probably a good idea to track them down and fix them. I wouldn't worry too much about the warnings but fixing the errors is probably a good idea.

Although having just validated a page from another V14 site that is properly indexed in Bing it shows exactly the same errors about Head and Body tags so I'm not really convinced that this is the problem.

Mike

These errrors are present on all Sellerdeck site I tested though, including your own. I have noticed it before, lots of Sellerdeck coding is objectional to HTML validators for some reason.

Thanks so much for helping guys, REALLY appreciate it

Originally Posted by jtaylor

Follow up... it may not be the answer, but the positioning of elements in your HTML is incorrect on some templates.

In our testing, cgi-bin templates are correctly positioned in the HTML head. On /acatalog pages (those not being indexed by Bing), they're within the HTML body.

Maybe a closing tag is missing somewhere in one of your templates?

Presume from this that fixing this problem may fix your Bing? (Plus fix the problem and tell Bing Webmaster tools and it should speed up the reindexing).

If you need help for this issue and can't resolve in house, please contact your Partner or our own services team would be able to assist.

Thanks,
Justin

Hi Justin,

Further to this, could you please tell me how/where you are seeing the cgi-bin templates in the body? I have searched extemsively and just can't find anything amiss. Would be grateful for your help.

Thank you again also Mike for all your help. You're a star and such an asset to the forum.

Originally Posted by KarenBM

Thank you again also Mike for all your help. You're a star and such an asset to the forum.

Thank you for the kind words Karen,

I'd feel more of a star if I could work out what the damn problem is.

Mike

Originally Posted by KarenBM

Hi Justin,

Further to this, could you please tell me how/where you are seeing the cgi-bin templates in the body? I have searched extemsively and just can't find anything amiss. Would be grateful for your help.

Thank you again also Mike for all your help. You're a star and such an asset to the forum.

Hi Karen,

I think you've misunderstood or I wasn't clear!

If you look at Bing you can see the only listed pages are cgi-bin (expect for the homepage).

Looking at the HTML source for pages which AREN'T cgi-bin, you can see how elements which are required within <head></head> are not in the correct position.

If you've an active Support contract, please give the team a call - mention that you've been speaking with me on the Community and we can get this sorted out for you. We'll need to access your screen using TeamViewer.

This is not something our Support team would usually help with, but on this occasion I believe it's a quick-fix in terms of your design which will at least aid in you trying to get your site back on track.

Kind regards,
Justin

Thank you Justin, I shall contact support.

Originally Posted by KarenBM

Thank you Justin, I shall contact support.

Hi Karen,

Whatever you did or was done - it looks fixed on Google! If you can get into your Bing Webmaster Tools, and advise Bing to re-index, that should sort the original issue :-)