Announcement

Collapse
No announcement yet.

Crawling Problems.

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Crawling Problems.

    Advice needed - Please !

    Goggle God has done again. Having just been crawled again www.cheapmowers.com has lost another 100 pages !! and none of the product has been indexed.

    The results within Yahoo are not much better.(partly indexed and old cached pages)

    MSN - has been spidered correctly.

    The web site was launched over a year ago and has never been properly indexed to date.

    I have now all lost faith in the company that designed the site. I am sure that there are fundamental errors within the coding and the way the site has been set up that is causing these problems.

    Can anyone recommend a reputable company that has plenty of experience designing and maintaining Actinic websites. I need this site soughting out as a matter of urgency.

    I am not looking for overnight miracles, but it would nice just to have my products and pages indexed !!

    I can be contacted on 0113 2819219 between 9-6pm

    e/mail - richardunsworth@btconnect.com

    Thanks in anticipation.

    Richard

    www.cheapmowers.com

    #2
    I am sure that there are fundamental errors within the coding and the way the site has been set up that is causing these problems...
    Are you qualified to make such an absolute statement?

    There's nothing wrong with your site Navigation.

    All your pages contain a link to the Site Map and that page contains a link to all the product pages.

    This means that there is nothing preventing search engines from finding all your product pages.

    I can also see a sensible title, and well defined meta Keyword and Description tags.

    In all, it seems like your site is competently put together.

    One area that seems somewhat overdone is the huge number of hard-coded links you have to your product pages. Everything after Quicklinks to Branded Products seems a bit too much.
    Norman - www.drillpine.biz
    Edinburgh, U K / Bitez, Turkey

    Comment


      #3
      Have you uploaded a sitemap to Google? Not your sitemap from your site, but an xml file for G to crawl in your root directory.

      If not, you might try downloading this free software to create the file.
      http://johannesmueller.com/gs/

      Then register on Google SiteMaps. They've got their own sitemap generator, but it's too complicated. The free software does it all. After the xml file is created you can upload it via FTP from the program or use your own FTP program and just drag it over into the root directory.

      Also, you must create an empty file that they want in your root directory to confirm to the crawlers that your are the registrant. You will be given a unique name for the file when you register. The G crawlers will be looking for that empty, named file before they will crawl your sitemap.
      https://www.google.com/webmasters/sitemaps/login?hl=en

      It worked for me. G has indexed more pages than I have pages. About 15,000 in less than five weeks. But having pages indexed doesn't mean anything unless you're getting ranking AND ORDERS.

      c

      PS I'm envious of your link popularity. How'd you do it?
      Attached Files

      Comment


        #4
        Having spent 2 hours on the phone with you only last week discussing your site you should know exactly where the problems lie. I am surprised that you are starting another thread on exactly the same subject as your last one - you will get the same answers! You never came back to me for a follow up after our last conversation.

        Comment


          #5
          Did you advise things he couldnt be bothered to do, so now he's seeing if theres something a bit easier?

          Comment


            #6
            Google is doing the same to me and another i know of, in the space of 50 minutes we lost over 300 pages, it seems that the results can differ depending on what time you do the search, It seems that the mighty G has not finnished what they started in December, trouble is how many people have to go out of business before they sort it!!

            One thing that does make me wonder - how can a page rank 5 site have less back links than a page rank 2 if links are so important, to how your site is treated by G surely the sites with a greater number of backlinks should fair better and be indexed properly?

            Regards
            Lee
            Many Thanks
            Lee
            Landscape Garden Machinery, Power Tools and Safety Equipment

            Comment


              #7
              how can a page rank 5 site have less back links than a page rank 2 if links are so important, to how your site is treated by G surely the sites with a greater number of backlinks should fair better and be indexed properly?
              Its not only the number of backlinks that count towards page rank but the page rank of the pages on the sites that link to you. So you you can end up with the situation above ie the page rank 5 site has less links but they are higher "quality" links.

              Comment


                #8
                Google Sitemap and Gsitecrawler Software Update

                Just an update to Charles' post above re: Gsitecrawler software and changes to Google Sitemap schema.

                Thanks for tip. Gsitecrawler is first class and easy to use although auto FTP doesn't work - no bother to manually FTP to site server.

                However, it seems from 26 March Google Sitemaps require additional header code that needs to be manually edited into the Gsitecrawler .xml file. In addition to the standard header <?xml version="1.0" encoding="UTF-8"?>
                [included in Gsite file]
                Google Sitemap now requires:
                <urlset xmlns="http://www.google.com/schemas/sitemap/0.84">
                <url>
                ....
                </url>
                </urlset> [end of sitemap file]

                Also, after several attempts found that masked comments in file made by Gsitecrawler and any reference to 'Projects' in file should be deleted for Google to accept your sitemap.xml file.

                Hope this will help others avoid 'Unsupported format' errors which are hitting previously OK sitemaps since Google changes.

                http://www.georgiajewellery.co.uk

                Comment


                  #9
                  Paul,

                  Have not encountered your errors and added a new sitemap just this week. In fact, I verified it today (thanks for the reminder). I use the automated FTP and ping on four sites - are you using the generate menu option and then following through? Do you have 1.06 v251?
                  Bill
                  www.egyptianwonders.co.uk
                  Text directoryWorldwide Actinic(TM) shops
                  BC Ness Solutions Support services, custom software
                  Registered Microsoft™ Partner (ISV)
                  VoIP UK: 0131 208 0605
                  Located: Alexandria, EGYPT

                  Comment


                    #10
                    Gsitecrawler and Google Sitemap

                    Bill

                    Yes, I'm using 1.06 v251 but only played with it for two days so clearly I've missed something.

                    At least my sitemap has been verified (yeah!), but my robots.txt file has gone from verified to 'Not Found' - Google working on this apparently.

                    What a game...

                    Paul
                    Last edited by Rocky; 29-Mar-2006, 09:47 PM. Reason: Update: Google verified robots.txt file again!

                    Comment


                      #11
                      Originally posted by Charles Bastari
                      PS I'm envious of your link popularity. How'd you do it?
                      If you use Adwords with content seach enabled some robots list your ad on other sites as a link back - thus boosting the so-called link popularity. When we were using content search our link popularity went up and stayed that way for about 2 months after stopping it. It made no difference to sales however.

                      Comment

                      Working...
                      X