GOOGLE CRAWL ERRORS

Site Errors:

DNS (Domain Name System) errors implications are huge to tackle. If Googlebot cannot able to connect to the domain of your website may be because of DNS Timeout or DNS lookup issue, its high time to take error fixing action. Fetch as Google Tool which is present in Search Console is a quick way to check DNS connection and you can also check with your DNS provider.

Server Errors happen when servers cannot handle too much traffic and they require urgent action to be taken. Googlebot can only wait for limited amount of time to load your website pages, after that it gives up. Fetch As Google tool can be used to see whether Googlebot can crawl and return homepage content with ease. Servers errors are of different types so do look out the type of error you must be facing.

Robot.txt file erroroccurs when Googlebot cannot retrieve this file located in domain name. Check for all the line of “Disallow: /”, use server header checker tool to check for 200 or 404 errors. Proper configuration is the best way to deal with it.

URL Errors:

Soft 404 errors display page with 200 found and not 404 not found. The content of the page should be such that it informs user that page they are looking for does no longer exist. According to Google, HTTP header response should return page with 404 (not found) or 410 (gone).Live and critical pages should not be listed as Soft 404 pages and they require immediate action. Do not redirect lots of non-existing pages to your home page; redirection should be to the appropriate pages.

404 error means page no longer exist. This error becomes important when a particular page is been requested by large amount of traffic, or it contains some important URL’s or links and backlinks.to fix it use your content management system for publishing content, ensure page is not deleted, check http v/s https version of this error, work on external links from URL Error section of Google Search Console and ensure whether 404 error is actually 404 error page and not other variation of 404.

Access denied occurs when Googlebot is been blocked from crawling the page at a very first place. Log in and authentication means used by you and hosting providers can block Googlebot.Robot.txt file can also block Googlebot to crawl whole website or any particular folders or URL’s.to fix it remove popup or in-page login from a page which you want to be indexed and gets crawled, check for robot.txt file content, user-agent switcher plug-in can be used or Fetch As Google to see how site appears to Googlebot.

Not Followed error occurs when Google because of Flash, JavaScript, or redirect issues cannot follow a particular page or URL. Lynx text browser, fetch as Goggle tool, user-agent switcher can be used and do not include redirected URL into sitemap, use destination URL.

Having different configuration for certain URL’s in your domain or isolated configuration for individual URL’s can cause Server and DNS errors under URL Errors sections and they need to be tackled in similar way as in Site Errors section.

It's only fair to share...Share on FacebookShare on Google+Tweet about this on TwitterShare on LinkedIn
Shares
Shares