July 28, 2015, several webmasters received an error from Google Search Console (Webmaster Tools) which read “Googlebot cannot access CSS and JS files on http://www.website.com/”.
The message reads:
To: Webmaster of http://www.website.com/,
What does this mean?
Google’s technical guidelines state that they want access to all of your files to thoroughly understand your website. This includes seeing potential advertisements on it and how it renders on a mobile device. To do this they need to crawl CSS and JS files.
How to fix the problem in WordPress
Many WordPress webmasters are inadvertently blocking CSS and JS files.
A few ways WordPress can block these files:
- robots.txt file is blocking crawling of this subfolder /wp-includes
- robots.txt is blocking crawling of JS and CSS files
- A security plugin such as iThemes Security is blocking JS and CSS crawling
To fix the error and allow Googlebot to crawl your JS and CSS files, take the following steps:
Check your robots.txt file. You can FTP in, use Google’s robots.txt tester, or simply visit yourwebsite.com/robots.txt. It should look something like this:
User-agent: * Disallow: /wp-admin/
If you’re using iThemes Security, uncheck the “disable directory browsing” box. Using it blocks access to JS and CSS files via the .htaccess file.
Once the issue is resolved, “Fetch and Render” your page with the “Mobile: smartphone” option selected to double-check that Googlebot for smartphones renders your content properly.
Note: Fetch and Render can be used to instantly crawl pages. Read about that here.
Testing for blocked resources
In search console, visit the blocked resources section.
Does blocking JS and CSS files hurt SEO?
Yes, but don’t freak out. I haven’t seen this affect traffic personally, however, it is well documented on the Yoast blog here. In that case study, the site bounced back to normal health once the issue was resolved.