I am not sure if Google is throttling content or not but after many many years of me using Google Search Console’s “Fetch and Render” to index new content (aka “Fetch as Google”) or using Google’s “Mobile Friendly Test” to crawl and index pages, these tools are suddenly no longer getting new pages indexed in a timely manner.
This change occurred in the middle of March 2018 about the same time the March 2018 Google Algorithm change rolled out. However, a few days ago Google introduced a new way to prevent “Fresh and Relevant” content from hitting search results.
A new 24 Hour “throttle”?
I am still testing this, but in the past many of the websites I operate had their content showing up in search within a few seconds of going live. These sites were very active in search.
So far my tests have confirmed that the content is indeed being crawled right away but the actual indexing of content (having the pages added to Google’s search engine results pages) is indeed taking 24 hours.
According to Google’s announcement, I assumed that freshness and relevancy was being overruled exclusively in Top Stories, but it appears that content is indeed being delayed from being indexed for 24 hours.
Here’s a screenshot of an article I posted onto my site on Monday, March 19, 2018 when many users were horrified to discover Google Finance had changed. Unfortunately, this post did not display in search results until the following day, March 20, 2018, one day later:
I hope the delay is temporary
At Telapost, I provide a lot of content to clients shortly after certain events take place. 24 hours later any webmaster is looking at a gigantic reduction in traffic. I’ve had posts receive 20,000 – 30,000 hits on the first day and as little as 15 the next day.
Throughout this week I have tested the crawling and indexing of content on 8 different sites all with the same result. I hope that this issue is temporary.
Is your content not displaying right away?
This issue has not yet been documented on the web anywhere, however I assume people will soon begin discovering this little issue towards the end of March or in April if it is not resolved soon. Hopefully this is a temporary bug and not a permanent “feature” of Google search as fresh and relevant content is mandatory for a healthy Internet.
Please feel free to comment below and share your observations or concerns.
It appears that this change, documented first by Telapost, is over! It was temporary. Google is rolling out the mobile-first index, and we believe that the failure to index pages timely was temporary.
- What Happened to ChicagoNow.com? - August 30, 2022
- The December 2021 Google Local Pack Algorithm Update - December 17, 2021
- Map Displaying to the Right of the Local Pack in Google Test in July 2021 - July 30, 2021
I can’t get new pages crawled at all and have dozens waiting. 🙁
Google is working properly once again.