How Google Indexing a very Large Websites
How Fast Crawling and Indexing Site with Millions of Pages?
Recently, I often hear questions about the rapid embedding in the index of large sites. And today in the chat they threw a link to the fresh millionaire — people-ua.biz.
people-ua.biz — database of citizens of Ukraine (population of about 42.5 million). The card page contains a Name, date of birth and residence address. Search can be made on any of this parameter.
We will review the site and analyze how they managed to break into the Google SE index.
To check the response of the server, I recommend using this service.
The domain is registered on October 20, 2016 whois#people-ua.biz:
The sitemap directory contains more than 800 sub-lists for 50,000 URLs each. Total on the site people-ua.biz more than 42,000,000 pages. That is close to the documented number of the population of Ukraine.
So how many pages were able to push to google index for the month of existence?
According to Google SERP, on request site:people-ua.biz issues only slightly more than 4 million pages, approximately 700k per week:
On my projects recorded a 6-fold increase in the number of crawled pages by the google-bot in comparison with the average value after adding a sitemap:
Of these, the index hit up to 98% of the pages in about two weeks.
Your record? How much pages did you get crawled & index by Google-bot? Or, maybe you know other cases?
Bonus tip: One of the tools that will help get your website indexed faster is Page Counter by Sitechecker. This tool will find all the website pages, check their indexing status and detect any technical problems that may be affecting the indexing.
-
Marketing Tips5 days ago
What is my Instagram URL? How to Find & Copy Address [Guide on Desktop or Mobile]
-
Cyber Risk Management22 hours ago
How Much Does a Hosting Server Cost Per User for an App?
-
Grow Your Business5 days ago
Best Instagram-like Apps and their Features
-
Outsourcing Development22 hours ago
All you need to know about Offshore Staff Augmentation
-
Software Development22 hours ago
Things to consider before starting a Retail Software Development
-
Marketing Tips5 days ago
B2B Instagram Statistics in 2024
-
Grow Your Business14 hours ago
The Average Size of Home Office: A Perfect Workspace
-
Solution Review14 hours ago
Top 10 Best Fake ID Websites [OnlyFake?]
Hello. impressive job. I did not imagine this. This is a remarkable story. Thanks!