.Google's John Mueller addressed a concern about why Google.com indexes webpages that are disallowed coming from crawling through robots.txt and why the it is actually risk-free to neglect the relevant Look Console reports concerning those crawls.Robot Website Traffic To Query Specification URLs.The person inquiring the question chronicled that bots were actually making hyperlinks to non-existent query criterion Links (? q= xyz) to pages along with noindex meta tags that are actually also blocked out in robots.txt. What triggered the inquiry is that Google is actually crawling the web links to those web pages, getting blocked by robots.txt (without noticing a noindex robotics meta tag) then acquiring shown up in Google.com Look Console as "Indexed, though blocked by robots.txt.".The person asked the observing concern:." However listed here is actually the big question: why would Google index webpages when they can not even see the material? What is actually the advantage during that?".Google.com's John Mueller verified that if they can not creep the page they can not observe the noindex meta tag. He likewise produces an appealing acknowledgment of the web site: hunt driver, recommending to dismiss the end results since the "ordinary" consumers won't see those end results.He composed:." Yes, you're right: if our company can't creep the page, our team can not see the noindex. That said, if our team can not crawl the webpages, then there is actually certainly not a whole lot for us to index. Thus while you might observe a number of those pages with a targeted site:- question, the average consumer will not view them, so I definitely would not bother it. Noindex is likewise great (without robots.txt disallow), it only implies the Links are going to wind up being crept (as well as wind up in the Look Console document for crawled/not catalogued-- neither of these standings lead to concerns to the rest of the website). The essential part is actually that you do not produce them crawlable + indexable.".Takeaways:.1. Mueller's response verifies the constraints in operation the Site: hunt advanced search driver for analysis causes. One of those main reasons is actually because it is actually not connected to the frequent search mark, it's a different point completely.Google's John Mueller discussed the web site search driver in 2021:." The brief response is that an internet site: inquiry is not implied to become full, nor made use of for diagnostics objectives.A site inquiry is a specific sort of hunt that limits the results to a certain web site. It's basically only words web site, a bowel, and afterwards the website's domain.This concern confines the end results to a specific web site. It is actually certainly not indicated to become a thorough collection of all the webpages coming from that site.".2. Noindex tag without utilizing a robots.txt is alright for these kinds of conditions where a bot is actually connecting to non-existent pages that are actually obtaining found out by Googlebot.3. URLs with the noindex tag are going to generate a "crawled/not listed" entry in Explore Console and also those will not possess a damaging impact on the remainder of the web site.Read the question and address on LinkedIn:.Why would certainly Google.com mark pages when they can't also observe the web content?Included Image through Shutterstock/Krakenimages. com.