Google is providing a service to the websites they crawl.
They try to not crawl when we don't want them (robots.txt, clear user-agent, no-index no-follow...).
reply
Yeah they're building an LLM and making it pointless to visit the websites.
Btw you can still block it.