Google said it will support a shared robots.txt signal that lets websites opt out of AI training by every participating AI company at once, joining Microsoft, OpenAI, and Anthropic, which committed to it the same day. The signal comes from a draft proposal at the Internet Engineering Task Force, and sites would list a single token, “__usage_ai-training,” instead of blocking each company’s crawler by name. Google plans to support it in early 2027 and will read an opt-out as a block on Google-Extended, its existing AI training control, unless the site addresses Google-Extended directly. Company-specific controls will keep working, so a site can opt out broadly while still letting Google-Extended in, which keeps it eligible to be cited in Gemini’s answers.
