Google, Microsoft and OpenAI back one robots.txt rule for refusing AI training
No company has said it honors the token yet, and only Microsoft has given a start date. The draft is an individual submission, not an IETF standard, and expires on May 3, 2027. It states a preference and does not technically prevent training. OpenAI's post refers to "foundational" generative AI training and does not say whether that is narrower than the draft's definition. Apple, Meta and Amazon are not among the companies that committed. This edition read the draft and Google's crawler page directly, and the Microsoft and OpenAI posts through PPC Land's report.
What was claimed?
On October 30, 2026, Google, Microsoft and OpenAI each said they will honor one robots.txt group, "User-Agent: __usage_ai-training", that lets a website say it does not want its pages used for generative AI training by any participating provider. The token is defined in an individual IETF Internet-Draft, "Machine-Readable Training Preference Signal", submitted the same day by an Anthropic engineer, who told the working group's mailing list that Anthropic has started adding support. PPC Land reports that Microsoft will begin honoring it on February 1, 2027, Google aims for early 2027 (as its crawler documentation now says), and OpenAI says "the coming months".
What does the evidence support?
Company claim, independent confirmation unresolved
Today a site that wants to refuse training has to list a separate token for each company's crawler. One shared rule makes that preference simpler to state, and to check, across the largest developers. Blocking a crawler still blocks search, while the new token states a preference about use only.
What are the limits?
No company has said it honors the token yet, and only Microsoft has given a start date. The draft is an individual submission, not an IETF standard, and expires on May 3, 2027. It states a preference and does not technically prevent training. OpenAI's post refers to "foundational" generative AI training and does not say whether that is narrower than the draft's definition. Apple, Meta and Amazon are not among the companies that committed. This edition read the draft and Google's crawler page directly, and the Microsoft and OpenAI posts through PPC Land's report.
Claim or evidence date: 2026-10-30. Last verified: .
Colors classify evidence; they do not rank truthfulness. Explore hype versus reality →