🔍 Read the full analysis: OpenAI, Google, Microsoft And Anthropic Back One AI Training Opt-Out on ThorstenMeyerAI.com
Get movie nights delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
OpenAI, Google, Microsoft and Anthropic have announced support for a shared robots.txt signal that lets website operators express a preference against having their content used to train generative AI models. The signal is voluntary, does not block crawling or AI answers based on live web content, and will take effect on different schedules; Meta expressed support without committing to honor it.
OpenAI, Google, Microsoft and Anthropic said on October 30 that they will recognize a shared instruction in websites’ robots.txt files asking that their content not be used to train generative AI models. The move could reduce the need for publishers to maintain separate training opt-outs for each company, but the signal is voluntary, does not cover every use of web content, and will arrive on different timelines.
The shared signal is described in an individual Internet-Draft called “Machine-Readable Training Preference Signal,” published by Kevin Kelley, an Anthropic standards lead. A site operator would add a group for the user agent __usage_ai-training and a disallow rule for the site’s content. The draft is associated with the Internet Engineering Task Force’s AI Preferences working group, but it is not an adopted internet standard.
The companies’ rollout plans differ. Anthropic says it has started implementing support, though it says support will take time to reach its services. OpenAI aims to implement the signal “in the coming months.” Google says it aims to support it in 2027, with its crawler documentation specifying early 2027. Microsoft says it will honor the training preference from February 1, 2027, as it crawls websites.
Microsoft also announced a separate page-level preference for AI Grounding—the use of web content to inform answers in Bing, Copilot and Microsoft 365. Microsoft says both preferences take effect February 1, 2027. Its existing NoArchive and NoCache tags will stop counting as preferences for training or grounding after March 1, 2027. Microsoft says the preferences do not guarantee inclusion, traffic or payment.
Infographic 01 · Who backs it, and when
One line in robots.txt, four companies, four timelines.
A shared signal against AI training. Only Anthropic says it has started; the others name dates or months. The standard itself is still a draft.
- 30 Oct 2026Draft published, four companies announce supportdoneIndividual Internet-Draft by Kevin Kelley (Anthropic)
- 30 Oct 2026Anthropic: “has already started implementing support”startedIts standards lead on the IETF list: “It’ll take us some time to add support across our services.”
- Coming monthsOpenAI aims to implement itplanExisting GPTBot and OAI-SearchBot rules keep applying
- 16 + 19 Nov 2026Working group meets at IETF 127, San FranciscoopenOn 21 Oct a chair said no consensus has been declared yet
- 1 Feb 2027Microsoft honors the new preferences as it crawlsplanTraining in robots.txt or per page; grounding per page
- 1 Mar 2027Bing stops reading NoArchive / NoCache as AI preferencesdeadlinePublish the new settings before this date
- 2027Google plans support; its docs say “early 2027”planA “no” maps to Google-Extended unless you set that token yourself
- 3 May 2027Draft version 00 expires unless updatedInternet-Drafts are valid for at most six months
Binding? No, voluntary
- OpenAI calls it a “voluntary effort”.
- The vocabulary draft does not “ensure that preferences are followed”.
- The draft “has no formal standing in the IETF standards process”.
- Private agreements may override it, the draft says.
Who else?
- Meta supports a common signal, but did not say whether it will honor it.
- Apple, Amazon, xAI, Perplexity: no statement in the sources reviewed for this article.
- The draft’s acknowledgments list contributors “for identification only”, not supporters.
Sources: IETF draft, 30 Oct 2026 · Anthropic on the IETF list · AIPREF minutes, 21 Oct 2026 · Google, 30 Oct 2026 · Google crawler docs · OpenAI, 30 Oct 2026 · Microsoft Bing, Oct 2026 · Meta (Andy Stone on X). Company plans are company statements.
Powered by Thorsten Meyer AI
One Rule, Several Company Timelines
For website owners, a shared signal could make it simpler to communicate a training preference across participating services. Under the old arrangement, publishers might need to manage separate crawler instructions, including rules for OpenAI’s GPTBot, Google-Extended and Anthropic’s ClaudeBot, as well as Microsoft-specific tags. A common line offers a more consistent way to state an objection, once each company supports it.
That convenience is not the same as enforcement. The draft is a request to participating crawlers, not a technical barrier that prevents a company from accessing a site or using content. Publishers will need to check each provider’s implementation and keep existing company-specific rules in place while support rolls out. The different dates also mean that adding the shared signal will not necessarily affect all four companies at once.
The announcement also draws a distinction between training and other AI uses. A site may want to prevent its material from being used to train a model while still allowing it to appear in search or inform an AI-generated answer. Microsoft’s separate grounding control makes that distinction explicit. The shared training signal alone does not tell companies to stop using a page as a source for live answers, and it does not establish a payment system for publishers.
Popular right now
Breville Bambino Espresso Machine, Brushed Stainless Steel (BES450BSS)
Ninja Foodi DualZone 10-Qt Air Fryer, 6-in-1, Black (DZ302)
KitchenAid Classic Series 4.5-Quart Tilt-Head Stand Mixer, Onyx Black
As an affiliate, we earn on qualifying purchases.
How Website Opt-Outs Work
Robots.txt is a file through which website operators give instructions to automated crawlers. Companies have used different crawler names and rules for different purposes, so publishers seeking to limit AI-related use have had to account for provider-specific controls. The new proposal aims to give companies a common machine-readable preference for one defined purpose: generative AI training.
The proposal is an individual Internet-Draft, not a completed IETF standard. At an October 21 working-group meeting, a chair said, “We haven’t declared consensus on anything yet.” That status matters: the draft may guide implementation, but it does not carry the formal standing of an approved standard. Existing controls such as GPTBot, Google-Extended and ClaudeBot remain relevant while companies adopt the shared signal.
Google says that if a site uses the new token but has no Google-Extended rule, it plans to interpret the preference as though the publisher had disallowed Google-Extended. Google says that control does not affect Google Search. Microsoft likewise says its preferences do not change eligibility for Bing search results. These company-specific explanations show why publishers should distinguish training, search and AI answer use rather than treat them as a single setting.
““has already started implementing support for this draft””
— Anthropic, in its announcement of support for the draft
Adoption and Coverage Remain Open
It remains unclear whether companies beyond the four named participants will honor the signal. Meta expressed support but made no implementation commitment in the statement cited in the source material. No statements from Apple, Amazon, xAI or Perplexity were identified in the source’s review, which does not establish what those companies may do later.
The draft is not an official IETF standard, and the precise reach of each company’s implementation may depend on its services and crawl schedules. The signal also does not block crawling, remove content from models that have already been trained, or cover AI answers generated using live web content. The source material does not establish how compliance will be monitored or what recourse a publisher would have if a company ignored the preference.
Publishers should also avoid treating the shared rule as a guarantee of traffic, inclusion or compensation. Microsoft explicitly says its controls guarantee none of those outcomes. More generally, the announcement sets out a way to express a preference; it does not settle broader disputes about how web content is used or whether publishers should be paid.
Watch for Rollout and Standardization
The next concrete milestone is implementation by each provider. Anthropic says it has begun work; OpenAI’s target is the coming months; Google is aiming for early 2027; and Microsoft’s stated start date is February 1, 2027. Website operators will need to follow provider documentation for confirmation that the relevant crawlers recognize the signal.
Microsoft’s separate transition for its existing NoArchive and NoCache tags also has a deadline: it says those tags stop counting as training or grounding preferences after March 1, 2027. Publishers relying on them for those purposes will need to review their instructions before then and decide separately how to handle training, AI grounding and search.
The IETF working group’s public process is another point to watch. The current draft has no formal standing in the standards process, and the group had not declared consensus at its October 21 meeting. Whether the proposal is revised, gains working-group support, or becomes a formal standard remains to be determined.
Key Questions
What does the shared AI training opt-out do?
It lets a website publish a robots.txt preference asking participating companies not to use its content to train generative AI models. It is a voluntary signal, not a technical block on access.
Which companies have said they will honor it?
OpenAI, Google, Microsoft and Anthropic announced plans to recognize the signal. Meta expressed support for publishers expressing preferences but did not say whether it would honor this specific signal.
When will the signal take effect?
The schedules differ: Anthropic says implementation has begun, OpenAI aims for the coming months, Google is targeting early 2027, and Microsoft plans to honor it from February 1, 2027.
Does the signal stop AI answers from using a website?
No. The shared signal concerns model training; it does not by itself cover AI answers based on live web content. Microsoft announced a separate AI Grounding preference for certain Microsoft services.
Is the opt-out an official internet standard?
No. It is an individual Internet-Draft connected to an IETF working group, and the source material says it has no formal standing in the IETF standards process.
Source: ThorstenMeyerAI.com
