Meta Drops "Train" From Its Website Crawling Terms but Keeps Broad Access Ahead of the 30 October Start

Ras Al Khaimah, United Arab Emirates, 21 October 2026.

Meta has rewritten the website crawling clause in the Commercial Terms it is introducing on 30 October 2026, removing the word "train" from the list of purposes for which it may use content collected from business websites. The change followed reporting by Nieman Lab that the original wording could let Meta scrape news sites for AI training. In its 17 October report on the revision, Nieman Lab said the clause still authorises crawling of domains "associated with your products or services", which legal experts described as the broad part. We covered the original terms on 18 October; this update concerns the revised wording.

What changed

	Original clause	Revised clause
Content use	Meta could use any content from the crawled domains	Crawling to provide, develop, improve, research and maintain advertising and business products
Training	"train" listed among the purposes	"train" removed
Scope of products	Meta Company Products generally	Advertising and business products, plus integrity and security of Meta products
Sites covered	Domains associated with the business's products or services	Unchanged
Effective date	30 October 2026	30 October 2026

Nieman Lab reported that business accounts have no way to opt out of the new terms before they take effect.

What Meta says

A Meta spokesperson told Nieman Lab that the first version was not clear that robots.txt preferences are respected, and that news sites, like other content owners, can still disable the crawler through robots.txt. In a later statement, Meta said the consolidated crawling terms only applied to its ads and business products and that the update did not give it new rights that override those preferences.

Why critics are not satisfied

Legal experts quoted by Nieman Lab said the revised clause can still be read broadly. Matt Topic, a media and intellectual property attorney at Loevy and Loevy, said the word develop is wide enough to cover training and that business product is not a defined term. James Grimmelmann of Cornell Law School said nothing limits the associated-with language that could let Meta crawl entire sites. Benjamin Sobel, a University of Wisconsin law professor, noted that robots.txt expresses a preference that is voluntary to follow unless made legally binding.

Publisher groups had objected to the original version. The European Publishers Council said the terms would allow Meta to scrape news sites to train AI products and would affect any company with a Meta account, and called on the European Commission to take notice.

The crawlers involved

Meta's web crawler documentation, last updated 21 May 2026, lists five user agents. The two most relevant here are separate and can be blocked separately:

• Meta-ExternalAds crawls for uses such as improving advertising and other business-related products and services, which matches the purpose in the revised clause.
• Meta-ExternalAgent crawls for use cases "such as training foundation AI models or improving products by indexing content directly".
• Meta-WebIndexer crawls to improve Meta AI search results, and Meta says allowing it helps Meta AI cite and link to your content.
• Meta-ExternalFetcher fetches links at a user's request and may bypass robots.txt, and FacebookExternalHit may bypass robots.txt for security or integrity checks.

Meta says changes to robots.txt can take up to 24 hours to apply because its crawlers may cache the file for that long.

What this means for SEO teams

• Decide per crawler. If you want Meta AI citations but not training use, allow Meta-WebIndexer and disallow Meta-ExternalAgent. Blocking Meta-ExternalAds is a separate choice that may affect ad-related features.
• Check which domains are tied to your business account. The revised clause still covers sites associated with your products or services, so review every domain linked to your Meta business assets.
• Update robots.txt before 30 October. Allow for Meta's 24-hour cache when timing the change.
• Verify in logs. Track requests by each Meta user agent after the terms start, since the documented bypass cases mean robots.txt will not stop every Meta fetch.

Conclusion

Meta's edit removes the most explicit reference to AI training from its crawling terms, but the clause still lets it crawl any domain associated with a business account for a broad set of advertising and business purposes. With the terms starting on 30 October and no opt-out before then, robots.txt rules for each Meta user agent remain the main control site owners have.

Questions and answers

What did Meta change in its Commercial Terms?

According to Nieman Lab, Meta removed the word "train" from the crawling clause and limited the stated purposes to providing, developing, improving, researching and maintaining advertising and business products, plus the integrity and security of Meta products.

When do Meta's new Commercial Terms take effect?

The new Commercial Terms take effect on 30 October 2026. Nieman Lab reported that business accounts cannot opt out before then.

Which Meta crawler is used for AI training?

Meta's crawler documentation says Meta-ExternalAgent crawls for use cases such as training foundation AI models or improving products by indexing content directly. It can be blocked in robots.txt with its own user-agent line.

How long do robots.txt changes take to affect Meta's crawlers?

Meta says to allow up to 24 hours, because its crawlers may cache robots.txt for up to 24 hours.

###