Meta's New TOS Greenlights Scraping Your Site for AI Training
Meta's updated commercial terms allow the company to scrape your linked business websites for AI training, potentially overriding your robots.txt file.
Meta updated its commercial terms to let it scrape your business website for AI model training, effective October 30. The new language authorizes automated scraping of any domains you link to from your business account. Your only real opt-out is to stop using Meta's business products, as these terms likely supersede technical blocks like `robots.txt`.
The change grants Meta a broad license to use crawlers and other tools on domains and pages associated with your business. This means if you link your company's website from your Instagram or Facebook business page, you are contractually agreeing to let Meta scrape that site's content. This isn't an implied permission; it's a condition of using their commercial services.
What breaks?
The core problem is the conflict between a legal agreement and a technical standard. Your `robots.txt` file is a directive, not a technical barrier. By accepting the new Terms of Service, you are giving Meta explicit legal permission to access your site's content. This contractual agreement likely voids any `Disallow:` directives in your `robots.txt` file as far as Meta's crawlers are concerned. A legal permission slip overrides a technical request to stay away.
Meta's response is that its crawler policies are not changing and that site owners can use "flags" to state their preferences. This is vague. The Nieman Lab report on **Meta's updated commercial terms** highlights that legal experts see the new language as a clear authorization for scraping. Relying on undefined "flags" when the legal text is explicit is a poor strategy.
Should you care?
Yes. This is a non-negotiable data grab hidden in a TOS update. You are trading access to Meta's audience for your proprietary website content, which will then be used to train their models. If your website content *is* your business, you're giving away the raw material for free. The lack of a clear, technical opt-out mechanism is the most telling detail. I would audit any site linked from a Meta business account and consider if the connection is worth the cost.