Four AI Giants Unite Behind a Single robots.txt Opt-Out Signal
Publishers who have spent the past two years juggling a separate blocking rule for every AI crawler finally have a simpler option on the table. Google, Microsoft, OpenAI and Anthropic have committed to honoring a single shared signal in robots.txt that lets a website refuse the use of its content for training generative AI models. … Read more