Cloudflare introduces tools for control over AI content use

Cloudflare has announced a new set of tools designed to help websites analyse and control how their content is used by artificial intelligence (AI) models.

Dubbed AI Audit, the tools promise to offer content creators a way to understand how AI model providers are using their content and to take control over access to it.

Additionally, Cloudflare is developing a feature that allows content creators to set a fair price for their content used by AI companies for training and retrieval augmented generation (RAG).

The announcement comes at a time when websites are frequently scanned by AI bots without the knowledge or compensation of the content creators, leading to a significant loss of value for both large and small businesses. These new tools aim to allow content creators to regain control and ensure fair compensation for their contributions to the internet.

Matthew Prince, co-founder and CEO of Cloudflare, highlighted the transformative impact of AI on online content and the need for control over this shift. "AI will dramatically change content online, and we must all decide together what its future will look like," he said. "Content creators and website owners of all sizes deserve to own and have control over their content. If they don't, the quality of online information will deteriorate or be locked exclusively behind paywalls. With Cloudflare's scale and global infrastructure, we believe we can provide the tools and set the standards to give websites, publishers, and content creators control and fair compensation for their contribution to the Internet, while still enabling AI model providers to innovate."

AI Audit aims to provide content creators with the information and tools necessary to manage how their content is accessed by AI bots. This includes the ability to:

• Automatically control AI bots, for free: This feature enables website owners to block all AI bots with a single click, which is particularly beneficial for smaller sites lacking the skills or bandwidth to manually block AI bots.
• Access analytics on AI bot activity: Cloudflare's new tools offer site owners analytics to understand why, when, and how often AI models access their website. This allows for differentiation between bots that credit the source of the data and those that do not.
• Better protect their rights: Advanced analytics will help site owners negotiate licensing agreements with model providers. Metrics such as the rate of crawling for certain sections or entire pages will be available, along with model terms of use to legally protect content creators' rights.
• Set fair prices for content scanning (in development): This feature aims to assist site owners who lack the resources to negotiate deals with AI model providers. It will facilitate seamless price setting and transactions, making it easier for model providers to access fresh content and for content providers to be compensated.

Current Cloudflare customers can access the AI Audit tools from their dashboard. These tools are designed to help website owners regain control over their content and secure fair compensation. The initiative aims to enable a transparent exchange between websites seeking control over their content and AI model providers in need of data sources.