Cloudflare to prevent artificial intelligence companies from bulldozing content without approval

Internet company Cloudflare It will begin to prevent artificial intelligence crawling from accessing content without the permit or compensation of the site, in a move that can significantly affect the ability of artificial intelligence developers to train their models.

Starting on Tuesday, every new web field will be questioned to Cloudflare if they want to allow AI’s crawls, which effectively gives them the ability to prevent robots from bulldozing data from their websites.

Cloudflare is the so -called content connection network, or CDN. It helps companies provide content and applications online faster by storing data closer to the final users. They play a An important role Ensure that people can access the web content smoothly every day.

Almost 16 % of global traffic on the Internet goes directly through Cloudflare’s CDN, estimated in 2023 a report.

“AI Crawles has ridiculously ridiculously. Our goal is to restore power in the hands of creators, with artificial intelligence companies not helping innovation,” Matthew Prince, co -founder and CEO of Cloudflare, said in a statement on Tuesday.

He added: “It comes to protecting the future of free and vital internet with a new model that works for everyone.”

Artificial intelligence crawls are automatic robots designed to extract large quantities of data from web sites, databases and other information sources to train large language models such as Openai and Google.

While the Internet previously rewarded creators by directing users to the original websites, according to Cloudflare, both AI Crawles break this model by collecting text, articles and images to create responses to inquiries in a way that users do not need to visit the original source.

The company adds that this company deprives vital traffic publishers, and therefore, revenue from advertising online.

On Tuesday, a step on a cloudflare tool is based on September last year, which gave publishers the ability to prevent artificial intelligence crawling with one click. Now, the company is running forward by making this default for all its web sites.

Openai says she refused to participate when Cloudflare inspected her plan to prevent artificial intelligence crawling on the basis that the content delivery network adds a mediator to the system.

The Microsoft -backed AI Laboratory stressed his role as a pioneer in using Robots.txt, a set of software instructions that prevent automatic drainage of web data, and said that its crawling respects the preferences of the publisher.

“AI’s crawls are usually seen as more invasive and selective when it comes to the data they are making consumers. They have been accused of overwhelming websites and greatly influencing the user experience,” Matthew Holman, a partner in the UK law firm, told CNBC.

He added: “If this is effective, the development will hinder the capacity of AI Chatbots to harvest data for training and research.” “This is likely to lead to a short -term effect on training on the artificial intelligence model and can, in the long run, the feasibility of models.”

He watches: Artificial intelligence engineers in high demand – but what is the job really?