Publishers Resist AI’s Insatiable Demand for News Training Data

The Battle Lines in the AI Data Scraping Controversy

In the rapidly accelerating AI revolution, iconic tech firms, Google, Meta, Anthropic, and OpenAI, compete rigorously for supremacy. Their insatiable desire for high-quality training data has led to dubious mass scraping of news articles. With what’s being described as the “largest theft in the United States,” journalism takes a stand against this infringement. The unfolding drama reveals intricate details.

Guarding Journalism Against AI Scraping

Recognizing the threat to journalism’s integrity and publishing businesses, media outfits, such as The New York Times, Graham Media Group, The Guardian, Hearst, and Hubbard Broadcasting, are taking precautions. Prominent AI chatbots, including Google’s Gemini and OpenAI’s ChatGPT, have been banned from mining their sites. The list of vigilant media houses continues to expand.

The alarm isn’t unfounded; there is a fear that unregulated use of news articles for training chatbots will amplify misinformation and synthetic content online. Vincent Berthier of Reporters Without Borders elaborates on this issue, warning about the potential for misusing AI models for nefarious purposes.

Media’s Defense Strategies Against AI Data Scraping

Media organizations are not standing idle; they are adopting diverse methods to safeguard their content against AI scraping. First, they have revised their terms of service to outlaw AI scraping. The New York Times led the way in August 2023. Although not foolproof, clarifying the terms against unauthorized data scraping provides a legal foundation.

Additionally, many are blocking web crawlers associated with AI chatbots using their robots.txt files. Despite its limitations, blocking these crawlers is a necessary initial protection line. OpenAI, for instance, only started respecting robots.txt rules in August 2023 after initially disregarding them.

Licensing Content and Developing Own AI Models

Another defense strategy is selling access to their content to AI companies for training data. The Financial Times and Axel Springer (owner of Business Insider) have opened this avenue. Startups like Dappier are also bridging the gap with a marketplace for publishers to license their content under their terms.

On the other hand, newsrooms are training LLMs on their content. CIO of the Local Media Association, Frank Mungeam, acknowledges that this approach not only preserves but also leverages the newsroom’s IP value.

The Legal Angle and the Role of Synthetic Data

To protect intellectual property, organizations like The New York Times have filed lawsuits against OpenAI and Google accusing them of illegal harvesting. Both AI giants maintain that their practices are fair use.

As publishers mount opposition to web scraping, AI companies may seek alternative sources of data. Synthetic data, generated by AI, is one option, although skepticism about its efficacy abounds.

Collaboration as the Path Forward

While legal battles brew, collaboration between media organizations and AI companies is seen as a more feasible solution. By licensing access to their content for training data, news organizations can protect their IP and create new revenue streams. In conclusion, a harmonious relationship between journalism and AI is crucial for their coexistence and mutual growth.