Publishers Look To Withhold Content As The Open Web Bargain Is Breaking

According to the Wall Street Journal, major news and media publishers are considering blocking Google crawlers, requiring registration, and renegotiating licensing agreements because AI-generated answers are using their reporting while sending fewer readers back to their websites.

For more than two decades, publishers accepted a basic bargain with Google. They allowed the search engine to crawl and index their reporting. Google then displayed links to that reporting and sent readers back to the publishers’ websites. That traffic is core to the economics of the open web. 

Publishers specifically used that traffic to generate advertising revenue, sell subscriptions, and build audiences. Now, some of the world’s largest publishers are questioning whether the bargain still makes sense.

USA Today Co., Politico, Reuters, The Economist, Reddit, People Inc., and other publishers are evaluating how much access Google should have to their content as AI-generated search answers reduce referral traffic. Some are considering blocking crawlers, placing more content behind registration walls, or renegotiating licensing agreements with Google and other AI companies.

Publishers Are Considering Withholding Their Content

Traditional Google search gave users a list of websites that might answer their questions. AI Overviews and AI Mode now synthesize information from multiple sources and provide a finished answer directly on the search results page. The sources may still be cited, but the user often has less reason to visit them.

The impact of Google’s AI Overviews is beginning to appear in publisher traffic figures. USA Today Co. reportedly experienced a nearly 50% decline in Google referrals to its national publication between June 2025 and June 2026. Politico’s Google traffic declined by approximately 23% during the same period. CNN, Business Insider, and other media companies have also experienced considerable losses.

Digital Content Next surveyed premium news and entertainment publishers in 2025 and found that most reported Google referral declines of between 1% and 25% following the expansion of AI Overviews. Median year-over-year Google search referral traffic among participants fell by 10% during the period studied.

Academic research has found similar effects. A 2026 study comparing more than 161,000 Wikipedia articles across multiple languages estimated that exposure to Google AI Overviews reduced daily traffic to the English-language articles by approximately 15%.

Another study found that AI Overviews appeared in almost 65% of question-based searches in its dataset. More than half of the pages cited by the summaries carried display advertising, meaning the publisher could lose revenue when Google answered the question without generating a click.

Killing Traffic Threatens More Than Publisher Revenue

It would be easy to treat this as a fight between media companies and Google. However, the implications are much larger. Quality information costs money to produce. Publishers pay:  Journalists, researchers, editors, photographers, legal teams, data analysts, and subject-matter experts. Industry publications invest in specialized reporting that may serve a relatively small but commercially important audience. Referral traffic helps fund that work.

When an answer engine uses the resulting information without sending the reader to the source, it weakens the economic incentive to produce the next investigation, analysis, or dataset.

Recent research describes this as a fundamental change to the web’s referral bargain. One July 2026 study found that ChatGPT produced an outbound click in only 5.2% of the conversation sessions examined. The researchers concluded that AI search can satisfy information needs within the platform while weakening the connection between search, referral traffic, and online content production.

The likely result is not that publishers stop creating content entirely. More may place their strongest work behind paywalls that require registration, limit crawler access, or reserve data for licensed partners. The downside is this creates a more restricted and fragmented information market. Users might receive convenient summaries, but fewer of the sources behind those summaries would remain freely available.

The Market Is Moving Toward Greater Control

Publishers are already taking action. Reuters and Time have reportedly adopted systems that block AI crawlers by default and allow only approved bots through controlled lists. Earlier research from the Reuters Institute found that approximately half of the tracked news websites blocked OpenAI’s crawler, while around 40% blocked Google’s AI crawler and Common Crawl.  

Cloudflare has also introduced tools that let website owners block AI crawlers, permit selected crawlers, or charge companies each time they access content. Beginning September 15, 2026, Cloudflare plans to block certain mixed-use crawlers from advertising-supported pages by default unless technology companies clearly separate traditional search crawling from AI training and agent activity.

Regulators are now intervening as well. The United Kingdom’s Competition and Markets Authority now requires Google to give publishers a way to opt out of having their content used in AI search features without removing the content from traditional Google search results. These developments suggest that unrestricted access to online content may no longer remain the default.

Why This Matters For B2B Marketing

B2B companies are not traditional publishers, but they rely on the same online traffic system. They publish technical guides, original research, and executive commentary. Search engines index the content. Buyers discover it, visit the website, and enter the company’s marketing and sales ecosystem. AI search weakens the connection between visibility and traffic.

A B2B company might provide the expertise behind an AI-generated answer without receiving the website visit, report download, or sales inquiry that previously justified the investment. 

Now this does not mean companies should hide all their content from AI or lock it behind registration walls. Buyers and answer engines still need enough public information to understand what the company does, who it serves, and why its expertise matters. It does mean B2B marketers should stop assuming that every valuable insight belongs in a completely open article.

Foundational content such as service pages, product information, and introductory guidance should remain easy to access. It helps buyers and machines understand the company. However, proprietary research, market intelligence, and original frameworks may warrant a different approach. A company could publish the headline findings openly while reserving the complete analysis for registered users, subscribers, or licensed partners.

Companies should also invest more heavily in channels they control. Subscriber databases, customer communities, and proprietary research programs that create direct relationships that do not depend entirely on Google sending the next visitor. B2B marketers now need to decide which content should generate visibility, which content should generate a relationship, and which intellectual property is valuable enough to protect.

Build A Content Strategy That Does More Than Generate Traffic

Search will remain an important part of B2B marketing, but companies can no longer assume that visibility will automatically lead to website traffic, audience growth, or sales opportunities. As AI-generated answers absorb more information directly into search results, marketers need a clearer strategy for deciding what to publish openly, what to gate, and how to build direct relationships with the people who value their expertise.

Borrowed Pen helps B2B companies develop content strategies built for this new environment. We create proprietary research, executive thought leadership, newsletters, gated reports, technical content, and owned-audience systems that help companies earn visibility without giving away all their value. We can help you decide which ideas should attract attention, which assets should generate leads, and which intellectual property deserves greater protection.

Connect with Borrowed Pen to build a content strategy that supports search visibility, audience ownership, and long-term demand.