Google Indexing Subscription Content

BetaNews has learned that Google is testing a premium service that will open up mainstream access to the "Deep Web," allowing webmasters with restricted or subscription content to let in Google and provide the masses with free previews.

The service is being tested server-side with a small number of sites that are under strict confidentiality agreements.

Premium content will be indexed and tagged as paid, and will be displayed in a special content area on the right side of Google's search results underneath the AdSense advertising links.

Some paid articles are "first click free" and will appear within Google's "natural" aggregated search results - if the participating publication decides that the content should be accessible when Google is the referrer.

Yahoo announced its own Yahoo! Search Subscription this month in an effort to surface normally restricted content. "This enables consumer to access their personal subscription content in one place via Yahoo! Search," a Yahoo spokesperson told BetaNews.

In contrast to Yahoo, Google has kept its efforts shrouded in secrecy. "We are under a strict confidentiality agreement. We've been told several times by Google that we are not allowed to share this information with anyone outside the involved organizations," a source close to testing told BetaNews.

Sources said that Google's Premium service creates a full sitemap of the paid site and then clears the Google IP block and Spider agent to show protected content. Publishers will be made aware that there was a Google referral.

"It doesn't surprise me that they are testing this kind of thing," Search Engine Watch editor Danny Sullivan told BetaNews. "Google has already had limited agreements with content providers to get into protected areas in one way or another. This is a natural extension of that, but was probably spurred along by Yahoo."

A Google spokesperson said the company doesn't typically comment on industry "speculation."