Google's Mediapartner Bot Index Creep Indicate AdSense Publisher SERPs Advantage?
We talked more this morning at the MarketSmart Interactive office about Greg Boser's recent findings that data from the AdSense bot's crawls actually makes its way into Google's main cache and index.

I had a bit of brain flash this morning while discussing this with Tansy OBryant: because AdSense publishers' pages are indexed more frequently than other content sites' pages does this indicate a SERPs advantage?

I followed up by chatting with Jon Revill some more.

We surmised that COMMENTS added to a given piece of content (whether a blog post, forum thread, comments enabled on an article, even wikis...) would create a reason for the AdSense bot to return regularly, to make sure that Google's ads remain relevant.

After all, a highly discussed forum thread or blog post can get lots of traffic - and more comments - for a long time.

So if the AdSense bot's finding FRESH, RELEVANT CONTENT (in the form of continuing conversations) it makes sense that Google would try to minimize work by including what the AdSense bot finds in Google's main cache and index.

The MAJOR ISSUE (if Boser's data indeed PROVES that there's creep from the AdSense index to the main index):

AdSense publishers' content is indexed and cached more frequently BECAUSE THEY ARE ADSENSE PUBLISHERS - not because they have demonstrated some measurable value to their community (which would incur the freshbot visit).

It seems that Google's SERPs will display fresher content from AdSense publishers that will align with emerging search terms, and that AdSense publishers will get more traffic... and Google will get more clicks on its AdSense ads.

This creep between the AdSense index and the main Google index creates a conflict of interest between pure relevance and Google's immediate financial return.

Google's always held that it concentrates on using pure relevance to drive financial return. Boser's findings - as I understand them and have framed them here - indicate otherwise.

I would love to hear YOUR thoughts 919-433-3139. Or email: selowdown@gmail.com.

UPDATE: Matt Cutts confirms AdSense media bot in natural search index
Garrett French - MarketSmart Interactive | permalink | +marktd |