Earlier this week, I sat down and wrote for about 40 minutes about hacked sites, then promptly lost that post forever because my webhost’s database machine was pokey right then. My fault for running Firefox 1.5 on my laptop instead of 2.0. Wordpress 2.1 will also have autosave built in. My breath is definitely bated for the Wordpress autosave feature. I tried not to let my inner snark out, but it’s been a couple days, so I figure I’ll let out just a little bit of grump.
Battelle mentioned a new metrics company / search engine called Compete. Yahoo! provides most of the backend power. What added value does Compete provide? Well, the search results have little icons beside them for things like spyware/phishing, coupon codes, and how popular the site is. How accurate is the spyware/phishing data? Well, for my site, I get a warning exclamation point and this message:

Use caution in providing any personal information or downloading software on mattcutts.com.

I’m not a spyware/phishing site, and my site has been around for over a year. So with a sample size of one, how accurate is that data? Not very.
Okay, how about offers or other deals? Compete says that they’re the “first service to identify available deal codes as you enter a retail website.” I can honestly say that of all the people I’ve talked to, I don’t remember anyone asking “Why don’t you tell me in the search results whether a site offers coupon codes?” I tried a search like [coupon codes] and none of the results triggered as offering promotion codes. So I did the search [fat wallet], and fatwallet.org isn’t listed as having any deals or coupon codes. That doesn’t seem right.
Let’s talk about Compete Picks. The idea behind the feature is for a search, Compete looks at the results and if there’s a lot of post-click page views or other activity on a site, then that site might be a better match. So for some queries, you’ll see “Compete Picks” where Compete thinks the results are especially helpful.
Except: go back to the search [coupon codes]. The Compete Picks are video game cheat codes, which are completely off-topic. And for [fat wallet], the three Compete Picks that I got for that query were “Bbw plumpers, Bbw mature thumb galleries,” “Fat Chicks in Party Hats,” and “Fat Face”:

That’s just obviously wrong.
Am I being too harsh on Compete? I’m guessing so–after all, I’m getting some snarkiness out. But on their launch announcement they claim to be the “first social search solution leveraging community click-stream information to enhance search results.” Really? Because Danny Sullivan wrote about DirectHit in 1998, and I distinctly remember when DirectHit leveraged community click-stream information for HotBot back in the day.
So what’s left? Metrics. That whooshing sound you hear is all the SEOs going to install the Compete Toolbar. Compete claims 2 million users help it compute metrics, and I’d be very surprised if all those participants were toolbar users. Yep, digging through the FAQ, they mention “ISP relationships,” which presumably means that they’re buying user data from ISPs. ISP relationships can be a huge source of metrics bias. For example, some ISPs partner with Yahoo, and users on those ISPs are probably more likely to visit Yahoo. Other ISPs partner with Google. And savvy users that use smaller providers such as Covad or Speakeasy are likely not counted at all.
Because you don’t know which ISPs are selling user data to companies such as Compete or Hitwise, you don’t know what biases are baked into those companies’ metrics–and the metrics companies won’t tell you. Maybe I’m cynical about metrics lately, but Rand Fishkin looked into this recently. He got data from 25 blogs (including mine) and then compared that data with a bunch of metrics services (including Compete). His conclusion? “Based on the evidence we’ve gathered here, it’s safe to say that no external metric, traffic prediction service or ranking system available on the web today provides any accuracy when compared with real numbers. … The sad conclusion is that right now, no publicly available competitive analysis tool we’re aware of provides solid value.” Go read his post–it’s a good one.
I don’t think Compete is even the first Bill Gross search company to look at using click data. This review of Snap.com quotes a previous version of Snap’s about page, and it sounds pretty familiar:

“Instead of just relying on computer algorithms to rank search results, Snap also uses click-stream information from a network of one million Internet users. By recording and processing which Web sites users spend time on, and which sites they quickly leave, Snap improves the likelihood that the search results you get will be the results you’re really looking for.”

You have to ask what Compete brings to the table if Snap.com already tried using clickstream data. Okay, I’ll stop. It’s cool that Compete is trying new ways to inform their users with different icons. I like that they link to their blog from their main page. And it’s nice that they offer free metrics for sites. I just wish I had more confidence in Compete’s metrics, and I wish I knew which ISPs they get data from. To the folks at Compete, I’ll try to be in a more positive mood the next time we meet.


Review: Compete - Read More...