The Curious Case Of Yahoo's App Reviews

Summly founder Nick D'Aloisio, in an interview with Yahoo's David Pogue, touted his new app's positive reviews. But who's writing them?

Yesterday, at the Consumer Electronics Show (CES), Yahoo announced two new products: Yahoo News Digest and The former is the result of last year's acquisition of Summly, a news summarization startup founded by a teenager, Nick D'Aloisio; the latter is a new tech site headed up by David Pogue, formerly of the New York Times.

The two men met on stage today, ostensibly to talk about D'Aloisio and his new app. The optics were admittedly strange — a Yahoo employee interviewing a Yahoo employee — and Pogue joked about them throughout. (Pogue was scheduled to interview Twitter co-founder Biz Stone, who had to drop out for emergency reasons.)

Regardless, this exchange, which occurs at the 19-minute mark, stood out.

Pogue: [News Digest] came out yesterday, and, how's it doing?

D'Aloisio: It's doing great. It's actually now the number-one app on the News category in the U.S. store, which is awesome, and it's four and a half out of five stars at the moment, so —

Pogue: Really!

D'Aloisio: Yeah, it's early days because we've only been on the store for about 12 hours or so, but it's good to see that the initial feedback has been strong.

Pogue: Four and a half stars...

D'Aloisio: It's a good start.

Pogue: I mean, that's hard to get.

D'Aloisio: Yeah.

Pogue: People are snarky out there.

The app, as of writing, is No. 1 in the News category — a rating decided not just by raw volume of downloads, but by changes in popularity (among other things). And D'Aloisio's claim that the app is rated at 4.5 stars is also true. But the substance of the reviews seems, even at first glance, to be questionable.

The app, it's worth stating up front, is competently designed. The interface is novel and interesting, and it doesn't crash or stutter. The summaries are serviceable, if dry and sometimes repetitive. They are sometimes awkward, but never nonsensical. It is not inconceivable that a normal person could like this app. But it seems unlikely that this theoretical person would come up with a line like this:

Glancing at the associated user profile, a pattern emerges:

And going through the profiles of other effusive reviewers, we see the same thing. A large share of this app's positive reviews are written by people who almost exclusively write reviews of Yahoo products, or people who have never written another review. At least one appears to be listed publicly as a Yahoo employee.

This doesn't appear to be a case of Yahoo buying fake reviews. While fake reviews are rampant in the App Store, they're also fairly easy to spot. What this looks like is Yahoo employees and/or D'Aloisio associates earnestly trying to give a new app a boost. Yahoo has about 12,300 employees; this new app has been reviewed about 250 times. Even without explicit guidance, a 12,000-strong army of enthusiastic downloaders could overwhelm an App Store category rankings for a few days, and boost an app's review score by a few stars. The reviews that appear to be organic are, in aggregate, mixed but skew somewhat positive. A Yahoo representative tells BuzzFeed, "Employees were not instructed to write reviews."

The fact that Yahoo employees might be reviewing their company's own app, in itself, isn't a scandal. But it's notable that both D'Aloisio, who knows how the App Store works and clearly looked at his reviews, and Pogue, a veteran journalist, decided to ignore it.

Update: One five-star iTunes reviewer, ultramookie, appears to be Steve Kong, a recent member of Yahoo's Mobile and Emerging Products team. He recently wrote a Tumblr post praising the new Digest app. His Twitter handle matches the first review listed for Yahoo News Digest when you filter by "most favorable." Over at TechCrunch, Matthew Panzarino also pegged reviews to Yahoo employees Jonathan Raspaud and Rahul Aneja.