How to cheat with statistics: CNN ad

In today’s New York Post (p. 31) runs a full page add by CNN. The ad itself looks like a PowerPoint presentation, that is, a dull layout driven by bullet points. Here are the first three:

1. #1 Most Watched Cable News Network Across Both DNC and RNC Conventions for P25-54, P18-49, & P18-34
2. #1 Most Watched Cable News Network at 10PM Across Both Conventions
3. #1 News and Information Site During Both Conventions-CNN.com

In the first bullet, what is that odd “P25-54”? What in the world could that be? The ad, even in the fine print, nowhere says. But we can guess is means “people aged 25 to 54”. OK, so people aged 25 to 54—a good slice of people—according to “sources”, liked CNN. Sounds impressive, but there are two misleading elements.

The first is that it was the most watched “cable news network”, which means that the “non-cable news networks” might have been watched by more people. We can guess that this is true else the ad would have touted that CNN was the most watched network period.

The second problem is the bizarre way they sliced the age groups. The 18-49 groups entirely contains the 18-34 group, does it not? So why mention the smaller-sized group? Does it mean that the 35-49 years olds did not prefer CNN? But the 35-39 year olds are certainly among the 25-54 years olds, which was the first group mentioned.

There is no making sense of any of this except by supposing that CNN scrounged through the data to find any hint of subgroups that supported their “#1” contention. Experience shows that you can do this for any statistical analysis, which is why so many rightly suspect whatever statisticians have to say.

The second bullet point is just as screwy. How many different networks did they compare anyway? The barely readable small print says “CNN, FNC, and MSNBC.” So they only had two competitors, the last of which, MSNBC, has always struggled for viewership. Coming in number 1 in a few categories with only one real competitors is not a laudable achievement.

But it also means that CNN must have lost, probably to Fox, in the 7p-8p slot, the 8p-9p slot, the 9p-10p slot, and the 11p-12a slot, where are 4 out of the 5 slots the small print says the “sources” checked. Losing 80% of the time hardly makes you number 1.

The third bullet is more tepid. The “source” says “Information Site” means “Current events and Global News Category.” We have no idea how many other sites were compared against CNN, nor how many other categories—say Analysis, Opinion, Politics, and so on—were checked.

Still, for cheating, the third bullet is best, because it’s a rare person will be pause much over the claim, nor will most browse the small print.

In any case, CNN should just have presented their fourth bullet, which was

• #1 Most Trusted & Credible Name in News

This bullet is so vague it can mean anything. It’s crafted so that readers can take any meaning they like from it. Most people will be left with a dull sense of the importance of CNN.

This ad, while fairly misleading, only earns a 4 (out of 10) on the Briggs Statistical Deception Scale.

For the teachers out there, these ads often make good homework problems for students. Chopping up an ad into component parts and reading it critically is always great fun for the students. Especially when you find deceitful ads.