Here’s a joke I hate: 84% of statistics are made up on the spot.
I hate it because it points out a problem that bothers me, a marketing writer, a lot.
Every marketer loves a good stat, especially when it undeniably supports the thing they’re marketing. Once found, it’s irresistible to fold it into some copy, somewhere.
Not all stats are made equal, though. There’s tons of information out there, and not all of it can be taken at face value. Statistics can be biased, outdated, misinterpreted, misleading or simply incorrect.
For example, here’s a fantastic pair of stats:
“Content marketing costs 62% less than traditional marketing and generates 3 times as many leads.”
I’d bet that every content marketer loves these stats. I’ve come to this conclusion not only because it’s obvious, but also because I’ve encountered it on tons of sites and articles that discuss the virtues of content marketing.
The problem with this stat is there is no source. Sure, plenty of sites have attributed some other source, but after some searching, I cannot confidently source it.
Humor me, and follow me down this rabbit hole.
A Google search of that statistic brings up plenty of trustworthy sources, including Neil Patel, Content Marketing Institute, WordStream and HubSpot, among others.
Most of them attribute this set of stats to this infographic from Demand Metric (the others have no source, or cite one another). There is no date on the graphic or page, so I don’t know how old it is. Of the articles I found it in, the earliest is a piece from a former Forbes contributor, published in October 2016.
There are four sources listed at the bottom of the infographic. Three of them, all list articles, result in 404 errors. One of them is dated 2012. The sole functional URL goes to Demand Metric’s own resource center. From what I can see, the only resource there that cites that stat is the very infographic I’m trying to fact-check.
My verdict: This statistic is almost definitely outdated, and virtually unprovable. In my opinion, it’s irresponsible to use it.
Common statistic sourcing problems
I come across situations like these a lot. I actually have a list called “I hate these stats,” and content marketing costing 62% less is at the top of it (true story).
There are numerous situations that deem certain statistics unusable, in my opinion. I’ve organized them into the following categories:
Offender No. 1: Revolving statistics
A revolving statistic (a term I made up) is a stat that is always relevant, but changes regularly.
My favorite example is that email marketing has a return on investment of $42:1. This totally usable stat comes from Litmus’s own research from 2019. The year before, email ROI was $38:1.
The problem here isn’t the actual stat. It’s when outdated versions of it are used and believed to be current, relevant information.
The key to a revolving stat is finding the most recent version of it. Unfortunately, sometimes annual reports and surveys get discontinued, or the focuses of these reports change to exclude your favorite revolving stats (what, you don’t have these?).
Offender No. 2: Inconclusive statistics
An inconclusive stat is one that has no discernible source, or the source is so weak, it’d be a stretch to consider it valid. Like this one: Humans process visuals 60,000 times faster than text.
In my search for a source on this one, I found what might be a kindred spirit. In 2012, blogger Alan Levine made it his mission to get to the bottom of where this stat came from. He couldn’t. He traced it back to a 2001 3M brochure, as well as a 1982 Business Week ad, with no supporting research to back it up.
As far as I can tell, Levine is still on the hunt for evidence that this is true, and he’s offering a cash reward for anyone who can prove it.
Offender No. 3: False statistics
False statistics are based on false premises. Like this: “65% of the population are visual learners.”
Here’s why that’s inaccurate: There aren’t actually different learning styles. This is a myth that has been debunked for years now, through numerous studies.
In this example, a bad stat doesn’t just make your marketing material misinformed. This myth has guided teaching curriculums, encouraged students to focus on specific study methods and basically put people into arbitrary boxes. The American Psychological Association even said the myth may be holding some learners back.
The lesson: Bad stats create problems. Don’t elongate their lives by perpetuating them through your marketing materials. That’s selfish and irresponsible.
Offender No. 4: Citing circles
Citing circles are, for me, the most frustrating sourcing problem because they are pointless time-wasters. But it’s hard to tell you’re in a citing circle until you get to the end – or back to the beginning – of it.
The rabbit hole I took you down earlier is an example of one. Ultimately, the most commonly cited source of that fact could only lead back to the original infographic.
Offender No. 5: Dead-end sources
A dead-end source is one where you finally reach the end of the citation line, and there’s still no conclusive evidence that the stat isn’t completely fabricated. For example, “whiteboard animations result in twice as many sales” (the “as what” is different, depending on the source).
A Google search shows Infographic World as having the featured snippet for this stat, as well as the first result (lucky guys). Infographic World links to an extremely short 2016 LinkedIn Pulse article with absolutely zero context. Here it is in its entirety:
The single commenter on the article asks for sources, with no response.
The twice-as-many-sales part is often tied to whiteboard animations being 3 times more likely to be shared on social media. Both data points are also found in an article from another former Forbes contributor, which attributed both stats to Carla Clark. She’s a good candidate to believe, as she’s a neuroscientist who advises for The Draw Shop, a whiteboard animation company.
Some digging leads to research done by Professor Richard Wiseman, which showed that participants were 66% more likely to say they’d share a whiteboard animation than a talking-head video. Clark actually references this study in an episode of Roger Dooley’s Brainfluence Podcast in 2016. This is a completely different stat than the one that’s perpetuated, and there’s no mention of influence over sales.
I also found this eBook from 2015, which has both the social shares and likelihood of purchase stats, and is attributed to Clark. However, there are no details about the research behind the claims, I can’t find this source anywhere except for the actual PDF showing up in SERPs, and why would Clark reference Wiseman’s research from three years prior if she did her own just one year before? I can’t answer these questions, but I also can’t in good faith tell someone that a whiteboard animation will generate 2 times more sales than another type of video or content format. If you ask me, neither can you.
How to avoid bad stats
We each have a responsibility to only use good stats, and to ignore the bad ones. Here are 3 tips on avoiding bad stats:
1. Ignore statistic listicles
I would recommend that any responsible marketing writer avoid stat list articles. You know the ones titled “87 incredible statistics to know about visual marketing in 2020” or something of the sort.
If you publish a stat list article: Cite your sources, provide context and keep the piece updated. Check out this article about voice search stats to see how we’ve done this on our blog.
If you find yourself reading a listicle, consider the publish date, whether it’s been updated recently and what the sources are. And this brings up my next point:
2. Do your research
If you look at a list article, research the statistic before you use it. Many of those list articles will have a handful of usable stats; you just have to do the work to weed them out.
3. Don’t cave into temptation
When you find a stat that sounds incredible, you’ll want to weave it into your content. Sometimes, it’s still tempting to believe and use the stat, even after you’ve fallen to the bottom of the rabbit hole and the fact still seems inconclusive. Fight the temptation. Only use facts that you can source in good conscience.
Tips on using statistics responsibly
Like I mentioned before, every marketer loves a good stat. This is for good reason: Numbers draw readers in, they support your claims and they generally just make good marketing copy.
Don’t let all the bad stats out there scare you away. You can find facts that are convincing and useful. Here are 4 tips for using stats responsibly:
1. Identify your favorite sources
Just like I have a list called “I hate these stats,” I also have one called “My favorite stats.” It has links to the most recent versions of reputable reports. I turn to this list often when I need a good data point.
I recommend that you also identify your winning sources, and make an effort to find updated versions of each report as they’re published (or as you need them). Here are a few of the sources I have on my list:
- B2B Content Marketing 2020 – Content Marketing Institute.
- The State of Video Marketing 2020 – Wyzowl.
- 2020 Social Media Marketing Industry Report – Social Media Examiner.
2. Ask whether a figure is essential
Before you use a stat, consider how you’d make your point without it. If you find yourself falling into a sourcing rabbit hole, it might be more worth your time to let it go and use your creative writing skills to produce convincing copy instead of leaning on a statistic.
3. Use original research
Original research is an excellent way to produce your very own data, and to demonstrate your thought leadership.
Depending on the resources at your disposal, your reach, the engagement of your audience and a number of other factors, your original research could be big projects or small surveys. Either way, you’ll have a piece of original data that others will want to cite (as a bonus, you’ll probably generate some backlinks).
Our version of this is Brafton Labs. We’ve created surveys on SurveyMonkey, sent them out to our newsletter subscribers and gleaned some interesting information from our readers. Not only were we able to gather great information to turn into original research pieces, but we also learned more about our subscribers.
4. Cite your sources, and provide context when necessary
This should go without saying, but it’s absolutely worth repeating: Always, always cite your sources.
Additionally, when it’s relevant, provide context into the stat you’re using. The person or group that conducted the research, time frame, sample size and more may all be relevant. Don’t extrapolate on a data point to make it fit your narrative. If you have to perform editorial gymnastics to get it to fit, you’d be better off finding a new way to say what you’re trying to say.
Statistics are wonderful things, but it’s important to not get too caught up in their allure. Numbers make for convincing marketing material, but a good marketer can make the case for their products/services/brand using only responsibly sourced stats and some creativity.
Finally: Stop saying content marketing costs 62% less than traditional marketing. You can’t possibly know that.
