When Numbers Mislead Us Part I: Consider the Source
If human enterprise is the soul of our capital markets, then its body is made up of numbers. It’s no surprise, then, that there is a never-ending supply of charts, graphs and tables seeking to translate all that mind-numbing market-driven data into a meaningful story at an approachable glance.
I was reminded of this by a recent Monevator blog posting: “Too Big to Scale: Long-term Stock Market Returns.” Monevator is a blog about money, written by two, presumably British authors using the anonymous monikers “The Investor” and “The Accumulator.” I don’t always agree with all of the material posted there but I enjoy their discourse.
In “Too Big to Scale,” The Investor steps us through various clever ways the same, long-term market data points can be sliced and diced to present stunningly different conclusions about past market growth and future expected returns. It’s no sleight of hand, it’s a matter of how you crunch and then visualize the data.
The details are relatively dense, so I’m not necessarily recommending you read the entire posting. The point is, well-crafted illustrations can truly speak a thousand helpful words. But like any power tool, if the data is misleadingly or ignorantly applied, it can be downright dangerous, especially if you let erroneous conclusions influence your financial decision-making.
If you are not a financial analyst, how do you know which illustrations are to be trusted? While there is no utterly fail-safe method, here is a rule of thumb that can help you gain a fighting chance at discriminating between helpful and misleading illustrations:
First and foremost, consider the source.
In other words, who is behind the work? With today’s search engines, it’s usually easy to find out more about an author, whether it’s a company or an individual. In fact, if it is hard to find out more, that’s a red flag in itself. Google the author’s name and see what comes up.
- What are her credentials? Does she have a consistent, credible, seasoned platform from which to be a voice of authority on the subject?
- What are his motives? Is the context in which the information is being provided a purely academic forum, a thinly veiled sales pitch, or somewhere in between? Even if someone knows what they’re doing with the data, they may have an agenda they’re seeking to advance that can color – or sometimes taint – the results.
- Has the work been rigorously peer reviewed? No matter how convincing an assertion may seem, if it has not withstood the gamut of respected, peer-reviewed scrutiny, it’s best to take it for what it is: one person’s or one firm’s potentially fallible opinions.
Speaking of peer review, another way we assist our clients is helping them sift through the mountains of financial information in the popular press and elsewhere. By working together with our clients as well as our network of subject matter experts, we stand the best chance of collectively separating visual facts from fallacy.
Still not convinced that you can’t always trust your own eyes? In my next blog post, I’ll review some of the many ways charts and graphs can confuse or deceive.