“Ninety eight percent of statistics are made up.”
Kim Harrington
“Statistics can be made to prove anything, even the truth.”
Anonymous
“Facts are stubborn things, but statistics are pliable.”
Mark Twain
A couple years ago I was involved with a council that was discussing water allocations for new development in an area that had experienced several years of below normal rainfall. Another member of the council provided precipitation statistics over the past twenty-five years. We in turn used those statistics in our discussions to work towards a solution. Later in that process I happened to pull the same statistics, but for some reason I pulled thirty years of data instead of twenty-five. The story changed dramatically!
I don’t believe there was any ill intent in the first set of data provided, and it was only chance that I ended up with a more complete data set. But even though innocent, this experience reiterated the danger in trusting statistics.
Even while I am a total statistical wonk, I used to use quotes such as those leading off this Insight on a regular basis. Stats lie, everyone knows it, right? I am no longer in that camp. Statistics are numbers – marks on a paper or a computer screen. They don’t have the ability to lie or provide effort towards manipulation. They only exist as one mundane thing: statistics. It is we, as humans, that take those statistics and use them for a purpose other than a number on a page. The purest use of statistics is to help define or solve problems. At the other end of the spectrum is the use of statistics to purposely mislead people from the truth. The area in between is filled with using statistics as part of storytelling or persuasion. The nobility of such use varies, depending on where in the spectrum between problem solving and outright dishonesty the use of the statistics lands.
It is not a surprise to anyone reading this Insight that politics are a playground on the most dishonest end of the spectrum. The Trump administration was often caught in partial or mistruths. More recently the Biden administration has used partial statistics to blame inflation on the war in Ukraine and to brag about this year’s fiscal spending, resulting in the largest deficit reduction in history. The Altus Insight in July 2021, and again in December 2021, called out the 2021 stimulus as the tinderbox of inflation well before Russia invaded Ukraine, and inflations in January (before the invasion) were already at thirty-year highs. And anything other than a massive reduction in the deficit would be a governmental malfeasance since 2020 and 2021 were by far the largest deficits in the history of the country – both specifically tied to COVID stimulus. With no additional COVID stimulus, and the administration’s other spending plans torpedoed in the Senate, there is no option but for spending to be reduced dramatically.
Other areas are less obvious but still dangerous. The Federal Reserve often provides guidance using cherry picked data. News outlet reporting can be almost criminal at times. But the truth is, we are all individually human, meaning we not only make mistakes (lots of them in my case), but we also have biases around desired outcomes. The more intellectually honest we are, the better we will be able to discern whether the benign statistics truly support or undermine any position we take. But even when intellectually honest, in cases where the statistics don’t obviously provide clear direction, it can be oh so easy to use the statistics at hand to support the argument we want to gain credence.
Many people don’t understand statistics or analysis and can make honest mistakes in applying statistics in analysis or decision making. Even those that have some level of experience in using statistics can easily be sidetracked.
I can use myself as a recent example. I was traveling shortly after the federal airplane/airport mask mandate was lifted. I was told that a certain age demographic was dramatically more likely to wear masks than other age demographics, so I decided to do my own observational tally in the airports to see if what I observed supported the thesis that had been shared with me. My observations overwhelmingly supported the thesis. Until I realized that I had been tallying wearers in that demographic as a percentage of total wearers, which was completely inaccurate. I wasn’t paying any attention to the total number of people in that age group dataset, so in theory the mask wearers could have been wearing masks at a LOWER percentage than the population at large, even if there were more wearing masks in total. The statistics (number of mask wearers within the age group as a portion of total mask wearers) were accurate, but not for what I was trying to observe (the percentage of mask wearers in the age group versus mask wearers in all other age groups). Since that trip I have taken other trips and been more careful in my observations. I believe the original thesis provided to me is indeed accurate, but I was still sobered by how easy it was to misuse statistics.
At Altus, we use statistics all the time. Choosing our preferred markets, setting lease rates, agreeing to sales prices…so much of what we do as a real estate investment company is based on interpreting statistics. If I could so easily make an analytical structure error on something as simple and silly as counting mask wearing, where else might I (and Altus as an organization) be making similar mistakes?
And if we want to avoid those mistakes, what are we to do? I have thought a lot about this since my own analytical error. I don’t know if I have all the answers, or even any of the answers, but my thinking has coalesced around two key items:
- If using statistics to solve or define a problem, or to verify the accuracy of someone else’s premise, it is important to define exactly what is trying to be determined. Are we trying to determine percentages (most useful for comparison purposes) or absolutes (most useful for determining levels of impact)? If percentages are the goal, are the inputs into the percentage equation the correct inputs to provide the output we need for our analysis?
- When using statistics provided by others it is important to clearly understand the motivations of whomever is providing the statistics. Politics live in the world of mistruths. I am skeptical of the accuracy of the use of any statistics provided in politics. Trade associations and market participants have better educated and smaller focus groups, so they have to be more accurate in their presentation of data. Yet they still have goals they are trying to achieve, so the data interpretation must be taken with a grain of salt. I find problem solvers (true problem solvers, not problem provers) to be much more accurate and dependable, but still not immune to errors of interpretation. The larger the importance of the outcome (usually financial) to the problem solver, the more likely any statistical interpretation is to be accurate. If a person or organization is making large investment decisions on the interpretation of data, it is darn important that interpretation is correct. Otherwise, careers, livelihoods, and retirements are at risk. Observing the action of well-run companies is quite illustrative in this regard. Not just listening/reading the quarterly investor calls, but watching the specific actions being taken. Words generally support the stock price. Actions usually support the business viability.
Statistics don’t lie. Humans lie. And humans make mistakes. Being aware of the information around us and the motivations of those providing the information can greatly help us reduce our exposure to incorrectly or imperfectly used statistics that can cause us to make bad decisions.
Happy Investing !
About the Author: Forrest Jinks is CEO of Altus Equity Group Inc and a licensed real estate broker. Forrest has decades of experience as principal in a variety of alternative investment segments including real estate (residential rehab, in-fill development, multi-family, office and retail), debt, and small business start-up (online marketing and site retail). He can be reached at fjinks@altusequity.com.