“What happened to the polls?” next to “What now?” was likely the most frequently asked question as soon as the first results started to roll in back on Election Day. The results shocked everyone as the polls, both national and state, had Hillary consistently pegged to take her seat in the Oval Office. So what went wrong? How could the polls be so far off? What’s the likelihood of getting Canadian citizenship before January? All valid questions but let’s focus on the first two. What’s fascinating about this is that so many polls, or let’s call them what they are, forecasts, were off the mark. All the different polling methodologies used by all the different polling organizations missed calling what was the biggest electoral spectacle in all of U.S. history. There are a number of theories floating around but all would agree the future is tough to predict. This is something supply chain practitioners live every day and not just during the election process. Like the pollsters, demand planners in particular, model the future. However, many planners are well versed in expecting the unexpected, considering multiple variables, modeling and comparing multiple scenario outcomes and quickly adjusting plans when needed. Unfortunately for the popular vote there’s nothing we can do about 2016 but here are three lessons for the pollsters come 2020. The lessons are actually borrowed from a post written by my colleague, Trevor Miles, from 2013 called Truth, Lies and Statistical Modelling in Supply Chain. I would recommend everyone, especially the pollsters, check out this three-part blog series where Trevor concludes “we model all of our manufacturing and supply chain systems using deterministic models, when in fact everything around us is stochastic.” You’ll quickly understand what he means when we get into the lessons.
Lesson 1: “Most systems that run our supply chain use precise mathematical models that assume complete identification and prediction of variables (Deterministic), yet we operate in a highly unpredictable environment (Stochastic).”
Statisticians like Nate Silver did a great job predicting the number of electoral votes President Obama would pick up in 2012 and this may have given the pollsters an edge of confidence and even arrogance going into the 2016 election. Taking a supply chain perspective, planners know that even if you were accurate once, future demand patterns, in this case an appetite for one side or the other of the political fence, remain highly volatile. Supply chains deal every day with new product proliferation, emerging technologies and the challenges of promoting dated supply in order to avoid excess and obsolete inventories. Supply chains with the ability to build multiple forecast streams, experiment with different demand patterns, and quickly sense and respond to variation in demand become the leaders in their fields. Imagine if the pollsters could simulate in seconds different demand scenarios and at least start the “what if” conversation. What if Obama’s supporters don’t’ show up at the polls for Hillary? How do we model respondents who don’t know how they’re going to vote? Can we model optimistic and pessimistic views for consideration? What if respondents don’t want to admit they’re voting for a “socially undesirable” candidate? Supply chain folks simply call this unexpected demand and are getting really good at managing it. The lesson is to take the unpredictable seriously.
Lesson 2: “Many supply chain models act as if events, over time, are evenly distributed (Normal Distribution), yet most incidents and their respective magnitude (e.g. demand spikes, supply delays) are highly random (LogNormal Distribution).”
Another factor the pollsters are now losing sleep over is what they call nonresponse bias. In this case people for one reason or another just don’t respond to surveys even though there’s equal opportunity across all parts of the electorate. As Trump has said, “I love the uneducated.” Some have speculated that groups like the less educated voter are hard to reach for polling but showed up when it was time to cast a ballot. In other words, the supply of respondents wasn’t as expected. For the supply chain participants this is what they would call a supply disruption. The human race has been dealing with the impact of supply disruptions since we began slicing bread, but it’s the supply chain community that has studied it and began to manage the effect of disruptions. From late deliveries to machine breakdowns, contamination and more catastrophic events like earthquakes and tsunamis, the best supply chains have been able to quickly understand impact and determine how to respond. As Trevor would say, if there’s a fire, the faster you can contain the fire the more you can minimize the damage. The best supply chains can quickly respond so material on allocation is distributed in a way to maximize customer service, revenue and profits in the face of difficult situations. I believe the pollsters knew the possible “supply disruption” was there. I’m not sure how many of them decided to conduct “what ifs” on multiple nonresponse bias scenarios. Imagine if the pollsters could quickly simulate optimistic and pessimistic views on who’ll show up to the polls or “supply” rather than just talk about it on CNN. I wonder if the political strategies may have been different for either party if there were more meaningful “what if” conversations. The lesson is to expect the unexpected.
Lesson 3: “The more variable the elements, the less effective the standard models are (the proof is in the math!)”
Similar to other problems with polling, there is still the issue of figuring out who will actually show up to vote. Pollsters develop models around who people will vote for as well as who will show up to vote. I wonder though how much consideration went into the number of Obama supporters that either didn’t support Hillary or simply didn’t vote. One reason Nate Silver may have been more accurate in 2012 is that the candidates themselves were more predictable. The 2016 race was so tight with two candidates that carried so much “unpredictability” with them. Were there democrats that didn’t vote for Clinton because they didn’t trust her? Are there republicans that didn’t vote for Trump because of what he said about [insert first thing that comes to mind]? Not too long ago it was easier for supply chain people to predict demand and align supply. For example, when I was a kid and the choices for a television were colored or black and white, some models would remain relevant for a few years. Now with product life cycles what they are, your new phone, tablet, television or computer feels like its obsolete the day after you purchase it. And on the supply side we’re getting hit with disruptions we’ve never expected before. It’s becoming even more challenging to manage because of globally distributed networks. For the pollsters, they got hit with more unpredictability than they’ve ever experienced during an election cycle and as Trevor so eloquently stated, this level of unpredictability means the standard models are less effective. The lesson is to think outside the standard models. Pollsters seem to be decades behind supply chain planners when it comes to planning, monitoring and responding to change. They will likely be spending months trying to figure out what went so wrong this time around. That being said polling and the pollsters still have an important place during the election process, in research and reflecting on public opinion. It may be time though that they take some supply chain lessons and look to experts outside their field for the innovation they need in their field. Let us know if you agree or disagree with the lessons in the comments section.
Leave a Reply