Winning at buzzword bingo (even when the board keeps changing)

Editor's note: This was originally published in Logistics Viewpoints on December 3, 2019.

Analytics, big data, data science, machine learning, artificial intelligence, deep learning, heuristics, optimization - do you feel beaten down playing buzzword bingo on a constantly changing board?

It can be hard to understand which new buzzwords bring business value and which are best left in the lab to mature. And yet this barrier hasn’t stopped many companies from adopting corporate initiatives around data science or artificial intelligence/machine learning (AI/ML). To keep it simple, I bundle these buzzwords and call them “fancy math,” and as passionate as I am about their power to make a positive impact on business, I also believe that starting with math misses the point.

Digital transformation is a strategic imperative for supply chains today, but math-driven technology alone will not drive transformative change, which also requires a strong business vision and strategy. The most strategic step is to start by setting your vision and identifying what problems you are trying to solve. Once the business problems are well-framed, leave the math under the hood for your data scientists, because which math method they use is a tactical decision. Leading with math amounts to letting the tail wag the dog.

The payoff from supply chain digital transformation can be big – McKinsey reports that these efforts can increase profits by 3.2 percent, larger than in any other business area. But McKinsey also points out that relatively few companies have yielded this payoff. Historically, immature technology to tackle supply chain complexity was a barrier, but they believe that the market has now closed that gap. So why haven’t companies succeeded when the technology is available?

Companies must establish a vision for how digitalization can transform their business, and then make the necessary process and organizational changes. McKinsey points to a failure to make those operational changes to support the vision because technology alone is not enough; people and process must accompany it.

Einstein is said to have quipped: “If I had an hour to solve a problem and my life depended on the solution, I would spend the first 55 minutes determining the proper question to ask, for once I know the proper question, I could solve the problem in less than five minutes.” If the venerable math genius prioritized problem formulation over math, shouldn’t the rest of us mere mortals?

The first step is to set the vision and frame the business problems and desired capabilities, which is the hardest step because business is prone to describe symptoms (e.g. we can’t get our orders fulfilled in time, we spend too much time waiting for supplies) but struggles synthesizing them into a clearly-defined business problem (we need to shorten our planning cycles and improve responsiveness).

Why does business problem definition matter so much? Too many expensive supply chain digital transformation initiatives fail because a team of data scientists is hired in a vacuum, with no vision for digital transformation or strategy. They scribble equations, sling code, and build a complex model…that solves the wrong business problem. Without planning the necessary operational changes McKinsey prescribes, they can trip along common paths to failure: ignoring the input of end users, forgetting to engage IT on deployment, or building a black box model neither executives nor regulators understand. Getting clear on what problem you’re trying to solve, how you define a better decision through the benefits you want to achieve, and what assumptions and criteria impact that solution are all part of framing the problem and preventing large data science investments from failing to deliver their promised value.

Defining the problem also entails defining the solution – what constitutes a better decision? It may seem that the objective of supply chain planning is a more accurate forecast, but can you tie this objective to tangible business benefits? Ultimately, the C-suite cares about objectives like maximizing revenue, minimizing cost, or improving customer satisfaction. Getting clear on metrics is an important step in ensuring that your problem is framed correctly and that you can measure success of a better business decision against your digital transformation vision.

Another pitfall involves data, a critical and very time-consuming step – do you have access to the data you need and is it well-prepared for analysis? Paying attention to data is not very sexy, but AI/ML models are predicated on consuming large amounts of data and will fail without being properly fed.

Too many businesses try to jumpstart their problem-solving journey by looking at the fancy math menu: “Let’s find a vendor who does AI/ML to help us.” But without being clear on your vision, what problem you are trying to solve, whether math can help solve it, and whether your data supports that approach, you put the fancy math cart before the problem horse.

While vendors are criticized for falling into the trap of being a hammer in search of a nail (“we build great optimization software”), buyers can fall into that same trap. If you have a corporate AI/ML initiative and compare vendors based on a list of algorithms, then you run the risk of letting a tactical choice of a math method drive a strategic decision of what problems you are trying to solve.

It’s easy to be seduced by arguments that a vendor (or fancy math menu item) will provide the “best” model. But have you defined “best” for your company for this problem? For example, is the best model the one with the most predictive accuracy? Consider the Netflix Prize, which offered a $1 million dollar prize to the team who could improve the accuracy of their movie recommendation model by 10%. In 2009 a winning team walked away with the cash, but Netflix later admitted that they never deployed the models. Their increased accuracy "did not seem to justify the engineering effort needed to bring them into a production environment.” They made a million-dollar mistake in assuming that the only criteria that would matter was accuracy.

The fancy deep learning models you may have read about indeed can dazzle with predictive accuracy. But this accuracy comes with costs you may not be willing to pay for your business problem. The models are “brittle” (requiring a lot of maintenance to continue to perform well), complex for IT to deploy, black box (lacking interpretability, meaning you get results without an explanation of the factors that drive them), and may be slow to run (due to computational complexity taking hours or even days to render a result).

I may sound like I have “algorithmic aversion,” but nothing could be further from the truth. I have spent the last 20 years working in analytics and am deeply invested in advanced analytics methods like machine learning and optimization. When properly applied, these methods have remarkable abilities to render better business decisions (see this article I wrote on “How machine learning can heal a supply chain”).

Not only can all this fancy math build more accurate models, it can find patterns and make decisions far beyond the cognitive capacity of the human mind. While we must hold our models accountable to the human biases they can reflect, if properly built they also have great potential to reduce human bias and yield more consistent, objective decisions. And math can automate many decisions, taking off our plate routine decisions and saving time for the exceptions that most need human skill and judgment.

But the key is the application of math to solve business problems, not the math as an end in itself. There is no universal standard for “best” – no one math method is best for every industry, every company, even every situation in the same company. Decide what problems you’re trying to solve, along with criteria and metrics to measure the results against your larger vision to transform your supply chain.

Occam’s Razor holds that the simplest solution to a problem is likely the best. It is important to hold your data scientists (and maybe even your board!) accountable to both your criteria and metrics and to this principle because left to their own devices they may prefer fancier math. Depending on your business problem, the savings from a more accurate answer may justify greater complexity, in spite of longer compute times or a black box method. But for other problems, a good-enough answer you can get quickly and explain widely may be the best choice.

Winning at buzzword bingo is all about changing the rules of the game – it’s not about the buzzwords, it’s about delivering business value to truly transform your supply chain. I believe in the power of fancy math to enable a smarter supply chain, but the best way to deliver this value is for leadership to start with the strategic decision of setting a vision and defining the business problems and criteria, and then (and only then) choose the math. Human intelligence and vision, combined with fancy math, can truly transform a supply chain. When you’ve taken this hard journey and delivered results, then you can truly call “bingo.”