Three Principles to Improve your Company's ROI on Predictive Analytics

William Ford, Director of Data Science, Chegg
William Ford, Director of Data Science, Chegg

William Ford, Director of Data Science, Chegg

Introduction

Have you seen any articles recently touting the virtues of Predictive Analytics to transform the business? I am sure that you have. Unfortunately, it’s possible that getting value from Predictive Analytics remains a situation of “the haves and have nots.”

Earlier this year, Gartner reported that of the nearly 200 companies that they surveyed less than 50 percent had reached a “transformational” level with Data and Analytics. They reported that common barriers for businesses to use data included:

• Determining how to get value from projects

• Defining data and analytics strategy

• Solving risk and governance issues

It is hard to find readily available advice that offers clear actionable steps for technical leadership to improve their ROI from predictive analytic investments.

Motivation

In previous roles, I made my living supporting and developing predictive analytic products for several customers. Across our clients, the common theme was that businesses wanted to gain a positive return on investment. This was typically understood as “we need a predictive model that does X.” In my experience, a deployed predictive model is not the first or last opportunity to derive real business value from predictive analytics. Wouldn’t it be valuable for your business to have multiple opportunities to gain value from a predictive analytics effort? If we think of predictive analytics as a value chain, we can find many opportunities to recognize a positive return on your investment.

At a high level, three principles can help you frame your critical conversations around improving the ROI of your predictive analytics initiatives:

• Success can be the journey

• All models are wrong

• Teach your business to fish

1st Principle: Success can be a journey

Traditional predictive modeling can be broken into loosely four phases.

• Pre-modeling

• Model development

• Model deployment

• Deployment (Impacting the business)

The first step in the predictive analytics value chain starts when your data scientists begin working with your data. More than 85 percent of time and effort required to bring a model to full deployment is spent in pre-modeling. This process requires the data scientist to think critically about what is going on in your business’s data. This is often the first opportunity to yield a return on your investment in predictive analytics.

When I worked in Data Science services/consultancy, we often found that our clients data had fundamental issues that pointed to breakage in their data operations. As businesses moved to big data infrastructures or cloud infrastructure providers, these issues seemed to crop up rather frequently. If there had not been an initial investment in developing Predictive Analytic methods, these issues might have gone unfixed for much longer.

As an information leader, how are you encouraging your organization to create value at this phase? How are you prioritizing keeping an eye on your business’s data?

2Nd Principle: All Models Are Wrong, But Some Are Useful

The next phase in the value chain focuses on the process of model development. In 1976 George Box, a famous statistician, wrote a paper that is often cited called “Science and Statistics.” He stated, “Since all models are wrong the scientist cannot obtain a “correct” one by excessive elaboration.” This line is often shortened to “all models are wrong, but some are useful.”

I have often seen data scientists toiling away to eke out small improvements in models before deploying them. This might be attributed to a focus on the model as the sole deliverable rather than the broader goal of improving the business and its operations. If the process of model development yielded an opportunity to improve your business, would you take it? When developing models, it is easy to forget that a dollar earned today is worth more than that same dollar a year or six months from now, this is summarized in the financial principle of the “time value of money” (TMV) and it applies to our investments in predictive analytics.

In predictive analytics we can call it the “time value of actionable insights.” The sooner the business can act on insights, the more options you have to create value and maintain a competitive advantage. As an information leader, do you value time to value over model perfection?

3rd Principle: Teach your business to fish

The final point in the predictive analytics value chain comes after the predictive model is deployed. Imagine that your data science team has partnered with engineering, product, marketing, and the business and has created and deployed a highly predictive model to determine which customers will churn. Knowing who will churn is great. And having a very strong model might seem great at first pass, but it also could indicate that there may be a fundamental issue with your business. Armed with the knowledge of why a customer might leave, you can take action to resolve those issues and improve the business.

At its core, predictive analytics does not promise to give the “why” behind a prediction, but it can if insight is prioritized. Certain data science methods and models are more amenable to introspection or explainability. As a leader of your corporation, have you made it clear that insight can be more valuable than the ability to predict an outcome?

Conclusion

I hope that this article evoked some questions that might inspire you to improve your return on investment in predictive analytics. From project conception to delivery, predictive analytic efforts are rich with opportunities to improve business outcomes. We are in an exciting time of innovation, and the job of the CIO is possibly more difficult than ever. As the leader of information, and likely technology for your organizations, it is up to you to cut through the noise. Good luck on getting the most out of your predictive analytics investments.

Read Also

The Fault in Prediction

The Fault in Prediction

Matt Hayes, Director of Strategic Operations, Baptist Health Medical Group
Predictive Analytics Key Component of Customer Experience Management

Predictive Analytics Key Component of Customer Experience Management

Richard Barton, CIO, American Management Association