Prescriptive Data Science: Beyond Predictive Data Science
(1) Descriptive, predictive, and prescriptive data science; (2) Required skillsets and key components in prescriptive data science
Data Science Problem Types | Required Skill Sets for Prescriptive Data Science | Key Components of Prescriptive Data Science
The focus of data science and machine learning has been mainly on predictive modeling and data engineering. As companies become more mature and sophisticated with data science and machine learning, predictive data science becomes more of table stakes. Prescriptive data science is gaining more importance in setting the leaders in the data science and artificial intelligence apart from other companies. More companies are asking for "prescriptive" recommendations on "what to do." Traditionally, "prescriptive decision recommendation" has been handled mostly by Operation Research and Economics policy researchers. However, optimization methods and reinforcement learning are now picked up by data scientists and machine learning engineers as one of core tools to go beyond predictive modeling.
As a personal anecdote, when I first built assortment (i.e., product line) decision support solutions 7 years ago, predictive modeling of product unit sales for a portfolio of products that retailer has when we either add or take out certain products were considered state-of-the-art. Business users were happy to be able to run scenario analyses and simulation based on predictive models for volume transfer. As they become more sophisticated, they start to ask "prescriptive" recommendation on what products to delete and add under shelf space constraint. This demonstrates that recommending "what to do" is more and more requested once the business end users become familiar with (predictive) model-based simulations. There are also close interdependencies between the predictive models and optimization methods for prescriptive recommendations. Thus, now is a good time to learn more about prescriptive data science and key methods.
In this article, I will discuss (1) different types of data science and their relevance with the organizational maturity in data science and analytics, (2) required skill sets to excel in prescriptive data science, and (3) key components of prescriptive data science and common misconception.
1. Data Science Journey – Descriptive, Predictive, Prescriptive
The business needs for different types of data science change with the analytical maturity of the organization. Initially, the focus is providing more "fact base" for day-to-day business decision making. The relevant business question is "What happened?" or "What is going on?" As an example, a retailer ran many different kinds of promotion events in the past, and this company wants to know what the ROI of each promotion event and what promotion type is driving more incremental profit.
Once an organization becomes more analytically mature in its data science journey, the business questions evolve into planning for the future. The business question here is "What is going to happen for Y if we change X?" As an example, a retailer is planning for Black Friday promotion events, and this company may consider a set of promotion options as candidates. Predictive data science can help to make forecasts under different promotion scenarios.
Having predictive capabilities is great, but most organizations get to realize that there are too many options to consider. At this stage, the business question is "What should we do (among the choices of X’s)?" As an example, a retailer would like to have a promotion recommendation engine, which gives recommendations on what promotion event to run (e.g., Buy 2 Get 1 50% off) at what discount level (50% vs. 30%), at what timing (2nd week of July), at what location (Denver), and what products (Coca Cola Regular 2L Single Bottle).
Figure 1 below summarizes the data science journey with organizational maturity with data science. Please note that the relevant data science toolkits also change along this journey, as shown on this graph.
2. Prescriptive Data Science: Required Skill sets
What are the required skills for data scientists when companies focus more and more on "prescriptive data science?" Many readers may be already familiar with the 3 circles of skill requirements for data scientists: (1) business domain knowledge, (2) statistics / machine learning (ML) / artificial intelligence (AI), and (3) software engineering. With "prescriptive data science," an additional skill on "optimization" is required to be more impactful data scientists. "Optimization" is usually in the domain of "Operations Research" or "Industrial Engineering." Please note that typical data science curriculums do not cover this topic in depth. Figure 2 below shows 4 major dimensions of skill requirements for the next generation of data scientists, with more details on each dimension.
3. Key components of prescriptive data science and common misconceptions
What is "optimization?" What are the key components of "(mathematical) optimization" for prescriptive data science? There is much confusion on the definition of "prescriptive data science" or "optimization." In case readers are not very familiar with (mathematical) optimization yet, I am describing key components of optimization below. In addition, I also show what is not "prescriptive data science."
- (1) Objective (function): This is a quantitative measure of the performance that the business uses to evaluate the business outcome. Most of time, total profit (i.e., bottom line) and total revenue (i.e., top revenue) are frequent candidates as an objective in optimization. However, other business metrics such as customer life-time value can be considered depending on the relevant time horizon and the nature of business questions. Please also note that it is possible to have multiple objectives with varying importance (e.g., 80% profit + 20% revenue)
- (2) Decision variable: These are a set of variables that the company can change. As an example, in the "joint price optimization" (i.e., pricing optimization) for retailers, a set of prices for 10 products in a given category (e.g., candy) are decision variables.
- (3) Constraints: Often times, businesses have a set of business rules or guardrails that need to be considered to make final recommendations for business actions. It is very common to apply these rules after initial attempt of optimization, which has unexpected consequences of over-riding optimization results. A more proper way is incorporating these sets of rules as "constraints" in constrained optimization. These rules can enter as "upper / lower bound" constraints, "equality" constraints, or "inequality constraints". As an example, in the "joint price optimization" problem, a group of products may need to have the same price due to the contractual agreement between a retailer and a manufacturer or due to the needs of simplification for business decision making – so called "line pricing." This line pricing rule can enter as "equality" constraints for the "constrained pricing optimization" problems. If product 1’s price should be the same as product 2’s price, this can be specified as: price (product 1) = price (product 2). If a retailer is reluctant to make more than a 10% increase in price from current the price level, this can enter as upper bounds for price increases: i.e., price (product 1) ≤ 1.10 x current price (product 1).
Figure 3 below summarizes key components for "prescriptive data science" (i.e., mathematical optimization.) In addition, it also shows "what it is not."
I hope this article helps you to better understand the data science journey with organizational maturity with data science, additional skill set for prescriptive data science, what the key components of "predictive data science" are, and what it is not. In the next article, I will give an overview of different types of optimization methods and what mathematical and analytical techniques are available. If you are a Python data scientist, a good starting place for "prescriptive data science" is SciPy. Optimize library.
- SciPy. Optimize Link
Share This Article
Towards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.
Write for TDS
