1. Business requirements
Great, now we will learn how to gather business requirements and scope out machine learning projects we reviewed before.
2. Scoping business needs
Now , there are three key topics to discuss when beginning the requirement collection process.
First, identify the business situation - for example maybe the company plans to expand to new markets.
Then, assess the opportunity size, and how much you expect it to improve with machine learning. For example, you might want to identify which markets or areas have the highest predicted demand for company's merchandise.
Finally, what business actions can you take? For example, prioritize and invest more in the markets with higher predicted demand.
3. Business scope - fraud example
Let's go through a few examples.
First - fraud prediction.
Here, the fraud rate has started increasing.
The opportunity here is to prevent an increase in costs due to fines, and to prevent losing customer trust.
The business action could be improving the fraud detection system, and introducing a manual review of transactions predicted as risky.
4. Business scope - churn example
Let's look into another example of customer churn prediction.
The business situation is that the customers started to churn more.
The opportunity is to reduce the churn and save a number of customers therefore mitigating the impact on revenue.
Finally, the actions the company might take could be identifying churn drivers and improving them. For example if we find that website errors, advertising levels or customer service issues strongly impact churn, then the company should invest time and resources to fix the identified issues - this is the use of the causal model. Also, the company could use the prediction model and introduce retention campaigns targeting customers who have been identified as being at risk of churning.
5. Business situation - asking the right question
Now, there are certain best practices when it comes to assessing the business situation. It starts with asking the right question.
We're back to inference vs prediction, and the questions are different.
First, you should always start with the inference questions.
For example, why has churn started increasing, which information indicates a potential fraud case, or how do our most valuable customers act differently in the first month compared to less valuable ones?
Then, the discussion should build on the inference question to define the prediction question.
For example, can we identify customers at risk of churning, or flag potentially risky transactions, or can we predict early on which customers are likely to become highly valuable?
6. Business opportunity
Obviously, you don't want to spend more than you will get. Therefore, it's very important to size up the opportunity.
This is essentially a cost/benefit analysis. First, we estimate how much we can impact the metric and what it's worth.
Then, we calculate the cost of doing that, and compare the cost with the opportunity size. Does it still make sense to pursue this project?
Estimating opportunity size is hard. For example, how much can we reduce churn levels?
The best approach answer is running experiments with the model predictions.
7. Actionable machine learning
Finally, the action part. Even if the model prediction are very accurate, they might not be actionable.
First, you should look at the historical levels of the metric you're trying to predict like churn or fraud percentage, or the number of new high value customers.
Then, run experiments multiple times, targeting customers who have been predicted and compare if your activities (campaigns or other strategies) show an impact on the targeted group.
If you get a positive answer, assess the level of lift or reduction, and calculate the opportunity size if this was rolled out to all markets, and all customers. Is it still worth it?
If you don't get meaningful results, then it's back to the business situation assessment, you need to collect more data, do qualitative research like surveys, and narrow down the business question.
8. Let's practice!
Great work! Now, let's do some exercises!