1. Assessing the POC
We know which metrics will be used to evaluate the AI POC, but what do we do with this information?
2. Stop and re-evaluate
First, if the POC doesn’t achieve the success criteria defined at the beginning of the phase, then the best idea is to pause and rethink the problem and solution. The worst idea is to waste company resources on a solution that clearly doesn’t work.
3. Adjust
If the POC is showing positive signs of success but hasn’t quite reached the bar, then adjust as necessary. The best thing about a POC is the ability to test quickly and learn what works and doesn’t work.
Is the UI fast and responsive but the rate of scheduled clients low? Evaluate the accuracy and relevance of the schedule suggestions, or the AI algorithm.
4. Adjust
Is the system not updating fast enough and suggesting blocked times? Evaluate ways to modify the data pipelines, compute, or other structure. A data analyst could prove invaluable in this phase.
Adjustments can happen at any level, depending on the metric not meeting the mark. Be sure to spend time on the metrics that matter, since it is only a POC. The goal is to show a working solution that shows potential of greater business impact.
5. Estimate potential ROI
There may have been small scale impact with big learnings from the POC, but a business rarely invests in initiatives that won't move the bottom line. Therefore, a successful POC
6. Estimate potential ROI
should help understand the potential value of
7. Estimate potential ROI
a full-scale implementation. Data from the POC can be used to estimate this, specifically, costs and return on investment.
This is why spending time to characterize the problem, determine the goals and objectives, and define success is crucial. It will help ensure that a POC hitting its success metrics is likely to also provide future value when fully implemented.
8. Feasible?
The POC may have potential for greater business impact but still not be a smart decision to scale to a full implementation. So in the conversation about next steps, it's important to talk about feasibility.
Here, feasibility is the likelihood for the company to execute given current and known future resources.
9. Questions to help determine scale decision
Here are some factors to consider regarding implementing the full-scale solution which should help determine feasibility.
Data, is there enough data to support the increased volume of users and decisions?
10. Questions to help determine scale decision
Infrastructure, is there the financial resources to pay for more compute, storage, networking capabilities, and other infrastructure components?
11. Questions to help determine scale decision
AI algorithm, will the algorithm handle the estimated volume? Some algorithms don’t scale well with larger datasets.
12. Questions to help determine scale decision
Robustness, test the solution for real-world situations and edge-cases which may not be caught in a POC.
13. Questions to help determine scale decision
Talent, more people will be needed to scale the solution - is there enough on the staff?
14. Questions to help determine scale decision
Responsible AI, Compliance, and Ethics, is there the expertise to create frameworks, policies, and guidelines for the development and use of the solution?
15. Finding the right choice
Spending intentional time to characterize the problem, determine the goals and objectives, and define success are crucial because it makes these scaling conversations easier. However, changes in the business happen, so determine if the value and full solution still align with business goals.
One useful method is to create a matrix with a score for feasibility on one axis and value on the other. Each data point can be a different version of the full implementation. These visuals can help the decision makers quickly identify plans that are feasible and high value.
16. Let's practice!
Now, let's test your knowledge on making scaling decisions.