Get Started

Calculating KPIs - a practical example

1. Calculating KPIs - a practical example

Congratulations! Now that we have mastered aggregating our data in interesting ways, let’s put these techniques into practice.

2. Goal - comparing our KPIs

We will start by examining the KPI of user conversion rate after the free-trial ends. In this case we will look in the first week after the trial ends.

3. Conversion rate : maximum lapse date

Consider our current date to be March 17, 2018. To start, we need to check what the maximum lapse_date in our data set is, which turns out to be today.

4. KPI calculation : restrict users by lapse date

Next we need to remove users who lapsed today, or any of the prior 7 days. This ensures everyone had a full seven days to potentially subscribe. We can do this by filtering on the condition of our lapse date being less-than the current date minus 7 days. Next we count how big this filtered group is by calling the count method on the filtered dataset.

5. KPI calculation: restrict subscription date

Then we need to find the number of users who subscribed within 7 days of lapsing. We check this by seeing who has a non-zero subscription price and whose purchase is within 7 days of lapsing. We can filter in a similar way to before and then count the size of the resulting group.

6. KPI calculation: find the conversion rate

Finally, we need to divide our subbing user number by our total number to see we have a first week conversion rate of 23%.

7. Cohort conversion rate

Now we want to check week-one and week-two conversion rates across different cohorts. We can exclude dates too near today as before.

8. Cohort conversion rate

Since we can only aggregate over one column, we will create a column containing our needed information. We add a column `sub_time` that is the days between the lapse data and the subscription date if the user subscribed and NaT otherwise. We use the `where()` method to do this, which takes a condition and a value to return if true and an alternative value to return if false.

9. Cohort conversion rate

I have created two functions `gcr7` and `gcr14` that take our sub_time column and perform the steps we performed previously to find the conversion rate in that period. First, we can group by some key demographic fields, and then call our functions on `sub_time`. As we can see we have fairly similar conversion rates with the exception of males on android being slightly higher generally.

10. How to choose KPI metrics?

To conclude our discussion on KPIs, it is important to note that while there an infinite number of KPIs we want to choose carefully which to rely on. One factor in determining this is how long it takes to gain insight on a metric. To find the monthly conversion rate, we would need to wait a month from the lapse date. This can make it impractical to monitor on an actionable time scale. Other ways to uncover KPIs include exploratory analysis which can reveal relationships between metrics and key results. Additionally, these metrics can be tied to the business metrics in important ways.

11. Why is conversion rate important?

In the KPIs calculated above, we may consider this metric important because if it changes, it serves as a warning of potential problems down the road.

12. Next chapter: continue exploring conversion rates

We will continue exploring this point in the next chapter by tracking this KPI over time. Further, measuring KPIs across groups is crucial because changes can impact groups in drastically different ways. There may be factors important to one group but not to another and this is crucial to understand.

13. Let's practice!

Now you have a sense of why KPIs are useful and how to think about integrating them into your work.