1. Identifying inconsistencies
Now that we have a few functions to work with, we can slice our data in a variety of ways and calculate retention rates with minimal effort. Let's leverage our new functions and make the kinds of reports stakeholders might not know to ask, but will be happy you took a look.
2. Day of week trends
One of the most common reasons for fluctuating metrics is due to differences in how customers behave on different days of the week. For example, some businesses consistently perform better during weekdays than weekends.
We pass the DoW column created in the first chapter to the retention_rate() function from the previous lesson.
3. Plotting the results
and plot the results.
4. Retention not affected by day of week user subscribes
As you can see, there does appear to be some relationship where retention is lower if users subscribe later in the week, but this difference is small and is likely indicative of something else, such as sending more emails on the weekend which converts lower intent users.
This is all to say, weekday fluctuations are common and do not necessarily merit raising a red flag even if you see a consistent pattern, but it might mean modifying when you attempt to market to customers most heavily.
5. Real data can be messy and confusing
Sometimes with real data, variations in metrics may be due to random chance and have no explanation. Or we might not have access to the kind of data that we would need to identify the cause of a change. Other times, it requires a bit of creative thinking to find the underlying reason. When you find concerning variations in the data, you must be creative and comfortable with ambiguity.
When you get stuck, it's a good time to brainstorm with coworkers. While this can be frustrating at first, over time, you will develop an intuitive sense of what kinds of problems can arise and how to identify these problems.
6. Let's practice!
Now, let's think creatively and try to find the root cause of House Ads’ conversion rate suddenly declining.