Adding evolutions

1. Adding evolutions

In the last exercises, you learned how to add aggregated variables to the basetable that express the donor's target behaviour in the past. In this video, you will learn to take it even one step further and capture evolutions in the basetable.

2. Motivation for evolutions (1)

Consider the problem where you want to predict whether a donor will donate more than 5 times next year. As you learned in the previous exercises, it is a good idea to add the number of donations for each donor to the basetable.

3. Motivation for evolutions (2)

However, let's have a look at these two donors: one donor donated 10 times the year before, and 20 times the year before that.

4. Motivation for evolutions (3)

Another donor donated 10 times the year before, and also 10 times the year before that. Which donor is most likely to donate more than 5 times next year?

5. Motivation for evolutions (4)

For the first donor, you see a decreasing trend in the number of donations, so you could assume that this trend continues, and that he is not that likely to donate at least 5 times next year.

6. Motivation for evolutions (5)

The second donor looks like a more stable donor and is probably more likely to donate again more than 5 times next year. To catch this evolution of the donor in a variable, you could for instance add the number of donations in 2016 divided by the number of donations in 2015 and 2016 as a variable. The higher this number, the more donations the donor has made recently compared to what he used to donate.

7. Adding evolutions to the basetable (1)

Adding evolutions to the basetable is straightforward and is very similar to adding aggregated variables to the basetable. First we select the gifts made in 2016, these are gifts made before January 1st 2017 and made after January 1st 2016. Next, we select gifts made in 2015 and 2016, these are gifts made before January 1st 2017 and after January 1st 2015.

8. Adding evolutions to the basetable (2)

We then count the number of gifts made in 2016 for each donor using the groupby command with the donor ID as argument, and using the size method to count the number of gifts. We rename the columns of the resulting dataframe appropriately. Next, we repeat this procedure, but now for gifts made in 2015 and 2016.

9. Adding evolutions to the basetable (3)

These aggregated values are then added to the basetable using merge with how equals left to obtain a left join of the basetable and the number of gifts dataframes. Finally, we calculate the evolution in the basetable by simply dividing the number of donations in 2016 and dividing them by the number of donations in 2015 and 2016.

10. Adding evolutions to the basetable (4)

Note that missing values are automatically taken care of: if a donor did not make any donations in 2016, the result of the division is a missing value as well.

11. Let's practice!

You're now ready to add some evolutions to the basetable yourself. Let's practice!

Create Your Free Account

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.