Leverage and conviction
1. Leverage and conviction
Market basket analysis is centered around the application of metrics and pruning. In this video, we'll add two more metrics to our toolkit: leverage and conviction.2. Building on simpler metrics
More complicated metrics in market basket analysis tend to build on simpler ones. We started by constructing support for a single item, X, which was just the total number of times X appears in transactions, divided by the total number of transactions. Similarly, we defined support for a two-item rule, such as "if X then Y" as the frequency of the two items, divided by the total number of transactions. Next, we constructed confidence for "if X then Y" as the support of "if X then Y," divided by the support of X. And finally, we defined lift as the support of "if X then Y," divided by the support of X multiplied by the support of Y.3. The leverage metric
Leverage is also constructed from a simpler metric: support. To compute the leverage of "if X then Y," we compute the support of "if X then Y" and then subtract the product of the support of X and the support of Y. Note that lift and leverage are similar. One advantage of using leverage is that it is bounded from below by minus one and from above by plus one, making it easy to identify high and low values. Lift, to the contrary, is bound from below by 0 and from above by infinity.4. Computing leverage
Let's return to the exercise where a library was considering whether to promote Harry Potter to readers who enjoyed Twilight. We found that both the confidence and lift measures suggested that we should. We ended by measuring lift and found that it was greater than 1. Now let's assume we've imported the same data and now want to compute leverage. The first three steps are the same. We compute the support for both books. We then compute the support separately for each book. Finally, we compute the leverage as the difference between the support of both and the product of the individual support values. The leverage value is 0-point-018, which is positive and equivalent to lift being greater than 1.5. The conviction metric
Conviction is also based on support, but is more complicated and less intuitive than leverage. The conviction of "if X then Y" is computed as the support of X multiplied by the support of NOT Y, divided by the support of X and NOT Y. The support of NOT Y is simply the share of all transactions that do not include Y. The support of X and NOT Y is the share of all transactions that contain X, but not Y.6. Interpreting conviction
In our library example, Twilight was X and Harry Potter was Y. So what was NOT Y? It was Twilight and the Hunger Games. If conviction is greater than 1, then Twilight and Hunger Games to occur together less frequently than expected, given their respective supports, providing evidence for "if Twilight then Harry Potter."7. Computing conviction
To compute conviction, we start by repeating the first two steps for leverage. We then deviate by computing the support for NOT Harry Potter, which is 1-point-0 minus the support of Harry Potter. After that, we compute Twilight's support and NOT Harry Potter's support by computing the support of Twilight and subtracting the support of Twilight and Harry Potter. Finally, we compute conviction, yielding a value greater than 1, which provides further evidence for the rule "if Twilight then Harry Potter."8. Let's practice!
Let's practice computing leverage and conviction in some exercises.Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.