Exercise

Testing perplexity

You have been given a dataset full of tweets that were sent by tweet bots during the 2016 US election. Your boss has identified two different account types of interest, Left and Right. Your boss has asked you to perform topic modeling on the tweets from Right tweet bots. Furthermore, your boss is hoping to summarize the content of these tweets with topic modeling. Perform topic modeling on 5, 15, and 50 topics to determine a general idea of how many topics are contained in the data.

Instructions 1/3

XP

1
- Perform LDA using the 'Gibbs' method on 5 topics and print the perplexity score for both the train and test datasets.
2
- Perform topic modeling with 15 topics.
3
- Perform topic modeling with 50 topics.