BaşlayınÜcretsiz Başlayın

TFIDF Practice

Earlier you looked at a bag-of-words representation of articles on crude oil. Calculating TFIDF values relies on this bag-of-words representation, but takes into account how often a word appears in an article, and how often that word appears in the collection of articles.

To determine how meaningful words would be when comparing different articles, calculate the TFIDF weights for the words in crude, a collection of 20 articles about crude oil.

Bu egzersiz

Introduction to Natural Language Processing in R

kursunun bir parçasıdır
Kursu Görüntüle

Egzersiz talimatları

  • Calculate TFIDF values for crude by article_id and by word. Save the resulting tibble as crude_weights.
  • Sort crude_weights with the arrange() function by descending tf_idf values.
  • Filter crude_weights to the lowest non-zero tf_idf values. Again, use the arrange function.

Uygulamalı interaktif egzersiz

Bu örnek kodu tamamlayarak bu egzersizi bitirin.

# Create a tibble with TFIDF values
___ <- crude_tibble %>%
  unnest_tokens(output = "word", token = "words", input = text) %>%
  anti_join(stop_words) %>%
  count(article_id, word) %>%
  ___(___, ___, n)

# Find the highest TFIDF values
crude_weights %>%
  ___(desc(___))

# Find the lowest non-zero TFIDF values
crude_weights %>%
  filter(___ != ___) %>%
  ___(___)
Kodu Düzenle ve Çalıştır