Tokenizing the Gettysburg Address

In this exercise, you will be tokenizing one of the most famous speeches of all time: the Gettysburg Address delivered by American President Abraham Lincoln during the American Civil War.

The entire speech is available as a string named gettysburg.

Deze oefening maakt deel uit van de cursus

Feature Engineering for NLP in Python

Cursus bekijken

Oefeninstructies

Load the en_core_web_sm model using spacy.load().
Create a Doc object doc for the gettysburg string.
Using list comprehension, loop over doc to generate the token texts.

Praktische interactieve oefening

Probeer deze oefening eens door deze voorbeeldcode in te vullen.

import spacy

# Load the en_core_web_sm model
nlp = ____.____(____)

# Create a Doc object
doc = ____(____)

# Generate the tokens
tokens = [token.____ for token in ____]
print(tokens)

Code bewerken en uitvoeren