word vectors with spaCy

In this exercise you'll get your first experience with word vectors! You're going to use the ATIS dataset, which contains thousands of sentences from real people interacting with a flight booking system.

The user utterances are available in the list sentences, and the corresponding intents in labels.

Your job is to create a 2D array X with as many rows as there are sentences in the dataset, where each row is a vector describing that sentence.

This exercise is part of the course

Building Chatbots in Python

Exercise instructions

Load the spaCy English model by calling spacy.load() with argument 'en'.
Calculate the length of sentences using len() and the dimensionality of the word vectors using nlp.vocab.vectors_length.
For each sentence, call the nlp object with the sentence as the sole argument. Store the result as doc.
Use the .vector attribute of doc to get the vector representation of each sentence, and store this vector in the appropriate row of X.

Hands-on interactive exercise

Have a go at this exercise by completing this sample code.

# Load the spacy model: nlp
nlp = ____

# Calculate the length of sentences
n_sentences = ____

# Calculate the dimensionality of nlp
embedding_dim = ____

# Initialize the array with zeros: X
X = np.zeros((n_sentences, embedding_dim))

# Iterate over the sentences
for idx, sentence in enumerate(sentences):
    # Pass each each sentence to the nlp object to create a document
    doc = ____
    # Save the document's .vector attribute to the corresponding row in X
    X[idx, :] = ____

Edit and Run Code

This exercise is part of the course

Building Chatbots in Python

IntermediateSkill Level

4.7+

Start Course for Free

In this chapter, you'll learn how to build your first chatbot. After gaining a bit of historical context, you'll set up a basic structure for receiving text and responding to users, and then learn how to add the basic elements of personality. You'll then build rule-based systems for parsing text.

Exercise 1: Introduction to conversational software Exercise 2: EchoBot I Exercise 3: EchoBot II Exercise 4: Creating a personality Exercise 5: Chitchat Exercise 6: Adding variety Exercise 7: ELIZA I: asking questions Exercise 8: Text processing with regular expressions Exercise 9: ELIZA II: Extracting key phrases Exercise 10: ELIZA III: Pronouns Exercise 11: ELIZA IV: Putting it all together

Here, you'll use machine learning to turn natural language into structured data using spaCy, scikit-learn, and rasa NLU. You'll start with a refresher on the theoretical foundations and then move onto building models using the ATIS dataset, which contains thousands of sentences from real people interacting with a flight booking system.

Exercise 1: Understanding intents and entities Exercise 2: Intent classification with regex I Exercise 3: Intent classification with regex II Exercise 4: Entity extraction with regex Exercise 5: Word vectors Exercise 6: word vectors with spaCy

Current Exercise

Exercise 7: Intents and classification Exercise 8: Intent classification with sklearn Exercise 9: Entity extraction Exercise 10: Using spaCy's entity recognizer Exercise 11: Assigning roles using spaCy's parser Exercise 12: Robust language understanding with rasa NLU Exercise 13: Rasa NLU Exercise 14: Data-efficient entity recognition

In this chapter, you'll build a personal assistant to help you plan a trip. It will be able to respond to questions like "are there any cheap hotels in the north of town?" by looking inside a hotel’s database for matching results.

Exercise 1: Virtual assistants and accessing data Exercise 2: SQL basics Exercise 3: SQL statements in Python Exercise 4: Exploring a DB with natural language Exercise 5: Creating queries from parameters Exercise 6: Using your custom function to find hotels Exercise 7: Creating SQL from natural language Exercise 8: Incremental slot filling and negation Exercise 9: Refining your search Exercise 10: Basic negation Exercise 11: Filtering with excluded slots

Everything you've built so far has statelessly mapped intents to actions and responses. It's amazing how far you can get with that! But to build more sophisticated bots you will always want to add some statefulness. That's what you'll do here, as you build a chatbot that helps users order coffee.

Exercise 1: Why statefulness is key Exercise 2: Form filling Exercise 3: Asking contextual questions Exercise 4: Dealing with rejection Exercise 5: Asking questions & queuing answers Exercise 6: Pending actions I Exercise 7: Pending actions II Exercise 8: Pending state transitions Exercise 9: Putting it all together I Exercise 10: Putting it all together II Exercise 11: Frontiers of dialogue research Exercise 12: Generating text with neural networks Exercise 13: Congratulations!