Creating a custom named entity in spaCy

If spaCy's built-in named entities aren't enough, you can make your own using spaCy's EntityRuler() class.

EntityRuler() allows you to create your own entities to add to a spaCy pipeline.

You start by creating an instance of EntityRuler() and passing it the current pipeline, nlp.

You can then call add_patterns() on the instance and pass it a dictionary of the text pattern you'd like to label with an entity.

Once you've setup a pattern you can add it the nlp pipeline using add_pipe().

Since Acme is a technology company, you decide to tag the pattern "smartphone" with the "PRODUCT" entity tag.

spaCy has been imported and a doc already exists containing the transcribed text from call_4_channel_2.wav file).

This is a part of the course

“Spoken Language Processing in Python”

Exercise instructions

Import EntityRuler from spacy.pipeline.
Add "smartphone" as the value for the "pattern" key.
Add the EntityRuler() instance, ruler, to the nlp pipeline.
Print the entity attributes contained in doc.

Hands-on interactive exercise

Have a go at this exercise by completing this sample code.

# Import EntityRuler class
from spacy.pipeline import ____

# Create EntityRuler instance
ruler = EntityRuler(nlp)

# Define pattern for new entity
ruler.add_patterns([{"label": "PRODUCT", "pattern": ____}])

# Update existing pipeline
nlp.add_pipe(____, before="ner")

# Test new entity
for entity in doc.____:
  print(entity.text, entity.label_)

Edit and Run Code

This exercise is part of the course

Spoken Language Processing in Python

AdvancedSkill Level

4.7+

Learn how to load, transform, and transcribe speech from raw audio files in Python.

Start Course for Free

Audio files are different from most other types of data. Before you can start working with them, they require some preprocessing. In this chapter, you'll learn the first steps to working with speech files by converting two different audio files into soundwaves and comparing them visually.

Exercise 1: Introduction to audio data in Python Exercise 2: The right frequency Exercise 3: Importing an audio file with Python Exercise 4: Converting sound wave bytes to integers Exercise 5: The right data type Exercise 6: Bytes to integers Exercise 7: Finding the time stamps Exercise 8: Visualizing sound waves Exercise 9: Staying consistent Exercise 10: Processing audio data with Python

Speech recognition is still far from perfect. But the SpeechRecognition library provides an easy way to interact with many speech-to-text APIs. In this section, you'll learn how to use the SpeechRecognition library to easily start converting the spoken language in your audio files to text.

Exercise 1: SpeechRecognition Python library Exercise 2: Pick the wrong speech_recognition API Exercise 3: Using the SpeechRecognition library Exercise 4: Using the Recognizer class Exercise 5: Reading audio files with SpeechRecognition Exercise 6: From AudioFile to AudioData Exercise 7: Recording the audio we need Exercise 8: Dealing with different kinds of audio Exercise 9: Different kinds of audio Exercise 10: Multiple Speakers 1 Exercise 11: Multiple Speakers 2 Exercise 12: Working with noisy audio

Not all audio files come in the same shape, size or format. Luckily, the PyDub library by James Robert provides tools which you can use to programmatically alter and change different audio file attributes such as frame rate, number of channels, file format and more. In this chapter, you'll learn how to use this helpful library to ensure all of your audio files are in the right shape for transcription.

Exercise 1: Introduction to PyDub Exercise 2: Import an audio file with PyDub Exercise 3: Play an audio file with PyDub Exercise 4: Audio parameters with PyDub Exercise 5: Adjusting audio parameters Exercise 6: Manipulating audio files with PyDub Exercise 7: Turning it down... then up Exercise 8: Normalizing an audio file with PyDub Exercise 9: Chopping and changing audio files Exercise 10: Splitting stereo audio to mono with PyDub Exercise 11: Converting and saving audio files with PyDub Exercise 12: Exporting and reformatting audio files Exercise 13: Manipulating multiple audio files with PyDub Exercise 14: An audio processing workflow

In this chapter, you'll put everything you've learned together by building a speech processing proof of concept project for a technology company, Acme Studios. You'll start by transcribing customer support call phone call audio snippets to text. Then you'll perform sentiment analysis using NLTK, named entity recognition using spaCy and text classification using scikit-learn on the transcribed text.

Exercise 1: Creating transcription helper functions Exercise 2: Converting audio to the right format Exercise 3: Finding PyDub stats Exercise 4: Transcribing audio with one line Exercise 5: Using the helper functions you've built Exercise 6: Sentiment analysis on spoken language text Exercise 7: Analyzing sentiment of a phone call Exercise 8: Sentiment analysis on formatted text Exercise 9: Named entity recognition on transcribed text Exercise 10: Named entity recognition in spaCy Exercise 11: Creating a custom named entity in spaCy

Exercise 12: Classifying transcribed speech with Sklearn Exercise 13: Preparing audio files for text classification Exercise 14: Transcribing phone call excerpts Exercise 15: Organizing transcribed phone call data Exercise 16: Create a spoken language text classifier Exercise 17: Congratulations!