The web is a rich source of data from which you can extract various types of insights and findings. In this chapter, you will learn how to get data from the web, whether it is stored in files or in HTML. You'll also learn the basics of scraping and parsing web data.

Importing flat files from the web

Importing flat files from the web: your turn!

Opening and reading flat files from the web

Importing non-flat files from the web

HTTP requests to import files from the web

Performing HTTP requests in Python using urllib

Printing HTTP request results in Python using urllib

Performing HTTP requests in Python using requests

Scraping the web in Python

Parsing HTML with BeautifulSoup

Turning a webpage into data using BeautifulSoup: getting the text

Turning a webpage into data using BeautifulSoup: getting the hyperlinks

Importing data from the Internet

In this chapter, you will gain a deeper understanding of how to import data from the web. You will learn the basics of extracting data from APIs, gain insight on the importance of APIs, and practice extracting data by diving into the OMDB and Library of Congress APIs.

Introduction to APIs and JSONs

Pop quiz: What exactly is a JSON?

Loading and exploring a JSON

Pop quiz: Exploring your JSON

APIs and interacting with the world wide web

Pop quiz: What's an API?

API requests

JSON–from the web to Python

Checking out the Wikipedia API

Interacting with APIs to import data from the web

In this chapter, you will consolidate your knowledge of interacting with APIs in a deep dive into the Twitter streaming API. You'll learn how to stream real-time Twitter data, and how to analyze and visualize it.

The Twitter API and Authentication

Streaming tweets

Load and explore your Twitter data

Twitter data to DataFrame

A little bit of Twitter text analysis

Plotting your Twitter data

Final Thoughts

Diving  deep into the Twitter API

Latitudes (XLS)

Tweets

Red wine quality

As a data scientist, you will need to clean data, wrangle and munge it, visualize it, build predictive models and interpret these models. Before you can do so, however, you will need to know how to get data into Python. In the prequel to this course, you learned many ways to import data into Python: from flat files such as .txt and .csv; from files native to other software such as Excel spreadsheets, Stata, SAS, and MATLAB files; and from relational databases such as SQLite and PostgreSQL. In this course, you'll extend this knowledge base by learning to import data from the web and by pulling data from Application Programming Interfaces— APIs—such as the Twitter streaming API, which allows us to stream real-time tweets.

Introduction to Importing Data in Python

Learn how to import data into Python from sources like the web and by pulling data from APIs, such as the Twitter streaming API to stream real-time tweets.

Intermediate Importing Data in Python

Improve your Python data importing skills and learn to work with web and API data.

Data Engineer in Python

Data Scientist in Python

Importing & Cleaning Data  in Python

Pop quiz: What exactly is a JSON?

Intermediate Importing Data in Python

Hands-on interactive exercise