1. Learn
    2. /
  2. Courses
    3. /
  3. Streamlined Data Ingestion with pandas
  • 1

    Importing Data from Flat Files

    Free

    Practice using pandas to get just the data you want from flat files, learn how to wrangle data types and handle errors, and look into some U.S. tax data along the way.

  • 2

    Importing Data From Excel Files

    Automate data imports from that staple of office life, Excel files. Import part or all of a workbook and ensure boolean and datetime data are properly loaded, all while learning about how other people are learning to code.

  • 3

    Importing Data from Databases

    Combine pandas with the powers of SQL to find out just how many problems New Yorkers have with their housing. This chapter features introductory SQL topics like WHERE clauses, aggregate functions, and basic joins.

  • 4

    Importing JSON Data and Working with APIs

    Learn how to work with JSON data and web APIs by exploring a public dataset and getting cafe recommendations from Yelp. End by learning some techniques to combine datasets once they have been loaded into data frames.

Initializing

Exercise

Joining and filtering

Just as you might not always want all the data in a single table, you might not want all columns and rows that result from a JOIN. In this exercise, you'll use SQL to refine a data import.

Weather exacerbates some housing problems more than others. Your task is to focus on water leak reports in hpd311calls and assemble a dataset that includes the day's precipitation levels from weather to see if there is any relationship between the two. The provided SQL gets all columns in hpd311calls, but you'll need to modify it to get the necessary weather column and filter rows with a WHERE clause.

pandas is loaded as pd, and the database engine, engine, has been created.

Instructions 1/2

undefined XP
    1
    2
  • Complete query to get the prcp column in weather and join weather to hpd311calls on their date and created_date columns, respectively.
  • Use read_sql() to load the results of the query into the leak_calls dataframe.