Handling exceptions when loading data
Sometimes, your data pipelines might throw an exception. These exceptions are a form of alerting, and they let a Data Engineer know when something unexpected happened. It's important to properly handle these exceptions. In this exercise, we'll practice just that!
To help get you started, pandas has been imported as pd, along with the logging module has been imported. The default log-level has been set to "debug".
This exercise is part of the course
ETL and ELT in Python
Exercise instructions
- Update the pipeline to include a
tryblock, and attempt to read the data from the path"sales_data.parquet". - Catch a
FileNotFoundErrorif the file is not able to be read into apandasDataFrame. - Create an error-level log to document the failure.
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
def extract(file_path):
return pd.read_parquet(file_path)
# Update the pipeline to include a try block
____:
# Attempt to read in the file
raw_sales_data = extract("____")
# Catch the FileNotFoundError
except ____ as file_not_found:
# Write an error-level log
logging.____(file_not_found)