Monitoring and alerting within a data pipeline
It's time to put it all together! You might have guessed it, but using handling errors using try-except and logging go hand-in-hand. These two practices are essential for a pipeline to be resilient and transparent, and are the building blocks for more advanced monitoring and alerting solutions.
pandas has been imported as pd, and the logging module has been loaded and configured for you. The raw_sales_data DataFrame has been extracted, and is ready to be transformed.
Questo esercizio fa parte del corso
ETL and ELT in Python
Esercizio pratico interattivo
Prova a risolvere questo esercizio completando il codice di esempio.
def transform(raw_data):
return raw_data.loc[raw_data["Total Price"] > 1000, :]
try:
# Attempt to transform DataFrame, log an info-level message
clean_sales_data = transform(raw_sales_data)
logging.____("Successfully filtered DataFrame by 'Total Price'")
except Exception:
# Log a warning-level message
____.____("Cannot filter DataFrame by 'Total Price'")