Time-triggered lambda code
Cody wants to be able to pull a full report of all speeders, forever, for all time - and she wants it updated nightly.
She asked you to create a nightly job
The current structure looks like this:
Write the code that would go into the lambda_handler
method of the function you created in the last exercise.
All boto3
, pandas
and awsrangler
have been imported for you, and the keys have been defined.
Let's roll!
This exercise is part of the course
Streaming Data with AWS Kinesis and Lambda
Exercise instructions
- Initialize the boto3 session with
AWS_KEY_ID
andAWS_SECRET
. - Pass the session to the AWS Wrangler
read_csv
method to get all the speeder dataframes. - Use AWS Wrangler to write the resulting csv with the key
"sd-vehicle-data/speeders-full/full.csv"
.
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
# Initialize boto3 session
session = boto3.____(aws_access_key_id = AWS_KEY_ID,
aws_secret_access_key = AWS_SECRET,
region_name="us-east-1")
# Read all records in the speeders folder
speeders_total = wr.s3.____('s3://sd-vehicle-data/speeders',
____ = session,
delimiter=" ")
# Write aggregated speeders file
____.____.to_csv(df = speeders_total,
boto3_session=session,
path="____://sd-vehicle-data/speeders-full/full.csv")