1. Creating a Kinesis data analytics application
Head to the console, and then Kinesis.
Go to data analytics, and create an application.
Create an application called gpsAnalytics.
Under source, connect streaming data, and select the gps-delivery-stream.
Note that we can use a transformational lambda in an analytics application as well to pre-process the records.
Then, click discover schema. Your Firehose stream should be receiving data at this point.
Kinesis will sample the stream and discover the columns of data you have coming in.
Click edit schema.
Let's change the column names so they make sense.
Col1 is record id
Col2 is record_time
Col3 is VIN
col4 is lon
Col5 is lat
Col6 is speed
Save schema and update stream samples. This will take a bit. Make sure your stream is still running.
You will see a table with the updated column names.
Click exit, then go to SQL editor.
Select "Add sql from template"
Select the Top-K items template. Change column1 to VIN, number of top items to 2,
and and tumbling window size to 10 seconds.
This will show us the top ten repeating pings every 10 seconds, letting us see if any transmitter is pinging too frequently.
Click save, and after a bit, you will see the results pop up below.
Your application is working. Now, head over to the destination, and you can send the results to another stream or a lambda function. But that's for next lesson.
Make sure to stop your application when you're done - otherwise it will keep running and costing money!
2. Let's practice!