Automating data pipelines

You are working for an online travel agency, and you are tasked with creating an Application Programming Interface (API) that will exchange information with an insurance company in real time to check if travelers have the correct insurance policy. This API will be tied to a microservice called insurance_checker.

Now, you want to…

1. establish data pipelines that will record the results of the insurance_checker API
2. automatically transform and record the data every night to another database

Which tool and what pipeline method can you use for these steps?