In this first chapter, you’ll be introduced to your role as a data engineer in a private equity fund. You'll be exposed to the whole ETL pipeline before deep-diving into its first phase: the extraction process.
In this chapter you're going to create some important foundations for our ETL pipeline. In particular, along with data transformation, you'll start setting up the components needed to communicate with the database.
This chapter is all about moving transformed data to a clean table, from which insights can be built. You'll explore how to create a unique key to perform insert and delete statements on SQLAlchemy. At the end of this chapter you'll complete the load process, the last step of the ETL pipeline.
This chapter will show you how the data the ETL pipeline processes every month is used to build insights, readable by the fund’s shareholders.
You'll explore key SQL components to build more complex queries and create these insights. You'll also explore libraries that will translate raw SQL queries into more readable Excel files.