1. Reviewing the input data
Throughout this course, we will use the US hourly demand for electricity to demonstrate the deployment process of a forecasting model. Let's review the input data.
2. The US hourly electricity demand
The US hourly demand for electricity is an hourly time series data. From the plot, we can see there is a strong seasonality component — high consumption during the summer and lower consumption during the fall and spring, with a minor pick-up during the winter.
3. Seasonality analysis
If we zoom in on the series, we notice additional seasonal patterns within the day of the week and the hour of the day.
4. Autocorrelation analysis
Those strong seasonal patterns are well pronounced on the series autocorrelation plot. We can see a strong correlation with the daily lags and the weekly lag. Those patterns will play a critical role in modeling the series in the next lesson.
5. The EIA API
This data is sourced from the U.S. Energy Information Administration website. The Energy Information Administration (or, in short, EIA) is part of the U.S. Department of Energy.
6. The EIA API
The US EIA provides access to more than one million series via an API. The API is free to use, but requires registration via the website to obtain an API key.
7. The EIA API
Let's see how to extract data from the API.
We start by loading the required Python libraries:
requests to send a GET request to the EIA API,
pandas to process the data,
and the os library to load the API key from an environment variable.
8. The EIA API
Next, we define the GET request by setting the API endpoint and the path of the series we want to pull. At the time of the course recording, the current API version is V2. We will set the path to the electricity data.
9. The EIA API
We concatenate the GET request components and set the request as a data request by adding the data argument and setting it to value. Here is the full path.
10. The EIA API
Last but not least, let's load the API key,
then add it to the GET request.
Note that the API key is personal, and you will have to register on the EIA website to obtain your own key.
11. The EIA API
Next, we use the requests.get method to send a GET request and reformat the output from JSON to pandas DataFrame.
12. The EIA API
Here is a preview.
While working with the EIA API is out of the course scope, I highly recommend checking the API.
13. The EIA API
A good place to start with the EIA API would be the API dashboard. The API dashboard enables you to explore and search the data that available on the API and filter by different facets and provides the GET request details.
14. Data preparation
We will work with the statsforecast and mlforecast libraries for forecasting, which have specific data requirements, as shown here. Let's load the data from a CSV file and prepare it.
We subset for the columns we need, then use the to_datetime function to modify the series timestamp to datetime object, and sort the series by its timestamp.
15. Data preparation
Next, we rename the period column to ds and the value column to y, then add a unique id column. Since we have only one series, we set the ID as one.
16. Data preparation
The data is now ready to use.
17. Let's practice!
Let's practice GET requests and data preparation. Afterwards, we will model our data using the Nixtla package.