1. Upsampling & interpolation with .resample()
In this chapter, you will dive deeper into pandas' capabilities to convert time series frequencies.
2. Frequency conversion & transformation methods
The resample method follows a logic similar to groupby:
It groups data within a resampling period, and applies a method to this group.
It takes the value that results from this method, and assigns a new date within the resampling period.
The new date is determined by a so-called offset, and for instance can be at the beginning or end of the period, or a custom location.
You will use resample to apply methods that either fill or interpolate missing date when up-sampling, or that aggregate when down-sampling. Let's first get the monthly unemployment rate.
3. Getting started: monthly unemployment rate
The 208 data points imported using read_csv since 2000 have no frequency information.
An inspection of the first rows shows that the data are reported for the first of each calendar month.
When looking at resampling by month, we have so far focused on month-end frequency.
In other words, after resampling, new data will be assigned the last calendar day for each month.
4. Resampling Period & Frequency Offsets
There are, however, quite a few alternatives as shown in the table.
Depending on your context, you can resample to the beginning or end of either the calendar or business month.
The example dates show how business dates may deviate from the calendar month due to weekends and holidays.
5. Resampling logic
Resampling implements the following logic:
When up-sampling, there will be more resampling periods than data points.
Each resampling period will have a given date offset, for instance month-end frequency.
You then need to decide how to create data for the new resampling periods.
The new data points will be assigned to the date offsets.
In contrast,
6. Resampling logic
when down-sampling, there are more data points than resampling periods.
Hence, you need to decide how to aggregate your data to obtain a single value for each date offset.
7. Assign frequency with .resample()
You can use resample to set a frequency for the unemployment rate. Let's use month start frequency given the reporting dates.
When you apply the resample method, it returns a new object called Resampler object.
8. Assign frequency with .resample()
Just apply another method, and this object will again return a DataFrame.
You can apply the asfreq method to just assign the data to their offset without modification.
The dot-equal() method tells you that both approaches yield the same result.
9. Quarterly real GDP growth
Let's now use a quarterly series, real GDP growth.
You see that there is again no frequency info, but the first few rows confirm that the data are reported for the first day of each quarter.
10. Interpolate monthly real GDP growth
You can use resample to convert this series to month start frequency, and then forward fill logic to fill the gaps.
We're using add_suffix to distinguish the column label from the variation that we'll produce next.
11. Interpolate monthly real GDP growth
Resample also let's you interpolate the missing values, that is, fill in the values that lie on a straight line between existing quarterly growth rates.
A look at the first few rows shows how interpolate averages existing values.
12. Concatenating two DataFrames
We'll now combine the two series using the pandas concat function.
pandas concat just takes a list of DataFrames with the default axis parameter set to 0, it stacks the two DataFrames, trying to align the columns that here don't match.
13. Concatenating two DataFrames
Using axis=1 makes pandas concatenate the DataFrames horizontally, aligning the row index.
14. Plot interpolated real GDP growth
A plot of the data for the last two years visualizes how the new data points lie on the line between the existing points, whereas forward filling creates a step-like pattern.
15. Combine GDP growth & unemployment
After resampling GDP growth, you can plot the unemployment and gdp series based on their common frequency.
16. Let's practice!
Let's now practice your new skills!