1. Reading date and time data in Pandas
In this chapter, you will use the Pandas library to work with dates and times. You should have encountered Pandas before, but now we will add datetimes to the mix.
2. A simple Pandas example
To start with, let's load data with Pandas. First, we import pandas, and as is customary we use the alias pd.
Our data is in a csv file, so we load it with the read_csv() function. pd-dot-read_csv() has one required argument, the name of the file to load, which in this case is capital-onebike-dot-csv. We save the result to the variable rides.
Let's print the first three rows to see what we've got.
3. A simple Pandas example
Note that the index, listed all the way to the left, starts with zero. Because the table is too wide, it wraps around. Each of these three rows has a start date, an end date, a start station, and end station, the bike number, and whether the ride was from someone who is a member or someone who walked up to the kiosk and bought a ride on the spot.
4. A simple Pandas example
We can also select a particular column by using the brackets, as here where we call rides['Start date']. And we can get a particular row with dot-iloc[], in this case row number 2.
Because we didn't tell Pandas to treat the start date and end date columns as datetimes, they are simply strings or objects. We want them to be datetimes so we can work with them effectively, using the tools from the first three chapters of this course.
5. Loading datetimes with parse_dates
If we want Pandas to treat these columns as datetimes, we can make use of the argument parse_dates in read_csv(), and set it to be a list of column names, passed as strings. Now Pandas will read these columns and convert them for us to datetimes.
Pandas will try and be intelligent and figure out the format of your datetime strings. In the rare case that this doesn't work, you can use the to_datetime() method that lets you specify the format manually. For more details, see the Pandas documentation.
6. Loading datetimes with parse_dates
Now when we again ask for the Start date for row 2, we get back a Pandas Timestamp, which for essentially all purposes you can imagine is a Python Datetime object with a different name. They behave basically exactly the same.
7. Timezone-aware arithmetic
Since our Start date and End date columns are now datetimes, we can deal with them the way we usually deal with datetimes. For example, we can create a new column, Duration, by subtracting Start date from End date. Because each of these columns are datetimes, when we subtract them we get timedeltas.
If we print out the first 5 rows, we get that the first ride lasted for only 3 minutes and 1 second, the second ride lasted for 2 hours and 7 minutes, the third ride lasted for 5 minutes 43 seconds, and so on.
8. Loading datetimes with parse_dates
Pandas has two features worth noting here. Let's see an example of converting our Duration to seconds, and looking at the first 5 rows.
First, Pandas code is often written in a "method chaining" style, where we call a method, and then another, and then another. For readability, it's common to break them up with a backslash and a linebreak at the end of each.
Second, you can access all of the typical datetime methods within the namespace -dot-dt. For example, we can convert our timedeltas into numbers with dot-dt-dot-total_seconds().
Now when we look at the results, we see that we've got seconds instead of timedeltas. Our first ride lasted 181 seconds, our second ride 7622 seconds, and so on.
9. Reading date and time data in Pandas
In this lesson, we discussed loading data in Pandas, and handling basic datetime elements. We talked about using slashes to continue lines, and selecting subsets of rows.
Time to practice!