APIs & JSON

1. APIs & JSON

In the previous video and exercises,

2. Other data formats

you've seen how you can get files from the web. In the last exercise, you also saw that there is also another format to represent data, namely JSON. The JSON format is very simple, concise and well-structured. On top of that, it is human-readable, but also easy to interpret and generate for machines. This makes it perfect to communicate with Web APIs.

3. API

What is an API, I hear you asking? Well, it's short for Application Programming Interface. Very generally put, it is a set of routines and protocols for building software components. It is a way in which different software components interact. This can happen in thousands of ways, but we will only focus on the web API. Typically, this is an interface to get data and processed information from a server or to add data to a server, through the HTTP methods you learned about earlier.

4. Twitter

As an example, take Twitter. They also have an API, which you can check out here. After authentication, you can simply request particular URLs from twitter with the GET request, allowing you to get the tweets for a particular person for example. Twitter does all the work for you on their servers, and spits out the information you need. You can also use the API to programatically place comments on somebody's tweets for example. The possible applications are endless. Say you're an avid tweeter and want to find out which tweets had the most impact. Maybe some of your more controversial tweets caused a lot of retweets and reactions? With the raw data from Twitter's API you can research this. But why are APIs and JSON useful in the first place? All the information is available on webpage already, right?

5. Info on Rain Man (1988)

Well, suppose you want to get some data on the movie Rain Man, from 1988. You could download the corresponding URL of IMDb, read it in, and then start programmatically searching your way through the HTML code, which is more than 4000 lines, to get the information you need. This is pretty error-prone and really slow. Luckily, there are alternatives. As it happens,

6. Rain Man JSON (OMDb API)

there's an API called the OMDb API, able to give you information on pratically any movie you can think of. It simply takes a URL, with some additional parameters such as the id of the movie you want to get info about and what type the response should be, that the API understands. If you simply visit this web page, you will see something like this. This is a JSON, containing everything we need to know in a nicely structured format. To convert this JSON into an R data structure,

7. jsonlite

you can use the jsonlite package by Jeroen Ooms. The package is an improvement of earlier R packages to handle JSONs, where converting from and to JSONs is more consistent and robust, such that it works nicely in all use cases.

8. Rain Man list in R

Let's install and load jsonlite, and then simply call fromJSON on the URL. This will download the JSON data for you and convert it to a named R list. We now have all the information in R, ready to do our analysis. This is way better than the messy HTML page where we would have to dig through to get the bits and pieces of interest, isn't it?

9. Let's practice!

I suggest you head over to some exercise, before we take a deeper dive into the JSON format. Enjoy!

Create Your Free Account

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.