Back to the office

You are still working as a data analyst for a web agency, and you've been asked to do web scraping. You have been given a list of URLs to analyze, an analysis you've already started in the previous chapter.

You expect this task to be recurrent: no doubt you'll be asked to do it again in a few weeks. In order to make your future work easier, you've decided to try and write clean code today, so that it will be easier to come back to it later.

We'll start by combining the two functions from httr we've seen in the previous chapter: GET(), for retrieving the webpage, and status_code(), to extract the status code, in order to create a status code extractor.

The urls vector is still available in your workspace. We have kept only the URLs that are reachable.

This exercise is part of the course

Intermediate Functional Programming with purrr

Exercise instructions

Launch purrr and httr.
Compose a status extractor with GET() and status_code().
Try this new function on "https://www.thinkr.fr" and "https://en.wikipedia.org".
Map this function directly on the vector urls.

Hands-on interactive exercise

Have a go at this exercise by completing this sample code.

# Launch purrr and httr



# Compose a status extractor 
status_extract <- ___(___, ___)

# Try with "https://thinkr.fr" & "https://en.wikipedia.org"
___("https://thinkr.fr")
___("https://en.wikipedia.org")

# Map it on the urls vector, return a vector of numbers
___(urls, ___)

Edit and Run Code

This exercise is part of the course

Intermediate Functional Programming with purrr

IntermediateSkill Level

4.9+

Start Course for Free

Do lambda functions, mappers, and predicates sound scary to you? Fear no more! After refreshing your purrr memory, we will dive into functional programming 101, discover anonymous functions and predicates, and see how we can use them to clean and explore data.

Exercise 1: purrr basics - a refresher Exercise 2: Refreshing your purrr memory Exercise 3: Another purrr refresher Exercise 4: Introduction to mappers Exercise 5: Creating lambda functions Exercise 6: Lambda functions Exercise 7: Using mappers to clean up your data Exercise 8: Clean up your data with keep Exercise 9: Split up with keep() and discard()Exercise 10: Predicates Exercise 11: What is a predicate?Exercise 12: Exploring data with predicates

Ready to go deeper with functional programming and purrr? In this chapter, we'll discover the concept of functional programming, explore error handling using including safely() and possibly(), and introduce the function compact() for cleaning your code.

Exercise 1: Functional programming in R Exercise 2: Everything that happens is a function call Exercise 3: Identifying pure functions Exercise 4: Tools for functional programming in purrr Exercise 5: Safe iterations Exercise 6: Create a function Exercise 7: Using possibly()Exercise 8: A possibly() version of read_lines()Exercise 9: Everything in one call Exercise 10: Handling adverb results Exercise 11: Purrrfecting our function Exercise 12: Extracting status codes with GET()

In this chapter, we'll use purrr to write code that is clearer, cleaner, and easier to maintain. We'll learn how to write clean functions with compose() and negate(). We'll also use partial() to compose functions by "prefilling" arguments from existing functions. Lastly, we'll introduce list-columns, which are a convenient data structure that helps us write clean code using the Tidyverse.

Exercise 1: Why cleaner code?Exercise 2: How to write compose()Exercise 3: Back to the office

Current Exercise

Exercise 4: Building functions with compose() and negate()Exercise 5: Build a function Exercise 6: Count the NA Exercise 7: Prefilling functions Exercise 8: A content extractor Exercise 9: Another extractor Exercise 10: List columns Exercise 11: About list-columns Exercise 12: Create a list-column data.frame

We'll wrap up everything we know about purrr in a case study. Here, we'll use purrr to analyze data that has been scraped from Twitter. We'll use clean code to organize the data and then we'll identify Twitter influencers from the 2018 RStudio conference.

Exercise 1: Discovering the dataset Exercise 2: Playing with tweets, round 1 Exercise 3: Identify profiles Exercise 4: Extracting information from the dataset Exercise 5: Counting favorites Exercise 6: Extracting mentions Exercise 7: Manipulating URLs Exercise 8: Analyzing URLs Exercise 9: Playing with URLs Exercise 10: Identifying influencers Exercise 11: Splitting the dataset Exercise 12: We have a winner!Exercise 13: Congratulations!