In which you learn how Spark and R complement each other, how to get data to and from Spark, and how to manipulate Spark data frames using dplyr syntax.

Getting started

Made for each other

Here be dragons

The connect-work-disconnect pattern

Copying data into Spark

Big data, tiny tibble

Exploring the structure of tibbles

Selecting columns

Filtering rows

Arranging rows

Mutating columns

Summarizing columns

Light My Fire: Starting To Use Spark With dplyr Syntax

In which you learn more about using the <code>dplyr</code> interface to Spark, including advanced field selection, calculating groupwise statistics, and joining data frames.

Leveling up

Mother's little helper (1)

Mother's little helper (2)

Selecting unique rows

Common people

Collecting data back from Spark

Storing intermediate results

Groups: great for music, great for data

Groups of mutants

Advanced Selection II: The SQL

Left joins

Anti joins

Semi joins

Tools of the Trade: Advanced dplyr Usage

In which you learn about Spark's machine learning data transformation features, and functionality for manipulating native DataFrames.

Two new interfaces

Popcorn double feature

Transforming continuous variables to logical

Transforming continuous variables into categorical (1)

Transforming continuous variables into categorical (2)

More than words: tokenization (1)

More than words: tokenization (2)

More than words: tokenization (3)

Sorting vs. arranging

Exploring Spark data types

Shrinking the data by sampling

Training/testing partitions

Going Native: Use The Native Interface to Manipulate Spark DataFrames

A case study in which you learn to use <code>sparklyr</code>'s machine learning routines, by predicting the year in which a song was released.

Machine learning on Spark

Machine learning functions

(Hey you) What's that sound?

Working with parquet files

Come together

Partitioning data with a group effect

Gradient boosted trees: modeling

Gradient boosted trees: prediction

Gradient boosted trees: visualization

Random Forest: modeling

Random Forest: prediction

Random Forest: visualization

Comparing model performance

Case Study: Learning to be a Machine: Running Machine Learning Models on Spark

Anti-join

Both-model-responses

Gbt-model-responses

Inner-join

Left-join

Predicted vs actual

Residual density

Semi-join

Timbre

Timbre parquet

Title text parquet

Track data parquet

Track data to model parquet

Track data to predict parquet

Track metadata

R is mostly optimized to help you write data analysis code quickly and readably. Apache Spark is designed to analyze huge datasets quickly. The sparklyr package lets you write dplyr R code that runs on a Spark cluster, giving you the best of both worlds. This course teaches you how to manipulate Spark DataFrames using both the dplyr interface and the native interface to Spark, as well as trying machine learning techniques. Throughout the course, you'll explore the Million Song Dataset.

<h2>Explore the Advantages of R, Spark, and sparklyr </h2>
R is mostly optimized to help you write data analysis code quickly and readably. Apache Spark is designed to analyze huge datasets quickly. The sparklyr package lets you write dplyr R code that runs on a Spark cluster, giving you the best of both worlds. This 4-hour course teaches you how to manipulate Spark DataFrames using both the dplyr interface and the native interface to Spark, as well as trying machine learning techniques. 
<br><br>
<h2>Load Data into Spark and Manipulate Spark DataFrames </h2>
You’ll start this Spark course by investigating how Spark and R work well together and practicing loading data, ready for cleaning, transformation, and analysis. You’ll use Spark frames and dplyr syntax to manipulate your data by filtering and arranging rows, and mutating and summarizing columns. 
<br><br>
<h2>Delve into Big Data Analysis with Spark MLib </h2>
This course focuses on building your skills and confidence in analyzing huge datasets. The final chapters take you through Spark’s machine learning data transformation features and offer you the chance to practice sparklyr’s machine learning routines by using it to make predictions using gradient boosted trees and random forests. "

Supervised Learning in R: Regression

Learn to use Apache Spark and the sparklyr package in R for big data analysis via machine learning. 

Introduction to Spark Course with sparklyr in R | DataCamp 

Learn how to run big data analysis using Spark and the sparklyr package in R, and explore Spark MLIb in just 4 hours. 

Introduction to Spark with sparklyr in R

Big Data in R

Machine Learning Scientist in R

Data Evangelist at DataCamp

The MLlib machine learning interface

Feature transformations

The Spark DataFrame interface

Let's practice!

https://s3.amazonaws.com/assets.datacamp.com/production/course_3309/subtitles/course_3309_2e7d7614b379d5cf3a8242d2ff1e9d1c.vtt

Going Native: Use The Native Interface to Manipulate Spark DataFrames - Two new interfaces

Two new interfaces

1. Two new interfaces

2. The MLlib machine learning interface

3. Feature transformations

4. The Spark DataFrame interface

5. Let's practice!