1. Learn
    2. /
  2. Courses
    3. /
  3. Scalable Data Processing in R

Exercise

Reading chunks in as a data.frame

In the previous example, we read each chunk into the processing function as a matrix using mstrsplit(). This is fine when we are reading rectangular data where the type of element in each column is the same. When it's not, we might like to read the data in as a data.frame. This can be done by either reading a chunk in as a matrix and then convert it to a data.frame, or you can use the dstrsplit() function.

Instructions

100 XP
  • In the function make_msa_table(), read each chunk as a data frame.
  • Call chunk.apply() to read in the data as chunks.
  • Get the total counts for each column by adding all the rows.