ChaptersBatch Processing

Batch Processing

How do you compute over all the data at once, reliably, at any scale?

1 lesson · 7 concepts · ≈ 30 min

Not started

  1. Lesson 1 · ≈ 30 min

    Batch Processing

    Take a pile of data that has stopped changing, run a pure function over all of it on as many machines as you like, and replace the old answer. From Unix pipes to MapReduce to Spark and DataFrames, how bulk computation works, and where it hurts.