Looping and Batch Processing
1. Looping and Batch Processing
Welcome back! So far in this course,2. Batch Processing
every workflow we've built has processed all its items in one shot. The data flows in, every item gets handled at once, and the workflow finishes. That works fine when you have five or ten items. But what happens when you have fifty? A hundred? A thousand? If we're pushing records to a CRM or sending notification emails for each item, blasting all of them at once can overload APIs, hit rate limits, or even get our accounts blocked. And if something fails halfway through, we've got no idea which items were processed and which weren't. That's why3. Batch Processing
batch processing exists — instead of processing everything at once, we break the data into smaller groups and handle them one batch at a time. The node that makes this possible in n8n is called Loop Over Items. Give it a dataset and a batch size, and it feeds the downstream nodes one batch at a time. Let's give this a go!4. Example: A Basic Batch Process
Here's a Manual Trigger with 15 customer contact records already pinned — each with an ID, name, email, and region. Instead of processing all of them at once, let's break them into batches. First, add a Loop Over Items node and set the batch size to 5. Then add an Edit Fields node and set up two fields: a processed flag set to true, and a batch_label. For the label, use the expression $runIndex — n8n tracks which batch is currently running, starting at zero and going up by one each time. Now connect the Edit Fields node back to Loop Over Items to complete the loop, and hit execute. The output shows each batch running separately — 15 items, three batches of 5. If a batch fails, it's immediately clear which items had the problem. Now let's see how batching helps with rate-limited APIs.5. Example: Batching API Requests
Many APIs have rate limits — maybe 10 requests per second, or 100 per minute. Blasting through all batches back to back will still hit those limits. That's where the Wait node comes in. We can place a Wait node inside our loop, right after the processing step, and it pauses execution for a configurable amount of time before the next batch starts. If we set it to 2 seconds, there's now two seconds of breathing room between each batch. The combination of Loop Over Items for batching and Wait for pacing is one of the most common patterns in production n8n workflows.6. Let's practice!
In the exercises, pay particularly close attention to the execution view as you run these workflows — watching batches process one at a time is the best way to understand how the loop actually works.Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.