1. Asynchronous processing
In this video, we'll explore how to implement asynchronous processing in FastAPI to improve performance and handle multiple requests efficiently.
2. What is asynchronous processing
Imagine a restaurant where each waiter can only serve one table at a time until the food is served to that table. That would be a disaster for the restaurant.
It is more efficient for waiters, in our case APIs, to handle multiple requests simultaneously. Asynchronous processing allows us to handle multiple requests concurrently.
3. Synchronous vs asynchronous requests
Handling concurrent requests can be critical to scaling APIs, especially when serving the requests require compute power that can take some time. For example, if a request typically takes two seconds to process, the fastest we can expect to process three of them synchronously is six seconds. If we handle them asynchronously, the total time for all three is equal to the longest request processing time, about two seconds.
4. Turning synchronous endpoints asynchronous
Here's the fundamental difference: the synchronous code blocks other operations until completion, while async allows other operations to continue.
We use asyncio, which is a Python library for processing concurrent requests using async and await syntax. Think of it like a manager that helps Python do multiple things at once, especially when waiting for slow operations.
'async' marks a function as a "coroutine" - meaning it can pause and resume (using await). Await tells Python to "wait here for this task to finish, but let other code run while waiting."
The asyncio.to_thread method ensures our CPU-intensive sentiment analysis doesn't block the event loop.
5. Implementing background tasks
If we have loads of comments to analyze, we can't let the users wait for the processing to be done sequentially. Background tasks manage the individual processing so we can process large batches of comments without making users wait for them to be run sequentially.
We import BackgroundTasks from FastAPI, which lets us process data after responding to the client.
Inside our async endpoint, we have declared a parameter background_task of type BackgroundTasks. It creates a new instance of BackgroundTasks for this request with multiple comments and injects this dependency into our function.
We then create an inner async function 'process_comments' that loops through each text. The background_tasks.add_task method schedules this processing to happen after we send our response.
This means users don't have to wait for processing to complete - they get an immediate response while analysis happens in the background.
6. Adding error handling
Robust error handling is crucial in async applications.
Here we have an endpoint that analyzes comments using the pre-trained sentiment_model.
We use asyncio.wait_for to set a timeout of 5 seconds - if our model takes more than 5 seconds, it raises a TimeoutError.
7. Adding error handling
We catch this and other exceptions using try-except blocks.
FastAPI's HTTPException lets us return HTTP status codes: 408 for timeouts and 500 for other errors.
This ensures our API fails gracefully and provides clear error messages to clients.
8. Let's practice!
We have learned how to process multiple requests asynchronously and how to process a big batch of workload in the background. Let's put all this into practice!