Monitoring and logging
1. Monitoring and logging
Now let's learn how to implement logging and monitoring to verify and troubleshoot our FastAPI apps.2. Why monitoring and logging?
First let's talk about why we should set up monitoring and logging. One reason is we usually can't debug in production, so we need information from the production server to reproduce issues in another environment. Also in production we are usually running our app with a script, so we need a quick and simple way for the script to verify when the app is up and running and continues to be healthy. Finally logging helps us measure and track key metrics over time to assess long-term stability. For example, we might want to log model inference time to assess server load over time to determine if we need to upgrade our server.3. Setting up custom logging
To start logging with FastAPI, we can simply load the uvicorn error logger. Now we can add custom logs, like this message after app startup. We can also add custom logs to our endpoints.4. Logging a when a model is loaded
A practical example of logging for AI applications is that often we want to load models into memory before starting the app. Here we instantiate the uvicorn error logger, then we load our model into memory and follow that with a log message to let us know the model loaded before we run the app.5. Logging process time with middleware
Logging can also be done with middleware to apply it systematically across different endpoint domains. Here we use the @app.middleware annotation to measure the time between receiving any http request and sending a response. The middleware function receives the request and a function call_next that will receive the request as a parameter. This function will pass the request to the corresponding path operation. Then it returns the response generated.6. Setting the logging level
The increasing Python logging levels and their underlying numeric values are shown here. The default logging level for FastAPI is "info," so by default FastAPI will only show logs with value 20 or higher. If we want to change that, we can start the uvicorn server with the --log-level flag set.7. Monitoring
Now let's talk about monitoring. Monitoring can mean many things, but in the context of FastAPI we are really only talking about monitoring the health of our app. In the case of APIs, it is standard practice to simply provide a GET endpoint at slash-health that basically says, "I'm ok!"8. Sharing model parameters with monitoring
In many applications it can be helpful to provide some extra data along with the application status. In AI apps, it's often a good idea to include information about the model that is running, so that systems and users can verify they are using the model they intended. Here we have an app that has loaded the penguin classifier model. We can add a health check endpoint that includes the scikit-learn model parameters along with the status, as if to say "I'm ok!" and "Here are some fun facts about me!"9. Let's practice!
Now that we've learned how to incorporate monitoring and logging into our applications, let's practice!Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.