Identify Bias - TFMA Tool

1. Identify Bias - TFMA Tool

The TensorFlow model Analysis library or TFMA library can help you analyze train model performance from multiple perspectives, including fairness. TFMA is designed to support TensorFlow based models, but can be easily extended to other frameworks such as pandas data frame or scikit learn models. For TF Keras models, it can automatically apply the training metrics at evaluation time and doesn't require additional steps to compute with pre calculated predictions. The results are automatically saved with the model. TFMA supports a wide variety of metrics, including all the standard TensorFlow metrics. This includes regression metrics and classification metrics, and fairness related metrics like flip rate, metric settings can be customized, and you can also define custom metrics entirely. Let's look at TFMA's capabilities in more detail. After training a model, you can first run model analysis on the model. This step will create an evaluation result that can be saved for later use. This result can be visualized by using a TFMA widget that enables you to interactively check model performance metrics. You can also slice the performance by using a sensitive feature such as racial group. With this capability, you can easily check if there is a critical performance gap among different groups. You can also automate the model performance validation process using defined metrics thresholds, for example, in an automated machine learning pipeline. If TFMA finds a critical performance gap, you can stop the model update process before deploying the model. This capability is important when you are building an automated machine learning system. You can compare the performance of two models often used to check the performance improvement of a new model over the preceding model. You can also link the TFMA analysis result to a supplemental library fairness indicators, where you can investigate a model using multiple fairness metrics. The fairness indicator also provides an interactive widget where you can investigate the model performance. This capability can also be incorporated into the what if tool.

2. Let's practice!

Create Your Free Account

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.