1. Bias mitigation
Let's look at our final topic, ways to mitigate bias.
2. We will cover
Bias mitigation involves implementing strategies throughout the project lifecycle, from preprocessing to modeling and further use to ensure fair and equitable outcomes across diverse groups.
Let's look at each project step's main bias mitigation strategies. We'll discuss re-sampling, re-weighting, re-labeling, sensitivity attribute removal, adversarial training, model re-calibration, user behavior, fairness constraints, and algorithm selection.
This list is not exhaustive; other strategies exist depending on your project.
3. Bias revisited
Recall that bias refers to systematic errors or prejudices in data collection, processing, model development, or application that lead to unfair outcomes for specific groups or individuals.
4. Bias across project
Data collection issues, such as non-representative samples or historical prejudices, can cause bias in raw data.
Bias in preprocessing often stems from imbalanced datasets or incorrect data handling, leading to skewed representations.
During modeling, bias can result from the algorithm's assumptions, feature selection, or model choices.
Post-deployment bias arises from the model's interaction with real-world data, and changes in data patterns result in model drift.
5. Bias in pre-processing
Bias mitigation in data preprocessing involves techniques like re-sampling, re-weighting, re-labeling, and sensitivity attribute removal to ensure the training data has minimal bias before model training commences.
6. Strategies in pre-processing
Let's have a close look at some of those techniques.
Re-sampling adjusts the training dataset's distribution to balance class representation.
Re-weighting uses class weights to correct biases without changing the dataset size.
Re-labeling updates data point labels for a more accurate target variables representation.
Sensitivity attribute removal eliminates protected attribute features to prevent the model from learning biases associated with these characteristics.
7. Bias in modeling
Bias in modeling arises not from the training data but from the algorithm itself.
Popular techniques to address this bias include:
First, adversarial training pairs the model with an adversary to highlight its biases, pushing the model to overlook these biases and make more equitable decisions.
Second, fairness constraints are added to the model specification, making fairness an explicit consideration during training.
Lastly, careful selection of algorithms helps minimize bias. This might involve opting for models less likely to overfit biased data, such as simpler models with fewer parameters or those specifically designed for robustness and generalization, like decision trees, regularized linear models, or ensemble methods.
Note that bias-mitigating techniques can inadvertently reduce model accuracy and worsen model performance. They also can lead to overcorrection and introduction of new biases.
8. Financial advisor
In our Financial Advisor project, we implement adversarial training by identifying patterns where the model's financial advice disproportionately favors users from more economically stable regions to address the bias related to regional economic disparities. We then create an adversarial model that intentionally targets these patterns, simulating inputs from less economically stable areas to expose the primary model's biases.
During training, the primary model receives feedback from the adversarial model, highlighting when and how its predictions are unfairly skewed towards more affluent regions. This forces the primary model to learn and correct these biases.
We use decision trees and regularized linear models to address the risk of overfitting to data from more economically stable regions. These models excel in generalizing across diverse scenarios.
9. Bias in post-deployment and use
When the model is deployed, we monitor bias to prevent model drift.
If bias is detected, we use model re-calibration to adjust model parameters to maintain equity as user demographics and economic contexts evolve.
We also need to consider improper application use by users — such as incorrect data inputs or misunderstanding recommendations. This can also impact model effectiveness but should be addressed by educating the user or improving the user interface.
10. Financial advisor
In the Financial Advisor project, if the model was initially trained with data that underrepresents the growing freelance economy, it may offer less optimized advice over time and require model re-calibration. However, performance drops due to users not updating their financial information in the app is incorrect use and does not require model changes.
11. Let's practice!
Let's practice!