Transformer models are notoriously trained on biased data, which they then propagate through the training, test, validation cycle and in production use cases. There are many types of biases at various stages of the process. The below highlight the different bias cases in the cycle that could evolve and provides a few suggestions for resolution.
Training Data is Collected and Annotated:
- Reporting Bias
- Selection Bias
- Stereotyping
- Racism
- Underrepresentation
- Gender Bias
- Human Bias
Model Trained:
- Overfitting
- Underfitting
- Default Effect
- Anchoring Bias
Media is Filtered, Aggregated, and Generated:
- Confirmation Bias
- Congruence Bias
People See Output:
- Automation Bias
- Network Effect
- Bias Laundering
How to Resolve Transformer Bias:
- Feedback Loop
- Model Cards for Model Reporting
- Open Review
- TLDR
