1. Automated decision making and profiling
Welcome back!
2. AI: Lawfulness, fairness, and transparency
If they process personal data, AI projects need a suitable legal basis.
If you use AI to infer data or information about people, your processing needs to be fair. That is, your AI should not assume wrong or biased information about people.
You need to be transparent about how you process personal data in an AI system, which may be complex given the black-box nature of some algorithms.
While AI is not explicitly mentioned in GDPR, several special provisions have been made to AI systems, generally specified as automated decision-making, including profiling, in Article 22 of GDPR.
3. Article 22
Article 22 requires organizations to ensure fairness, non-discrimination, and transparency in automated decisions.
It states the rights of data subjects especially related to automated decision-making, including profiling.
It also empowers individuals to seek human intervention and an explanation for fully automated decisions.
4. Profiling
Profiling means any form of automated processing of personal data to predict future behavior.
You are carrying out profiling if you:
Carry out large-scale processing of personal data using AI algorithms: build links between different behaviors and attributes; create profiles based on the processing and associations and predict individuals' behavior based on their assigned profiles.
Profiling may be used for good, like machine learning to predict treatment success rate for a patient based on their assigned profile.
On the other side, it could have ethically complex consequences.
5. Ethical concerns related to profiling
AI algorithms may propagate or reinforce harmful biases when used for profiling,
This is due to a complex combination of their design choices, biased datasets, and the inherent complex or black-box nature.
There have been many examples of discriminatory AI-based profiling related to gender and ethnicity. Examples include women being profiled as a group with low credit worthiness, and persons of color being attributed a higher risk of re-offense.
6. The famous profiling example
One of the most notorious examples of profiling is the Cambridge Analytica scandal.
In 2016, about 320000 users took an online personality test on Facebook and were told that the test was for academic research.
They also collected the personal data of all the friends of the test takers, up to 50 million of them, without their explicit consent.
They combined all the information to create personality/political profiling to identify potential voters likely to change their voting behavior and targeted political ads to those groups.
This profiling-based targeting influenced several election results worldwide, most prominently in the US and the UK.
Laws like GDPR would have heavily deterred such profiling activities. Now let's learn about automated decision-making.
7. Automated decision making
Automated decision-making is the ability to make decisions using AI algorithms without human involvement. Automated decisions can be based on any data, such as responses to a survey, location data, or inferred data from profiling, like a credit score.
Automated decision-making may or may not be based on profiling.
Examples include an automated screening of CVs for candidate selection. Let's find out more about automated decision-making's fairness and transparency implications with an example.
8. Banking example
A bank may use an automated system to decide whether an individual gets a mortgage. Such a system usually gives only an yes or no answer due to the black-box nature of AI models. However, banks can adopt explainable AI mechanisms to explain why the model took such a decision.
This not only ensures that the mortgage applicant is not affected by an inherent bias in AI but also allows human intervention to detect and correct if AI makes a mistake.
9. Let's practice!
In the next video, we'll learn more about the risks and opportunities of AI. Before that, let's try out your new knowledge.