We analyse the space where the rules around automated decision-making intersect with the deployment of AI in connection with personal data.
Article 22(1) of the General Data Protection Regulation (GDPR) provides individuals with the right not to be subjected to decisions based solely on automated processing that have a legal or similarly significant effect on them, otherwise known as “automated decision-making”.
The concept of artificial intelligence (AI) is not readily distilled into a universal definition. However, it can be described as the “umbrella term for a range of algorithm-based technologies that solve complex tasks by carrying out functions that previously required human thinking”.
What exactly is automated decision-making and what are the rules around it?
Automated decision-making is a purely automated process; there cannot be any meaningful human involvement. The decision must have a legal or similarly significant effect, for example the refusal of an online credit application or the outcome of an online recruitment process.
Automated decision-making is permissible where it is based on explicit consent, contractual necessity, EU law or EU member state law, and where the following measures are in place:
- Suitable safeguards - these should go beyond compliance with the core data protection principles and include, at the very least, the right to obtain human intervention in relation to the decision, and contest or otherwise express views on the decision.
- Transparent information – this should include at a minimum, meaningful information on the logic involved and the significance and envisaged consequences of the decision.
What does this mean in the context of the deployment of AI?
AI is often deployed without engagement with the data subject, on the basis of legitimate interests under Article 6(1)(f) of the GDPR, a legal basis which is not available for automated decision-making. Therefore, care must be taken in selecting the appropriate legal basis for AI-driven automated decision-making.
The ICO guidance on AI and data protection explores a number of other difficulties associated with AI in the automated decision-making context, including:
- Human review being rendered impossible or prohibitively difficult because of interpretability issues due to the sophisticated, impenetrable nature of the underlying AI.
- AI based on machine learning being unable to properly identify “outlier” cases that don’t match the profile of the data used in its training.
- Transparency being undermined by the complex nature of the underlying technology, with the effect that the data subjects involved are “overloaded” with information that is not readily comprehensible. The Article 29 Working Party guidelines on automated decision-making and the ICO guidance on automated decision-making are both aligned in the view that a detailed explanation of the technical processes is not required. Rather, the focus should be on explaining what data is used, why and how the decision was reached in a manner that equips the data subject with the information they need to request human intervention or otherwise “appeal” the decision.
Given the nature of AI, internal understanding, accessibility and control regarding the underlying technology and the data it uses are vital elements of being able to explain the decisions made by that technology. With this in mind, the ICO recommends data processing in an “explanation-aware” manner. This includes having sound practices in place for:
- Data labelling and categorisation
- Data storage
- Access and extraction
- Reviews and audits, and
- Record keeping, including records in relation to decisions made and the allocation of functions / designation of roles, for example clarity on the individuals responsible for different phases of the processing.
Automated decision-making is a processing activity that is fraught with risk from a data protection perspective. When complex, sophisticated AI is added to the mix, these risks are all the more heightened. This is especially true in circumstances where both data subjects and individuals behind the decision-making are unable to fully comprehend, and by extension contest or control, the technology underlying the processing. Organisations engaging in automated decision-making that involves the deployment of AI should be aware of these issues, and their attendant obligations in this space.
For more information, contact a member of our Technology team.
The content of this article is provided for information purposes only and does not constitute legal or other advice.
 See recital 71 of the GDPR. See also the Article 29 Working Party guidelines on automated decision-making and profiling which contain non-exhaustive examples of decisions producing “serious impactful effects”.
 Where special categories of personal data under Article 9(1) of the GDPR are involved, Article 22(4) provides that the automated decision-making can only be based on express consent or where it is necessary for reasons of substantial public interest.
Sophisticated governance models and oversight will be particularly important where AI processing is outsourced or deployed with the engagement of third-party service providers.