On 15 March 2023, the UK ICO published an update to its Guidance on AI and Data Protection (the “Guidance”), following requests from the UK industry to clarify requirements for fairness in artificial intelligence (“AI”). The Guidance contains advice on the interpretation of relevant data protection law as it applies to AI, and recommendations on good practice for organisational and technical measures to mitigate risks caused by AI.
The Guidance was first published in September 2020. The updated Guidance has been restructured using the GDPR data protection principlesas the pillars for the structure. According to the ICO, this will make updating the Guidance in the future more efficient and also “makes editorial and operational sense”.
The update supports the UK Government’s “vision of a pro-innovation approach to AI regulation” and its intention to “embed considerations of fairness into AI”. The ICO also noted that the Guidance will require further updates in the future to keep up with the “fast pace of technological developments” and confirmed it will be supporting the implementation of the UK Government’s white paper on AI. The White Paper set forth a principles-based framework to regulating AI for existing regulators to interpret and apply to AI within their own remits, making this guidance all the more important.[1]
In this blog post, we outline some of the key updates to the Guidance.
Accountability Principle
The update with respect to the accountability principle includes a new section on data protection impact assessments (“DPIA”), which sets out what a company should take into account when conducting a DPIA on AI. In particular, companies must include evidence to demonstrate that “less risky alternatives” were considered and reasoning on why those alternatives were not pursued. When considering the impact of data processing on individuals, companies must consider both allocative harms and representational harms. Allocative harms are the result of a decision to allocate goods and opportunities among a group (e.g. an AI system in recruitment that disproportionally classifies applications from male candidates as suitable compared to women). Representational harms occur when systems reinforce the subordination of groups along identity lines (e.g., image recognition system assigns labels to ‘selfie’ photos uploaded by an individual belonging to an ethnic minority group that are denigrating or reflecting racist tropes).
If the result of an assessment indicates residual high risk to individuals that an organisation cannot sufficiently reduce, the organisation must consult with the ICO prior to starting the processing in question.
Transparency Principle
A new chapter has been added in the updated Guidance, containing high-level content on the transparency principle as it applies to AI. The Guidance cross-refers to the co-badged guidance on transparency produced by the ICO and the Alan Turing Institute, which provides practical advice to companies to help them explain to data subjects how their personal data is used throughout the AI lifecycle.
As a rule, under the UK GDPR, where data is collected directly from individuals, privacy information must be provided to those individuals at the time of collection of their data. In an AI context, this means that the information must be provided before the data is used to train a model or apply that model on those individuals, including the purposes for processing the individuals’ personal data; retention periods for that personal data; and who the personal data will be shared with.
If an organisation collects the personal data from other sources (i.e. not directly from individuals), it must provide this information within a reasonable period and no later than within one month, or even earlier if it contacts that person or discloses that data to someone else. There are exceptions to this requirement under the UK GDPR, which, according to the ICO, would also apply in an AI context. These exceptions include where providing the information would involve a disproportionate effort, but this requirement is strictly interpreted and the fact that the costs of compliance would be high may not always be a factor that is taken into account (see, for example, the First-Tier Tribunal’s decision in Experian v Information Commissioner (2023), subject to appeal).
Lawfulness Principle
A new chapter (which also includes existing content) has been added regarding lawfulness in AI. The new content relates to AI and inferences, affinity groups and special category data. The updated Guidance notes that it may be possible, using AI, to infer or guess details about a person, which may constitute special category data. Whether or not this counts as special category data and triggers the associated provisions of the UK GPDR is dependent on how certain the inference is and whether it is drawn on purpose. In relation to affinity groups, the Guidance states that if an AI system makes inferences about a group, thereby creating ‘affinity groups’, and linking these to a specific individual, then data protection law will apply at multiple stages of the processing (including both the development and deployment stage).
Accuracy Principle
With respect to the accuracy principle, the updated Guidance differentiates between ‘accuracy’ in the context of data protection (as one of the fundamental principles) and “statistical accuracy in AI”. It also states that the accuracy principle applies to all personal data, whether it is information about an individual used as an input to an AI system, or an output of such system. This does not mean, however, that an AI system needs to be 100% statistically accurate to comply with the principle.
Fairness Principle
The update adds a new chapter (which also includes existing content) regarding the fairness principle and how such principle applies to AI. The new content discusses: (i) a non-exhaustive list of legal provisions to consider; (ii) the difference between fairness, algorithmic fairness, bias and discrimination; (iii) high-level considerations when thinking about evaluating fairness and inherent trade-offs; (iv) processing personal data for bias mitigation; and (v) technical approaches to mitigate algorithmic bias.
In addition, a new annex has been added regarding fairness in the AI lifecycle. The annex details data protection fairness considerations across the AI lifecycle, from problem formulation to decommissioning. It also explains how fairness can be impacted by fundamental aspects of AI, identifies the different sources of bias that can lead to unfairness and lists possible measures to mitigate the same.
Security and Data Minimisation, and Individual Rights
No updates have been made to the Guidance chapters on individual rights or security and data minimisation.
The existing Guidance on security and data minimisation provides that companies deploying AI should follow certain data minimisation criteria. The data processed should be adequate (i.e. sufficient to fulfil the stated purpose of processing) and relevant, which requires a rational link between the data processed and the purpose of such processing. Finally, the data processed should be limited only to what is necessary, as organisations should not hold more data than is needed for their stated purpose. Guidance is also provided for security measures for AI systems, describing certain security standards for ensuring that training data is secure and how to ensure the security of externally maintained software. Companies should actively monitor and take into account state-of-the-art security practices when using personal data in an AI context. The Guidance also addresses protection and content of individual rights falling within AI and data protection. Individual rights include the right to rectification, the right to erasure, the right to data portability and the right to be informed. These rights apply wherever personal data is used at any of the various points in the development and deployment lifecycle of an AI system, and hence cover personal data contained in the training data, personal data used to make a prediction during deployment (and the result of the prediction itself), and personal data that might be contained in the model itself.
[1] For additional information on the UK Government’s white paper setting out its proposed approach for regulating AI, please see our article “A “pro-innovation” agenda: the UK Government’s Approach to AI and Digital Technology” (available here).