On April 12, 2024, the Information Commissioner's Office (ICO) announced that it had launched the third chapter of its consultation series on generative artificial intelligence (AI). In particular, the third chapter of the consultation focuses on how the accuracy principle of data protection applies to the outputs of generative AI models and the impact that accurate training data has on the output.

Should the outputs of generative AI models be accurate?

The consultation explains that the level of accuracy required of the outputs of generative AI models depends on how the model will be used, with high accuracy needed for models that are used to make decisions about people or that are relied on by users as a source of information. 

On the other hand, the consultation notes that organizations developing and using generative AI models that have a purely creative purpose are unlikely to need to ensure that the outputs are accurate as their first priority. By way of example, the consultation highlights that models used to triage customer queries would need to maintain higher accuracy than models used to help develop ideas for video game storylines.

Obligations of developers

To ensure the accuracy of the training data used to develop generative AI models, the consultation recommends that developers should:

• know whether the training data is made up of accurate, factual, and up-to-date information;
• understand and document the impact that the accuracy of the training data has on the generative AI model outputs;
• consider whether the statistical accuracy of the generative AI model output is sufficient for the purpose for which the model is used and how that impacts data protection accuracy; and
• clearly communicate the three points above to deployers and end users to ensure that the lack of accuracy at the training stage does not result in negative impacts on individuals at the deployment phase.

Obligations of deployers

The consultation highlights that deployers of generative AI are responsible for clear communication with end users, and are expected to:

• consider how a potential lack of accurate training data and outputs could impact individuals and mitigate those risks ahead of deployment;
• provide clear information about the statistical accuracy of the application and its intended use; and
• monitor how the application is used to inform and, if necessary, improve the information provided to the public and restrictions on the use of the application.

Comments for the third consultation can be submitted online or via email at [email protected] until May 10, 2024.

You can read the press release here and the consultation here.