Clifford Chance

On 30 March 2023, the Italian Data Protection Authority (the Garante) issued an interim emergency decision ordering OpenAI LLC (OpenAI) to immediately stop the use of ChatGPT to process the personal data of data subjects located in Italy, pending further investigation.

In an interview with Politico, the leading global media platform for policy and politics, Dessislava Savova, head of the Continental Europe Tech Group at Clifford Chance, explains: “This is a wake-up call. It will trigger a dialogue in Europe and it will accelerate a position being taken by other regulators.” To Reuters, Dessislava comments: “The points they raise are fundamental and show that GDPR does offer tools for the regulators to be involved and engaged into shaping the future of AI.”

Andrea Tuninetti Ferrari, Counsel at Clifford Chance, comments: "Generative AI presents a new set of challenges to the application of fundamental privacy principles and rights, such as accuracy and rectification of data. With guidance arising from this investigation, the Garante may lead the way in establishing how regulators will apply the GDPR in relation to this type of technology."

In this article we look at the Garante's order and the legal issues it highlights in relation to generative AI.

BACKGROUND

The Garante's order follows a data breach suffered by ChatGPT on 20 March. OpenAI said the number of users who were impacted by the breach was "extremely low." The platform explained that, due to a technical bug, a few users were able to see titles of other active users' chat history and certain payment-related information data regarding 1.2% of the ChatGPT Plus subscribers who were active during a specific nine-hour window.

Prominent Garante board member Mr Guido Scorza explained to the press that ChatGPT was found to be GDPR non-compliant for three main reasons: (1) OpenAI has collected personal data of billions of data subjects – with the aim to train the algorithm – without informing them in advance and, based on the evidence available to the Garante, without a proper legal basis; (2) ChatGPT's users did not receive information on the purposes for which their personal data is processed and how such data is used; and (3) the AI-generated responses may process inaccurate personal data.  

THE ORDER

The Garante's order – which is relatively short (two pages) – found that there is a material risk that ChatGPT would breach the GDPR on a number of grounds:

• The users of ChatGPT and other data subjects whose data is processed by OpenAI are not provided with a privacy notice (breach of Art. 13 GDPR)
• The use of personal data as a training set for the AI software is unlawful due to the absence of an adequate legal basis (breach of Art. 6 GDPR)
• The processing is not accurate, in that the information contained in ChatGPT's responses to users' queries is not always correct (breach of Art. 5 GDPR)
• Although OpenAI's terms and conditions forbid access to users below the age of 13, OpenAI has not implemented measures to detect the users' age and block access accordingly (breach of Art. 8 GDPR). The order further outlines the risk that the responses provided by ChatGPT may be unsuitable for children.

The order contains a disclaimer, clarifying that the order is an interim decision based on preliminary findings and does not represent a full assessment of the case. The Garante stated that the absence of age verification measures had a pivotal role in the Garante's decision to issue the order. This said, the Garante clarified that the order does not prevent ChatGPT from operating in Italy at all; rather, it prevents OpenAI from processing personal data through ChatGPT (until ChatGPT is made fully consistent with the GDPR obligations).

OpenAI has a right to file written arguments within 20 days of the order, (i) outlining which measures it implemented to comply with it and (ii) providing clarifications on the potential breaches outlined in the Garante's order. The order expressly mentions that failure to comply with the GDPR may result in administrative fines under the GDPR, i.e. up to EUR 20 m or 4% of the annual turnover of the preceding financial year. Mr Scorza envisaged a remediation plan where the Garante and Open AI investigate and discuss the matter jointly and announced that a videocall is scheduled for 5 April between representatives of OpenAI and the Garante to discuss and tackle the issues connected with the processing of personal data using this type of AI.

As an alternative to cooperating with the Garante, OpenAI may challenge the order within 60 days of the date it was issued.

In response to the order, OpenAI has suspended use of ChatGPT by users located in Italy.

WIDER REACTIONS                  

While the order has already generated debate at political level in Italy and elsewhere, one may expect other privacy authorities to assess ChatGPT's processing operations in other EU Member States. At the time of writing, the press has reported that the two complaints about ChatGPT have now been filed with the French data protection authority, and that data protection authorities in France, Germany and Ireland have reached out to counterparts in Italy to find out more about the basis of the ban. The Spanish regulator has reportedly not received any complaint about ChatGPT but did not rule out a future investigation. Outside of Europe, on 4 April 2023, the Office of the Privacy Commissioner of Canada announced it launched an investigation into ChatGPT, in response to a complaint alleging the collection, use and disclosure of personal information without consent. There have also been complaints filed with the US Federal Trade Commission in relation to ChatGPT.

Hence, although the Garante has made it clear that it investigated the matter on its own initiative following the data breach, the Garante's order is likely to be a trigger for others to look more closely at privacy risks and concerns connected with AI, including generative AI. Indeed, the European Consumer Organisation (BEUC) is calling for EU and national authorities to launch an investigation into ChatGPT and similar chatbots.

The fast-paced development and adoption of generative AI such as ChatGPT has also reportedly impacted the legislative progress of the EU AI Regulation, with discussions regarding how to regulate such forms of AI given the variety of uses to which it can be put.

AI AND DATA: A COMPLEX INTERPLAY

The Garante's order is a reminder of how important it is to address data compliance and governance when developing or using AI.  With AI-specific legislation expected to be enacted soon in the EU and China, the order also highlights the role that the GDPR currently plays in regulating AI (and will continue to play even after the enactment of AI-specific legislation).

More broadly, the order is a reminder of the complex legal landscape for AI and the key role played by AI compliance and risk management frameworks in addressing the significant legal, commercial and public relations risks that can arise in relation to the development and use of AI.   Alongside possible data protection concerns, those risks include other forms of data misuse (e.g. in breach of confidentiality restrictions), unintentionally embedding biases in the decision-making process, breach of intellectual property rights, inadequate contractual arrangements, anticompetitive conduct as well as breaches of employment, human rights or financial regulatory laws. Even where the AI systems work as intended, there is increasing pressure for organisations to be able to explain how their AI works and justify its use. 

The EU Courts' demand for transparent AI

Courts and regulators are now subjecting AI to increasing scrutiny and several major fines have already been issued for unethical AI use. This is particularly true of the Italian courts and regulators, which in recent years have issued some groundbreaking decisions sanctioning uses of AI that were deemed to lack transparency, or to be discriminatory or biased, based on employment or privacy laws, and consistent with the risk-based approach envisioned in the draft EU AI Regulation. See our article, The Italian courts lead the way on explainable AI, for more details.

Data accuracy is the new standard for AI

The ChatGPT order touches upon some of the issues that were at the heart of the abovementioned cases in Italy and in other EU Countries – in particular, the requirement that data subjects be informed of the purposes of personal data processing. The order also considers AI from a different angle, highlighting a challenge that providers of AI services will likely be increasingly expected to address in the future, i.e. how to ensure that AI software abides by high data accuracy standards. Failure to do so, as suggested by the Garante, may result in a breach of the GDPR.  In other contexts, inaccurate data processing could also have wider ramifications, for example under AI-specific regulations and in relation to legal and contractual liability for harms resulting from inaccurate output.

The issue is a complex one because AI software's interaction with data is, in many cases, at least two-fold: data is inputted to train AI (input data); and AI also generates data (output data).

Given the preliminary nature of the order, the Garante does not provide extensive details on why it considers that ChatGPT's processing may be "incorrect," however the focus seems to be on the output data, in that, according to the Garante, the answers ChatGPT provides to its users "do not always correspond to real facts."

Building on the Garante's reasoning, one may conclude that, when output data is personal data, Art. 5(1)(d) GDPR continues to apply, whereby data must be "accurate and, where necessary, kept up to date." However, there can be challenges relating to the explainability of some AI systems and their accuracy, especially in machine learning. Since machine learning systems do not use traditional ‘logic’ it can be difficult to illustrate exactly why a particular outcome has been reached in any given situation, as well as technical challenges in correcting the data such that these inaccuracies do not reappear in subsequent output data.

FINAL REMARKS

The Garante's order falls within a broader range of urgent interim measures the Italian regulator has adopted in recent years, especially in the aftermath of cyber or real-life incidents concerning data-fuelled businesses such as ChatGPT.

The Garante's approach is to 'stop the processing first – investigate later'. This may seem to be conservative and restrictive, but arguably it has proven to be an effective way of engaging businesses and open a dialogue, in lieu of simply issuing a sanction. From this perspective, it can be viewed as an open regulatory approach, where the Garante is seeking dialogue with controllers to understand how their business goals can be achieved while ensuring privacy compliance. We expect the next chapters in this case to provide useful guidance in relation to important topics such as age verification, data accuracy and rectification in the context of AI, legal basis for processing (including in the context of re-use of publicly available data) and expectations regarding provision of privacy notices (and related exemptions) - hopefully laying down practical advice on how platforms like ChatGPT, and a range of other data controllers, can implement effective mechanisms.

LATEST UPDATES 

Following the interim decision of the Garante which on 30 March 2023 ordered the immediate interruption of personal data processing by OpenAI and the discussions between OpenAI and the Garante in April, OpenAI has proposed remedial measures which have been analysed and approved by the Garante.

On 28 April 2023 the Garante has declared that OpenAI has implemented the first batch of proposed measures at a satisfactory extent which include:

• a privacy notice displayed on the website and addressed to users and non-users. The notice for users outlines that (i) the processing of certain personal data for the purpose of enabling the performance of services is carried out on a contractual basis and (ii) the processing for training algorithms is based on legitimate interest
• the right for all individuals in Europe, including non-users, to opt-out from processing of their data for training of algorithms
• mechanisms to enable (i) data subjects to obtain erasure of information that is considered inaccurate, although OpenAI notably stated that it is technically impossible, as of now, rectifying inaccuracies and (ii) all European users to opt-out from the processing of their personal data and thus to filter out their chats and chat history from the data used for training algorithms.

Since one of the main concerns of Garante was the absence of a control over the users' age, OpenAI implemented mechanisms aimed at ensuring that only those users who comply with the age requirements (e.g. users +18 and +13 with the parents' consent) are allowed to access.

As said, the Garante stated it is overall satisfied of OpenAI's action, nevertheless it highlighted that further efforts are expected to (i) ensure an effective system of age verification and (ii) implement a public campaign informing Italians on their right to opt-out from the processing of their personal data for training purposes.