UK GDPR and language models: where your data actually goes

When a business asks whether it can put personal data through a language model, the question is almost always “will they train on it?” For business use of the major APIs, that was settled some time ago. OpenAI stopped using API data for training by default on 1 March 2023, and Anthropic’s commercial terms bar training on customer content. It was a reasonable worry. It is no longer the one that matters.

What matters is how many copies of the data a single request creates, where each copy sits, who controls it and when it is deleted. UK GDPR asks those questions of every copy, not just the one you were thinking about.

Draw every copy, not the API call

The usual architecture diagram shows an arrow from the application to the model and back. A realistic one for a document-processing or retrieval feature has at least six places where personal data now lives:

• The provider’s abuse-monitoring logs, which exist whether or not you use any stored feature
• Provider-side state you asked for: stored responses, uploaded files, vector stores, batch jobs
• Your application logs, which capture the full prompt the first time someone turns on debug logging to chase a bug
• The tracing or evaluation tool, usually a separate SaaS product with its own processor terms and its own region
• Your vector database, holding embeddings of every document you indexed
• The consumer chat app a member of staff pasted the same document into last Tuesday

Each of those is processing under UK GDPR. Each needs a lawful basis, a retention period that satisfies the storage-limitation principle in Article 5(1)(e), and, where a third party holds it, an Article 28 processor contract. A DPIA that describes only the API call is describing the least interesting part of the system.

The provider keeps more than the request

Standard API retention is not zero. OpenAI generates abuse-monitoring logs for all API usage and keeps them for up to 30 days, and those logs can contain prompts and responses. Anthropic’s API baseline is also 30 days. Both offer zero data retention, but only to approved customers, and only on endpoints that do not need to store anything.

That last condition matters more than it sounds, because stateful features sit outside zero retention by design. On OpenAI, vector stores are kept until you delete them and the conversations endpoint is excluded from zero retention. On Anthropic, the Files API, batch processing and code execution are all ineligible, and inputs and outputs flagged by its trust and safety systems can be kept for up to two years even under a zero-retention arrangement. Signing a zero-retention amendment and then building on the stored-files feature quietly undoes it.

Every one of these policies also carries the phrase “unless required by law”, and that phrase has been exercised. In May 2025 a US federal court, in the New York Times copyright case against OpenAI, ordered the company to preserve output logs that would otherwise have been deleted. Until that obligation ended on 26 September 2025, standard API traffic was kept regardless of the 30-day policy. Customers on zero data retention were outside it, because there was nothing stored to preserve.

A retention policy is a statement of what a provider intends to delete. The only data a court cannot order kept is data that was never written down.

Your own stack is the bigger store

Provider retention is bounded and documented. Your own is usually neither.

Logging is the easiest one to get wrong. Prompts are long, they are the first thing an engineer wants when an output looks wrong, and observability tools are built to capture them in full. It takes very little for the most complete record of every customer document you have processed to end up in a tracing tool that is missing from the record of processing, retained for as long as that vendor’s default plan allows.

Embeddings are the misunderstood one. They look like arrays of numbers and are often treated as anonymised. They are not. Morris and colleagues showed in 2023 that an inversion method could reconstruct 92 per cent of 32-token inputs exactly from their embeddings, and on clinical notes it recovered 89 per cent of full names. A vector database built from personal documents holds personal data, and it needs the same access control, retention and deletion handling as the documents themselves. That includes erasure: when someone asks to be deleted, their chunks have to come out of the index too.

Residency is not the same as no transfer

Keeping data in the UK is now an option. OpenAI offers UK data residency to eligible API customers, and the major clouds serve some models from London regions. It is worth using. It is also narrower than it looks: OpenAI’s own documentation states that data residency “does not apply to system data, which may be processed and stored outside the selected region”.

Where data does leave the UK, the legal test has changed. Since 5 February 2026 the Data (Use and Access) Act 2025 has replaced the old adequacy standard with a “data protection test”: whether protection in the destination is not materially lower than in the UK. For the United States, the UK-US data bridge covers transfers only to organisations certified under the UK Extension to the Data Privacy Framework. A provider or sub-processor that is not certified still needs an International Data Transfer Agreement or the UK Addendum, with a transfer risk assessment behind it. Check every vendor on the data-flow map, not just the model provider.

What we put in place before the first real record

None of this argues against using language models on personal data. It argues for doing the plumbing first. Before a feature touches live personal data, this is the minimum we would expect to exist:

1. A DPIA. The ICO’s position is that “in the vast majority of cases” the use of AI will involve processing likely to result in a high risk, which makes one a legal requirement rather than good practice
2. A data-flow map listing every copy above, each with an owner, a region, a retention period and the contract that governs it
3. Zero data retention where the provider will grant it, with the stateful features that would break it deliberately avoided and the state held in your own systems instead
4. Redaction or pseudonymisation before the prompt wherever the task does not need the identity. Extracting line items from an invoice rarely needs the customer’s name
5. Prompt logging off by default in production, with a short-lived, sampled and access-controlled alternative for debugging
6. A written position on consumer AI tools for staff, backed by a sanctioned business tool so the policy is not simply routed around

One change in the law cuts the other way. The same Act relaxed the rules on solely automated decisions from February 2026: a significant decision no longer has to fit one of three narrow exceptions unless special category data is involved, provided safeguards are in place. Those safeguards include telling people the decision was automated, and letting them contest it and reach a human. That makes more automation lawful. It does not make the safeguards optional, and they have to be designed into the workflow, not added after launch.

The question worth asking first

Ask for a list of every system that will hold a copy of a customer’s data after the feature has run once, with a deletion date against each. If nobody can produce it, the gap is not the model. It is the diagram.

Common questions

Written by the Voxalone team. If this raised a question about your own systems, get in touch — you will get a straight answer from an engineer, with no obligation attached.