Can You Train AI on Your Company Data? The Legal Question First

Can you train an AI model on your company's data? Often, yes. UK data protection law does not ban it, consent is not automatically required, and guidance from the ICO, the UK's data protection regulator, says you could base your processing of personal data for both development and ongoing use of AI on the legitimate interests lawful basis. The catch comes in two parts. You have to be able to justify the training before it starts, and once personal data has gone into a model, you may not be able to get it back out.

Few businesses have had to think about this yet. In the government's UK Business Data Survey 2026, 6% of businesses that handled digitised data and used AI said they used data to develop, train or improve AI or automated decision systems, rising to 13% of large businesses.

By training I mean changing a model's weights, usually by fine-tuning an existing model on your own examples, which is a different thing from giving a system a memory it reads from.

This post sets out three questions in the order they bite, which are whose data it is, whether you can justify this use of it, and whether you could get it out again. It then puts the EU AI Act in proportion and lists what we would keep out of a model altogether. It is how we think about the subject, not legal advice.

Whose data is it?

Company data is a loose phrase. A good deal of what sits in a business's systems is not the business's to reuse, so start by sorting it by the position you hold it in.

Where you decide how customer or staff records are used, you are the controller, and the next section is yours to answer. Data you hold for a client is a different matter. An agency, a software firm or an outsourcer that handles personal data on a client's behalf is usually a processor, and UK GDPR says a processor handles that data only on documented instructions from the controller. The same article carries the consequence. If a processor infringes this Regulation by determining the purposes and means of processing, the processor shall be considered to be a controller in respect of that processing.

This is the sharpest trap in the subject, because nobody has to decide to take a risk. A delivery team sits on years of client work and wants a model that writes the way that work is written. As soon as the model is for the firm's own use, the firm has arguably become a controller of those people's data, with no relationship with them, and has probably breached its client contract too. The obvious defence is that clients benefit as well, and the ICO has already rejected it for AI developers. It says that if a developer is processing data for their own purposes, they are a controller for that processing, and that the fact that clients may also benefit is not enough to make the client the sole controller.

There is a legitimate route. The ICO's AI guidance describes a provider that keeps a separate model for each client, and says that as long as the isolation between different controllers is complete and auditable, the client will be the sole controller and the provider will be a processor. A model trained on one client's data, on that client's instructions and for that client alone can therefore keep you where you should be, provided the agreement says so before the run.

Personal data is not the only limit. Company files also hold bought-in reports, licensed datasets and documents under NDA. The UK's copyright exception for text and data mining is for the sole purpose of research for a non-commercial purpose. Check the licence or the NDA first, and do not treat silence as permission.

Can you justify this use of it?

Where you are the controller, training still has to be justified in its own right. The ICO's guidance, which it says is under review following the Data (Use and Access) Act, tells organisations that you must break down and separate each distinct processing operation, and identify the purpose and an appropriate lawful basis for each one. It treats developing a system and deploying it as distinct and separate purposes, with different circumstances and risks. Customer data collected to deliver a service already has a purpose, and training a model is a new one.

Is the new purpose compatible with the old one? The ICO says a new purpose is likely to be incompatible if it is very different from the original purpose or would be unexpected, and that you are then likely to need to ask for specific consent.

What is your lawful basis? Contract is unlikely to stretch this far, because the ICO says this may not be an appropriate ground for processing personal data to develop an AI system. Consent may be an appropriate lawful basis in cases where you have a direct relationship with the individuals, but it sits awkwardly with training. Writing about web-scraped data, the ICO observed that people are unlikely to be able to revoke their consent if removing their data requires model re-training, and that reasoning travels to a company's own records.

That leaves legitimate interests for most businesses, with the three-part test behind it. The best indication of how hard that test will be read is the ICO's report of October 2026. It says broad interests such as "training our models" are unlikely to meet this requirement without specification or justification, and that the necessity step needs analysis of any less intrusive alternatives, such as synthetic data. It was written for foundation model developers rather than a 200-person firm tuning a small model, so it is the direction of travel and not a rule made for you. I would still work to it.

HR files and CRM notes are where special category data hides, and where you have no Article 9 condition, the ICO's advice to developers is to prevent the collection of special category data, filter it out or avoid processing it. A data protection impact assessment is also likely, because the ICO says that in the vast majority of cases, the use of AI will involve a type of processing likely to result in a high risk.

I have found no UK fine for a company training on its own users' data, which is not proof that none exists. The two best-known cases ended in change rather than penalty. LinkedIn suspended training on UK users' data after the ICO became involved, and later resumed, with the ICO saying that organisations should put effective safeguards in place before they start using personal data for model training, including providing a clear and simple route for users to object to the processing. Meta paused and then resumed too, and the ICO noted that it has not provided regulatory approval for the processing. The ceilings are still real. Breaching the basic principles can reach £17.5 million or 4% of worldwide turnover, whichever is higher, and a bare breach of the processor rules sits in the lower tier of £8.7 million or 2%.

Could you get it out again?

This is where training differs from almost every other use of company data. A record in a CRM can be corrected or deleted, and the next report changes. A model works differently, and the ICO's October report says that AI models themselves can contain personal data and can therefore be subject to data protection law, which requires a case-by-case assessment of the model. On erasure it says it can be difficult to erase their information from the model itself without retraining the entire model, and it refers to cases where the processing is effectively irreversible. The time to read those words is before the run, the last point at which the decision is cheap.

Not every model is caught. The ICO's AI guidance says erasing training data does not entail erasing all ML models based on this data, unless the models themselves contain that data or can be used to infer it. The trouble is knowing which side you are on, and the October report says there are currently limited technical means of identifying whether, and to what extent, a specific person's personal data is represented within a trained model.

The research adds a twist. A study accepted at EACL 2026, working with small open models and both synthetic and real-world data, describes a substantial risk of unintended memorization and leakage when fine-tuning on sensitive datasets. Its specific concern was the exposure of PII that appears only in model inputs, not in training targets, so personal data sitting in the examples you feed in can stick, even when it never appears in the answers you are teaching. Removing data afterwards is an active research field, but one NeurIPS 2025 paper sets out why unlearning is not a general-purpose solution, and it is not yet something to plan around.

Why we keep knowledge out of the weights

Our position is that a business should fine-tune a model on how it should behave and never on what it knows. The commercial reasons are made elsewhere, including that knowledge in weights cannot be read, checked or carried across to another model. There is a legal version of the same argument. No regulator has stated it and the law does not forbid the alternative, so what follows is our reasoning.

Necessity. Legitimate interests requires you to ask whether a less intrusive route exists. If a system can retrieve the answer from a document you can read, it is hard to argue that learning the same personal facts into a model was necessary.

Rights. In a store, you delete the record and the next answer changes. In weights, the same request may mean retraining.

Accuracy. UK GDPR requires personal data to be accurate and, where necessary, kept up to date. The ICO accepts that individual inaccuracies in training data may be less important where a model is finding general patterns. A fact learned as a fact is a different case, and facts about customers and staff go out of date.

The limits matter as much as the argument. Teaching behaviour is no safe harbour. A fine-tune on real customer emails is still processing personal data, so the clean version uses scrubbed or synthetic examples. Retrieval is no compliance pass either, since it processes personal data as well and only keeps people's rights workable. And a hosted fine-tuning service does not move your own position, because the lawful basis and the compatibility assessment stay with you. That route is also less settled than it looks, since OpenAI says it is winding down the fine-tuning platform, and that from 6 January 2027 active existing customers will no longer be able to create new fine-tuning jobs.

I had to apply this to my own work when planning the model programme I am running this autumn. The client dataset available to me had the best real examples, and I ruled it out. The plan is to generate training pairs from our own published writing, by deliberately corrupting it so a model can learn to correct it. That is less realistic, but there is nobody's personal data in it and no client's confidence to answer for. The work is at an early stage, so this is a decision and not a result.

The EU AI Act, in proportion

The EU AI Act is often quoted on this subject, and often wrongly. Article 10 says high-risk AI systems which make use of techniques involving the training of AI models with data shall be developed on the basis of training, validation and testing data sets that are relevant, sufficiently representative, and to the best extent possible, free of errors and complete in view of the intended purpose. Its reach is much narrower than that wording suggests.

It covers high-risk systems only, so a tone-of-voice fine-tune is not on the list. Annex III does include AI systems intended to be used for the recruitment or selection of natural persons and systems used to evaluate the creditworthiness of natural persons or establish their credit score, and those are the ones a mid-market business should watch.

The duty falls on providers, who must ensure that their high-risk AI systems are compliant with the requirements set out in Section 2. A deployer's own duty is narrower and applies only to the extent the deployer exercises control over the input data.

It does not apply yet. The 2026 Digital Omnibus moved the dates, to 2 December 2027 for high-risk systems such as recruitment and to 2 August 2028 for AI embedded in products. The penalty tier that reaches these duties is up to €15 million or 3% of worldwide turnover, with the lower figure capping smaller firms. The €35 million that often gets repeated belongs to prohibited practices.

Fine-tuning is how a business can walk into the provider role. Someone who changes the purpose of a general-purpose system so that it becomes high-risk shall be considered to be a provider of a high-risk AI system. Take advice before doing that. Being a UK company is no answer if the output produced by the AI system is used in the Union, so EU candidates or customers can bring you in.

What data should never go into a model

No regulator publishes this list, so it is ours.

• Client data you hold as a processor, unless your instructions cover it and the model is isolated for that client.
• Special category data where you have identified no Article 9 condition. Treat criminal offence data the same way.
• Credentials and secrets. The ICO's warning about scraped training data mentions email signatures, API keys and passwords, and a company's own mailboxes and shared drives hold the same things.
• Data collected on consent for something else, without fresh consent.
• Bought-in reports, licensed data and documents under NDA, unless the terms allow it. The model's own licence is a separate question.
• Facts that will change, which belong in a store the system retrieves from.

Practical takeaways

1. Sort the data by your role first. Client data under a processing agreement is a different question from your own customer records.
2. Write the purpose down specifically. "Training our models" will not do.
3. Run the less intrusive test honestly. If retrieval or synthetic examples would do the job, that is your answer.
4. Do the paperwork before the run, including a written plan for an erasure request.
5. Train behaviour and retrieve knowledge, and take legal advice before any fine-tune touching recruitment, credit, special category or client data.

The ICO is developing a statutory code of practice on AI and ADM, so check the position again before a run rather than relying on this post or any other.

Frequently asked questions

Can we train an AI model on our customer data?

Often yes, if you hold that data as a controller and can justify the new use. Training is a separate purpose from delivering your service, so you need a lawful basis for it, a compatibility assessment against why you collected the data, and usually a data protection impact assessment. If you hold the data for a client as a processor, the answer changes. You can act only on that client's documented instructions, and training a model for your own use can make you a controller of data you were never meant to decide about. Take legal advice on anything sensitive.

Do we need consent to fine-tune on personal data?

Not automatically. The ICO accepts that legitimate interests can be the lawful basis for developing and using AI, provided you pass its three-part test. Consent becomes the realistic route in three cases, namely when you originally collected the data on consent, when the new use is incompatible with your original purpose, and when special category data is involved and no other condition applies. Consent also has a practical weakness for training, because people can withdraw it, and removing their data from a trained model may mean retraining it.

What does the EU AI Act require of training data?

Article 10 requires training, validation and testing data to be relevant, sufficiently representative and, to the best extent possible, free of errors and complete, with data governance covering matters such as its origin. It applies to high-risk AI systems, and the duty sits with their providers rather than the businesses using them. Following the 2026 amendments it applies from 2 December 2027 for high-risk uses such as recruitment and credit scoring, and from 2 August 2028 for AI embedded in regulated products. A business that fine-tunes a model for a high-risk purpose can become the provider.

Should you fine-tune a model on company knowledge?

Our view is no, though no regulator forbids it, so this is our reasoning and not a rule. If retrieval from a readable store can answer the same question, it is hard to argue that learning personal facts into a model's weights was necessary, and the law asks about less intrusive alternatives. A record in a store can be corrected or deleted, while erasing something from a model may require retraining it. Facts also change and weights do not. Use fine-tuning for behaviour, such as format and tone, on scrubbed or synthetic examples, and keep what the business knows where it can be read and changed.

What data should never go into a model?

No regulator publishes a list, so this one is ours. Leave out client data you hold as a processor unless your instructions cover it and the model is isolated for that client. Leave out special category data where you have no Article 9 condition, data collected on consent for something else, and credentials and secrets. Leave out bought-in reports, licensed data and documents under NDA unless the terms allow it, along with any fact that will change. Scrub what remains, because personal data that appears only in a training example's input can still be memorised.