artificial intelligence (AI) tools can help teachers plan a lesson, rewrite a parent letter, generate questions or find a better way to explain a difficult concept. Used without thought, the same tools can also become an accidental route for sending a child's personal information to a service the school has never approved.
That distinction matters. A pupil's name, a behaviour incident, an Education, Health and Care Plan (EHCP) extract or a description of a child's health needs is not just useful context for a chatbot. It is personal data, and some special educational needs and disabilities (SEND) information may also be special-category data under the UK General Data Protection Regulation (UK GDPR).
The practical rule is simple: if you have not checked your school's approved-tool policy and data flow, do not paste identifiable pupil or SEND information into a general-purpose AI chat.
This is not an argument that teachers must never use ChatGPT, Claude or similar tools. It is an argument for using them with the same care you would apply to any other service handling children's information.
Can teachers use ChatGPT or Claude at school?
Sometimes, but the answer depends on the exact product, account, settings, contract and use case.
For schools in England, the Department for Education says personal data should not normally be used in generative AI tools. If a school considers it necessary, staff should check with the data protection officer or information technology (IT) lead, use an approved tool, understand how it uses personal data and ensure that the data is not used to further train the AI. The DfE warns that entering pupil data without those checks could result in a data breach [1][2].
That means a personal account opened by an individual member of staff is not automatically equivalent to a school-procured education or enterprise service. It may have different retention rules, different settings, different support arrangements and no contract that reflects the school's responsibilities.
The question is not simply, "Does this AI tool work?" It is also:
- What account and product are we using?
- What information will leave the school?
- Who can access it and for how long?
- Is it used for model improvement, safety review or feedback analysis?
- Who is the controller and who is the processor?
- Has the DPO or IT lead approved this particular use?
If nobody at the school has answered those questions, the safe answer is not to upload the pupil information while you find out.
Why SEND information needs extra care
SEND records often contain much more than a name and a year group. They may include diagnosis, disability, health information, medication, therapy, specialist reports, family circumstances, safeguarding concerns, attendance, behaviour, professional opinions and a child's own words.
Information concerning health is special-category data under the UK GDPR. Not every piece of SEND information will fall into that category, but a school should not leave the decision to a quick judgement made while copying text into a chat box [3]. Children are also a group that needs particular care, and the Information Commissioner's Office (ICO)'s guidance for education technology providers explains who determines the purposes and means of processing children's information [4].
Removing a name does not automatically make a case anonymous. A rare combination of year group, need, incident, school context and family detail may still identify a pupil to someone who knows the setting. The ICO has also made clear that data protection law applies to personal data processed "incidentally" or without deliberate intent [5].
The fact that a teacher did not mean to disclose a child's identity is not the same as the information being anonymous.
Does the school need a Data Protection Impact Assessment (DPIA)?
Not every use of an AI tool automatically requires a Data Protection Impact Assessment. The ICO says a DPIA is required where processing is likely to result in a high risk to people's rights and freedoms. Its risk indicators include innovative technology, sensitive data, vulnerable individuals, profiling, automated decision-making and large-scale processing. AI combined with other risk factors may therefore require a DPIA, and the ICO recommends erring on the side of caution where there is doubt [12].
That decision belongs in the school's data-protection process. A teacher should not be expected to decide it while trying to finish a report before the end of the day.
Does ChatGPT or Claude train on what teachers enter?
This is where a lot of school advice becomes too blunt to be useful.
It is not accurate to say that every prompt entered into every AI service instantly becomes part of a model that anyone else can query. The National Cyber Security Centre says that a deployed language model does not automatically add every query to its model for other users. The query is still visible to the organisation providing the service, however, and may be stored or used to develop the service depending on the provider's terms [6].
There are four separate questions hiding inside the phrase "the AI trains on your data":
First, inference. The prompt, pasted text or uploaded file is sent to a model so that it can produce a response. That is already processing. It may involve logs, support access, safety checks or connected services before anyone discusses model training.
Second, model improvement. Some products may use eligible conversations, feedback or flagged content to improve future models. Whether that happens depends on the product, plan, settings and the action taken by the user.
Third, retention and history. A service may retain chats, files, conversation history, memories, metadata or feedback even when a particular conversation is not used to train a general model. Turning off training does not automatically answer how long the data remains accessible or who can access it.
Fourth, outputs and copies. The generated answer may contain sensitive details, an inaccurate inference or an invented statement. It may then be saved in the account, copied into a document, shared with colleagues or downloaded to a device.
That is why turning off a training setting is not, by itself, a school approval. It does not settle retention, access, deletion, international transfers, controller and processor roles, the Data Protection Act (DPA), a DPIA or the school's own policy.
Consumer ChatGPT, business ChatGPT and Claude are not the same thing
The provider matters, but the product and account matter just as much.
OpenAI says that users of personal ChatGPT accounts can turn off "Improve the model for everyone". New conversations then remain in chat history but are not used to train ChatGPT. Temporary Chats are not used to train models, do not create memories and are deleted from OpenAI's systems after 30 days, although they may be reviewed to monitor abuse [7]. Those controls are not the same as a school-wide contract and approval process.
OpenAI's published business data information says that ChatGPT Business, Enterprise, Edu, ChatGPT for Teachers and its application programming interface (API) platform do not use inputs or outputs to train models by default. It also describes data retention controls for qualifying organisations [8]. That is materially different from assuming that a free personal account has the same protections. It still needs to be assessed against the school's use case, contract, data map and policy.
Anthropic makes a similar distinction. Its consumer Claude products may use chats to improve Claude when a user allows it, when a conversation is flagged for safety review or where the user otherwise opts in. Anthropic says that Incognito chats are not used to improve Claude [9]. Its commercial products do not use inputs or outputs to train models by default, unless the customer provides feedback or gives permission [10].
Anthropic also offers zero-data-retention arrangements for eligible API features, but its documentation is explicit that coverage varies. Consumer products, third-party integrations, flagged content and different API features do not all fall under the same arrangement [11].
The lesson is not that one provider is good and another is bad. The lesson is that a brand name is not a data-flow assessment. A school needs to know which product staff are using and what its current terms actually say.
Provider policies change. Before approving a tool, the school should check the current documentation and contract rather than rely on a screenshot, a staff briefing from last year or a setting that an individual user can change themselves.
What should teachers never paste into an unapproved AI tool?
Do not paste identifiable information such as:
- pupil names, unique pupil numbers (UPNs), dates of birth, addresses or contact details;
- initials combined with distinctive details that make the child recognisable;
- diagnoses, disability information, health details, medication or therapy information;
- EHCP sections, SEN Support evidence, specialist reports or professional observations;
- safeguarding, social-care, attendance, behaviour or incident records;
- identifiable work samples, photographs, recordings or screenshots;
- parent or carer names, phone numbers, email addresses or correspondence;
- a supposedly anonymous case where the combination of details could identify the child.
A safer starting point is a genuinely fictional case or deliberately synthetic information. Remove details that are not needed for the task. Use a school-approved system where the data flow and contractual position have been reviewed. Then check the output yourself. Fluent wording is not evidence that the content is accurate, fair or appropriate for a pupil's record.
For example, a teacher asking an AI tool to "rewrite this email about Sam's missed medication and recent self-harm disclosure" has not created a generic writing task. They have disclosed sensitive information about a particular child. The safer approach is to ask for a neutral structure using fictional details, then write the real communication within the school's approved systems.
What if pupil data has already been pasted?
Do not panic, but do not quietly delete the chat and hope the problem has gone away either.
Stop entering further information. Record the account and product used, the date and time, what was pasted or uploaded, what output was returned and who may have had access. Tell your line manager and the school's DPO or IT lead promptly, following the school's suspected-breach process. Preserve the relevant evidence and follow the school's instructions about deletion, notification and risk assessment.
Deleting a conversation may be part of the response, but it does not by itself tell the school what was processed, whether a copy exists elsewhere or whether the supplier's terms require a different action. A prompt response gives the school the best chance of understanding the risk properly.
What a purpose-built SEND AI should do differently
The strongest alternative to an open-ended chat box is not a bigger promise about AI. It is a controlled workflow designed around the job the school is actually trying to complete.
MeritDocs is designed for SEND documentation, with staff reviewing and approving the result. Development uses fictional cases rather than identifiable pupil records. Pupil inputs are not used to train our AI, and a generated document is not treated as a finished professional record.
That approach was shaped through ongoing consultation with data protection officers, special educational needs coordinators (SENCOs) and practising teachers. Their input affected the evidence handling, document structure and review expectations, not just the colour of a button. The longer explanation is in How MeritDocs builds SEND documents from real school examples.
This does not mean that a school can skip its own checks. No proprietary AI system is a substitute for a school's DPA, DPIA, policy or professional judgement. It does mean the product starts from a different question: how do we help schools create and manage SEND documents without treating live pupil records as casual material for a general-purpose chat?
That is also why the buyer due-diligence guide for AI SEND software asks suppliers about data flows, retention, model training, access, export and exit before a school signs up.
Frequently asked questions
Is using ChatGPT at school automatically unlawful?
No. The answer depends on the specific use, product, account, controls, contract and information involved. Using an approved service for a generic lesson idea without personal data is a different situation from putting an identifiable SEND record into a personal account. Schools should follow their own policy and ask the DPO or IT lead where the position is unclear.
Is Temporary Chat safe for pupil information?
Temporary Chat may change how a provider stores or uses a conversation, but it does not make an unapproved use acceptable by itself. The school still needs to understand the full data flow, the account, the provider's terms and its own responsibilities.
Does turning off model training make ChatGPT or Claude suitable for SEND data?
Not automatically. It may address one question about model improvement while leaving retention, access, deletion, feedback, connected services, contractual roles and school approval unanswered.
Are enterprise or education accounts automatically safe?
No. They can offer materially stronger controls and contractual arrangements, but the school must still assess the exact product, configuration, use case and supplier terms. An enterprise label is not a substitute for due diligence.
Is AI banned in schools?
No. The DfE recognises useful staff-facing applications, including planning and administrative work, while warning that schools must assess risks and staff must use professional judgement. The sensible boundary is controlled use, not blind use.
A chatbot can help with a lesson plan. A child's SEND record is not prompt material. Before any pupil information leaves the school's approved systems, someone responsible for data protection should be able to explain where it goes, what happens to it and why the use is justified.
Sources
[1] Data protection in schools: generative artificial intelligence (AI) and data protection in schools
[2] Generative artificial intelligence (AI) in education
[3] ICO: What is special category data?
[4] ICO: The Children's code and education technologies
[5] ICO: Tackling misconceptions about generative AI and data protection
[6] NCSC: ChatGPT and large language models: what's the risk?
[8] OpenAI: Business data privacy, security and compliance
[9] Anthropic: Is my data used for model training? Consumer products
[10] Anthropic: Is my data used for model training? Commercial products