# Who Is Accountable When a School's AI Tool Gets It Wrong? Canonical URL: https://blog.theaieducator.io/posts/who-is-accountable-when-ai-gets-it-wrong Publication: Dan Fitzpatrick Insights Author: Dan Fitzpatrick Topic: AI Governance and Responsible Leadership Published: 2026-09-20T07:17:18.000Z Modified: 2026-09-20T07:17:19.775Z AI does not create a new accountable person. It removes a checking step that used to be automatic. Here is who answers when a school's AI tool gets something wrong, in what order to put it right, and the seven lines your incident log needs. ## In brief When a school's AI tool gets something wrong, accountability does not move. It stays with the professional and the organization that used the tool, as England's Department for Education states in its generative AI guidance and as the EU AI Act requires of deployers. What AI changes is the checking step: the moment a named person applies judgment to what the tool produced, which used to be welded to the doing and is now detachable. Dan Fitzpatrick's Repair Order sets the sequence for the days afterward: tell the person it landed on before you know why, put it right and say by when, find the step where a person was supposed to look rather than the person who typed the prompt, then change one thing and write it down, or say plainly that nothing will change and why. ## Key takeaways - AI does not create a new accountable person. Accountability stays with the professional and the organization that used the tool, whatever the tool produced. - Responsibility can be shared and delegated. Accountability cannot be divided without disappearing, because two accountable people is nobody accountable. - The genuinely new thing is the checking step: the moment a named person applies professional judgment to what the tool produced. Before AI it was welded to the doing. Now it detaches unless somebody names it. - The Repair Order sets the sequence after an error: tell the person it landed on, put it right, find the step rather than the culprit, then change one thing and write it down. A school that starts by investigating has left the family waiting. - Ask of every approved AI use: name the last person who could have stopped this before it reached a child, and say whether they knew that was their job. No name means the tool was approved and the accountability was not. - The EU AI Act requires deployers of high-risk systems, which includes education uses, to assign human oversight to natural persons with the necessary competence, training, authority and support. The Digital Omnibus moved the Annex III compliance date from August 2, 2026 to December 2, 2027. - CoSN's U.S. State of EdTech 2026 found 79 percent of districts reporting AI guidelines in place, up from 57 percent a year earlier. In Dan Fitzpatrick's reading, almost all published guidance answers approval and acceptable use and stops before the response to an error. In August 2026, Jefferson County Public Schools in Louisville had to write to families at Farnsley Middle School about AI-generated content in student agendas that carried errors. The district's stated position was that human oversight should remain central to how it uses AI. Local reporting, in [JCPS Insider's account of the incident](https://jcpsinsider.com/articles/ai-errors-found-in-farnsley-middle-student-agendas-who-was-supposed-to-catch-them-jcps), put the harder question in one sentence: "The question isn't whether AI made a mistake. It's why the mistake reached students before a person caught it." That is the governance question, and almost no school has answered it. ## Who is accountable when a school's AI tool gets it wrong? The same person who was accountable before the tool existed. AI does not create a new accountable person and it does not move accountability to the supplier. It moves the work, and it leaves a gap in the middle of that work where somebody's judgment used to sit automatically. When the report was written by a teacher, the teacher read it as they wrote it. When the report is drafted by a tool, reading it becomes a separate act that somebody has to be told to perform. If nobody was told, the accountability did not vanish. It simply arrived at an incident with no name attached. England's Department for Education says this plainly in its guidance on [generative AI in education](https://www.gov.uk/government/publications/generative-artificial-intelligence-in-education/generative-artificial-intelligence-ai-in-education), last updated on August 12, 2025: "The quality and content of any final documents remains the responsibility of the professional who produced it and the organisation they belong to, regardless of the tools or resources used." No exemption for the tool. No shared custody with the vendor. So the honest answer to "who is accountable?" is not a name you go looking for after an incident. It is a name you write down before one. ## Accountability, responsibility and the checking step are three different things Schools collapse these three into one word and then cannot find the person they need. Separate them and you stop hunting for a person who does not exist. **Responsibility** is who does the work. It can be shared, delegated, rotated and outsourced. A tool can hold a large share of it. **Accountability** is who answers for the outcome to the person it lands on. It cannot be divided without disappearing. Two accountable people is nobody accountable. **The checking step** is the new thing. It is the specific moment, in a specific piece of work, when a named person applies professional judgment to what the tool produced and, in doing so, takes the output on as their own. Before AI, that step was welded to the doing. Now it is detachable, and if you do not name it, it detaches. Most school AI policies name the first two implicitly and skip the third entirely. That is why the question feels unanswerable in the week after something goes wrong. The school is searching for an accountable person when what is actually missing is a step. The distinction matters for the supplier too. The vendor is answerable for the product: whether it does what the documentation says, what it does with data, whether it is fit to sell. You are answerable for the decision to use it here, on these children, for this job. An indemnity clause reallocates liability. It does not move that. ## The Repair Order When something has gone wrong, the order of what you do next carries more meaning than the speed of it. This is the sequence I ask schools to work through, and I offer it as a way of thinking rather than a tested procedure. The Repair Order is the sequence I ask schools to follow in the days after an AI tool has got something wrong, and the order matters more than the speed: tell the person it landed on, before you know why; put it right for them, and say by when; find the step where a person was supposed to look, not the person who typed the prompt; then change one thing and write down what you changed, or say plainly that nothing will change and why. A school that works in that order has been accountable. A school that starts at step three has begun an investigation, and the family is still waiting. **One. Tell the person it landed on, before you know why.** The instinct is to establish the facts first and communicate once the picture is complete. That instinct is what turns a mistake into a grievance. A family can absorb "your daughter's report contained two sentences that were not true of her, we are looking into how, and I will call you Thursday". They cannot absorb silence followed by a polished account three weeks later, because by then they have worked out that the school knew and said nothing. **Two. Put it right for them, and say by when.** Repair precedes explanation. Reissue the report, correct the record, remove the wrong mark, restore the place on the trip. Attach a date. Vagueness at this step reads as reluctance, whatever you intend by it. **Three. Find the step, not the culprit.** Ask where a person was supposed to look, and whether anyone had ever told them that was their job. That question produces a fixable answer. "Who typed the prompt?" produces a frightened member of staff and a school that learns nothing, because the next error will come from a different person at the same unguarded step. **Four. Change one thing, and write it down. Or say that nothing will change.** One change, named, owned, dated. The second option is legitimate and is almost never used: sometimes the honest conclusion is that the process was sound and a person made an ordinary error that no reasonable system would have caught. Saying so, out loud, is more credible than a token new procedure, and it protects you the next time something genuinely does need to change. Four moves, in that order: tell, repair, trace, change. ## The one question that finds your gaps before an incident does Take any AI use you have already approved and ask it: **name the last person who could have stopped this before it reached a child, and say whether they knew that was their job.** Two names come back, or one, or none. None means the tool was approved and the accountability was not. One name that the person themselves would not have given you means you have a checking step that exists only in the leadership team's imagination. It is a two-minute question and it is the smallest piece of governance work available to you this term. Run it on your three highest-stakes uses. Not your three most-used: your three highest-stakes. Anything touching a child's record, a parent's understanding of their child, or a judgment that follows a student forward. ## What the regulations require, and where they bite Two jurisdictions matter to most readers here, and they say compatible things. **England.** The DfE guidance quoted above places the responsibility on the professional and the organization, whatever tool was used. [Keeping Children Safe in Education](https://www.gov.uk/government/publications/keeping-children-safe-in-education--2), whose 2026 version came into force on September 1, 2026, remains the statutory frame around anything that reaches students, and the DfE guidance is explicit that schools must comply with those wider statutory obligations when using generative AI. **The European Union, which reaches international schools and any supplier selling into the bloc.** Education uses sit in the AI Act's Annex III as high risk. Article 26 puts obligations on the deployer, which for a school-purchased tool is the school. It requires deployers to "assign human oversight to natural persons who have the necessary competence, training and authority, as well as the necessary support". Note all four words: competence, training, authority and support. A teaching assistant told to "check the AI output" with no time, no authority to stop a mailing and no training in what a wrong output looks like satisfies none of them. Article 26 also requires deployers to keep the system's automatically generated logs, for at least six months in most cases, and to inform people when they are subject to a high-risk system's use. The timing has moved and leaders are working from the old date. The original compliance deadline for Annex III high-risk obligations was August 2, 2026. The EU's Digital Omnibus, agreed in May 2026 and since published in the Official Journal, pushed it to December 2, 2027. [Orrick's July 2026 analysis](https://www.orrick.com/en/Insights/2026/07/EU-AI-Act-Update-Digital-Omnibus-Finalizes-8-Compliance-Changes) sets out the revised dates. The obligations did not soften. They arrived later, which is a gift of about fifteen months and nothing more. That fifteen months is the interesting part. The standard the EU will eventually enforce, a named human with competence, training, authority and support, is a description of decent practice that no school needs a regulation to justify adopting now. ## What I Tell Boards and Governors I sit on a board myself, and the question I ask when AI appears on the agenda is not whether the school has a policy. Nearly everyone has one now. [CoSN's U.S. State of EdTech 2026](https://www.cosn.org/wp-content/uploads/2026/05/U.S.-State-of-EdTech-2026.pdf), surveyed between January 14 and March 1, 2026, found 79 percent of districts reporting AI guidelines in place, up from 57 percent a year earlier, with 56 percent holding an acceptable use policy for generative AI. The survey measures whether guidelines exist. It does not measure what is inside them, and that is my own reading rather than the report's finding: the published guidance I have worked through overwhelmingly answers two questions, what staff may use and how a tool gets approved, and stops at the point where something goes wrong. Approval is the easy half of governance. It is a decision made calmly, in advance, by people who are enjoying the conversation. So the three questions I put to a governing body are these. When an AI tool produces something wrong and it reaches a family, who tells them, and how quickly? Where in our approved uses is the checking step, and does the person standing at it know they are standing there? And when did we last look at an actual error rather than an actual policy? The third one is where boards find out what kind of school they govern. A board that has never seen an incident either has a school that reports nothing upward or a school with nothing to report, and only one of those is true. ## The AI incident log, in seven lines The artifact worth building this term is small. Seven lines per entry, kept wherever your approved tools list is kept, reviewed once a term by whoever owns [approvals for AI tools](https://blog.theaieducator.io/posts/approving-ai-tools-for-schools). 1. The date it happened, and the date it was noticed. These are usually different, and the gap between them is the most useful number in the log. 2. The tool, the model or version if the supplier names one, and who approved it. 3. Who it reached, and what it took from them. 4. The checking step: where a person was supposed to look, and whether that had been written down anywhere before today. 5. What we did to put it right, and when the person affected was told. 6. The one thing that changed, with a name and a date. Or the sentence explaining why nothing changed. 7. Who signed this entry. Line four is the one that earns the log its place. Read six months of those lines together and you will see the same unguarded step appearing under different tools and different staff, which is a system telling you where to put your attention. A caution on line two: [an approved tool that changes under you](https://blog.theaieducator.io/posts/when-an-approved-ai-tool-changes) is a common source of errors nobody can explain, because the thing that was checked in June is not the thing running in October. ## The Mistake I See Most Often Across the leadership teams I work with, the most common error is treating the first incident as a communications problem. The meeting fills with wording. What will we say to the family, what goes to staff, what if the local paper calls. All of that matters, and none of it is the work. The work is the sentence nobody wants to write: this is the step where a person looks, and this is the person. It is uncomfortable because it turns a general good intention, we keep humans in the loop, into a specific name with specific time attached to it, and then somebody has to find that time in a week that is already full. That is precisely why it gets deferred, and why the same school has the same conversation twice. There is a second pattern worth naming. Schools that handle a first incident well tend to have decided, in advance and in writing, [which decisions AI should never make](https://blog.theaieducator.io/posts/which-decisions-should-ai-never-make). Not because the list covers every case, but because a team that has drawn one line has practiced drawing lines, and the second one comes faster. ## Where to start this term Three things, in this order, none of which takes a working group. **This week.** Take your three highest-stakes AI uses and run the question on each: who is the last person who could stop this, and do they know? Write the answers down even where the answer is nobody. **This month.** Add lines four and five of the incident log to your existing approval record, so every future approval names its checking step at the moment it is granted rather than in the week after a mistake. This is a change to a form, not a change to a process, which is why it actually happens. **This term.** Agree with your leadership team, in one sentence, who tells a family when an AI tool has got something wrong about their child. Then tell the staff who that is. The value is not in the sentence. It is in the fact that nobody has to invent an answer on the worst morning of their year. None of this requires you to know where AI is going. It requires you to know [who owns the decisions you have already made](https://blog.theaieducator.io/posts/who-should-own-ai-strategy), which is a question schools could always answer before a tool arrived to blur it. The reason this is urgent has little to do with regulation. Public patience is thinner than it was. In the first week of September 2026, [Axios reported](https://www.axios.com/2026/09/07/ai-schools-backlash-bans-new-york-los-angeles) that New York City had blocked AI use for roughly 600,000 students from pre-kindergarten to eighth grade, and that Los Angeles had imposed a districtwide moratorium on AI on school devices for 2026-27. Whatever you make of those decisions, they describe a climate in which one badly handled incident does more damage than it would have done a year ago. The schools that keep their permission to use this technology will be the ones that can show what they do when it fails. If your leadership team is working through these questions, this is the kind of work I support through [AI strategy sessions and advisory work](https://theaieducator.io/ai-strategy-for-schools?utm_source=blog.theaieducator.io&utm_medium=referral&utm_campaign=who-is-accountable-when-ai-gets-it-wrong), and I write about it each week in [the newsletter](https://theaieducator.io/?utm_source=blog.theaieducator.io&utm_medium=referral&utm_campaign=who-is-accountable-when-ai-gets-it-wrong#newsletter). ## Sources and further reading - Department for Education, "Generative artificial intelligence (AI) in education", GOV.UK policy paper, published March 29, 2023, last updated August 12, 2025. https://www.gov.uk/government/publications/generative-artificial-intelligence-in-education/generative-artificial-intelligence-ai-in-education - European Union, Regulation (EU) 2024/1689 (the AI Act), Article 26, obligations of deployers of high-risk AI systems. https://artificialintelligenceact.eu/article/26/ - Orrick, "EU AI Act Update: Digital Omnibus Finalizes 8 Compliance Changes", July 2026, on the revised December 2, 2027 date for Annex III high-risk obligations. https://www.orrick.com/en/Insights/2026/07/EU-AI-Act-Update-Digital-Omnibus-Finalizes-8-Compliance-Changes - CoSN, "U.S. State of EdTech 2026", published May 2026, surveyed January 14 to March 1, 2026. https://www.cosn.org/wp-content/uploads/2026/05/U.S.-State-of-EdTech-2026.pdf - JCPS Insider, "AI Errors Found in Farnsley Middle Student Agendas. Who Was Supposed to Catch Them?", August 13, 2026. https://jcpsinsider.com/articles/ai-errors-found-in-farnsley-middle-student-agendas-who-was-supposed-to-catch-them-jcps - Josephine Walker, "Parents, students and educators are revolting against AI in classrooms", Axios, September 7, 2026. https://www.axios.com/2026/09/07/ai-schools-backlash-bans-new-york-los-angeles *Dan Fitzpatrick is the founder of The AI Educator, a school trustee, and an adviser to government bodies including the UK Department for Education, KHDA in Dubai and the Ministry of Education in Kazakhstan on leading through AI. [More about Dan](https://www.theaieducator.io/about).* ## Frequently asked questions ### Who is responsible if an AI tool makes a mistake in a school? The professional who used the tool and the organization they belong to. England's Department for Education states that the quality and content of any final document remains their responsibility regardless of the tools used. AI changes who does the drafting. It does not change who answers for what goes out. ### Can a school blame the AI vendor when a tool gets something wrong? No. The supplier is answerable for the product: whether it does what the documentation says and what it does with data. The school is answerable for the decision to use it here, on these children, for this job. An indemnity clause reallocates liability without moving that decision. ### What should a school do first after an AI tool produces something wrong? Tell the person it landed on, before you know why it happened. Families can absorb an early, incomplete account with a date attached. They cannot absorb silence followed by a polished explanation weeks later, because by then they know the school knew and said nothing. ### What is a checking step, and where should it sit? It is the specific moment when a named person applies professional judgment to what a tool produced and takes the output on as their own. It sits immediately before the output reaches anyone outside the school, and the person standing at it has to know that is their job. ### Does the EU AI Act apply to schools using AI? Yes, where the school is in the European Union or a supplier is selling into it. Education uses sit in Annex III as high risk, and Article 26 obliges deployers to assign human oversight to competent, trained, authorized and supported people, to keep system logs, and to inform those affected. ### What should be in a school's AI incident log? Seven lines: the date it happened and the date it was noticed; the tool, version and approver; who it reached and what it took from them; the checking step and whether it was written down before; what was done to put it right and when the person was told; the one change made; and who signed the entry. ### Do we need a separate AI incident procedure, or will our complaints process do? Your existing complaints process handles the family's route in. What it does not handle is line four of the incident log, the question of where a person was supposed to look. Read six months of those answers together and you see the unguarded step that keeps producing errors. --- Source: [Who Is Accountable When a School's AI Tool Gets It Wrong?](https://blog.theaieducator.io/posts/who-is-accountable-when-ai-gets-it-wrong) Publisher: [The AI Educator](https://theaieducator.io)