Forming a sensible view of AI would be considerably easier if it would stop changing for five minutes. New capabilities arrive almost weekly, accompanied by no shortage of bewildering predictions about what AI will transform, improve, or make obsolete. For education leaders, the question is no longer whether or even when to engage with AI; that decision has largely been made for us. The harder questions are what we should use it for, what we should trust it with, and what might we be giving up in the process. Those questions only become more urgent as AI becomes more capable.
We asked AI to write a school policy…
At SchoolDocs, we decided to explore some of these questions through work we know very well. Four hundred years before ChatGPT, Shakespeare offered the rather useful advice to go “wisely and slow; they stumble that run fast.” It seems a useful principle to bring to AI, so rather than getting too carried away by what it could do, we decided to test it on something we already understand deeply. We asked it to produce an Education Outside the Classroom (EOTC) policy for a New Zealand primary school. If you know the myriad intersecting regulations, requirements and guidelines attached to EOTC, you will know this is not a task for the faint-hearted. Or, as it turned out, for AI.
At first glance, the results were impressive. Spun up in seconds, the AI-produced EOTC policy was polished, confident and apparently comprehensive. Not a shock really. Generative AI is exceptionally good at producing content that looks finished, and fluency carries its own impression of authority. Then our policy experts started pulling it apart. Did it connect properly with the school’s wider policy framework? No. Did it accurately and sufficiently reflect New Zealand legislation and guidance? Sort of. Had it picked up the surrounding requirements involving volunteers, inclusion, emergency procedures and the many practical realities of EOTC? Crickets. What it had produced, very competently, was what we had explicitly asked for. No more and no less.
Oddly enough, this is when the exercise became most interesting. AI had not really failed; it had done what we asked. The problem was that it had treated the policy as a discrete information task when school policy simply does not work like that. Any policy operating in a New Zealand school sits within a web of legislation, guidance, other policies, procedures, practice and real-world decision-making. Understanding that web requires more than access to information. It requires context, interpretation, connections and the judgement to recognise what might be missing. Sometimes expertise is simply knowing enough to be suspicious of an answer that looks entirely convincing.
…It taught us the value of human expertise
It’s worth thinking about where that expertise comes from in the first place. After more than twenty years of growing big policy muscles we know expertise does not magically appear at the point when we need to exercise judgement. We build that muscle by doing the work: reading the source material, writing and rewriting our drafts, following an argument to its logical conclusion, encountering contradictions, making connections and occasionally disappearing down a rabbit hole before discovering that it really did matter after all. The finished policy may be the visible product of that work, but the knowledge and judgement developed while producing it are part of the product too. Which raises a slightly uncomfortable question about all the work AI is now rather obligingly offering to save us from. If AI summarises the key material before we read it, if it drafts the paper before we have wrestled with the argument, or if it analyses the evidence before we have made sense of it, we certainly gain time. That is attractive, and sometimes entirely sensible, and the productivity gain is easy to see and easy to measure. What is much less visible is what we lose in the process: knowledge, connections, understanding and, ultimately, judgement. Some of what we call inefficiency may simply be the cognitive work involved in becoming good at something. If we optimise all of that away, we may become considerably more efficient at producing work while becoming progressively less equipped to judge whether the work is any good.
…That extra really is for experts
For education, this feels particularly important because learning has never been only about producing the answer. We ask students to read, write, calculate, research and reason partly because of what those activities produce, but also because of what happens to the person doing them. The same must surely be true of professional expertise. There is no particular virtue in preserving tedious work for its own sake, and plenty that AI can and should take off our hands. The much more interesting challenge is distinguishing between work that is merely laborious and work whose difficulty is actually doing something useful to our thinking. That seems a more worthwhile conversation for schools than whether we are “for” or “against” AI. The question is not simply what AI can do for us, but which parts of our work we can safely stop doing ourselves. So perhaps we need to think more carefully about the work we still need to do for ourselves. Reading, writing, analysing and puzzling things out may feel like the long way round, but they are also how we acquire and retain the knowledge and judgement to recognise when an AIrecommended shortcut has taken us somewhere we never intended to go.
… And that trust is not negotiable
Our EOTC experiment began as a fairly simple test of what AI could produce, but the answer turned out to be the least interesting part of the exercise. Far more revealing was the work our experts had to do to recognise what was missing, and the knowledge and judgement that made that possible. For us at SchoolDocs, this reaches into something much more fundamental than how, or how much, we use AI. It asks us to think about what defines us and what lies at the heart of the work we do. Our relationships with schools have been built over many years on trust: trust that we understand the complexity behind school policy, that we will get it right, and that those policies schools rely on will help keep their people safe. There may be opportunities to use AI to make our work faster and more efficient, and we may take those opportunities where they genuinely add value. But that will never be at the expense of the knowledge, judgement and expertise on which that trust depends.
Those are things we will never give away.
About the author
Dr Jane Gregg is Chief Executive of LIGHTN Ltd, the New Zealand company behind SchoolDocs.
SchoolDocs provides an online policy platform and support service, helping schools and boards navigate changing legislation, compliance requirements and governance responsibilities. This year, SchoolDocs celebrates 20 years supporting New Zealand schools and now works with more than 90% of schools across Aotearoa, including approximately 79% of the independent school sector.
The expertise developed through SchoolDocs has since been extended into other sectors through LIGHTN, with ECEDocs supporting early childhood education services and GPDocs supporting general medical practices.
Together, LIGHTN’s three specialist platforms support more than 4,000 schools, early childhood services and medical practices across New Zealand. All are developed and maintained in New Zealand by the team behind SchoolDocs, with the same focus on making complex policy and compliance requirements easier to understand and manage.
For more information: team@schooldocs.co.nz | 03 977 8639 | schooldocs.co.nz | lightn.co.nz
