The law reform and human rights charity, JUSTICE, is proposing ‘a rights- and rule of law-based framework’ for the use of AI in the justice system – Ensuring technology strengthens justice and the rule of law.
We’re glad someone is!
A briefing paper, Social work use of transcription AI, harmful hallucinations and the deletion of original audio, has now been published. This relates directly to family court proceedings.
JUSTICE has concerns about the routine use by social workers of transcription software such as a programme called ‘Magic Notes’, that uses a large language model (LLM).
We can see from this article by a social worker more than a year ago that the software can save her a lot of time but also has limitations, e.g.
“Sometimes it records things that weren’t actually said, so you need to be super cautious about copying and pasting.”
“If you’re a detailed writer, you’ll notice it omits crucial information. For example, it once assumed there were no debts when a client had clearly disclosed financial difficulties.”
As research by the Ada Lovelace Foundation explains:
Transcription AI is distinct from more traditional methods of automated transcription due to its incorporation of generative AI in the form of large language models (LLMs). LLMs help fill in gaps and infer content when audio quality is poor or contains long silences. However, by doing so, modern AI transcription tools have integrated a hallucination risk as a design choice to improve performance: there is an increased likelihood of plausible text output because of the LLM, but also an increased likelihood of fabrications.
They say:
At present, social workers are fully accountable for their records and professional judgements regarding an adult, child or family who draw on their care. Any AI-produced inaccuracies that enter these records may have far-reaching impacts, such as a social worker making an incorrect decision about a child’s care, which could lead to harm for the child and professional consequences for the social worker.
Family courts
Of course, any note-taking or recording by a professional, however conscientiously intended, can be subject to errors. There’s a risk of inaccuracy in any method of contemporaneous recording or relating from memory. But if this record is to be relied on as evidence in court, everyone wants it to be as error-free as possible.
The problem with this AI transcription method seems to have two aspects:
The LLM will ‘improve’ the recording to transcribe it, rather than literally reproduce what the social worker recorded, word-for-word. It will fill in gaps and silences with its own inferences.
The audio recording is automatically deleted after 30 days. So, if a transcribed note is later contested by a party in the court case, the social worker won’t have their ‘own’ note; they will only have the transcribed (and possibly enhanced or inaccurate) version.
There could be a very long interval between the recording and it being used in court.
JUSTICE ask:
if harmful mistakes do make it into care records or care assessments which are then submitted into court proceedings, what safeguards are needed to ensure the court can do its job and ensure fair and safe proceedings?
They conclude:
It is unrealistic to expect any design change in these tools to be instigated by the vendors … any change in design will be a response to the demands of their customers. Those customers – social workers – need an agreed set of standards for the use of AI transcription in social work which reflects system-wide perspectives of what safeguards are needed, including safeguards for the courts to do their job.
System-wide discussions should take place to identify these standards, and JUSTICE of course recommends that audio retention is considered as a critical piece of the puzzle. Such discussions should incorporate the perspectives of the judiciary, people affected by care processes, and regulators who establish standards for social work.
We have a small favour to ask!
TEN YEARS A CHARITY
The Transparency Project is a registered charity in England and Wales run by volunteers who mostly also have full-time jobs. Although we’ve now been going for a decade, we’re always working to secure extra funding so that we can keep making family justice clearer for all who use the court and work in it.
We can’t do what we do without help from you!
We’d be really grateful if you were able to help us by making a small one-off (or regular!) donation through our Just Giving page