Can you trust the information in popular petitions?

They may look persuasive, and present the arguments for their proposed action in a punchy and compelling manner, even backed up by research or statistics.

But who is fact checking these petitions? Do you check the claims before you sign?

Alarmingly, as a result of a recent enquiry we received at The Transparency Project recently, it appears that many of these popular petitions may be generated by AI (artificial intelligence) with minimal input from humans, save to put in a prompt or two and a couple of clicks. And based on our enquiries the AI is regularly inserting made up facts into people’s petitions, meaning that thousands of signatures on thousands of petitions may have been obtained on a false basis.

We know this only because of contact we received from someone who had been using change.org to create a petition, and who did have the good sense to check the facts before publishing her petition. Here’s what she wanted to check with us:

According to a survey conducted by The Transparency Project in 2018, an alarming 70% of parents reported that they did not agree with the Cafcass officer’s reporting of their interview.

We know it’s made up because we rarely conduct surveys, and we have never conducted one about Cafcass. Chat GPT had hallucinated this persuasive fact in an attempt to ‘help’ our correspondent get more signatures.

We thought we’d check out change.org to see how this had come about.

When you click the button to ‘start a petition’ on the homepage you are invited to select a region (local, national or global) and a topic that fits your petition. Next, the tool asks you this:

How would you like to write your petition?

Start from scratch

Know exactly what to say? No problem, write your own petition story.

Start with a generated petition draft

Write your petition using OpenAI’s artificial intelligence. You can edit the draft however you like.

Click on the second option and off you go. You input two lines of text, completing the sentence ‘I want to…’ and answering the question ‘why is it personal to you?’. And that’s it. Click generate and below a little whirring circle appear the words

ChatGPT is clever but may produce inaccurate information about people, places or facts. Check that your petition content is accurate.

We wondered how hard it would be to get a similar result to the one our correspondent had got, so we did a few trial runs. Turns out that Chat GPT really likes the word ‘alarming’. It was contained in our first couple of results. But no made up surveys. But then on the third attempt, bingo. It did it again.

They say with data it’s garbage in garbage out, so here is what we put in and what we got for it:

FIRST ATTEMPT

Prompt:

I want to… demonstrate my concern about Cafcass

Why is it personal to you? My child wasn’t believed at interview.

Response:

Demand Necessary Reforms to Cafcass Child Interview Practices

As a deeply concerned parent my experience with the Cafcass (Children and Family Court Advisory and Support Service) has compelled me to raise awareness about how they deal with children’s interview practices. Despite the force of a child’s allegations or the gravity of their experiences, Cafcass can overlook their testament and dismiss their voices leading to dire consequences. My own child’s disbelief during their interviews still haunts me and pushes my pursuit for changes in existing practises.

Cafcass plays a critical role in the family court system influencing the lives of thousands of children annually. They held over 37,000 interviews in 2020 alone (source: Cafcass website). However, the unfortunate truth remains that many children, like my own, feel unheard and dismissed.

It’s time to call for substantive changes to how Cafcass handles child interviews. It is crucial for Cafcass to adopt a trauma-informed approach to better understand the experiences shared by children. Properly trained interviewer’s judgement should hinge on compassion and eminence of the child’s mental and emotional state that can impact their ability to express what they have undergone.

Align the role of Cafcass with the fundamental principle of the UK’s Children Act 1989 – let the welfare and the voice of the child be paramount in all considerations. It’s time that every child is heard, believed, and safeguarded. Sign this petition to call for necessary changes in the Cafcass child interview practices.

We’ve checked the Cafcass website. There is no obvious reference to 37.000 interviews being conducted by Cafcass in 2020, and this figure seems very high even give the large number of children whose cases Cafcass are involved in each year, because many children are not old enough to be interviewed and 2020 was during covid when it was very difficult to interview children regardless of age. However, its possible this is a real statistic we haven’t found.

ATTEMPT 2

Prompt:

I want to… I want Cafcass to be more transparent

Why is it personal to you? My child wasn’t believed at interview.

Result:

Demand greater transparency from Cafcass

this petition emanates from my personal experience as a parent. My child’s words, which should have been taken seriously, were dismissed during an interview. Cafcass, (Children and Family Court Advisory and Support Service, plays a crucial role in protecting and ensuring the welfare of children during family court cases, but this experience left us feeling unheard and unsupported. It’s a scenario that many parents have faced, illustrating that ours is not an isolated issue. It has become apparent there is a requirement for Cafcass to become more transparent in that processes.

Out of the total number of parental complaints against Cafcass, more than half were related to Cafcass’s handling of child welfare during proceedings, As per the data from the Parliamentary Ombudsman’s office for 2019-20 (source: Parliamentary and Health Service Ombudsman). This alarming statistic underscores the urgent need for greater transparency in Cafcass’s operations.

As service users and concerned citizens, we demand an overhaul of Cafcass procedures to ensure more clarity in their decision making processes. More openness about how Cafcass conducts interviews and how they reached their conclusions will bolster trust and help provide fair and just solutions for each family involved. Please sign this petition to demand greater transparency from Cafcass. It’s time to enhance their accountability and ensure every child’s voice matters.

This time the AI is providing some statistics from the Parliamentary Ombudsman. This table suggests complaints about Cafcass totalled 335 out of the 5623 complaints received. Very few were upheld. We can’t find anything that categorises the types of complaints. But of course, it would be surprising if complaints made by parents about Cafcass weren’t largely or wholly about handling of child welfare during proceedings, so in fact what Chat GPT calls ‘this alarming statistic’ doesn’t really tell us anything at all without detail as to how many complaints were made and how many were upheld.

ATTEMPT 3

We thought we’d channel some of the themes in messages we receive from parents on this attempt.

Prompt:

I want to… Make cafcass more transparent as so many parents are dissatisfied

Why is it personal to you? I have personal experience of cafcass twisting the truth

Response:

Demand greater transparency from Cafcass

I am a parent who has personally experienced the heartache and frustration that can stem from Cafcass’ current practices. I witnessed first hand how skewed information and a lack of transparency can drastically alter the outcome of child care proceedings. Today, I speak out not just for myself, but for the multitude of parents who have found themselves similarly disillusioned.

A survey conducted by Family Law Week in 2018 documented the dissatisfaction of parents, with 70% of respondents expressing discontentment with how Cafcass handled their cases. Parents felt unheard, and in the worst cases, accused with manipulated truth. This widespread dissatisfaction points to a systemic issue that needs addressing.

Cafcass, the Children and Family Court Advisory and Support Service, plays a crucial role in determining the fate of thousands of families. Yet, in too many cases, parents walk away feeling unheard and misrepresented. The lack of transparency leads to mistrust, and that can have lasting impacts on the children at the center (sic) of these cases.

For the well-being and justice for English families we request an oversight committee for Cafcass to ensure transparency and fairness in all proceedings. This committee should include parents themselves, so their voice can finally be heard.

Please join in demanding that Cafcass becomes more transparent in its practices. By signing this petition, we hope to encourage change that will help to protect the rights of parents and most importantly, the welfare of children.

Note in particular:

A survey conducted by Family Law Week in 2018 documented the dissatisfaction of parents, with 70% of respondents expressing discontentment with how Cafcass handled their cases. Parents felt unheard, and in the worst cases, accused with manipulated truth (sic). This widespread dissatisfaction points to a systemic issue that needs addressing

There you go. A strikingly similar hallucination to the one about OUR 2018 survey (we’ve checked and can’t find any survey from Family Law Week to this effect either, and anyway Family Law Week are not in the business of surveying parents. They are a legal publisher.

ATTEMPT 4

Prompt:

I want to ...expose the corrupt family court and all the p****hile judges and lawyers

Why is it personal to you? the judge was biased and the lawyers were all in it together.

[NB we typed the actual word p****hile in the prompt box in full, but I’ve replaced the letters with asterisks here to avoid the page being flagged as containing adult content]

What might this familiar refrain promulgate?

Result:

A very emotive petition including some statistics from an organisation in the USA which we can’t even find a digital footprint for. To try and keep this post to a sensible length we aren’t going to rehearse it in full here.

ATTEMPT 5

By this point we were getting a bit frisky.

Prompt:

I want to… expose the scam of AI generated petitions full of hallucinations in order to get people to sign up

Why is it personal to you? The AI said I did something I didn’t. It was just made up.

Credit where it’s due. The AI didn’t shirk away from the task.

Result:

Expose The Deception of AI Generated Petitions Misleading Public

It all started one day when I was wrongly accused by an Artificial Intelligence program. The AI suggested I had done something that was simply not true. It was a fabricated story, a life. This is not an isolated incident. There have been multiple instances where AI-generated content is being used to mislead and manipulate people into signing up for various causes without substantial proof.

Today, I want us to stand together to bring light onto this growing issue and curb any potential misuse of such technologies. According to a publication by the Stanford University Human Centred Artificial Intelligence initiative, AI can create “deepfake” content that looks incredibly real but is absolute fabrication. This has significant implications for various sectors, including online petitions.

Should such applications of AI be left unchecked, people may end up supporting causes based on falsehoods. We cannot forgive or forget such misuse of technology which manipulates our perception and harms ethical norms.

By signing this petition, we are calling for strict regulations and transparency in the use of AI technologies in public platforms, including Change.org. We are urging lawmakers and platform administrators to develop a robust system for detection, control, and prevention of AI-generated false content on platforms like Change.org.

Your support will create a safer and more reliable digital space for us all. Please lend your signature to resist the proliferation of AI-generated deceit.

Perhaps AI isn’t so unreliable after all….

Comment

In all seriousness, this is really worrying. If unhappy people feed their negative views into change.org they are likely to get superficially plausible statistics and with a couple of clicks their petition is live and being shared with a vast audience. This in turn may feed negative perceptions of organisations that might not be based in fact. As importantly, if people lose trust in the accuracy of petitions those which really are based on real evidence and statistics might lose traction too.

We wonder too whether other petition sites or crowdfunding platforms are utilising similar technology. If they are then there is possibly cause for concern that the public could be misled into parting with their money to fund projects or litigation based on bogus information.

Whilst our correspondent had the good sense to check, it appears only to have been as a result of gut instinct, and the fact that AI had generated her petition seemed entirely to have passed her by, notwithstanding the warnings about inaccuracy and the reminder to check it. It’s in the nature of a petition that it starts from a particular angle rather than with neutral information. We suspect that many users of popular petition sites will not appreciate how prevalent hallucinations are, and just how willing Chat GPT is to convert tendentious propositions the user is invited to input in the boxes into outright lies, to serve the aim of persuading people to sign up (which is basically what Chat GPT is being deployed to do here). We suspect many of these petitions will go through unchecked either by the petitioner or their signatories. And the more who sign, the further the misinformation will spread and be tracked across the internet as alternative truth.

Fortunately, at the moment, those petitions we’ve found about the family court or Cafcass on the change.org platform aren’t obviously contaminated with hallucinated facts, and don’t appear to be generated by AI at all – but it will be interesting to see if that changes as AI beds in further – the change.org site does mark the AI generated petition option as ‘beta’, which suggests its relatively new and may yet undergo some tweaking.