‘Scary’: how misinformation and AI hallucinations are infiltrating Australia’s parliament

The useful part

Getty Images/Guardian design View image in fullscreen In some cases, committee reports have cited submissions in which the majority of sources appear to be – because they do not exist. Getty Images/Guardian design 'Scary': how misinformation and AI hallucinations are infiltrating Australia's parliament Exclusive: Guardian Australia can reveal that the system is being flooded with material, which is inventing studies and attributing nonexistent research to real academics and authors.

How it works

• Show An AI hallucination takes place when an LLM like ChatGPT or Claude outputs text that is plausible but incorrect, or confidently generates information that appears relevant but is actually unrelated.
• They simply predict the next most likely bit of text based on their training data, with a certain degree of creativity or randomness involved.
• Hallucinations can become more common when the AI is asked to generate content on topics that are not well covered in its training data.
• The situation is made more confusing by AI services including Google search.
• Haslam, who researches child and family adversity and maltreatment, says the reference was "scary" because it looked so real that someone taking a cursory look could be convinced.

What to take from it

This happens because these AI systems use complex statistics to figure out the text they generate and they have no innate ability to distinguish between correct and incorrect content. Now Google's AI summary will sometimes create summaries of a fake reference as though it were genuine. In some instances, Google's AI summary and ChatGPT will also cite the inquiry submission with the fake reference as a source, creating an ongoing cycle of misinformation.

Example or evidence

• Sign up for the Breaking News Australia email "It's very frustrating to … to have invested time, money, effort, expertise in rigorous research," she says.
• These are tags added by ChatGPT when it provides links to users and can persist when copied into a document.
• Transparency and contestability are a vital part of the research and peer-review process, which is why citations are used as evidence to back up claims, says Gurran, a professor of urban and regional planning.
• One document submitted to an inquiry into family violence and suicide included a hallucinated reference attributed to Divna Haslam, a University of Queensland associate professor and clinical psychologist,...

Details worth keeping

In some cases, committee reports have cited submissions in which the majority of sources appear to be "hallucinations", when large language models (LLMs) invent content that looks real but doesn't actually exist. These hallucinations are not bugs and although the frequency can be reduced by increasing the amount of training data or adding web-search results and other information, they are an inherent feature of LLMs and can never be fully eliminated. If the phenomenon continues, there is a significant risk of parliamentarians "making decisions based on evidence that doesn't exist", says Christian Downie, a professor in the Australian National University's school of regulation and global governance.

Related coverage

• Theguardian: Guardian Australia's reporting on content in inquiries has highlighted its dangers, politicians say Get our breaking news email, free app or daily news podcast Chairs of key federal parliamentary committees...
• Theguardian: A machete-wielding robber, a 'tsunami of debt' … campaigns echoing opposition talking points outspend major parties Get our breaking news email, free app or daily news podcast A man clad in a balaclava and...
• Washingtonpost: Artificial-intelligence tools increasingly allow political actors to fool voters — and help candidates cast doubt on reality.
• Accountingtoday: AI's infamous tendency to occasionally generate false information, also known as "hallucinations," remains untamed by even the world's largest accounting firms.