Safeguarding Adolescents Using Large Language Models for Mental Health Support

The mental health of Australian adolescents (12-18 years) is in crisis. Of the 2 in 5 Australian young people currently experiencing mental ill-health most are not accessing professional support despite the proliferation of digital mental health services, including mobile applications and web-based platforms. Instead an estimated 1.35 million Australian adolescents have turned to generative Artificial Intelligence (AI) for their mental health, typically general purpose Large Language Models (LLMs) e.g., ChatGPT and Character.AI. These LLMs were not designed to provide mental health support and pose a significant risk with use linked to recent suicides and the exacerbation of psychotic symptoms. Robust independent governance of LLMs is critical to mitigate the risk for vulnerable young Australians – yet none exists. In Australia governance is fragmented and relies largely on general harm-prevention laws, leaving significant gaps around duty of care, accountability, and oversight. Young people are more vulnerable to the risks associated with LLMs because their because their social, emotional and cognitive skills are developing. Emerging studies on social and epistemic trust demonstrate developmental periods where young people over-attribute socialness to AI systems (e.g.. see an AI system as a friend) and at other times over-trust AI systems even after witnessing blatant failures. There are also sub-groups of young people who may be more susceptible to the risks. Australian young people with neurodevelopmental conditions, such as autism, ADHD or both, have a 5-fold increase of co-occurring mental health conditions. LLMs can offer low-pressure social interactions and consistent acceptance but they can also amplify social withdrawal and encourage harmful behaviours. Project: co-design the first integrated digital governance infrastructure for LLM-based mental health support and operationalise it into a standalone safety application to provide: 1) real-time support for adolescents; and 2) analytics to enhance the decision making of clinicians working with consumers who seek mental health support from LLMs. The project includes an interdisciplinary team integrating expertise in AI, youth mental health and neurodevelopmental conditions, bioethics, law, and intervention mapping. Guidance from consumers, we will ensure outcomes are aligned with the needs of adolescents, families and clinicians.