NTU Singapore doctors find mental health chatbots are effective in helping treat symptoms in people with depression

Clinician scientists from Nanyang Technological University, Singapore (NTU Singapore) have found that mental health chatbots are able to effectively engage people with depression in empathetic conversations and assist in the treatment of their symptoms.

Chatbots or conversational agents are computer programmes that simulate human conversations.  They are increasingly used in healthcare, for example, to help manage mental health conditions such as depression and anxiety and for general well-being.

A 2021 survey[1] by Woebot Health, one of the leading therapeutic chatbot companies in the US, found that 22 per cent of adults have used a mental health chatbot, with nearly half (47 per cent) saying they would be interested in using one if needed.

This study by doctors from NTU’s Lee Kong Chian School of Medicine (LKCMedicine) is among the first to analyse user-Chatbot dialogues to evaluate their effectiveness.

The researchers analysed nine mental health chatbots from leading app stores, of which five had at least 500,000 downloads, to see whether they offered self-help for people with depression.

Nine mental health chatbots were included in the study, four of which, Marvin, Serenity, Woebot, 7 Cups, are free-to-use, while Happify, InnerHour, Wooper, Wysa and Tomo, required a subscription or one-time purchase to be used.  

The chatbots were evaluated by the NTU research team through scripted user personas that were created to reflect different cultures, ages, and genders. The personas also presented behaviours that reflect varying degrees of depressive symptoms. 

This study published in December in the peer-reviewed Journal of Affective Disorders found that all the chatbots engaged in empathetic and non-judgmental conversations with users and offered support and guidance through psychotherapeutic exercises commonly used by psychologists and counsellors.

Through examination of the app interfaces and their privacy policy legal statements, the researchers observed that all the chatbots kept the confidentiality of the user’s personal information and did not transfer or store any of it. This information includes chat history, names, or addresses, which they might divulge during chat sessions.

Depression affects 264 million people globally and is undiagnosed and untreated in half of all cases, according to the World Health Organisation.[2]  In Singapore, the COVID-19 pandemic has led to an increase in mental health concerns, which include depression.[3]

Professor Josip Car, Director of the Centre for Population Health Sciences at NTU’s LKCMedicine, who led the study, said, “There are still a lot of stigmas surrounding mental health disorders and the COVID-19 pandemic has significantly increased the number of people affected by mental health issues. Worldwide, healthcare systems are struggling to cope with the increased demand for mental health services. Digital health tools, including chatbots, could assist in providing timely care to individuals who may be unwilling or unable to consult a healthcare provider. Through this study, we have shown how chatbots are being used and how they engage in therapeutic conversations.”

Chatting up chatbots to test their effectiveness

Although international research has shown that chatbots could help people, previous studies have not evaluated the dialogues between chatbots and users.

The NTU team’s content analysis evaluated the quality and effectiveness of the chatbots’ responses and looked at the level of personalisation, appropriateness in supporting self-management in users with depression, and how they conveyed empathy to users.

The study also monitored how the chatbots guided users to engage in or complete mood-boosting activities, how they monitored moods and managed suicide risks. 

The researchers said that all the chatbots displayed a “coach-like” personality that is encouraging, nurturing, and motivating. However, their analysis showed that while chatbots could engage in empathetic conversations with users they were not able to deliver personalised advice. This in-depth analysis of the conversational flow may be useful to help app developers design future chatbots.

First author Dr Laura Martinengo, a research fellow from LKCMedicine said: “Chatbots are not yet able to provide personalised advice and do not ask enough personal questions – possibly to avoid breaching user anonymity.  However, these chatbots could still be a useful alternative for individuals in need especially those who are not able to access medical help. For some people, it’s easier to talk to a machine than a human being.”

While chatbots may support the self-management of depression and other mental health disorders, the researchers said that further research is needed to improve chatbots for individuals at risk of suicide and to evaluate the long-term effectiveness of chatbot-led interventions for mental health.

The researchers will be conducting further studies to advance the scope, quality and safety of their research looking into the effectiveness of other digital methods for mental well-being.

***

Notes to Editor:

The research paper titled “Evaluation of chatbot-delivered interventions for self-management of depression: Content analysis” was published in Journal of Affective Disorders in Dec 2022. DOI 10.1016/j.jad.2022.09.028

*** END ***

Media contact:

Mr Joseph Gan

Manager, Media Relations

Corporate Communications Office

Nanyang Technological University, Singapore

Email: joseph.gan@ntu.edu.sg

About Nanyang Technological University, Singapore

A research-intensive public university, Nanyang Technological University, Singapore (NTU Singapore) has 33,000 undergraduate and postgraduate students in the Engineering, Business, Science, Medicine, Humanities, Arts, & Social Sciences, and Graduate colleges.

NTU is also home to world-renowned autonomous institutes – the National Institute of Education, S Rajaratnam School of International Studies, Earth Observatory of Singapore, and Singapore Centre for Environmental Life Sciences Engineering – and various leading research centres such as the Nanyang Environment & Water Research Institute (NEWRI) and Energy Research Institute @ NTU (ERI@N).

Under the NTU Smart Campus vision, the University harnesses the power of digital technology and tech-enabled solutions to support better learning and living experiences, the discovery of new knowledge, and the sustainability of resources.

Ranked amongst the world’s top universities, the University’s main campus is also frequently listed among the world’s most beautiful. Known for its sustainability, over 95% of its building projects are certified Green Mark Platinum. Apart from its main campus, NTU also has a medical campus in Novena, Singapore’s healthcare district.

For more information, visit www.ntu.edu.sg

[1] Woebot Health. Large-Scale Study Finds Mental Health App Forms Bond with Users (2021).

[2] World Health Organisation.  Depression, a fact sheet (2021)

[3] Institute of Mental Health, Singapore. Novel Coronavirus, Population Well-being and Resilience Cross-Sectional Study (2021)