For patient questions about cancer, artificial intelligence (AI) chatbots can generate quality, empathetic, and readable responses, according to a study published online May 16 in JAMA Oncology.
David Chen, from the Princess Margaret Hospital Cancer Center in Toronto, and colleagues examined the competency of AI chatbots (GPT-3.5 [chatbot 1], GPT-4 [chatbot 2], and Claude AI [chatbot 3]) to generate high-quality, empathetic, and readable responses to questions about cancer. AI chatbot responses were compared to responses from six verified oncologists for 200 patient questions about cancer from a public online forum.
The researchers found that compared with physician responses, responses generated by chatbot 3, which was the best-performing AI chatbot, were consistently rated higher in overall measures of quality, empathy, and readability. There was no significant difference noted between the mean Flesch-Kincaid Grade Level of physician responses and chatbot 3 responses (mean, 10.11 versus 10.31); however, the grade level was lower than that seen for chatbot 1 and chatbot 2 (mean, 12.33 and 11.32).
“Further research is required to investigate the implementation of AI chatbots into clinical workflows with consideration of chatbot scope, data security, and content accuracy in the age of digital health care,” the authors write.
More information:
David Chen et al, Physician and Artificial Intelligence Chatbot Responses to Cancer Questions From Social Media, JAMA Oncology (2024). DOI: 10.1001/jamaoncol.2024.0836
Copyright © 2024 HealthDay. All rights reserved.
Citation:
AI chatbots show higher empathy, readability in responding to cancer questions, study finds (2024, May 20)
retrieved 20 May 2024
from https://medicalxpress.com/news/2024-05-ai-chatbots-higher-empathy-readability.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no
part may be reproduced without the written permission. The content is provided for information purposes only.
