JMIR Publications

ABSTRACT

Background:

COVID-19 pandemic has raised the issues of reduction of human contact and healthcare works load. AI chatbot can provide preliminary assessment and follow-on directions to users, while reducing the healthcare workload and increasing user satisfaction. Ongoing research aims to identify, implement, and study the impacts of design factors for AI chatbots and conversational recommender systems that increase reported user satisfaction.

Objective:

This study evaluated user perceptions of an AI chatbot that was offered free to the public in response to COVID-19. The chatbot engaged patients, provided educational information and the opportunity to report symptoms, understand personal risk, and receive referrals for care. Materials and

Methods:

Chi-square tests and multinomial logistic regression analyses were conducted to assess the relationship between reported risk factors and perceived chat helpfulness using chats started between April 24th, 2020, and April 21st, 2022.

Results:

A total of 82,222 chat series were started with at least 1 question/response on record; 53,805 symptom checker questions with at least one COVID-19 related activity series were completed, with 5,191 individuals clicking further to receive a video virtual visit, and 2,215 to make an appointment with a local physician. A total of 9,931 answered the question “did you find this chat helpful”. Those patients who were over 65 years old, reported comorbidities, had been in contact with a COVID-19-infected person in the last 14 days, and responded to symptom checker questions that placed them at higher risk of COVID-19 were 1.8 times more likely to report the chat as helpful than those who reported lower risk factors. Users who engaged the chatbot to conduct a series of activities were more likely to find the chat helpful (P<.001), including seeking COVID-19 information (3.97-4.07 times), in person appointments (2.46-1.99 times), telehealth appointments with a nearby provider (2.48-1.9 times), or seeking vaccination (2.9-3.85 times) over those who did not perform any of these activities. Discussion: The chatbot program in this study provided a remote and helpful experience for the at-risk public to learn about COVID-19, assess risk, and find follow-on care options while minimizing COVID-19 exposure and in-person health care utilization. The results identified and validated significant design factors for conversational recommender systems design: 1. Triangulating a high-risk target user population and, 2. Providing relevant actionable items for users to choose from, as a part of user engagement.

Conclusions:

AI chatbots that are designed to target high risk user groups and provide relevant actionable items may be perceived as a helpful and satisfactory approach to early contact with the health system for assessing communicable disease symptoms at home prior to virtual or in-person contact with healthcare providers.