(L to R) Jacaranda Health's measurement and evaluation associate Teresa Ogolla, research assistant Winnie Ngigi and DGHI student Mary Brannock after a design focus group.
Published December 2, 2018, last updated on December 3, 2018 under Research News
By Nick Steenwyk
As artificial intelligence is on the rise, a few Duke Global Health Institute researchers have recognized its potential to improve access to healthcare. Artificial intelligence (AI) can come in many different forms, but Master of Science in Global Health student Mary Brannock and assistant professor Eric Green are focusing on chatbots.
A chatbot is a computer program that automatically replies to user messages based on a decision tree or artificial intelligence algorithm. Increasingly, chatbots are being used to give technology a more human face and make data more accessible to people. At the Duke Global Health Institute (DGHI), two teams are using chatbots to bring maternal health information and mental health tools to people in Kenya.
Understanding Attitudes toward Chatbots in Kenya
Mary Brannock’s master’s thesis research focuses on understanding attitudes toward social media-based chatbots that provide pregnancy support in Kenya. To assess women’s preferences and determine what information women need during pregnancy, she built and tested a chatbot to provide information and reminders on timely prenatal care visits, best breastfeeding practices and postnatal family planning options.
In Kenya, about 86 percent of people have access to the internet, typically through their smartphones. However, most phones have very limited storage space. For this reason, Brannock decided to build her chatbot in a Facebook Messenger platform, which is widely accessible and doesn’t require any extra space or added expense since most people already have the free app downloaded.
What she didn’t know was how users wanted the chatbot to look, sound and feel. For four weeks this summer, 26 women tested prototypes and participated in focus groups to help Brannock determine the characteristics of an ideal chatbot.
Under the mentorship of DGHI associate professor Krishna Udayakumar and adjunct associate professor Nicholas Pearson, Brannock used a human-centered design cycle to develop the chatbot. The cycle involved three stages: listening to users to determine needs, creating a prototype and soliciting user feedback on the prototype.
Brannock built the bot using Textit, a program she learned about through a course with Lavanya Vasudevan, an assistant professor in community and family medicine. She then partnered with Jacaranda Health, a social enterprise in Nairobi founded by Pearson, to collect user feedback and make improvements.
Brannock worked closely with Jacaranda’s chief information officer, Sathy Rajasekharan, and research coordinator Rachel Jones to develop the study design and coordinate data collection. Their goal was to determine how icons should look, how to structure dialogue and whether users wanted dialogue-based reminders from the app.
Brannock also had to decide how free-flowing the chatbot dialogue should be. Many chatbots offer semi-structured conversation—that is, the bot will ask a question and provide a list of possible responses, similar to an automated customer service phone call. Other chatbots have started to incorporate natural language processing, a type of AI that enables the bot to respond organically to any text users send it. Brannock found that structured conversation flow that included questions laid out on buttons was especially appreciated by new mothers that didn’t necessarily know what questions to ask.
“We also had to consider more subtle factors,” said Brannock. “For example, we asked the women, ‘Do you want to feel like you’re talking with a friend, or like you’re talking with your doctor?’ and ‘Do you want emojis and memes, or do you want to feel like you’re speaking with a professional?’”
Their data, which haven’t yet been fully analyzed, are already yielding some preliminary insights into women’s chatbot preferences. For example, the team found that women liked icons best when they didn’t represent race or stage of pregnancy and were neither fully human nor fully robot. Focus group participants also appreciated prompts that reminded them to schedule timely prenatal care appointments or keep on track with family planning services.
Overall, Brannock found that women were most receptive to a compassionate, friendly voice that gave them physician-approved advice on practical matters like diet and family planning.
Designing and Testing a Chatbot to Treat Perinatal Depression
A second team, led by assistant professor Eric Green, designed, built and tested an SMS-based chatbot that screens and provides treatment for perinatal depression. Green was struck by how chatbots could be used for task shifting—that is, delegating healthcare responsibilities to less specialized health workers. “Even with modest impacts, the cost-effectiveness of these digital tools could be quite high relative to traditional or task-shifting models,” he wrote.
Green’s team used an artificial intelligence program called Tess to deliver Thinking Healthy, a cognitive behavioral therapy-based curriculum published by the World Health Organization. The Thinking Healthy curriculum provides a script for a community health worker to use while interacting one-on-one with a depressed woman, so Green’s first task was to convert this script into a series of engaging 120-character conversation prompts. He then built these prompts into a decision tree and created the chatbot using X2’s Tess platform. His team is also working on a paper-based journal for women to document their progress through the curriculum.
After it was built, Green’s team also tested their bot at Jacaranda Health in Nairobi, Kenya. The team recorded video and screen captures of women interacting with the chatbot in 23 testing sessions over three weeks with ten women recruited from Jacaranda’s maternity hospital. “The biggest and most important takeaway from our user testing sessions is that women liked talking with our Tess mock-up. Their enthusiasm for the idea really motivated us to learn about how to make it better,” said Green.
The team is currently preparing for a pilot trial with 20 depressed women recruited from Jacaranda Health’s maternity hospital. Meanwhile, the team at X2 is finalizing their Tess SMS integration before Green’s team completes one more round of testing and refinement.
Learning to Listen
Both teams found that while chatbots promise to revolutionize the way people receive information, it can be challenging to make the technology personalized to specific user groups. “Being open to change is important with any type of innovation. You have to be able to quickly adapt to your user. What might make sense to me might not make sense to the people actually interacting with our product,” said Brannock.
Brannock and Green are excited about chatbots’ ability to provide social support in areas with limited human resources. “Just as technology helps all complex systems operate more efficiently and effectively, it seems reasonable to assume that technology can help scale up traditional and task-sharing models of care,” said Green.
Read more about Green’s chatbot in two Medium articles.
