Type 1 diabetes (T1D) is one of the most common chronic diseases of childhood with significant morbidity and mortality. High-level evidence supports the optimization of blood sugar control to prevent chronic complications [1]. Even with remarkable advances in diabetes technology, such as continuous glucose monitoring systems and automated insulin delivery pumps, a majority of patients fail to meet glycemic targets [2].

Diabetes self-management and education play a major role in glycemic control. However, several challenges limit the potential of traditional educational models: shortages of diabetes educators, reduced access to care for some patient groups (exacerbated by the recent pandemic), and inherent limitations to addressing patients’ and caregivers’ individual needs.

Chatbots are tools powered by artificial intelligence (AI) that have shown promise in many aspects [3]. They are computer programs designed to respond to questions in natural language as if they were a real person. It is anticipated that such tools will be increasingly used by patients and their families because of their ease of access. Clinicians nonetheless remain a key source for trusted health information, and pediatricians and other front-line health care providers must understand the strengths and limitations of such tools.

A recent paper in Diabetes Care [4] noted strengths and inaccuracies of the recent GPT-4 model developed by OpenAI when asked questions on diet/exercise, glycemic management, and insulin storage and administration in adult patients. Children with T1D have unique needs regarding self-management when compared to adults. In this brief report, we sought to evaluate the strengths and weaknesses of an AI tool for pediatric diabetes management.

We asked a multidisciplinary team at the Pediatric Diabetes Clinic of the Montreal Children’s Hospital at McGill University Health Centre, a tertiary care center caring for 900 pediatric patients with T1D, to help craft a comprehensive list of questions regarding education and self-management of children with T1D spanning 4 key categories (Textbox 1). The multidisciplinary team included a pediatric endocrinologist, a pediatric emergency room physician, a medical student with previous experience in pediatric diabetes research, 3 certified diabetes educators, and 1 parent partner, all affiliated with the Montreal Children’s Hospital.

The questions were asked in both French and English to the chatbot, and its answers were evaluated qualitatively based on whether they reflected what is routinely taught to families and whether the answers were complete, accurate, and correct.

Textbox 2 compiles the feedback given by the multidisciplinary diabetes team, highlighting strengths and limitations of ChatGPT for diabetes self-management and patient education. We found that the answers provided by ChatGPT had the same limitations in both French and English.

When asked common questions on T1D, ChatGPT provided useful answers, though they were at times lengthy and lacked pediatric-specific details. It used clear terminology that had the potential for immediate translation to different languages. One important deficiency was noted regarding counseling for hyperglycemia, as it omitted the need for ketone measurement at home. The chatbot also provided inaccurate information on the type of insulin delivered by pumps (Textbox 2).

AI chatbots for diabetes education of patients and families can free precious health care provider time, allowing clinicians to support patients with the greatest need for more frequent human touchpoints. A human-AI interaction that integrates safeguards such as easily interpretable care algorithms can thus enable more frequent technology-enabled contact.

To understand the limitations of AI tools in diabetes education, we need to consider the way that they are developed and trained. They use an extraordinarily large text database from the internet and generate answers based on the likelihood of phrases following each other. They are not trained on databases specific to medical information and are not capable of evaluating the reliability of the answers they provide. Thus, they are prone to the phenomenon of “hallucination”: answers that sound reasonable but are factually incorrect [5].

As with any new technology in health care, it is important to consider ChatGPT’s strengths, limitations, and ethical implications prior to implementing its use in routine clinical practice. It is essential to ensure that AI-based resources are easily accessible to our patients to prevent worsening existing health care disparities. Moreover, transparency in how information is generated and communicated is critical, so that patients can interpret responses with appropriate caution. With the growing ease of access to these constantly evolving tools, front-line providers must advocate for use that is responsible and promotes value-based care [6].