Abstract
Background: Digital health solutions (DHS) are technologies with the potential to improve patient outcomes as well as change the way care is delivered. The value of DHS for people with diabetes is not well understood, nor is it clear how to quantify this value.
Objective: We aimed to summarize current literature on the use of patient-reported outcome measures (PROMs) in diabetes as well as in selected guidelines for Health Technology Assessment (HTA) of DHS to highlight gaps, needs, and opportunities for the use of PROMs to evaluate DHS.
Methods: We searched PubMed and ClinicalTrials.gov to establish which PROMs were most used in diabetes clinical trials and research between 1995 and May 2024. HTA guidelines on DHS evaluation from France, Germany, and the United Kingdom were also assessed to identify PROMs for DHS evaluation in general.
Results: A total of 46 diabetes-specific PROMs and 16 nondiabetes-specific PROMs were identified. The most used diabetes-specific PROMs were (1) Diabetes Distress Scale, (2) Problem Areas in Diabetes, (3) Diabetes Empowerment Scale, (4) Diabetes Quality of Life, and (5) Diabetes Treatment Satisfaction Questionnaire. The most used nondiabetes-specific PROMs were Beck Depression Inventory, Sickness Impact Profile, EuroQol 5-Dimension, and Short Form 36-Item Health Survey. In HTA guidelines, the most prominent domain was health-related quality of life, for whose assessment there are well-established measures (Short Form 36-Item Health Survey and EuroQol 5-Dimension).
Conclusions: Of the many PROMs used in diabetes care, few are currently used to evaluate DHS, and certain domains of value in diabetes are not mentioned in HTA guidelines. A common, comprehensive DHS-specific HTA framework could facilitate and accelerate the evaluation of DHS.
doi:10.2196/52909
Keywords
Introduction
Despite innovative technologies and major advances in drug discovery, treatment goals in diabetes have not been fully met [
] and access for populations at risk is still lacking [ ]. More holistic and integrated treatment modalities are needed to improve treatment goals and access to care for people with diabetes worldwide [ ]. Digital Health Solutions (DHS) may offer a viable way to tackle these challenges. The potential of DHS for improving health and well-being is becoming increasingly evident [ ], and health authorities have begun to acknowledge the benefits of these solutions for patients and health care systems [ - ].The concept of digital health has been defined by the US Food and Drug Administration (FDA) as the use of technologies for health care and related purposes [
]. “Technologies” encompass mobile health, health information technology, wearable devices, telehealth and telemedicine, and personalized medicine, while “health care purposes” include preventing and treating disease, improving diagnosis, and enhancing health care delivery. The UK National Institute of Health and Care Excellence (NICE) characterizes digital health as technological solutions that improve (1) the efficiency of health systems, (2) understanding and communication about health, or (3) health interventions [ ]. Digital health interventions provide health care stakeholders with a means to address unresolved health system challenges, according to the World Health Organization (WHO) [ ].Traditionally, outcomes of health interventions have been measured using “narrow” clinical, biological, and metabolic endpoints. However, the WHO’s definition of health is broad: “a state of complete physical, mental and social well-being and not merely the absence of disease or infirmity” [
]. By improving outcomes related to patients and health care systems, DHS may benefit patient’s health and well-being beyond clinically measurable values [ , ]. Assessing these patient-reported perceived elements, for example, empowerment, self-efficacy or health literacy, is central to holistically understanding the effects of health interventions and capturing the full range of benefits from DHS [ ].Compared with existing interventions, DHS could provide near-constant monitoring and feedback, helping patients to better understand their disease and supporting them in making health care decisions [
]. At a system level, DHS could automatically collect, manage, and store health data. By providing this information to health care professionals, the quality of care could improve, and encounters could become more effective. There is an emerging need to provide validated tools that reliably assess these parameters of health and health care benefits. Patient-reported outcome measures (PROMs) are expected to play a key role in understanding patients’ perspectives of the mentioned outcomes to fully understand the range of benefits from DHS.PROMs are instruments to capture the impact of treatment on relevant patient perspectives and health-related outcomes usually outside the scope of clinical or biological endpoints. Examples include standardized, validated questionnaires on health status, perceived level of impairment, empowerment, or health-related quality of life PROMs used in health technology assessment (HTA) [
, ], which is defined by the WHO as “the systematic evaluation of properties, effects, and impacts of health-care technology” [ ]. In this research, we will focus on the potential value of PROMs for assessing DHS for people with diabetes.The objectives of this research are to (1) review current literature on the use of PROMs in diabetes and in selected HTA guidelines for DHS (2) and describe the challenges, needs, and opportunities for the use of PROMs to evaluate DHS.
Methods
Use of PROMs in Diabetes: Literature and HTA Guidelines Review
We conducted a literature review to identify the PROMs used in diabetes. In addition, 3 databases were scanned—ProQolid, ClinicalTrials.gov, and PubMed—as these are the main sources where research on PROMs is being published. ProQolid was used to identify diabetes- and nondiabetes-specific PROMs. Findings were complemented by data from literature reviews of PROs used in prominent diabetes outcome consortiums, such as the International Consortium for Health Outcomes Measurement. Furthermore, PubMed and ClinicalTrials.gov were searched to establish which PROMs were most used in diabetes clinical trials and research between 1995 and May 2024 to identify the use of these PROMs for evaluating DHS (
and ).In PubMed, the PROs name and “diabetes” were used to search for relevant results and in ClinicalTrials.gov the PROs name and “type 1 diabetes” and “type 2 diabetes” were used. Then, to determine associations with DHS, PROs name and “diabetes” were used in conjunction with “mobile application,” “telemedicine,” “telehealth,” “health digital solutions,” and “e-health” in PubMed. Not all articles mentioning DHS were selected; 2 data scientists independently read all abstracts to identify the peer-reviewed publications where PROs were used in relation to DHS. Therefore, in the final list of selected publications, a DHS was used, as well as a PRO in relation to this DHS.
PROM type (diabetes-specific or nondiabetes-specific [generic]), domain and objective, and occurrences in literature were collected for each PROM. The number of items, format, and administration time were collected for disease-specific PROMs, and copyright and language versions were collected for generic PROMs. Any mentions of PROMs used in reimbursement studies were noted. The PROMs most used in diabetes were analyzed, by determining which PROMs appeared in all 3 PubMed and ClinicalTrials.gov searches and were mentioned in HTA guidelines, as described next.
Alongside this, we retrieved and assessed guidelines about DHS evaluation from health authorities in France (HAS), Germany (BfArM), and the UK (NICE) [
- ]. These countries were chosen because of recent initiatives by their HTA authorities offering guidance on DHS assessment ( ).Health authority Country (Year) and Guidelines
Haute Autorité de Santé France [
] (2019)- Assessment of medical devices: Assessment principles established by the Medical Device and Health Technology Evaluation Committee (CNEDiMTS) to determine the reimbursement eligibility of medical devices for individual use
Haute Autorité de Santé France [
] (2019)- 2019 Prospective Analysis Report - Digital technology: what (R)evolution? [in French]
German Federal Institute for Drugs and Medical Devices (BfArM) Germany [
] (2020)- The Fast-Track Process for Digital Health Applications (DiGA) according to Section 139e SGB V - A Guide for Manufacturers, Service Providers and Users (version 1.0)
National Institute of Health and Care Excellence UK [
] (2019)- Evidence standards framework for digital health technologies
Results
Overall, 62 PROMs were identified from the literature review: 46 diabetes-specific PROMs and 16 nondiabetes-specific PROMs (
). The diabetes-specific PROMs most used in diabetes clinical trials and research are shown in .PROM | Domain or objective | Items | Format | Admin time (min) | Occurrences | Mentions HTA | guidelines or DHS reimbursement||||
ClinicalTrials.gov | PubMed | |||||||||
Diabetes, n | T1D | , nT2D | , nPROMs and diabetes | , nPROMs and diabetes for digital solutions | , n||||||
Summary of Diabetes Self-Care Activities (SDSCA) [ | ]Diabetes self-management | 11 | Self-report questionnaire with scores | <10 | 159 | 9 | 105 | 306 | N/A9 | N/A |
Diabetes Distress Scale (DDS) [ | ]Psychosocial distress | 17 | 6-point Likert Scale | 10 | 243 | 105 | 124 | 277 | 10 | N/A |
Areas in Diabetes (PAID) scale [ | ]Emotional functioning | 1, 5, 20 | Questions with values from 0 (“no problem”) to 4 (“serious problem”) | NA | 255 | 112 | — | 400 | 10 | United Kingdom |
Diabetes Empowerment Scale (DES) & DES-sf [ | ]Diabetes-related psychosocial self-efficacy | 8, 28 | Response categories ranging from “strongly disagree” to “strongly agree” | NA | 92 | 21 | 47 | 99 | 9 | N/A |
Diabetes Quality of Life (DQOL) [ | ]Relative burden of an intensive diabetes treatment regimen | 46 | 5-point Likert scale in 3 main domains: “satisfaction”, “impact”, and “worry” | NA | 92 | 52 | 39 | 274 | 3 |
|
Diabetes Treatment Satisfaction Questionnaire (DTSQ) [ | ]Satisfaction with diabetes treatment regimens and changes in satisfaction with treatment | 8 | 7-point scale ranging from 0 to 6. The questionnaire assesses treatment satisfaction and burden from hyper- and hypoglycemia | — | 219 | 135 | 83 | 110 | 8 |
|
aPROM: patient-reported outcome measurement.
bThe keywords used were: “full name of PROM” AND diabetes; “full name of PROM” AND type 1 diabetes; “full name PROM” AND type 2 diabetes.
cHTA: health technology assessment.
dDHS: digital health solution.
eT1D: type 1 diabetes.
fT2D: type 2 diabetes.
gThe keywords used were: “full name of PROM” AND diabetes AND (“mobile application” OR telemedecine OR telehealth OR “health digital solutions” OR “e-health”).
hnot available.
iHaute Autorité de Santé. Assessment of medical devices: assessment principles established by the Medical Device and Health Technology Evaluation Committee (CNEDiMTS) to determine the reimbursement eligibility of medical devices for individual use (La Plaine Saint-Denis, France; 2019).
These PROMs cover a wide range of outcomes, including diabetes self-management (SDSCA [
]), psychosocial distress (DDS [ ]) and diabetes distress (PAID [ ]), diabetes-related psychosocial self-efficacy (DES [ ]), relative burden of an intensive diabetes treatment regimen (DQOL [ ]), and satisfaction with diabetes treatment (DTSQ [ ]). Furthermore, 3 PROMs were mentioned in HTA guidelines: PAID [ ] in UK guidelines [ ], and DQOL [ ] and DTSQ [ ] in French guidelines [ ].shows the most prominent nondiabetes-specific PROMs used in diabetes clinical trials and research. The domains covered by these PROMs include severity of depression (BDI [ ]), patient dysfunction assessed through everyday behavior (SIP [ ]), health outcome from interventions on a common scale (EQ-5D [ ]), and generic health concepts (SF-36 [ ]). Two PROMs were mentioned in 3 HTA guidelines: EQ-5D [ ] in French guidelines [ , ], and SF-36 [ ] in French and German guidelines [ , ].
PROM | Domain or objective | Copyright | Language | Mentions | |||||
ClinicalTrials.gov | PubMed | ||||||||
Diabetes | T1D | T2D | PROMs and diabetes | PROMs and diabetes for digital solutions | HTA | guidelines or DHS reimbursement||||
Beck Depression Inventory (BDI) [ | ]Severity of depression in adults and adolescents | Aaron T. Beck | English +73 others | 82 | 13 | 37 | 581 | 3 | — | N/A
Sickness Impact Profile (SIP) [ | ]Patient dysfunction measured via everyday behavior, generally related to disease | Johns Hopkins University, 1977 | English | 2 | 1 | 1 | 397 | 2 | —N/A |
EQ-5D [ | ]Health outcome from interventions on a common scale, for evaluation, allocation, and monitoring | EuroQoL Group | English +181 others | 312 | 47 | 157 | 895 | 8 |
|
Short Form (SF-36) [ | ]Generic health concepts relevant across age, disease, and treatment groups | Medical Outcomes Trust (MOT), Dr J. Ware | English +160 others | 392 | 39 | 195 | 1387 | 10 |
|
aPROM: patient-reported outcome measure.
bThe keywords used were: “full name of PROM” AND diabetes; “full name of PROM” AND type 1 diabetes; “full name PROM” AND type 2 diabetes.
cT1D: type 1 diabetes.
dT2D: type 2 diabetes.
eThe keywords used were: “full name of PROM” AND diabetes AND (“mobile application” OR telemedecine OR telehealth OR “health digital solutions” OR “e-health”)
fHTA: health technology assessment
gDHS: digital health service.
hNot available.
iHaute Autorité de Santé. Assessment of medical devices: assessment principles established by the Medical Device and Health Technology Evaluation Committee (CNEDiMTS) to determine the reimbursement eligibility of medical devices for individual use (La Plaine Saint-Denis, France; 2019).
jFood & Drug Administration Center for Devices and Radiological Health. Value and Use of Patient-Reported Outcomes (PROs) in Assessing Effects of Medical Devices - CDRH Strategic Priorities 2016-2017. (Silver Spring, MD; 2017).
kFederal Institute for Drugs and Medical Devices (BfArM). The Fast-Track Process for Digital Health Applications (DiGA) according to Section 139e SGB V - A Guide for Manufacturers, Service Providers and Users (version 1.0) (Bonn, Germany; 2020).
The review of the 3 HTA guidelines identified 12 recommended patient outcomes and 5 outcome categories (
). The most prominent domain, Quality of Life, was recommended by the 3 HTA guidelines and there were well-established nondiabetes-specific PROMs for its assessment, for example, SF-36 and EQ-5D [ , ]. EQ-5D is directly linked to reimbursement processes in many countries and is used to derive quality-adjusted life year. The next most prominent domain was acceptability, from the DHS outcome category. Acceptability was highly relevant in both French and UK HTA guidelines [ , ]. Two diabetes-specific PROMs were mentioned in French HTA guidelines: DTSQ [ ] for the assessment of acceptability, and DQOL [ ] for quality of life (QOL) assessment.Outcome category | France (HAS | )United Kingdom (NICE | )Germany (BfArM) | |
Assessment of medical devices: assessment principles established by the CNEDiMTS | [ ]Evidence standards framework for digital health technologies | [ ]The Fast-Track process for digital health applications (DiGA) | , [ ]||
Acceptability | ||||
DHS | ++ (DTSQ | [ ])++ | ||
User satisfaction | ||||
DHS | + | + | ||
Engagement | ||||
DHS | + | |||
Patient empowerment | ||||
Patient sovereignty | ++ | |||
Health literacy | ||||
Patient sovereignty | + (HLS-EU-Q | [ ])|||
Quality of life | ||||
QoL/disease management | +++ (EQ-5D [ | ], SF-36 [ ], DQOL [ ])+++ | +++ (SF-36 | [ ])|
Symptom severity | ||||
QoL/disease management | + | +++ (NRS | [ ], SCL-90 [GSI , PSDI , PST] [ ])||
Autonomy | ||||
QoL/disease management | + | + | + | |
Coping with illness-related difficulties | ||||
QoL/disease management | + | |||
Reduction of therapy-related effort and strain (for patients or relatives) | ||||
QoL/disease management | + | + | ||
Adherence | ||||
Adherence | ++ (MAQ | [ ], Morisky Score [ ])|||
Enhanced safety | ||||
Safety | + |
aHAS: Haute Autorité de Santé.
bNICE: National Institute for Health and Care Excellence.
cBfArM: [German] Federal Institute for Drugs and Medical Devices.
dCNEDiMTS: [French] Medical Device and Health Technology Evaluation Committee.
eDHT: digital health technology.
fAll PROMs mentioned were given in specific examples.
gDiGA: digital health application.
hDHS: digital health solution.
iDTSQ: Diabetes Treatment Satisfaction Questionnaire.
jApplies to DHTs in tiers 1, 2, 3a and 3b.
kDomain covered satisfactorily by existing diabetes-specific PROMs.
lApplies to DHTs in tiers 3a (disease prevention and management) and 3b (treatment, diagnostic, active monitoring, and calculation tools with measurable user benefits) only.
mHLS-EU-Q: European Health Literacy Survey Questionnaire.
nQoL: quality of life.
oDQOL: Diabetes Quality of Life [instrument].
pSF-36: Short Form (36) Health Survey.
qNRS: Numerating Rating Scale.
rSCL-90: Symptom Checklist 90.
sGSI: Global Severity Index.
tPSDI: Positive Symptom Distress Index.
uPST: Positive Symptom Total.
vMAQ: Medication Adherence Questionnaire.
Discussion
Principal Findings
In this targeted literature review, we identified 46 diabetes-specific and 16 nondiabetes-specific PROMs that were most used in diabetes clinical trials and research between 1995 and May 2024. In addition, this review shows that HTA guidelines on DHS evaluation from France, Germany, and the United Kingdom primarily reflect well-established PROMs for health-related quality of life.
Challenges in Using PROMs for the Evaluation of DHS
Our review highlights that major HTA bodies acknowledge the importance and emerging needs of accepting PROMs as valuable outcomes to evaluate digital health interventions. However, by comparing the patient outcomes recommended in these guidelines to the number of prominent PROMs listed in the literature, there is still a lot of room for increasing the adoption and recommendation of PROMs by HTA bodies. Overall, in the literature as well as in the guidelines we identified a gap for PROMs that are relevant in evaluating DHS, such as disease knowledge (eg, carb counting in diabetes), eHealth literacy, digital burden (eg, data overload, fear of digital surveillance, and adverse effects associated with using digital technology), sexual life, family life, or well-being at work.
PROMs that were developed decades ago, as many of the PROMs we found in the literature, are likely to require updating before they can be used to assess the value of DHS. This is because they were developed to capture the impact of other types of interventions, and they may not be relevant for DHS and how these might affect patients’ overall well-being. In addition, more research needs to be done into associations between the domains measured by the PROMs and other outcomes, for example, clinical and economic outcomes that are important in a reimbursement process.
Although we see PROMs being used more frequently in modern diabetes interventions [
], their use is currently neither widespread nor consistent. One of the reasons that few DHS-specific PROMs have been used until now is the relative “youth” of digital health. The term “digital health” was coined in 2000 [ ], and the WHO published a guide to harmonize the use of digital health terminology as recently as 2018. By contrast, diabetes-specific PROMs first appeared in the late 1980 s—Self-Efficacy for Diabetes (SED-D) [ ] and the Hypoglycemic Fear Survey (HFS) [ ] appeared in 1987—while generic, nondiabetes-specific PROMs used in diabetes first appeared even longer ago in the 1960 s, for example, the Beck Depression Inventory (BDI) [ ] in 1961 and the Affect Balance Scale (ABS) [ ] in 1969. The difference in PROMs and development lifecycles of digital products means that DHS may require existing PROMs to be adapted to a digital environment [ ] or for new, bespoke PROMs to be created to adequately characterize the value and impact of DHS on patients’ health perceptions, QOL, and general well-being [ ]. Either way, the lack of PROMs related to specific attributes of DHS needs to be addressed.Specific, validated PROMs related to diabetes DHS have already been developed [
, ]. For new PROMs, patient communities emphasize the importance of assessing outcomes such as numeracy or health literacy, coping with diabetes, knowledge in diabetes management, evaluating problems with DHS (eg data overload), and evaluation of trust in health care providers.The HTA bodies of France, Germany, and the UK generally agree that PROMs are valuable for the evaluation of DHS, but the scope of current HTA guidelines is limited to specific types of DHS, eg, medical devices or digital health applications. For these HTA bodies, there is broad consensus about the importance of assessing DHS based on functional, technical, and organizational characteristics, such as data security, practicality, quality, interoperability, and safety. Moreover, not only the investigated HTA bodies and countries focus on reimbursing and evaluating DHS, but most developed countries are working on frameworks and policies to provide faster access to DHS for patients and people with diabetes [
]. Obtaining a consensus on the best way to use PROMs to evaluate DHS is a necessary next step.In France, although different types of DHS have been approved for reimbursement (mobile apps, telemonitoring systems, etc), the principles of evaluation published by the French health authority are specific to medical devices [
]. Deliberation about future frameworks for digital health interventions (including medical devices with artificial intelligence) is currently ongoing at the request of the French government and pharmaceutical companies. In the guideline specific to medical devices, PROs are highly recommended for supporting claims for reimbursement. Specific PROs are listed to demonstrate the impact on QOL [ ].In 2019, Germany adopted the Digital Health care Act (DVG), to promote the use of telehealth, mobile apps, and other digital solutions, as well as the use of health data for research purposes. The DVG entitles all individuals covered by statutory health insurance to reimbursement for certain digital health applications. The manufacturer must provide evidence of the positive effect of the digital application on care. In the Fast Track process guideline [
], which details how mobile app manufacturers can apply for reimbursement, the BfArM defines a full set of requirements for DiGAV (digital health applications), such as the types of study expected, and provides examples of PROs that would be suitable for the evaluation of certain endpoints. However, the scope of the DVG is limited to lower-risk medical devices, and many potentially valuable digital health applications are therefore not covered by the provisions of the DVG.Early in 2019, NICE published the Evidence Standards Framework for Digital Health Technologies [
]. It is not suitable for all digital solutions, as it excludes mobile applications directly downloaded on app stores by users and solutions incorporating artificial intelligence using adaptive algorithms. In this guideline, digital solutions are classified in 4 categories or tiers: the higher the category, the more important the request for evidence is. Endpoints may vary depending on the category. Using PROs is highly recommended.Needs and Opportunities of Using Patient-Reported Outcome Measures for the Evaluation of DHS
A major need in diabetes care is to improve behavior change techniques. Existing evidence shows that behavioral change can improve disease trajectories and reduce the risk of severe complications [
]. PROMs are at the core of understanding patient-relevant endpoints and how these relate to behavioral change as well as helping us understand where the unmet needs for people with diabetes are. By systematically assessing these outcome domains, therapy and behavioral decisions can be personalized for people with diabetes and therefore maximize the benefit of diabetes treatment. DHS may be a key catalyst for improving adherence to behavior change techniques and treatment overall [ ]. By comprehensively understanding the effects of digital health interventions on both metabolic and patient-reported outcomes, people with diabetes may feel better informed about and included in treatment decisions, which has been shown to positively affect adherence and long-term outcomes [ ].As technologies quickly develop and become more available, it is vital to identify the unmet needs, through PROs, to properly assess the value of these solutions. As touched upon before, personalization of treatment plays a major role in increasing quality-of-care standards. The identification of unmet needs in outcome domains such as patient empowerment, QOL or health literacy should be a major deciding factor when tailoring treatment to people with diabetes. Much research is still needed in this area to provide evidence for unmet needs and PROMs tailored for DHS.
The Need for a Holistic Treatment Approach
A holistic approach to improving health in diabetes looks beyond the attainment of established clinical endpoints and aims at including patient relevant dimensions. For example, improving metabolic control by reducing HbA1c can enhance QOL in diabetes, but it is not the only way of doing so. Decreased physical functioning and well-being of people with diabetes leads to a diminished QOL, but people with diabetes can actively improve QOL themselves through empowerment and self-efficacy [
, ]. Empowerment is the process by which patients gain the information and confidence to make independent, educated decisions about their diabetes using solid reasoning skills. Empowerment helps patients feel involved in treatment decisions and supports the ideal doctor-patient relationship, as defined in this quote, “The doctor is there to give the patient all the information the patient needs in order that the patient can make a decision, and the doctor should then implement the decision once the patient has made it” [ ]. Self-efficacy is a measure of patients’ ability to manage their own diet, exercise, and medical treatment with assurance. DHS offer people with diabetes a way to increase empowerment and self-efficacy and thus both QOL and clinical outcomes. The number of DHS available to people with diabetes is increasing [ , ], which bodes well for patient-driven approaches to improving QOL and clinical outcomes in diabetes.Harmonizing the Evaluation of DHS
A common, comprehensive HTA framework to guide the use of PROMs in the context of DHS would help harmonize the evaluation of DHS. This, in turn, would facilitate and expedite the path of DHS to market and ultimately enhance the quality of support and education for people with diabetes. At present, HTA guidelines differ in terms of scope, types of DHS evaluated, and the role PROMs play in DHS reimbursement and evaluation [
- ]. These differences make the evaluation of DHS more complex and slower than it needs to be. Difficulties with evaluation may partly explain why DHS are struggling to be accepted by health systems despite promising results, and the harmonization of outcome measures may be one critical stepping-stone to simplify and accelerate the evaluation of DHS [ ].Limitations of the Targeted Literature Review
While interpreting the results of this targeted literature review, the following limitations need to be considered. First, it is difficult to describe DHS as one category as they vary substantially in terms of disease area, treatment regimen, type, and their regulatory risk classification. Second, as this review reflects on the two databases PubMed and Clinical Trial.gov trials that are listed in other databases may not be captured. Third, the patients’ perspective is missing in this research although it is a key topic around the use of PROMs for evaluating DHS. Finally, our assessment focused only on HTA-driven European countries.
The Impact and Challenges of PRO Data Collection
A crucial prerequisite for the use of PROMs is the cooperation of patients in data collection. Patients may not be willing to share this data, for example for data privacy reasons or because the respective PROMs are too tedious to fill in. For example, Skovlund et al [
] described that patients ask for shorter questionnaires with dichotomous items instead of Likert-scale response formats to minimize complexity, to lower the cognitive burden, and to reduce the risk of questionnaire fatigue. Moreover, it is important that patients’ data is collected accurately and the entry of wrong data is avoided. Therefore, the items’ wording and terminology needs to be clear to prevent misleading or misinterpretation [ ]. Ultimately, the data collection method can also influence the use and the validity of PROMs as there might be a difference between data collected by digital tools (eg smartphone apps and SMS text messaging) used by the patients on their own versus paper-based questionnaires filled in at the doctor’s office.Conclusions
Many PROMs are used in diabetes care, although few currently exist with the aim to evaluate DHS. Certain domains of value to people with diabetes have few or no PROMs to evaluate them at present. Generally, key HTA bodies are acknowledging the value of PROMs, but there is a need to harmonize the outcomes and evaluation processes for DHS between countries. In diabetes, PROMs can help provide a more holistic assessment of patient health beyond the control of clinical and metabolic outcomes. Therefore, the value of DHS may be best captured through PROMs, which will increase our understanding of the full range of benefits from these interventions.
Acknowledgments
Professional medical writing and editorial assistance was provided by John Plant, PhD, (BPharm) and Florence Boulmé, PhD, (Cilas) on behalf of Else Care (Paris, France). Generative artificial intelligence was not used in any portion of the manuscript.
Authors' Contributions
PC and MG contributed to conceptualization, funding acquisition, methodology, project administration, supervision and writing. PV and OW assisted with methodology, investigation, project administration, and writing.
Conflicts of Interest
MJ has received fees for advisory/speaking services and research grant form Abbott, Air Liquide Santé International, Amgen, Astrazeneca, Bayer, BMS, Boehringer-Ingelheim, Dexcom, Glooko, Lifescan, Lilly, Medtronic, MSD, Nestle HomeCare, NovoNordisk, Orkyn’, Roche Diabetes, Sanofi, Vitalaire, and Voluntis. NO has received research funding from Dexcom, Roche Diabetes and Medtronic Diabetes and has participated in advisory boards for Dexcom, Roche Diabetes and Medtronic Diabetes. SG at Imperial College London receives research funding from Roche and Prova Health receives consulting payments from Roche. PV and OW are or were full-time employees for Else Care (Carenity) who received funding from Roche Diabetes Care to conduct the literature review. PC works for Roche Diabetes Care.
List of all assessed PROMs.
PDF File, 454 KBSearch strategy per database.
DOCX File, 16 KBPROMs, assessed HTA guidelines and expert opinions.
PDF File, 99 KBReferences
- Prigge R, McKnight JA, Wild SH, et al. International comparison of glycaemic control in people with type 1 diabetes: an update and extension. Diabet Med. May 2022;39(5):e14766. [CrossRef] [Medline]
- Addala A, Auzanneau M, Miller K, et al. A decade of disparities in diabetes technology use and HbA1c in pediatric type 1 diabetes: a transatlantic comparison. Diabetes Care. Jan 2021;44(1):133-140. [CrossRef] [Medline]
- Kazemian P, Shebl FM, McCann N, Walensky RP, Wexler DJ. Evaluation of the cascade of diabetes care in the United States, 2005-2016. JAMA Intern Med. Oct 1, 2019;179(10):1376-1385. [CrossRef] [Medline]
- World Health Organization (WHO). Global strategy on digital health 2020-2025. 2021. URL: https://apps.who.int/iris/bitstream/handle/10665/344249/9789240020924-eng.pdf [Accessed 2023-09-15]
- National Institute for Health and Care Excellence. Evidence standards framework for digital health technologies. 2018. URL: https://www.nice.org.uk/corporate/ecd7/resources/evidence-standards-framework-for-digital-health-technologies-pdf-1124017457605 [Accessed 2023-09-15]
- Federal Institute for Drugs and Medical Devices (BfArM). The fast-track process for digital health applications (DiGA) according to Section 139e SGB V - a guide for manufacturers, service providers and users (version 10). 2020. URL: https://www.bfarm.de/SharedDocs/Downloads/EN/MedicalDevices/DiGA_Guide.pdf;jsessionid=926F2482BE2E611FA8B45F25727E3447.intranet351?__blob=publicationFile [Accessed 2023-09-15]
- Haute Autorité de Santé (HAS). Assessment of medical devices: assessment principles established by the Medical Device and Health Technology Evaluation Committee (CNEDiMTS) to determine the reimbursement eligibility of medical devices for individual use. 2019. URL: https://www.has-sante.fr/upload/docs/application/pdf/2019-11/assessment_principles_established_by_cnedimts.pdf [Accessed 2023-09-15]
- Haute Autorité de Santé (HAS). Prospective analysis report - digital technology: what (R)evolution? [in French]. 2019. URL: https://www.has-sante.fr/jcms/p_3238368/en/has-proposes-the-first-classification-of-digital-solutions-used-in-healthcare [Accessed 2023-09-15]
- Food & Drug Administration (FDA). What is Digital Health. URL: https://www.fda.gov/medical-devices/digital-health-center-excellence/what-digital-health [Accessed 2023-09-15]
- World Health Organization (WHO). Classification of digital health interventions v10: A shared language to describe the uses of digital technology for health (WHO/RHR/1906). 2018. URL: https://www.who.int/reproductivehealth/publications/mhealth/classification-digital-health-interventions/en [Accessed 2023-09-15]
- World Health Organization (WHO). Basic Documents (including amendments adopted up to 31 May 2019). 2020. URL: https://apps.who.int/gb/bd/pdf_files/BD_49th-en.pdf [Accessed 2023-09-15]
- Van Der Wees PJ, Nijhuis-Van Der Sanden MWG, Ayanian JZ, Black N, Westert GP, Schneider EC. Integrating the use of patient-reported outcomes for both clinical practice and performance measurement: views of experts from 3 countries. Milbank Q. Dec 2014;92(4):754-775. [CrossRef] [Medline]
- World Health Organization (WHO). Health technology assessment of medical devices: WHO medical device technical series. 2011. URL: https://apps.who.int/iris/rest/bitstreams/53048/retrieve [Accessed 2023-09-15]
- Bradley C, Plowright R, Stewart J, Valentine J, Witthaus E. The Diabetes Treatment Satisfaction Questionnaire change version (DTSQc) evaluated in insulin glargine trials shows greater responsiveness to improvements than the original DTSQ. Health Qual Life Outcomes. Oct 10, 2007;5(57):57. [CrossRef] [Medline]
- Toobert DJ, Hampson SE, Glasgow RE. The summary of diabetes self-care activities measure: results from 7 studies and a revised scale. Diabetes Care. Jul 2000;23(7):943-950. [CrossRef] [Medline]
- Polonsky WH, Fisher L, Earles J, et al. Assessing psychosocial distress in diabetes: development of the diabetes distress scale. Diabetes Care. Mar 2005;28(3):626-631. [CrossRef] [Medline]
- Polonsky WH, Anderson BJ, Lohrer PA, et al. Assessment of diabetes-related distress. Diabetes Care. Jun 1995;18(6):754-760. [CrossRef] [Medline]
- Anderson RM, Funnell MM, Fitzgerald JT, Marrero DG. The Diabetes Empowerment Scale: a measure of psychosocial self-efficacy. Diabetes Care. Jun 2000;23(6):739-743. [CrossRef] [Medline]
- Jacobson AM, de Groot M, Samson JA. The evaluation of two measures of quality of life in patients with type I and type II diabetes. Diabetes Care. Apr 1994;17(4):267-274. [CrossRef] [Medline]
- Beck AT, Ward CH, Mendelson M, Mock J, Erbaugh J. An inventory for measuring depression. Arch Gen Psychiatry. Jun 1961;4:561-571. [CrossRef] [Medline]
- Bergner M, Bobbitt RA, Kressel S, Pollard WE, Gilson BS, Morris JR. The sickness impact profile: conceptual formulation and methodology for the development of a health status measure. Int J Health Serv. 1976;6(3):393-415. [CrossRef] [Medline]
- EuroQol Group. EuroQol - a new facility for the measurement of health-related quality of life. Health Policy. Dec 1990;16(3):199-208. [CrossRef]
- Ware JE, Sherbourne CD. The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Med Care. Jun 1992;30(6):473-483. [Medline]
- Sørensen K, Van den Broucke S, Pelikan JM, et al. Measuring health literacy in populations: illuminating the design and development process of the European Health Literacy Survey Questionnaire (HLS-EU-Q). BMC Public Health. Oct 10, 2013;13:948. [CrossRef] [Medline]
- Hartrick CT, Kovan JP, Shapiro S. The numeric rating scale for clinical pain measurement: a ratio measure? Pain Pract. Dec 2003;3(4):310-316. [CrossRef] [Medline]
- Derogatis LR, Lipman RS, Covi L. SCL-90: an outpatient psychiatric rating scale--preliminary report. Psychopharmacol Bull. Jan 1973;9(1):13-28. [Medline]
- Knobel H, Alonso J, Casado JL, et al. Validation of a simplified medication adherence questionnaire in a large cohort of HIV-infected patients: the GEEMA Study. AIDS. Mar 8, 2002;16(4):605-613. [CrossRef] [Medline]
- Krapek K, King K, Warren SS, et al. Medication adherence and associated hemoglobin A1c in type 2 diabetes. Ann Pharmacother. Sep 2004;38(9):1357-1362. [CrossRef] [Medline]
- Iversen MM, Igland J, Smith-Strøm H, et al. Effect of a telemedicine intervention for diabetes-related foot ulcers on health, well-being and quality of life: secondary outcomes from a cluster randomized controlled trial (DiaFOTo). BMC Endocr Disord. Oct 21, 2020;20(1):157. [CrossRef] [Medline]
- Frank SR. Digital health care--the convergence of health care and the Internet. J Ambul Care Manage. Apr 2000;23(2):8-17. [CrossRef] [Medline]
- Grossman HY, Brink S, Hauser ST. Self-efficacy in adolescent girls and boys with insulin-dependent diabetes mellitus. Diabetes Care. 1987;10(3):324-329. [CrossRef] [Medline]
- Cox DJ, Irvine A, Gonder-Frederick L, Nowacek G, Butterfield J. Fear of hypoglycemia: quantification, validation, and utilization. Diabetes Care. 1987;10(5):617-621. [CrossRef] [Medline]
- Bradburn NM, Noll CE. The Structure of Psychological Well-Being. Aldine; 1969. URL: https://www.norc.org/content/dam/norc-org/pdfs/BradburnN_Struc_Psych_Well_Being.pdf
- Field J, Holmes MM, Newell D. PROMs data: can it be used to make decisions for individual patients? A narrative review. Patient Relat Outcome Meas. 2019;10:233-241. [CrossRef] [Medline]
- Reaney M, Black P, Gwaltney C. A systematic method for selecting patient-reported outcome measures in diabetes research. Diabetes Spectr. Nov 2014;27(4):229-232. [CrossRef] [Medline]
- Steed L, Cooke D, Hurel SJ, Newman SP. Development and piloting of an acceptability questionnaire for continuous glucose monitoring devices. Diabetes Technol Ther. Apr 2008;10(2):95-101. [CrossRef] [Medline]
- Weissberg-Benchell J, Shapiro JB, Hood K, et al. Assessing patient-reported outcomes for automated insulin delivery systems: the psychometric properties of the INSPIRE measures. Diabet Med. May 2019;36(5):644-652. [CrossRef] [Medline]
- Essén A, Stern AD, Haase CB, et al. Health app policy: international comparison of nine countries’ approaches. NPJ Digit Med. Mar 18, 2022;5(1):31. [CrossRef] [Medline]
- Farmer A, Jones L, Newhouse N, et al. Supporting people with type 2 diabetes in the effective use of their medicine through mobile health technology integrated with clinical care to reduce cardiovascular risk: protocol for an effectiveness and cost-effectiveness randomized controlled trial. JMIR Res Protoc. Feb 21, 2022;11(2):e32918. [CrossRef] [Medline]
- Baradez C, Liska J, Brulle-Wohlhueter C, Pushkarna D, Baxter M, Piette J. Brief digital solutions in behavior change interventions for type 2 diabetes mellitus: a literature review. Diabetes Ther. Apr 2022;13(4):635-649. [CrossRef] [Medline]
- Burton L, Rush KL, Smith MA, et al. Empowering patients through virtual care delivery: qualitative study with micropractice clinic patients and health care providers. JMIR Form Res. Apr 27, 2022;6(4):e32528. [CrossRef] [Medline]
- Łuczyński W, Głowińska-Olszewska B, Bossowski A. Empowerment in the treatment of diabetes and obesity. J Diabetes Res. 2016;2016:5671492. [CrossRef] [Medline]
- Feng Wang H, Chen YC, Hua Yang F, Juan CW. Relationship between Type 2 diabetes self-efficacy and quality of life: analysis under varying glycated hemoglobin conditions. Fam Med Med Sci Res. 2017;06(2). [CrossRef]
- Charles C, Gafni A, Whelan T. Shared decision-making in the medical encounter: what does it mean? (or it takes at least two to tango). Soc Sci Med. Mar 1997;44(5):681-692. [CrossRef] [Medline]
- Greenwood DA, Gee PM, Fatkin KJ, Peeples M. A systematic review of reviews evaluating technology-enabled diabetes self-management education and support. J Diabetes Sci Technol. Sep 2017;11(5):1015-1027. [CrossRef] [Medline]
- Bassi G, Mancinelli E, Dell’Arciprete G, Rizzi S, Gabrielli S, Salcuni S. Efficacy of eHealth interventions for adults with diabetes: a systematic review and meta-analysis. Int J Environ Res Public Health. Aug 26, 2021;18(17):8982. [CrossRef] [Medline]
- Fatehi F, Menon A, Bird D. Diabetes care in the digital era: a synoptic overview. Curr Diab Rep. May 10, 2018;18(7):38. [CrossRef] [Medline]
- Skovlund SE, Renza S, Laurent J, Cerletti P. Identification of core outcome domains and design of a survey questionnaire to evaluate impacts of digital health solutions that matter to people with diabetes. J Diabetes Sci Technol. Jan 2025;19(1):136-142. [CrossRef] [Medline]
Abbreviations
ABS: Affect Balance Scale |
BDI: Beck Depression Inventory |
CNEDiMTS: Medical Device and Health Technology Evaluation Committee |
DHA: Digital Health care Act |
DHS: digital health solutions |
FAD: Food and Drug Administration |
HFS: Hypoglycemic Fear Survey |
HTA: health technology assessment |
MOT: Medical Outcomes Trust |
NICE: National Institute of Health and Care Excellence |
PROM: patient-reported outcomes measure |
QOL: quality of life |
SED-D: Self-Efficacy for Diabetes |
SIP: Sickness impact profile |
WHO: World Health Organization |
Edited by Ricardo Correa; submitted 19.09.23; peer-reviewed by Esse Evbuomwan, Hareesh Veldandi; final revised version received 18.06.24; accepted 27.01.25; published 04.06.25.
Copyright© Paco Cerletti, Michael Joubert, Nick Oliver, Saira Ghafur, Pasquale Varriale, Ophélie Wilczynski, Marlene Gyldmark. Originally published in JMIR Diabetes (https://diabetes.jmir.org), 4.6.2025.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Diabetes, is properly cited. The complete bibliographic information, a link to the original publication on https://diabetes.jmir.org/, as well as this copyright and license information must be included.