We conducted a literature review to identify the PROMs used in diabetes. In addition, 3 databases were scanned—ProQolid, ClinicalTrials.gov, and PubMed—as these are the main sources where research on PROMs is being published. ProQolid was used to identify diabetes- and nondiabetes-specific PROMs. Findings were complemented by data from literature reviews of PROs used in prominent diabetes outcome consortiums, such as the International Consortium for Health Outcomes Measurement. Furthermore, PubMed and ClinicalTrials.gov were searched to establish which PROMs were most used in diabetes clinical trials and research between 1995 and May 2024 to identify the use of these PROMs for evaluating DHS (Multimedia Appendices 1 and 2).

In PubMed, the PROs name and “diabetes” were used to search for relevant results and in ClinicalTrials.gov the PROs name and “type 1 diabetes” and “type 2 diabetes” were used. Then, to determine associations with DHS, PROs name and “diabetes” were used in conjunction with “mobile application,” “telemedicine,” “telehealth,” “health digital solutions,” and “e-health” in PubMed. Not all articles mentioning DHS were selected; 2 data scientists independently read all abstracts to identify the peer-reviewed publications where PROs were used in relation to DHS. Therefore, in the final list of selected publications, a DHS was used, as well as a PRO in relation to this DHS.

PROM type (diabetes-specific or nondiabetes-specific [generic]), domain and objective, and occurrences in literature were collected for each PROM. The number of items, format, and administration time were collected for disease-specific PROMs, and copyright and language versions were collected for generic PROMs. Any mentions of PROMs used in reimbursement studies were noted. The PROMs most used in diabetes were analyzed, by determining which PROMs appeared in all 3 PubMed and ClinicalTrials.gov searches and were mentioned in HTA guidelines, as described next.

Alongside this, we retrieved and assessed guidelines about DHS evaluation from health authorities in France (HAS), Germany (BfArM), and the UK (NICE) [5-8]. These countries were chosen because of recent initiatives by their HTA authorities offering guidance on DHS assessment (Textbox 1).

In this targeted literature review, we identified 46 diabetes-specific and 16 nondiabetes-specific PROMs that were most used in diabetes clinical trials and research between 1995 and May 2024. In addition, this review shows that HTA guidelines on DHS evaluation from France, Germany, and the United Kingdom primarily reflect well-established PROMs for health-related quality of life.

Our review highlights that major HTA bodies acknowledge the importance and emerging needs of accepting PROMs as valuable outcomes to evaluate digital health interventions. However, by comparing the patient outcomes recommended in these guidelines to the number of prominent PROMs listed in the literature, there is still a lot of room for increasing the adoption and recommendation of PROMs by HTA bodies. Overall, in the literature as well as in the guidelines we identified a gap for PROMs that are relevant in evaluating DHS, such as disease knowledge (eg, carb counting in diabetes), eHealth literacy, digital burden (eg, data overload, fear of digital surveillance, and adverse effects associated with using digital technology), sexual life, family life, or well-being at work.

PROMs that were developed decades ago, as many of the PROMs we found in the literature, are likely to require updating before they can be used to assess the value of DHS. This is because they were developed to capture the impact of other types of interventions, and they may not be relevant for DHS and how these might affect patients’ overall well-being. In addition, more research needs to be done into associations between the domains measured by the PROMs and other outcomes, for example, clinical and economic outcomes that are important in a reimbursement process.

Although we see PROMs being used more frequently in modern diabetes interventions [29], their use is currently neither widespread nor consistent. One of the reasons that few DHS-specific PROMs have been used until now is the relative “youth” of digital health. The term “digital health” was coined in 2000 [30], and the WHO published a guide to harmonize the use of digital health terminology as recently as 2018. By contrast, diabetes-specific PROMs first appeared in the late 1980 s—Self-Efficacy for Diabetes (SED-D) [31] and the Hypoglycemic Fear Survey (HFS) [32] appeared in 1987—while generic, nondiabetes-specific PROMs used in diabetes first appeared even longer ago in the 1960 s, for example, the Beck Depression Inventory (BDI) [20] in 1961 and the Affect Balance Scale (ABS) [33] in 1969. The difference in PROMs and development lifecycles of digital products means that DHS may require existing PROMs to be adapted to a digital environment [34] or for new, bespoke PROMs to be created to adequately characterize the value and impact of DHS on patients’ health perceptions, QOL, and general well-being [35]. Either way, the lack of PROMs related to specific attributes of DHS needs to be addressed.

Specific, validated PROMs related to diabetes DHS have already been developed [36,37]. For new PROMs, patient communities emphasize the importance of assessing outcomes such as numeracy or health literacy, coping with diabetes, knowledge in diabetes management, evaluating problems with DHS (eg data overload), and evaluation of trust in health care providers.

The HTA bodies of France, Germany, and the UK generally agree that PROMs are valuable for the evaluation of DHS, but the scope of current HTA guidelines is limited to specific types of DHS, eg, medical devices or digital health applications. For these HTA bodies, there is broad consensus about the importance of assessing DHS based on functional, technical, and organizational characteristics, such as data security, practicality, quality, interoperability, and safety. Moreover, not only the investigated HTA bodies and countries focus on reimbursing and evaluating DHS, but most developed countries are working on frameworks and policies to provide faster access to DHS for patients and people with diabetes [38]. Obtaining a consensus on the best way to use PROMs to evaluate DHS is a necessary next step.

In France, although different types of DHS have been approved for reimbursement (mobile apps, telemonitoring systems, etc), the principles of evaluation published by the French health authority are specific to medical devices [7]. Deliberation about future frameworks for digital health interventions (including medical devices with artificial intelligence) is currently ongoing at the request of the French government and pharmaceutical companies. In the guideline specific to medical devices, PROs are highly recommended for supporting claims for reimbursement. Specific PROs are listed to demonstrate the impact on QOL [7].

In 2019, Germany adopted the Digital Health care Act (DVG), to promote the use of telehealth, mobile apps, and other digital solutions, as well as the use of health data for research purposes. The DVG entitles all individuals covered by statutory health insurance to reimbursement for certain digital health applications. The manufacturer must provide evidence of the positive effect of the digital application on care. In the Fast Track process guideline [6], which details how mobile app manufacturers can apply for reimbursement, the BfArM defines a full set of requirements for DiGAV (digital health applications), such as the types of study expected, and provides examples of PROs that would be suitable for the evaluation of certain endpoints. However, the scope of the DVG is limited to lower-risk medical devices, and many potentially valuable digital health applications are therefore not covered by the provisions of the DVG.

Early in 2019, NICE published the Evidence Standards Framework for Digital Health Technologies [5]. It is not suitable for all digital solutions, as it excludes mobile applications directly downloaded on app stores by users and solutions incorporating artificial intelligence using adaptive algorithms. In this guideline, digital solutions are classified in 4 categories or tiers: the higher the category, the more important the request for evidence is. Endpoints may vary depending on the category. Using PROs is highly recommended.

A major need in diabetes care is to improve behavior change techniques. Existing evidence shows that behavioral change can improve disease trajectories and reduce the risk of severe complications [39]. PROMs are at the core of understanding patient-relevant endpoints and how these relate to behavioral change as well as helping us understand where the unmet needs for people with diabetes are. By systematically assessing these outcome domains, therapy and behavioral decisions can be personalized for people with diabetes and therefore maximize the benefit of diabetes treatment. DHS may be a key catalyst for improving adherence to behavior change techniques and treatment overall [40]. By comprehensively understanding the effects of digital health interventions on both metabolic and patient-reported outcomes, people with diabetes may feel better informed about and included in treatment decisions, which has been shown to positively affect adherence and long-term outcomes [41].

As technologies quickly develop and become more available, it is vital to identify the unmet needs, through PROs, to properly assess the value of these solutions. As touched upon before, personalization of treatment plays a major role in increasing quality-of-care standards. The identification of unmet needs in outcome domains such as patient empowerment, QOL or health literacy should be a major deciding factor when tailoring treatment to people with diabetes. Much research is still needed in this area to provide evidence for unmet needs and PROMs tailored for DHS.

A holistic approach to improving health in diabetes looks beyond the attainment of established clinical endpoints and aims at including patient relevant dimensions. For example, improving metabolic control by reducing HbA1c can enhance QOL in diabetes, but it is not the only way of doing so. Decreased physical functioning and well-being of people with diabetes leads to a diminished QOL, but people with diabetes can actively improve QOL themselves through empowerment and self-efficacy [42,43]. Empowerment is the process by which patients gain the information and confidence to make independent, educated decisions about their diabetes using solid reasoning skills. Empowerment helps patients feel involved in treatment decisions and supports the ideal doctor-patient relationship, as defined in this quote, “The doctor is there to give the patient all the information the patient needs in order that the patient can make a decision, and the doctor should then implement the decision once the patient has made it” [44]. Self-efficacy is a measure of patients’ ability to manage their own diet, exercise, and medical treatment with assurance. DHS offer people with diabetes a way to increase empowerment and self-efficacy and thus both QOL and clinical outcomes. The number of DHS available to people with diabetes is increasing [45,46], which bodes well for patient-driven approaches to improving QOL and clinical outcomes in diabetes.

A common, comprehensive HTA framework to guide the use of PROMs in the context of DHS would help harmonize the evaluation of DHS. This, in turn, would facilitate and expedite the path of DHS to market and ultimately enhance the quality of support and education for people with diabetes. At present, HTA guidelines differ in terms of scope, types of DHS evaluated, and the role PROMs play in DHS reimbursement and evaluation [5-7]. These differences make the evaluation of DHS more complex and slower than it needs to be. Difficulties with evaluation may partly explain why DHS are struggling to be accepted by health systems despite promising results, and the harmonization of outcome measures may be one critical stepping-stone to simplify and accelerate the evaluation of DHS [47].

While interpreting the results of this targeted literature review, the following limitations need to be considered. First, it is difficult to describe DHS as one category as they vary substantially in terms of disease area, treatment regimen, type, and their regulatory risk classification. Second, as this review reflects on the two databases PubMed and Clinical Trial.gov trials that are listed in other databases may not be captured. Third, the patients’ perspective is missing in this research although it is a key topic around the use of PROMs for evaluating DHS. Finally, our assessment focused only on HTA-driven European countries.

A crucial prerequisite for the use of PROMs is the cooperation of patients in data collection. Patients may not be willing to share this data, for example for data privacy reasons or because the respective PROMs are too tedious to fill in. For example, Skovlund et al [48] described that patients ask for shorter questionnaires with dichotomous items instead of Likert-scale response formats to minimize complexity, to lower the cognitive burden, and to reduce the risk of questionnaire fatigue. Moreover, it is important that patients’ data is collected accurately and the entry of wrong data is avoided. Therefore, the items’ wording and terminology needs to be clear to prevent misleading or misinterpretation [48]. Ultimately, the data collection method can also influence the use and the validity of PROMs as there might be a difference between data collected by digital tools (eg smartphone apps and SMS text messaging) used by the patients on their own versus paper-based questionnaires filled in at the doctor’s office.

Many PROMs are used in diabetes care, although few currently exist with the aim to evaluate DHS. Certain domains of value to people with diabetes have few or no PROMs to evaluate them at present. Generally, key HTA bodies are acknowledging the value of PROMs, but there is a need to harmonize the outcomes and evaluation processes for DHS between countries. In diabetes, PROMs can help provide a more holistic assessment of patient health beyond the control of clinical and metabolic outcomes. Therefore, the value of DHS may be best captured through PROMs, which will increase our understanding of the full range of benefits from these interventions.