Remote Foot Temperature Monitoring Among Veterans: Large Observational Study of Noncompliance and Its Correlates

Introduction

Diabetic foot ulcers are common, debilitating, and costly diabetes complications. Over 10 percent of US adults [1] and nearly a quarter of veterans enrolled in Veterans Health Administration (VHA) have diabetes [2]. In patients with diabetes, lifetime risk of ulceration is estimated to be 14% [3]. Ulcerations negatively impact mobility, mental health, and quality of life, and have high recurrence rates. Nearly two-thirds of patients have a recurrence within 5 years of ulcer healing [4]. Loss of pain sensitivity, foot deformity, and peripheral artery disease place individuals at high initial and subsequent risk of ulceration; these conditions do not resolve after healing.

Five systematic reviews and meta-analyses [5-9] have been conducted that each included the same 4 or 5 [10-14] randomized controlled trials of foot temperature monitoring. While there were slightly different analytic approaches in each meta-analysis, all estimated a substantially lower risk of ulceration in the groups assigned to monitor plantar foot temperatures compared to the usual care groups (meta-analysis odds ratios [ORs] or relative risks ranged from 0.37 [7] to 0.53 [8]). Based on this research, several clinical organizations have endorsed foot temperature monitoring [15-17], but it is rarely practiced because measuring temperatures on multiple parts of feet daily and then calculating differences between the feet is time consuming and burdensome. New technologies, including temperature sensing mats, have eliminated much of the burden, and made foot temperature monitoring easier [18,19].

In 2019, the VHA, the largest integrated health care system in the United States, began national implementation of remote foot temperature monitoring (RTM) using SmartMats. The VHA Innovation Ecosystem launched the Initiative to End Diabetic Limb Loss [20] in partnership with the VHA Podiatry Service, Office of Health Equity and Office of Connected Care to design new care models that incorporated emerging technologies such as the SmartMat in early detection of diabetic foot ulcers.

We are aware of a single study that published data on compliance with use of the SmartMat [21]. This study included 132 people with diabetes and a prior diabetic foot ulcer who were recruited from 7 outpatient sites in the United States. During 34 weeks of follow-up, patients used the device 5.0 days per week on average (SD not reported) and 86% of patients used the SmartMat ≥3 times per week [21]. Data on characteristics associated with use were not reported.

Understanding compliance outside of a study is valuable to assess the real-world potential for effectiveness. Additionally, identifying which patients might be less likely to comply with RTM recommendations could be helpful to identify barriers and determine which patients might benefit from additional support to improve compliance. Thus, the aims of this study were to estimate noncompliance in patients who were enrolled in RTM nationwide across VHA between 2019 and 2021 and to evaluate characteristics associated with noncompliance.

Methods

Study Data

Demographic, geographic, medical, and use data came from the corporate data warehouse (CDW). Race and ethnicity are determined based on self report. During this study’s period, when a patient was enrolled in RTM, their Department of Veterans Affairs (VA) provider placed an order for the device through the Prosthetics Department. As the company making the devices only had 1 product, we were able to determine enrollment in RTM based on the Data Universal Numbering System number (DUN & Bradstreet). The company provided information on the average number of times per week that patients used the mat each calendar month from January 2019 until October 2021.

Study Population

We included individuals who were enrolled in RTM in VA between January 2019 and June 2021 (as identified in the CDW) and for whom we were also able to obtain SmartMat use data. Of those enrolled in January 2019 or later (n=1675), we were able to link 1641 individuals to other data in the CDW. We excluded 504 people with less than 12 months of follow-up, leaving 1137. Due to how the variables were constructed, individuals for whom we could not determine a home facility (n=123) were included in most analyses, but were missing for area deprivation index (ADI), VA district or region, and facility complexity.

RTM in VA

The device under investigation is a daily-use telemedicine foot temperature monitoring SmartMat made by Podimetrics. Patients with a high risk of ulceration (mainly due to a history of foot ulceration; lower limb, foot, or toe amputation; or Charcot foot) are eligible for RTM (Kyle Nordrum, DPT, personal communication). The device is ready to use without any configuration or set-up. A temperature scan takes 20 seconds, and the temperature data are transmitted to the cloud using an embedded cellular component. All scans are timestamped, allowing for objective measures of use. The software detects “hot spots,” defined as asymmetries of ≥2.2˚C between the same region on the left and right foot or different regions on the same foot. Temperature asymmetries that persist for at least 2 days are predictive of ulceration [21]. When hot spots are detected, either clinical support staff from the company or the VA provider follows-up with the patient to evaluate risk factors and make recommendations about actions to take. Patients are also called if they fail to use the mat for several days in a row to assess reasons for nonuse and encourage re-engagement.

Definition of Noncompliance

While daily use is recommended, using the device at least 2 times per week is thought to be sufficient for the detection of hot spots (Jon Bloom, MD, personal communication). As we had a relatively long follow-up (12 mo), we defined noncompliance as using the mat less than 2 times per week for at least a quarter of the months under observation. Specifically, patients were considered noncompliant if they used the mat less than 2 days per week for 4 or more months out of 12.

Compliance Correlates or Predictors

We evaluated demographic, geographic, clinical, and facility factors, as well as health care use as potential correlates of nonadherence. Details about the data sources, definitions, and categories are included in Table 1.

Statistical Analyses

We calculated the percentage of patients who were noncompliant, with 95% CIs. To evaluate associations between characteristics and noncompliance, we estimated OR and corresponding 95% CIs using a logistic regression model that included all covariates. The Akaike information criterion (AIC) was used to assess contributions of each covariate and groups of covariates to model fit along with a likelihood ratio test for a model that excluded the covariate or group of covariates [25]. AIC helps to quantify how well a model fits the data it was generated from relative to other models fit on the same data. AIC penalizes models that use more parameters to reduce the potential for overfitting. Lower AIC scores are considered evidence of better model fit.

Missing data were recovered using multiple imputation by chained equations using all covariates and the outcome and results displayed are from pooling the 20 imputed datasets [26-28]. Generalized variance inflation factors [29] for each of the covariates were calculated to assess correlation between covariates, and the impact it may have had on regression results. A variance inflation factor of 4 or more was used as evidence of substantial collinearity [30]; there was no evidence of substantial collinearity.

We conducted several sensitivity analyses. First, because there is no empirical basis for our definition of noncompliance [31], and some prior studies have used a higher cutoff, we conducted analyses using different cut points of minimum days per week on average (2 and 3) and months in the past 12 (11, 9, and 6) for defining noncompliance. Second, we graphically explored the association between hospitalization and separately, amputation, on SmartMat use by examining use in the 6 months prior to, and the 6 months after a hospitalization or amputation (separately). In these analyses, we categorized days per week into 4 categories: no use, <2 days per week, 2 to <5 days per week, and 5‐7 days per week. We also graphed use over time in those with a hospitalization or amputation (separately) relative to those who had neither a hospitalization nor an amputation.

Ethical Considerations

This program evaluation qualified as a nonresearch quality improvement activity conducted under the authority of VHA operations. It complies with the VHA definition of “non-research operations activities” outlined in section 5a of the 2019 VHA Program Guide 1200.21: VHA Operations Activities That May Constitute Research, meeting both specified conditions: (1) the evaluation was designed and implemented for internal VHA purposes and (2) not designed to produce information to expand the knowledge base of a scientific discipline.

Results

The sample was predominantly male (n=1125, 98.94%); 21.1% (n=230) were Black and 75.7% (n=825) were White (Tables 2 and 3). Just over half (n=595, 53.8%) of the patients were married. Nearly half (n=525, 46.2%) of the patients were aged between 70 and 79 years and 96.9% (n=1102) had diabetes. In the 2 years prior to baseline, 82.0% (n=932) had a diabetic foot ulcer, 40.4% (n=459) had osteomyelitis, 41.6% (n=473) had chronic kidney disease or end stage renal disease, 30.7% (n=349) had depression, and 53.3% (n=606) were hospitalized. Over a third (n=409, 36.0%) had a Gagne comorbidity of 4 or greater and 59.8% (n=660) had a BMI>30 kg/m². About two-thirds of patients had poorly controlled diabetes based on a hemoglobin A_1c greater than or equal to 7.0, including 7% (n=80) with a hemoglobin A_1c greater than or equal to 10. Further, 31.2% (n=181) of patients were current smokers, though smoking status was missing for nearly half of the participants. Lastly, 74.9% (n=851) of the patients lived in urban areas and less than 20% (n=143, 12.6%) lived more than 30 minutes’ drive time from primary care or 60 minutes’ drive time from specialty care (n=182, 16%). Most of the patients came from the Midwest (n=361, 35.6%) or the Pacific region (n=335, 33.0%), while less than 5% (n=48) of the patients were from the Southeast. Multimedia Appendix 1 compares those included in analyses to those who were excluded because of insufficient follow-up. Briefly, the individuals with insufficient follow-up who were excluded from analyses had fewer ulcer risk factors, were predominantly from the Continental region, and were more likely to live in rural areas where they had longer drive times to primary and specialty care.

Overall, 37.6% (428/1137) of patients were classified as noncompliant (Tables 4-6 and Figure 1). Mat use declined over time; by month 12, over 30% of patients never used the mat in the prior month. In descriptive analyses, the factors associated with higher noncompliance (5 percentage points or more above the mean) included race other than Black or White (43% noncompliance), Hispanic or Latino ethnicity (48%), widowed (47%), ADI <24 (47%), age <60 (47% for those aged <50 years and 44% for those aged 50‐59 years), osteomyelitis (45%), major lower limb amputation (44%), Gagne comorbidity index >4 (48%), depression (44%), BMI <25 kg/m² (49%), inpatient visit in the 2 years before baseline (43%), more than 28 telehealth encounters (43%), hemoglobin A_1c ≥10 (60%), current smoking (54%), substance use disorder (53%), living in the Pacific region (44%), and receiving care at a high-complexity facility (44%). The factors associated with lower noncompliance (5 percentage points or more below the mean) included being a female (33% noncompliance) and living in an area with an ADI between 50 and 74 (31%); not having a recent history of ulceration (32%), osteomyelitis (33%), or hospitalization (31%); and having a lower comorbidity index score (≤0: 29% noncompliance; 1‐2: 30% noncompliance); BMI ≥30 kg/m² (30‐39.9 kg/m²: 33% noncompliance; ≥40 kg/m²: 30% noncompliance) fewer than 12 telehealth encounters (<6: 28% noncompliance, 6‐12: 31% noncompliance); and hemoglobin A_1c between 7 and 7.9 (32% noncompliance). Lastly, noncompliance was lower among those from the Midwest (32%) and at low-complexity facilities (22%). In the multivariable model, compared to an ADI <25, intermediate ADI was inversely associated with noncompliance (ADI 50‐74: OR 0.56, 95% CI 0.35-0.89); associations for the other categories were not statistically significantly different from the lowest ADI category. Recent history of osteomyelitis (OR 1.44, 95% CI 1.06‐1.97), Gagne comorbidity index score ≥4 (vs ≤0: OR 1.81, 95% CI 1.15-2.83), telehealth encounters (13‐27 vs <6: OR 1.65, 95% CI 1.01-2.70; 28+ vs <6: OR 1.70, 95% CI 1.02-2.84), hemoglobin A_1c ≥10 (vs <5.7: OR 2.67, 95% CI 1.27-5.58), and current smoking (vs never smoking: OR 2.06, 95% CI 1.32-3.20) were statistically significantly and positively associated with noncompliance. Using AIC to help inform the contribution of different variables to model fit, behavioral factors (hemoglobin A_1c and smoking), and to a lesser extent, health conditions or comorbidities (eg, osteomyelitis and Gagne comorbidity index) most contributed to model fit (Table 7). The results were not meaningfully different in the sensitivity analyses that used different cut points for minimum number of days per week and months in the past 12 to define noncompliance (data not presented).

‎

When considering changes in use around the time of a hospitalization or amputation, we found that the fraction of people with 5‐7 days per week of SmartMat use decreased, while the proportion of those with no SmartMat use increased in the month of the hospitalization or amputation (Multimedia Appendices 2 and 3). The proportion with no SmartMat use increased slightly in the 6 months after the hospitalization or amputation (Multimedia Appendices 4 and 5). Specifically, there was a larger increase in the proportion of patients with no use (and decrease in the proportion of patients with 5‐7 d of use) in the month concurrent with a hospitalization or amputation as well as in the 6 months after, compared to the reference group of patients with no hospitalization or amputation during follow-up.

Discussion

Principal Findings

Even though the requirements of monitoring are minimal (20 s per day to obtain a scan), nearly 4 in 10 patients did not use the SmartMat at least 2 days per week for at least 75% of the months under observation. Based on AIC differences, the strongest factors associated with noncompliance were behavioral factors (poor glucose control [as measured by hemoglobin A_1c] and smoking), and to a lesser extent, factors such as a recent history of osteomyelitis and an elevated Gagne comorbidity index, indicating a high comorbidity burden. Patients who are unable to manage their blood glucose levels or quit smoking, as well as those with osteomyelitis and numerous chronic comorbidities may also have challenges regularly using a home temperature monitoring device. Noncompliance was also higher among those who had a hospitalization or amputation. Although some results were different in descriptive analyses (eg, BMI and ADI) or statistically significant (eg, ADI) in our multivariable model, the absence of large AIC differences indicated that they did not contribute importantly to model fit and therefore may not be important factors for compliance. Additional research that replicates these findings and that can help us understand reasons for noncompliance in patients would be helpful to inform future interventions.

We are only aware of a single study [21] that evaluated compliance with a SmartMat for foot temperature monitoring, and that study found lower noncompliance using a more stringent definition. Unlike our study, the Frykberg et al [21] study excluded days during which a participant had a contraindication to using the mat (eg, for an open plantar wound) and considered those who did not use the mat for >28 consecutive days as lost to follow-up (18.6% of the sample) [21]. As our sensitivity analyses indicated a reduction in use following a hospitalization or amputation, differences in use between the prior study and ours may be partly because we did not exclude any days. In any case, noncompliance in this study is similar to prior studies of foot temperature monitoring involving handheld thermometers. For example, in a trial of 151 people randomized to daily foot temperature monitoring using a handheld thermometer, 62% of participants measured foot temperatures at least 70% of days (equivalent to 38% noncompliance) [14]. Likewise, a study of daily temperature monitoring in Peru observed 60% compliance when treating those who did not return logbooks as nonadherent (vs adherence of 87% when leaving them out of the analysis, equivalent to 40% and 13% noncompliance) [32]. Estimates for foot temperature monitoring are also similar to general adherence to a variety of self-management activities in people with diabetes observed in meta-analyses [33]. In summary, even though RTM is a relatively low burden intervention, noncompliance rates are strikingly similar despite different definitions of adherence and different activity or intervention burdens, suggesting that factors other than time burden likely impact compliance.

The Unified Theory of Acceptance and Use of Technology 2 (UTAUT2) [34] is a helpful framework for understanding why individuals accept and use technology and may help to understand factors impacting compliance with RTM. UTAUT2 outlines how 7 constructs influence consumers’ intention to use a technology: (1) performance expectancy (does it help?), which is analogous to relative advantage in the diffusion of innovation literature [35] and perceived usefulness in the technology acceptance model [36], (2) effort expectancy (how easy is it to use?), (3) facilitating conditions (are there resources available to support use?), (4) social influence (do those close to the individual support their use of the technology?), (5) hedonic motivation (is it fun?), (6) price value (is it worth it financially?), and (7) habit (does it become a habit?).

Because we lack data from patients’ perspectives, it is unclear whether patients perceived a benefit, especially those who did not have any hot spots, or had many hot spots. Unlike a scale or a blood pressure monitoring device, which provides direct feedback with each use, and a sense of accomplishment for those who are losing weight or lowering their blood pressure, patients were not routinely provided direct feedback. They were called if they did not use the mat, or if temperature asymmetries were detected. If perceived benefit is low, use may decrease over time. Even though the apparent time burden is low, there may be steps (such as removing socks and shoes) that may be challenging and prevent people from using the mat more regularly. Given that an annual SmartMat subscription includes access to company personnel to answer questions, the third construct from the UTAUT2, facilitating conditions, may be high. As social influence is known to be important in diabetes management [37], social influence may be an important factor for this technology. Unfortunately, we had no direct measures of social influence, and marital status (a poor proxy) was not associated with compliance, which is understandable because marital status does not provide a direct measure of whether someone has a positive social influence. RTM is intended for disease management and was not designed to be fun, so hedonic motivation may be low. Gamification (eg, points and badges for streaks or other goals) could make it more fun and might improve compliance [38]. Price value is likely low since VA provides the SmartMat free of charge to patients. Future studies that collect self-report information from patients, including patient interviews, could help elucidate the extent to which factors in the model facilitate use.

Strengths and Limitations

A major strength of our study was its large size and ability to examine associations between various characteristics and SmartMat noncompliance. A potential limitation is that we did not have data on each day’s use, but instead an average number of days per week per month of follow-up. Though less precise, this level of detail is informative to understand trends. Third, there are no standard definitions of adherence or noncompliance, and our definition was different from the 1 used in a prior study [21], though it was based on expert opinion. Fourth, we lacked information on factors not readily available in the medical record such as patient preferences, perceived benefits, beliefs, attitudes, social support, and environmental factors that may have impacted use of the mats. This information would be useful to collect in future studies. Finally, because our sample included VA patients, individuals were primarily older White males, so results may not generalize to more diverse samples. Future research, particularly randomized trials testing different approaches to increase compliance (eg, gamification, incentives, patient and caregiver education, motivational interviewing, and reminders or alerts) in different patient populations, will be valuable to informing how to make technologies such as these more impactful.

Conclusion

Our study found that a large fraction of patients did not use the SmartMat as directed, and thus they would be unlikely to benefit from it. Testing approaches to proactively provide additional support for self-monitoring to patients with poor glucose control, current smoking, or high comorbidity burden (factors associated with high noncompliance) is an important area of future research. Future research should also seek to understand patients’ perspectives on their experience with SmartMats and why they may have routinely used, rarely used, or stopped using the mat. Reducing the risk of ulcer recurrence and amputation could have enormous benefits for individual patients and lower health care costs. Thus, ensuring that patients effectively employ tools to reduce the risk of ulcer recurrence is paramount.