This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Diabetes, is properly cited. The complete bibliographic information, a link to the original publication on http://diabetes.jmir.org/, as well as this copyright and license information must be included.
There is a growing focus on the potential uses, benefits, and limitations of social media in the context of health care communication. In this study, we have sought to evaluate an initiative pioneered at a hospital in Denmark that uses Facebook to support and enhance patient-provider communication about diabetes.
This paper aims to evaluate the success of the trial according to its initial objectives and to assess its potential scalability.
The study was undertaken in a clinic for diabetes and hormonal diseases at a large regional hospital in Denmark. Using a realist evaluation approach, we identified 4 key components in the program theory of the initiative, which we formulated as context-mechanism-outcome configurations (eg, complex and iterative chains of causality). These configurations informed data gathering and analysis. Primary data sources were the activity and content in the Facebook group, in the form of posts, likes, and comments, and interviews with patients (n=26) and staff (n=6) at the clinic.
New developments in diabetes technology were the most popular posts in the forum, judged by number of likes and comments. Otherwise, information specific to the clinic received the most attention. All 4 components of the program theory were compromised to varying degrees, either as a result of failings in the anticipated mechanisms of change or contextual factors derived from the mode of implementation.
Social media serves well as a conduit for imagining positive change, but this can be a strength and weakness when attempting to enact change via concrete interventions, where stakeholder expectations may be unreasonably high or incompatible. Nonetheless, such initiatives may possess intangible benefits difficult to measure in terms of cost-effectiveness.
Diabetes mellitus is a complex and multifarious health condition that impacts millions of people globally, giving rise to both personal and societal costs on a large scale [
Social media platforms are oft-touted as one possible area of innovation that can be of benefit within health care [
The recent emergence and growth of the diabetes online community (DOC) presents opportunities and challenges to health care professionals and health care systems [
Traditional roles in health care communication are thrown into flux by the advent of social media [
There is a growing focus on the potential uses, benefits, and limitations of social media in the context of health care communication [
Aside from the ethical and legal concerns associated with social media–facilitated health care communication [
In this study we have sought to evaluate an initiative, pioneered at a hospital in Denmark, to use Facebook to communicate directly from HCP to people with diabetes. At the time our evaluation was undertaken, the Facebook group being used to facilitate this initiative had been active for approximately 18 months. At the outset, the initiative was not designed as an intervention, the impact of which might be directly or indirectly measured. Nonetheless, after seeing membership of the Facebook group grow substantially from its inception and in view of the effort required to maintain the group, the owner of the initiative (the head physician) considered that it was timely to determine whether the group was achieving the objectives for which it was developed. In view of the difficulties noted above concerning evaluation of such initiatives, it was agreed by the partners involved in this work that the optimal approach would be to undertake a theory-driven evaluation. Theory-driven evaluation represents an ideal approach to the appraisal of complex real-world interventions [
In this study, we chose to apply a particular form of theory-driven evaluation known as realist evaluation (RE) [
This study aims to evaluate the success of a concrete intervention using Facebook as a means to support and enhance ambulatory care among people with type 1 and type 2 diabetes. Additionally, we sought to identify more generic factors influencing the use and uptake of social media in the context of health care. Finally, we sought to apply and exemplify the use of realist evaluation as a methodology for apprehending complex outcomes within a complex, real-world intervention.
The study was undertaken in a clinic for diabetes and hormonal diseases, which is part of a large regional hospital in provincial Denmark. The outpatient clinic caters to people with both type 1 and type 2 diabetes, with a capacity for approximately 2500 consultations per annum for people with diabetes. The clinic employs 3 chief consultants, 2 residents, 5 diabetes nurses, and 5 dieticians.
The Facebook group, Diabetes Viborg (DIAVIB), was established by a consultant endocrinologist in the clinic in January 2017. DIAVIB was not established with an explicit set of aims and objectives, but the initiative was motivated by the interests and concerns of this consultant endocrinologist regarding the use of social media by people with diabetes. It was set up as a closed group, requiring registration by potential members, and it targeted people with diabetes, their family members, and anyone with an interest in diabetes. The Facebook group focused primarily on users of the clinic but also stated that it was open to anyone with an interest in diabetes. At the time of our evaluation, there were approximately 500 registered members, of whom approximately two-thirds were women (at the time of writing, this figure is now 630, with the sex distribution unchanged). In terms of age distribution, the lowest proportion of users was seen in the age range of 18 to 24 years, with the next lowest in the 65+ years age range. The majority of users lived in the catchment area of the clinic. Communication on DIAVIB was almost exclusively conducted in Danish, although some links were provided to external content that was only available in English.
The study draws upon 2 primary data sources: the activity and content on DIAVIB, in the form of posts, likes, and comments observed over the period from June 1, 2017, to August 22, 2018, and interviews with patients and staff at the clinic. Interview participants were sought via posts on DIAVIB, a leaflet posted in the clinic, and by a nurse in the clinic, who phoned people visiting the clinic on the days on which interviews were planned. We sought to recruit a representative sample of the clinic’s overall population, seeking variation according to age, gender, social class, and both users and nonusers of DIAVIB. Patient characteristics can be seen in
Two potential participants declined the invitation to participate when asked directly, primarily due to a general dislike of social media and practical issues with available interview times. In addition to the consultant who founded DIAVIB, other HCPs in the clinic were also interviewed, namely 2 nurses, 2 consultants, and 2 dieticians.
Study participant characteristics (N=26).
Characteristic | Value | ||
Age (years), median (range) | 48.1 (19-77) | ||
Diabetes duration (years), median (range) | 13 (3-57) | ||
|
|
||
|
Type 1 | 15 (58) | |
|
Type 2 | 11 (42) | |
|
|
||
|
Male | 12 (44) | |
|
Female | 15 (56) | |
BMI (kg/m2), median (range) | 28.7 (21-42) | ||
|
|
||
|
Type 1 | 67.6 (46-89) | |
|
Type 2 | 57.2 (40-71) | |
Existing DIAVIBb member (yes), n (%) | 11 (42) | ||
|
|
||
|
In employment | 15 (58) | |
|
Unemployed | 1 (4) | |
|
Pensioned | 3 (12) | |
|
Disability pensioned | 5 (19) | |
|
Student | 2 (8) |
aHbA1c: glycosylated hemoglobin (used to measure average blood glucose levels over time).
bDIAVIB: Diabetes Viborg.
In cases where there is no clear set of theoretical principles explicitly coupled to an intervention, the first task for evaluators working with theory-driven approaches is to articulate a program theory. This is undertaken in collaboration with those who have developed the intervention, in this case the consultant at the clinic. With numerous informal discussions and a 2-hour semistructured interview, we initially identified 4 distinct objectives, which we were then able to investigate and assess.
In addition to the interview data, we also gathered and analyzed data from DIAVIB itself. These data were analyzed in terms of their general characteristics (eg, a comment, a question, a like, etc) and in terms of their content. BC undertook the first analysis and thematization of the content, and this was subsequently discussed and consensually verified within the author group. It was relatively straightforward to achieve high levels of consensus within the author group because the content being analyzed was, for the most part, very concrete and prosaic in what it was addressing.
Theory-based evaluations that draw upon the realist evaluation approach take it as axiomatic that context is a key mediator between desired objectives and actual outcomes. Context contains numerous dimensions and is not easily demarcated, but in the case of our evaluation, its impact is seen in at least 3 levels: social, organizational, and individual. A further crucial dimension of RE is the mechanism, or what might be deemed the underlying causality that explains why certain actions lead to particular outcomes. The overarching model for RE is context-mechanism-outcome (CMO) configurations, that is, the causal but often convoluted relationship between conditions and outcomes. In undertaking RE , therefore, we have sought to identify and gauge which contextual factors have influenced the outcomes, whether these contexts were anticipated in the design of the intervention, and to what extent mechanisms of change imagined at the outset were confirmed in the outcomes.
Ethical approval for the study was obtained from Region MidtJylland’s research board (May 18, 2018). All data extracted from the DIAVIB group were anonymized before being put to use. Likewise, all interview participants were required to sign an informed consent form, guaranteeing their anonymity but allowing researchers unhindered access to the interview transcripts.
In the period observed, the administrator of the site initiated 109 unique communication threads across a wide range of subjects related to diabetes. In 30 of these threads, the message was accompanied by a link to some external source of information. In 14 cases, the administrator initiated a thread to conduct a poll among the members of DIAVIB. The 109 threads received a total of 780 likes from members of the group, and there were 232 follow-up comments. Many members of the group commented on multiple occasions and the 232 comments were authored by 76 individual members of the group.
The topics attracting the most likes and comments were related to both general diabetes information and information pertaining specifically to the clinic. New developments in diabetes technology were by far the most popular posts in relation to general diabetes information, judged by number of likes and comments. For example, a post about the implantable glucose sensor Eversense XL (Senseonics Holdings) received 54 likes and was commented on 25 times. Of the information specifically pertaining to the clinic, it was personal information about staff members joining or leaving the team that was the most popular on the metric of likes and comments. For example, a thread about a nurse who was leaving the clinic to take retirement received 44 likes and 18 comments.
At the outset of the project, we identified 5 objectives that represented the underlying program theory of DIAVIB. We have subsequently discarded one of these objectives, relating to peer support, on the basis that the setup of DIAVIB was not actually suited to facilitate peer support and the data we acquired from participants reflected this fact. As such, it was deemed to be something that could not be reasonably evaluated. From the 4 remaining objectives, we posited 4 different CMO configurations.
A nonexhaustive summary of this process is exemplified in table form, as seen in
Context-mechanism-outcome configuration 1: Diabetes Viborg as source of knowledge about diabetes.
CMOa | Objective | Context | Mechanism | Outcome |
CMO 1: |
DIAVIBb should provide people with a reliable source of knowledge about diabetes. | Individual: People with diabetes and their relatives. |
People feel overwhelmed by amount of available information about diabetes and have doubts about its veracity. |
DIAVIB is used as a primary information source about diabetes by its users. |
aCMO: context-mechanism-outcome.
bDIAVIB: Diabetes Viborg.
cHCPs: health care professionals.
In our interview data, participants did express concerns relating to the volume of information about diabetes, both in general and on the internet, and the challenge of determining its veracity:
I’ve been on the internet and looked at different things, but I think people say a lot of different things there. Some say something, and others say something else. That can make things all a bit more confusing.
The extent to which this was viewed as a problem varied, but there was a clear distinction in the degree to which people with diabetes viewed it as a problem and the degree to which health care professionals saw it as such. Rightly or wrongly, people with diabetes did not experience it as essentially problematic because they felt able, in one way or another, to find a way to normalize things for themselves:
Once you’ve had it for a while you get more and more information, so you just learn. I don’t think there is too much.
I’ve learnt to filter it out. I’ve grown up with diabetes, so I know what I need to relate to.
If there is anything you are in doubt about then you can always look it up. You can look up everything these days.
In contrast, every HCP interviewed expressed concern about people being exposed to inaccurate information and the consequences this might have.
However, while interview participants did not indicate a sense of being overwhelmed by information, there was recognition that information provided through DIAVIB did carry extra credibility compared with other more random sources. In fact, for some participants, their contact with the clinic was perceived to provide them with all the information that they needed about diabetes:
I get [information about diabetes] from here [the clinic]. It's not something I read about. If you start to read about it, you will immediately get 10 more symptoms and I don’t want that. I trust what they do here, and I do what they say and I’m fine with that.
The next CMO configuration we identified was DIAVIB as a forum for patient-provider interaction, exemplified in
Context-mechanism-outcome configuration 2: forum for patient-provider interaction.
CMOa | Objective | Context | Mechanism | Outcome |
CMO 2: |
DIAVIBb should be a forum in which people with diabetes and HCPsc can interact with one another. | Individual: |
People with diabetes are interested in communicating with their HCPs in online forums because they have needs and concerns that are not addressed in the conventional point of contact with the health care system. |
People with diabetes and HCPs use DIAVIB to interact with one another. |
aCMO: context-mechanism-outcome.
bDIAVIB: Diabetes Viborg.
cHCPs: health care professionals.
The possibility of two-way communication between HCPs and people with diabetes was, in principle, something that could be facilitated by DIAVIB. However, this possibility was limited by the fact that it was only the administrator of the group (the consultant) who could initiate posts. So, while it was possible for members to comment on posts, they were not able to determine the topics under discussion. For some, this represented a limitation that lessened the appeal of engaging with the group:
Yes, I think it would be a good thing. I know that there are other Facebook groups with people who share experiences, so ... for me it’s not likely I’d join the group if they only share information because I think that I can do this myself, also with respect to being critical of the sources. So, if there are no elements besides that in the Facebook group, then I don’t think it’s so interesting for me.
Others voiced a wish for more communication, expressing dissatisfaction with the way in which dialogue had been handled within DIAVIB:
I think it could be better in the way that, if there are questions in there, then they should make sure to answer them. They should be a bit more active. Sometimes there are long gaps before anything gets posted.
In general, however, there was uncertainty about opening up DIAVIB to more direct two-way communication, expressed as a concern about the type and quality of exchanges that would ensue:
I think it’s a professional tool. I think it’s important that the things that get written are based on professional knowledge. The things posted in here should come from doctors or nurses, so there isn’t any misunderstanding about what is and isn’t true.
It was also seen as open to question what kind of communication someone would want to have:
Is this really the right forum, if I’ve got a need to get in touch with my Doctor? Then it would be more about me and not something that I would want to share in an open group.
An underlying factor in the general ambivalence toward the use of DIAVIB as a forum for direct interaction with HCPs could also be inferred from the fact that participants did not express frustration with the degree of contact that they had with HCPs. The interview data presented an overwhelmingly positive impression of a clinic and clinic personnel that were attentive and accessible.
DIAVIB was conceived to inspire HCP engagement with health communication, and the CMO configuration derived from this objective is exemplified in
Context-mechanism-outcome configuration 3: Diabetes Viborg supports health care professional engagement with health communication.
CMOa | Objective | Context | Mechanism | Outcome |
CMO 3: |
DIAVIBc should foster an interest in innovative health care communication among HCPs | Individual: |
HCPs are challenged by the expansion of publicly available knowledge and by time limitations in their encounters with people with diabetes. | The opportunities for direct communication offered by DIAVIB will motivate HCPs to engage more in the dissemination of valid and relevant knowledge, which addresses the everyday needs of patients with diabetes. |
aCMO: Context-mechanism-outcome.
bHCP: health care professional.
cDIAVIB: Diabetes Viborg.
All the HCPs interviewed acknowledged that the advent of the internet had made some impact on their interactions with people with diabetes. This was viewed as something with both positive and negative consequences. It was, however, primarily the negative consequences that were emphasized by HCPs, who felt that inaccurate information could lead to false expectations and even dangerous actions among people with diabetes. While the notion that there is a need for innovative approaches to health communication is supported in these observations, not all HCPs agreed that posting on Facebook was viable. One concern expressed related to the complexity of the information being conveyed and the challenge of supplying information at a general level, thereby omitting the more personal judgements involved when conveying information to people with diabetes:
I mean, when you’re talking about diet, there’s all sorts of information that you can write about which the patient will see. And I just think, the things we write should be quite specific when you know that a lot of patients are going to read it and you don’t know how they’re going to react to it. That’s something we talk about a lot—what we should and shouldn’t say—where you need to make a judgement based on the individual and that’s just easier when you’re in an individual consultation with the patient.
More prosaically, reservations were voiced in relation to the time needed to maintain the group. Even though the clinic’s personnel were sympathetic to the initiative, it also evoked more negative emotions:
So, it’s a bit like there is a mild pressure to contribute, and that’s fair enough, but it’s like, argh, when is there going to be time for that, to actually sit down and provide something worthwhile… We could do more, but I don’t know when or how it should be.
Although there had been discussions within the clinic about making DIAVIB a collective responsibility, because it was not something that was integrated into the clinic’s everyday practice, it emerged as an exclusively individually driven initiative. DIAVIB was only officially supported to the extent that it existed nominally under the auspices of the regional hospital and clinic, but the time used to set up and maintain the site was not financially reimbursed.
Aside from the issue of time and reimbursement, personnel at the clinic also felt that Facebook imposed limitation in terms of what could be communicated. An important aspect of this related to privacy, both that of the HCP and users of the clinic:
My first thought was, that’s an innovative and visionary initiative. My second thought was, I don’t want to be personally part of that…And I mean, it’s clear that we can’t have personal information. If there’s anything that’s even remotely identifiable they have to use their digital post box, so it just doesn’t work on Facebook. So, I don’t really know what I could help them with, apart from really general information, like insulin can’t cope with 30°C heat.
The personnel interviewed in our evaluation were aware of the innovative and unrealized potential of Facebook to communicate, such as the consultant who imagined it might serve to capture the hardly reached:
I thought it might be a good way to reach those people who don’t come [to the clinic], but who are always on their Facebook pages, you know.
Despite this, interviews with the clinic’s personnel ultimately left an overriding sense of DIAVIB falling short in activating the potential to reach hardly reached patients, a feeling captured by the same consultant reflecting on his own idea:
So that could be a way to get some more people in. But, it’s not really integrated into my way of working. It’s not like I sit here and say, you should do this and this and you can see it on our Facebook group. I’m not there yet… I don’t know, maybe it’s because I don’t really use Facebook much myself.
The final CMO configuration we identified anticipated that DIAVIB would help patients to achieve improved empowerment and outcomes, as exemplified in
Context-mechanism-outcome configuration 4: Diabetes Viborg improves empowerment and clinical outcomes.
CMOa | Objective | Context | Mechanism | Outcome |
CMO 4: |
DIAVIBb should enable people with diabetes to achieve an improved illness understanding. | Individual: |
People’s diabetes management practices are related to their level of knowledge about diabetes. Enhanced knowledge will enable improved diabetes management. | People with diabetes in the clinic will become better at diabetes management and thereby improve their diabetes-related outcomes (eg, HbA1cc) |
aCMO: context-mechanism-outcomes.
bDIAVIB: Diabetes Viborg.
cHbA1c: glycosylated hemoglobin, type A1c.
This CMO configuration was ultimately the most abstract to evaluate, since there were no means by which to measure participants’ level of engagement with DIAVIB and equate it to changes in clinical outcomes. It was, however, possible to investigate the premise for the proposed mechanism of change and find support for the notion in principle:
You should try to know more about your illness. Knowing more about it, you’re better able to control it.
At the same time, we also identified individual strategies relating to diabetes knowledge that pushed in the opposite direction. Information overload in relation to diabetes does not only come from what one can hear and read about it. Dealing with diabetes on a day-to-day basis can also be experienced as a type of information overload. In view of the potentially endless information that is available, it is also important that people can delimit what they do and do not need to know:
Our evaluation of DIAVIB indicates that while acceptance of the initiative was apparent in the numbers who joined the group and the overall positive attitude expressed during the interviews, levels of direct engagement were much lower. This possibly reflects more fundamental challenges in health care communication, where there is generally a lack of clear guidelines for how best to generate content and strategies for communication and engagement with people living with chronic health conditions such as type 1 and type 2 diabetes [
Although many HCPs express concerns about the veracity of online information in general, there is no overwhelming evidence that clinically inaccurate information is flooding online diabetes forums [
The nature of the communication on DIAVIB was also influenced by the setup of the group (ie, a closed Facebook group associated with a physical diabetes outpatient clinic in a regional hospital, in which only the administrator can initiate topics for discussion and which is primarily being maintained by one individual, for the most part as a hobby rather than something being officially recognized and rewarded). These architectural affordances of the group inevitably impact the type and level of interaction and the respective roles of people with diabetes and HCPs [
Online interaction between patients and providers has previously been shown to be problematic, with a discrepancy between the concerns being voiced by patients and the nature of replies being provided by HCPs, particularly in regard to the use of inclusive and supportive language [
It remains unclear whether participation in online support groups serves to establish collective empowerment or whether the collective identity fostered in such groups only serves to generate individual empowerment [
At the current time, there remain concerns about whether the advance of social media and its increasing pervasiveness in all aspects of life may also engender a situation in which certain groups of people are actually disempowered. This applies, for example, in the case of engaging older people with diabetes via social media, where more support is often needed to overcome the barriers they experience [
DIAVIB was very clearly a complex intervention involving various stakeholders located in diffuse contexts and, as such, it was suited to a theory-driven approach to its evaluation. For reasons highlighted above, we adopted a realist evaluation framework to structure our investigation. RE’s approach is not always easy to follow in relation to mechanisms of change, and it assumes a rationality regarding these mechanisms that is not necessarily in place [
DIAVIB was an initiative that was inspired by motives rooted in genuine and contemporary concerns about supporting people with diabetes in the best possible way. It sought to exploit the potential for new modes of patient-provider interaction seemingly allowed by social media and, at the same time, aimed to provide support for people with diabetes in a world in which flows of information are not necessarily anchored in conventional understandings of knowledge and truth. However, from the perspective of the objectives it was anticipated to address, the success of the initiative is limited. Part of this rests in the expectations, which were highly ambitious. Social media serves well as a conduit for imagining positive change, but this can be a strength and weakness when attempting to enact change via concrete interventions. This is especially true of initiatives like DIAVIB, which are developed organically rather than systematically.
Having stressed the limited extent to which DIAVIB represents a successful intervention when seen through the lens of a theory-driven evaluation, it should finally be noted that such an evaluation does not necessarily capture more intangible benefits. Whatever its limitations, the fact that more than 600 individuals have actively sought membership in DIAVIB suggests that it has tapped into a seam of interest in the possibilities allowed by social media in the context of health care that, as of yet, are not fully realized. Although our study suggests that there remain numerous and serious obstacles on the path towards the realization of such potential, pioneering initiatives such as DIAVIB and the lessons that can be drawn from them represent important milestones along this seemingly inexorable route.
context-mechanism-outcome
Diabetes Viborg
diabetes online community
health care professional
realist evaluation
The authors would like to express their appreciation to Anna Kristina Moeller, who provided invaluable assistance in recruiting participants to the study. Appreciation is also due to the staff at Viborg Regional Hospital for their willingness to contribute to the project. Finally, and most importantly, the project could not have been undertaken without the people living with diabetes who volunteered to contribute their time in order to provide us with invaluable insights, and for this we are extremely grateful.
None declared.