This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Diabetes, is properly cited. The complete bibliographic information, a link to the original publication on http://diabetes.jmir.org/, as well as this copyright and license information must be included.
Considering the increasing incidence and prevalence of diabetes worldwide and the high level of patient involvement it requires, diabetes self-management is a serious issue. The use of mobile health (mHealth) in diabetes self-management has increased, but so far research has not provided sufficient information about the uses and effectiveness of mHealth-based interventions. Alternative study designs and more rigorous methodologies are needed. Mixed-methods designs may be particularly useful because both diabetes self-management and mHealth studies require integrating theoretical and methodological approaches.
This scoping review aimed to examine the extent of the use of mixed-methods research in mHealth-based diabetes management studies. The methodological approaches used to conduct mixed-methods studies were analyzed, and implications for future research are provided.
Guided by Arksey and O’Malley’s framework, this scoping review implemented a comprehensive search strategy including reviewing electronic databases, key journal searches, Web-based research and knowledge centers, websites, and handsearching reference lists of the studies. The studies focusing on mHealth technologies and diabetes management were included in the review if they were primary research papers published in academic journals and reported using a combination of qualitative and quantitative methods. The key data extracted from the reviewed studies include purpose of mixing, design type, stage of integration, methods of legitimation, and data collection techniques.
The final sample (N=14) included studies focused on the feasibility and usability of mHealth diabetes apps (n=7), behavioral measures related to the mHealth apps (n=6), and challenges of intervention delivery in the mHealth context (n=1). Reviewed studies used advanced forms of mixed-methods designs where integration occurred at multiple points and data were collected using multiple techniques. However, the majority of studies did not identify a specific mixed-methods design or use accepted terminology; nor did they justify using this approach.
This review provided important insights into the use of mixed methods in studies focused on diabetes management via mHealth technologies. The prominent role of qualitative methods and tailored measures in diabetes self-management studies was confirmed, and the importance of using multiple techniques and approaches in this field was emphasized. This review suggests defining specific mixed-methods questions, using specific legitimation methods, and developing research designs that overcome sampling and other methodological problems in future studies.
Increases in diabetes incidence and prevalence are a major concern in today’s health care system. There are nearly 385 million diabetic patients in the world, and almost 90% of them have type 2 diabetes, which can be treated with appropriate lifestyle and nutrition changes. Diabetes is a complicated disease and requires a high level of patient involvement; 95% of its management is patient initiated. Considering the additional 235 million patients with type 2 diabetes expected worldwide by the year 2035, the development of diabetic patients’ self-management skills is critical [
Mobile health (mHealth) apps that offer personalized, fast, cost-effective, and engaging services to patients have been used increasingly in diabetes self-management [
Despite the increase in the diabetes cases and rapid growth in the mobile app market, research does not yet provide sufficient information about the factors fostering or hindering the adoption of these apps, patients’ attitudes toward using them, or their effectiveness in terms of health behavior change [
There is a growing interest in using mixed-methods research in mHealth-based diabetes management studies because both diabetes self-management and mHealth studies require using different approaches, techniques, and measures cooperatively within an integrated perspective [
Mixed-methods research grew out of the “paradigm” controversy between positivist (quantitative) and constructivist (qualitative) research traditions in the 1980s and 1990s [
The main goal of mixed-methods research is to use both quantitative and qualitative approaches to provide a better understanding of the research phenomena [
Mixed-methods research is generally used for (1) triangulation of different approaches and methodologies to explain a single phenomenon; (2) complementarity of the research methods where one method is used to elaborate, illustrate, enhance, or clarify the results from another method; (3) sequential development of a study in which the results of one method are used to inform the other; (4) initiation of a study by using one method to find contradictions and paradoxes in findings from another method; and (5) expansion of a research study by using different methods to gain new perspectives or insights about a research problem [
Selecting an appropriate research design is a crucial step in a mixed-methods research. The purpose of a study and the nature of the research question help shape the design used in a mixed-methods study. In addition, using either concurrent or sequential time orientation affects study design, sampling, and data collection.
In a concurrent time orientation, data collection is completed for quantitative and qualitative phases of the study at approximately the same time to answer the same question. Both datasets are processed during data analysis and interpretation stages.
In a sequential time orientation, data can be obtained in stages, so the data from the first stage are used to shape the selection of data in the second stage (exploratory or explanatory sequential design). On the basis of their study purposes, questions, designs, and resources, researchers might place equal emphasis on both methods or use one of them primarily [
Integration is the central issue of mixed-methods research because it enables researchers to examine both types of data intensively [
The 5-stage scoping review framework developed by Arksey and O’Malley [
The research questions addressed in this review are as follows: (1) what is the extent and nature of mixed-methods research in mHealth-based diabetes management studies and (2) what are the current methodological approaches for designing and conducting mixed methods in these studies?
As a scoping review, this study did not intend to evaluate the scientific rigor of the selected studies as seen in systematic reviews [
Because the key purpose of the study was scoping the area of research, a comprehensive search strategy using multiple sources was used to identify all the relevant studies. Content-specific electronic databases, key journals, Web-based research and knowledge centers, websites, and handsearches of reference lists from studies and reviews were included in the search. The search was run between May 15, 2016, and June 30, 2016, and included all articles published by July 1, 2016.
Sources used to search the literature in this scoping review.
Source type | Literature sources used |
Electronic databases | PubMed, CINAHLa, Web of Science, PsycINFO, Google Scholar |
Key journal search | Journal of Medical Internet Research, The Journal of mHealth, The Diabetes Educator, The Journal of Mixed Methods Research, Journal of Diabetes, Journal of Diabetes Research, Journal of Telemedicine and Telecare, Telemedicine and e-Health |
Research and knowledge networks | mHealth Evidence |
Reference lists search | Systematic reviews, meta-analyses, narrative reviews, and similar article searches of the Web-based publishers |
aCINAHL: Cumulative Index to Nursing and Allied Health Literature.
bNIDDK: National Institute of Diabetes and Digestive and Kidney Diseases.
Three sets of search term combinations were used in the search, and they were entered and combined using Boolean operators where applicable. Combination 1 was the most detailed and was used in each electronic database and on the mHealth Evidence website. It included “mobile health,” “mHealth,” “m-Health,” “mobile,” “mobile phone,” “smartphone,” “cellular phone,” “texting,” “text messaging,” “SMS,” “telemedicine,” “telehealth,” “telecare,” “telemonitoring,” “diabetes,” “diabetes mellitus,” “diabetic,” “mixed method,” “multi-method,” and “mix methodology.”
Because some studies do not explicitly indicate their methodologies as mixed methods, mix methodology, or multimethod, combination 2 was applied without using the methodology terms (mixed methods, multimethod, mix methodology). Because of the excessive number of records that resulted in the search with combination 2 in Google Scholar, PubMed, and Web of Science, this search was only completed in CINAHL (Cumulative Index to Nursing and Allied Health Literature) and PsycINFO databases.
Combination 3 used the terms most relevant to our research question. “Mobile health,” “diabetes,” and “mixed methods” search terms were used in US NIDDK (National Institute of Diabetes and Digestive and Kidney Diseases) Diabetes Research Center and in Google Scholar. Wide search options were used in terms of text availability, publication dates, and document formats, and only the English-language and academic journal filters were applied in electronic sources. ResearchGate was only used to search for the authors’ personal files and to obtain full-text articles.
Search combinations used in electronic sources.
Search terms | Web of Science | CINAHLa | PubMed | PsycINFO | Google Scholar | mHealth Evidence | US NIDDKb Diabetes Research Center |
Combination 1 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | |
Combination 2 | ✓ | ✓ | ✓ | ✓ | ✓ | ||
Combination 3 | ✓ | ✓ |
aCINAHL: Cumulative Index to Nursing and Allied Health Literature.
bNIDDK: National Institute of Diabetes and Digestive and Kidney Diseases.
Several inclusion and exclusion criteria were applied to the identified studies. The review included the primary research studies published in English-language academic journals that focused on using mHealth technologies in the context of type 1, type 2, or gestational diabetes management and included a combination of qualitative and quantitative methods.
In this review, the term “mobile health technologies” was used to cover mobile phones, portable monitoring devices (ie, accelerometer), and wireless devices used in medical care (ie, cell phones). Studies whose focus was Internet (computer)-based, telephone (landline)-based, or home-based monitoring were excluded. Examples of such studies include a mixed-methods diabetes telemonitoring study [
Study selection process.
The descriptive-analytical method [
Authors, year of publication
Main purpose of the study
Recognition of mixed methods (mixed-methods terminology used or not)
Purpose of mixing
Formal mixed-methods research question formulated (yes or no)
Prioritized method
Design type
Stage of integration
Legitimation methods used
Sources of qualitative data
Sources of quantitative data
Limitations described
Common design types used in mixed-methods research.
Designsa | Objectives | |
Concurrent | This design aims to compare and contrast the results of both quantitative and qualitative findings, or to validate or expand quantitative results with qualitative data. This design is also labeled as parallel or convergent design. The researchers use different methods complementarily to investigate the same topic. | |
Exploratory | If a study has one dataset built on the results from the other, it is classified as a sequential design. It is a 2-phase mixed-methods design that collects only one type of data at a time. This design aims to explore first by placing a qualitative phase before a quantitative phase, and the data from the first phase are used to develop the second phase. | |
Explanatory | This design aims to clarify or interpret unexpected or confusing results from the quantitative phase with a follow-up qualitative phase. It can also be used to form groups based on quantitative results and monitor the groups through follow-up qualitative research. | |
Embedded | This includes the combination of both quantitative and qualitative data, but one data type has a supportive, secondary role within the overall design. One type of data is embedded within a methodology adapted by the other data type. It can be either a 1- or a 2-phase study. Unlike the conventional mixed-methods researchers, who think both methods should answer the same question in the research, some researchers have different questions requiring different types of data. Some complex interventions and experimental studies need embedded designs because this design is more manageable in terms of time and resources. | |
Multiphase | This design includes multiple qualitative, quantitative, or mixed measurement phases conducted over time and linked together so that one phase builds on another with a common overall objective. They usually include convergent and sequential elements. |
aDeveloped based on the frameworks presented in Creswell [
Integration through methods.
Approacha | Description |
Connecting | One dataset links to the other through sampling. |
Building | One dataset informs the data collection approach of the other. |
Merging | The two datasets are brought together for analysis. |
Multiple integration | Data collection and analysis are linked at multiple points. |
aAdapted from Fetters et al [
Methods of legitimation.
Approacha | Description |
Inside-outside legitimation | Insider (emic) viewpoint refers to viewpoint of a group member while outsider (etic) viewpoint refers to an objective viewpoint gathered from an external source. Peer reviews, expert reviews, participant views, and team members’ views are used for legitimation. |
Paradigmatic or philosophical validity | Successful integration of philosophical and methodological beliefs, defining the paradigm assumptions explicitly, and conducting research accordingly are the main indicators of legitimation. |
Commensurability legitimation | This type of validity is obtained when researchers develop a third, mixed view, which helps them make broader and richer explanations about their study conclusions. |
Weakness minimization | This type of validity is related to the integration of the research; researchers must continuously work to have nonoverlapping weaknesses while planning and designing their study. |
Sequential legitimation | This type of validity is used to understand whether the sequential order of qualitative and quantitative phases in a study influences the results. |
Conversion legitimation | By quantifying the narrative descriptions and creating a narrative profile for quantitative results (qualitizing), researchers can interpret their data in a broader perspective. |
Sample integration | This type of validity refers to making appropriate generalizations from mixed samples. The relationship between the sampling designs in quantitative and qualitative phases is an important indicator of validity. |
Sociopolitical legitimation | In order to achieve this type of legitimation, mixed-methods researchers should advocate pluralism of perspectives and try to build a practical theory or result that research consumers will find valuable. |
Multiple legitimation | This type of validity indicates the extent to which all the pertinent validities (quantitative, qualitative, and mixed) are addressed and resolved successfully. |
aAdapted from Onwuegbuzie and Johnson [
The database search revealed 8 articles; with handsearches of reference lists and key journals, a total of 14 articles were identified. The final sample included studies focusing on feasibility and usability of mHealth diabetes technologies (n=7), behavioral measures related to mHealth apps (n=6), and challenges in patient recruitment, fidelity, and intervention delivery in the context of mHealth (n=1).
For this scoping review, the first author derived the data from the articles and completed the initial coding, which was verified by the second author. In addition, an independent researcher (an experienced researcher and doctoral student in social sciences) separately coded the articles based on the study design framework used in this review. Comparing the results revealed an interrater reliability level of more than 90%.
The characteristics of mixed-methods research used in the reviewed studies and the summary of the findings are presented in
The characteristics of mixed-methods research in mHealth-based diabetes management studies examined in this review (part 1).
Author, year | Main purpose of the study | Recognition of MMa | Purpose of mixing | Formal MM research question | Prioritized method | Stage of integration | Design type |
Allen et al, 2009 [ |
To assess feasibility and acceptability of continuous glucose monitoring and accelerometer technology in exercising type 2 diabetic patients | Yes, as multimethod | Complementarity | No | Equal | Sampling and interpretation | Explanatory sequential |
Baron et al, 2015 [ |
To identify the challenges related to recruitment, fidelity, implementation, and context of mobile telehealth interventions targeting diabetic patients | Yes | Complementarity: different measures for different parts of the research phenomenon | No | QUANb | Sampling, data collection, data analysis, and interpretation | Embedded |
Baron et al, 2016 [ |
To examine the behavioral effects of a mobile phone–based home telehealth intervention in diabetic patients | No | Triangulation | No | QUAN | Sampling and interpretation | Embedded |
Burner et al, 2013 [ |
To explore the attitudes of inner-city Latino patients toward TExT-MEDd program and other health information sources | Yes | Initiation: the qualitative study was conducted to understand the contradictory findings of the quantitative method | No | Equal | Sampling and interpretation | Explanatory sequential |
Carroll et al, 2007 [ |
To evaluate user satisfaction with an mHealth diabetes monitoring system | Yes | Sequential development | No | QUAN | Discussion or interpretation | Exploratory sequential |
Franklin et al, 2008 [ |
To explore the interactions of type 1 patients with SMS text messaging support | Yes | Triangulation | No | Equal | Sampling, data analysis, and interpretation | Concurrent |
Froisland et al, 2012 [ |
To explore ways mobile apps can be used to monitor adolescents with type 1 diabetes | Yes | Triangulation | No | QUALc | Sampling, data collection, and interpretation | Embedded |
Georgsson and Staggers, 2016 [ |
To test the feasibility of a multimethod approach for patients’ experienced usability of a diabetes mHealth system | Yes, as multimethod | Triangulation | No | Equal | Sampling, data analyses, and interpretation | Concurrent |
Grindrod et al, 2014 [ |
To examine the usability and usefulness of mobile medication apps with older adults | Yes | Triangulation | No | Equal | Sampling, interpretation, and data analysis | Concurrent |
Jones et al, 2015 [ |
To evaluate the attitudes of American Indian women toward postpartum intervention approaches (including mHealth) and risk factors for developing gestational diabetes | Yes | Complementarity: different measures for different parts of the research phenomenon | No | QUAL | Sampling and interpretation | Embedded |
Nundy et al, 2014 [ |
To investigate the behavioral effects of a theory-driven mobile phone–based intervention using an automated, interactive SMS text messaging system | Yes | Triangulation | No | QUAN | Sampling and interpretation | Embedded design |
Osborn and Mulvaney, 2013 [ |
To examine the capability of an SMS text messaging and interactive voice response intervention for low-income adults with type 2 diabetes mellitus | Yes | Sequential development | No | Equal | Sampling, data analysis, and interpretation | Embedded |
Verwey et al, 2016 [ |
To examine the reach, implementation, and satisfaction with a counseling tool combining an accelerometer, a mobile phone, and a Web application | Yes | Triangulation | No | Equal | Sampling, interpretation, and data analysis | Embedded |
van der Weegen et al, 2014 [ |
To test the usability of a monitoring and feedback tool targeting diabetic patients | Yes | Sequential development | No | Equal | Sampling, data collection, data analysis, and interpretation | Multiphase study |
aMM: mixed methods.
bQUAN: quantitative.
cQUAL: qualitative.
dTExT-MED: Trial to Examine Text Message–Based mHealth in Emergency Department Patients With Diabetes.
The characteristics of mixed-methods research in mHealth-based diabetes management studies examined in this review (part 2).
Author, year | Legitimation methods described | Sources of qualitative data | Sources of quantitative data | Limitations described |
Allen et al, 2009 [ |
Inside-outside legitimation | A 1-hour, structured, focus group interview (n=7) following the completion of the quantitative phase. Field notes were also taken on key discussion points and observations (eg, body language and group mood). | Descriptive measures of the sample (n=9). Wearable Continuous Glucose Monitoring System, activity monitor data, and activity counts reviewed for each participant. | Small sample size and lack of control group setting in quantitative phase were reported. |
Baron et al, 2015 [ |
Inside-outside legitimation | Interviews, meetings, field notes, and communications between team members. | A 9-month randomized controlled trial (n=81) to assess intervention delivery and fidelity (patients and nurses). | A possible sample selection bias was indicated. |
Baron et al, 2016 [ |
Inside-outside legitimation and conversion legitimation (quantizing the qualitative data) | Semistructured interviews (n=26) on perceived effects of mobile telehealth system on diabetes self-management. | A randomized controlled trial (n=81) with intervention and control groups was conducted. Self-report measures of self-efficacy, illness beliefs, and self-care were taken at baseline and 3- and 9-month points. | Sample size was indicated as insufficient to make generalizations. |
Burner et al, 2013 [ |
Inside-outside legitimation | Two focus groups of 90-minute duration, one in English, one in Spanish (n=8), were conducted with a structured guide. | A 1-month bilingual diabetes SMS text messaging intervention (n=23). Quantitative data included demographic, clinical, and biometric data of patients, and measures of health behaviors, knowledge, and beliefs were taken. | Small sample size was indicated as a limitation to the generalizability of the results. |
Carroll et al, 2007 [ |
Weakness minimization legitimation: large focus groups to support small-scale usability test | A series of focus groups (10; n=59) was conducted before testing a prototype cell phone with a glucose monitoring system. | A pilot usability test to evaluate satisfaction with the new system (n=10). A 15-item questionnaire is used. | Sample size and sample selection, intervention duration, and incentives to the participants were seen as barriers to generalizability of findings. |
Franklin et al, 2008 [ |
Inside-outside legitimation and conversion legitimation | Content analysis of text messages and messaging patterns of a 12-month Sweet Talk intervention period. | A 12-month 3-armed randomized controlled trial of a text messaging support system, Sweet Talk (n=64), was conducted. Observational data on messaging patterns were triangulated with patient clinical and demographic data. Post hoc analyses combining qualitative data and demographic variables were made. | Small sample size was indicated as a limitation to generalizability of the results. |
Froisland et al, 2012 [ |
Inside-outside legitimation | Semistructured in-depth interviews lasting between 45 and 90 minutes (n=12) were conducted at the end of the quantitative phase. | A pilot test of 2 mobile apps (n=12), after a 3-month trial. | Possible sampling bias, small sample size, and short intervention period were indicated. |
Georgsson and Staggers, 2016 [ |
Inside-outside legitimation, conversion legitimation | Think aloud protocol and open-ended interviews (15-20 minutes) were conducted (n=20). | First, a brief demographic questionnaire, and, at the end of the intervention, a posttest questionnaire measuring the usability of an interactive SMSa- text messaging system for a randomly selected sample of patients with diabetes (n=10) were conducted. | Using a convenient sample frame and the novelty of the system to patients were indicated as a limitation to the generalizability of the findings. |
Grindrod et al, 2014 [ |
Outside legitimation, sequential legitimation, sample integration legitimation | A 10-minute group discussion of what medication management meant before each usability evaluation and 30-minute focus group discussion after each session (n=35). | A 2-hour usability testing (n=35) of different mobile apps using a 10-item system usability scale and a visual analog scale was used. | Short intervention period was indicated as a limitation of the study. |
Jones et al, 2015 [ |
Inside legitimation | Four focus groups consisting of 2-5 participants (n=11 in total) were conducted, maximum duration of 60 minutes. Individual interviews (n=15) ranged from 25-45 minutes. | A cross-sectional study (n=26) was conducted with eligible group of patients. The questionnaire included measures for personal and family health history and technology feasibility and acceptability. | The purposive sampling and small sample size were seen as barriers to the generalizability of the results. |
Nundy et al, 2014 [ |
Inside-outside legitimation | Approximately 1-hour, semistructured, in-depth interviews (n=14) based on topic guides and open-ended questions after the intervention. | A longitudinal observational cohort study (n=74) was conducted and data were collected at baseline, 3 months (mid-intervention), and 6 months (end of intervention). Diabetes self-care, medication adherence, self-efficacy, health beliefs, and social support measures were used. | The limitations were described as the lack of control group measure and sole use of SMS text messaging intervention, which may create a causality problem. Sample size was small to make proper generalizations. |
Osborn and Mulvaney, 2013 [ |
Inside-outside legitimation | Motivational interviewing, face-to-face interviews (n=20) before and after trial, at baseline, and after week 3. | Secondary research: previous descriptive data on target population obtained, and self-administered daily text messages and interactive voice response calls are collected for analysis (n=20). | Sampling size was indicated as small to generalize results. |
Verwey et al, 2016 [ |
Inside and outside legitimation, conversion legitimation, sample integration | 30-Minute semistructured telephone interviews with the nurses about the receipt of intervention and the evaluation forms regarding consultations were used. | A longitudinal 3-armed cluster randomized controlled trial in a total of 24 family practice locations; evaluation questionnaire after intervention with practice nurses (n=20) and patients (n=131; 71 with type 2 diabetes and 42 with chronic obstructive pulmonary disease). | A possible sample bias was indicated as a limitation of the study. |
van der Weegen et al, 2014 [ |
Multiple legitimation: weakness minimization, conversion legitimation, inside-outside legitimation | Heuristic evaluation with 6 experts, thinking aloud procedure and video recordings of 5 patients at two different stages, a series of interviews with patients in the pilot test. | A usability test with 5 patients, a pilot test in real-life settings with 20 patients, and a poststudy system usability test were conducted. | The small sample size was indicated as a limitation to the generalizability of the findings. |
aSMS: short message service.
According to Creswell and Plano Clark [
Only a few studies (4/14, 29%; Verwey et al [
In other studies, information about the purpose of mixing methods had to be extracted from the methodology and discussion sections of the studies. Particularly for the embedded design studies, a decision rule was used based on the phases of a study because they could signify the purpose of mixing the methods in a study [
In total, the purpose of mixing methods was triangulation in 7 studies [
Because the studies did not use a formal mixed-methods terminology when describing their design types, stages of integration, and priority of the methods they used, information on these categories was extracted from the methodology sections of the studies.
The most common design used by the studies was embedded design (n=7). These studies (ie, [
There were 3 concurrent design studies in the review [
The only multiphase design study in this review was the study by van der Weegen et al [
All the studies enabled integration by connecting samples. The studies used either identical or nested sampling methods. On the basis of their designs and time orientation, they used samples sequentially or concurrently. Some studies (ie, [
Some studies also enabled integration through the merging of both datasets in the data analysis stage. In an explanatory sequential design study on diabetes self-management [
The studies also presented various forms of data integration at the stage of interpretation or discussion. They identified themes or concepts and integrated their qualitative and quantitative findings into these themes [
More than half the studies evaluated their quantitative and qualitative methods equally; neither method was prioritized (8/14, 57%). Some studies with randomized controlled trial data [
In another study [
The most frequently used qualitative method in the studies was in-depth interviews (n=9). The studies conducted semistructured interviews [
Surveys were the most common quantitative data collection method (n=10), such as pre- and posttest questionnaires used to measure the usability of an interactive SMS text messaging intervention developed for patients with diabetes [
Quantitative data were also generated from descriptive measures, such as glucose and exercise levels of patients after a 72-hour trial of a continuous glucose monitoring system [
Although the studies in this review explicitly mentioned their procedures and strategies to increase the objectivity and credibility of their research, they did not use formal mixed-methods terminology. Therefore, the information on legitimation methods was extrapolated from the methodology and discussion sections of the studies.
Almost all the studies in this review applied an “inside and outside” legitimation method (13/14, 93%). For example, Nundy et al [
In addition to inside-outside legitimation, different methods of legitimation were identified in the studies. For instance, the qualitative and quantitative methods were used to complement each other and to minimize weaknesses (weakness minimization method) in a small-scale usability test study [
Because the studies used either identical or nested sampling methods, sample integration legitimation was another frequently observed method of legitimation. Using similar samples in each phase minimizes lack of representativeness in qualitative findings and increases overall generalizability of the study [
Although various legitimation methods were used, they were also the source of study limitations. Many of the limitations were related to insufficient sample sizes and possible generalizability problems associated with them (ie, [
This scoping review identified comprehensive mixed-methods applications in the growing field of mHealth-based diabetes management. Mixed methods research can be effectively used if there is a strong and appropriate design dictated by research questions [
This study confirms the prominent role of qualitative methods in mixed-methods research. Qualitative measures were very important for the evolution of the majority of the papers reviewed, and they strongly influenced study findings. Because of the growing need for interpreting complex longitudinal studies and interventions [
The studies in this review presented various forms of integration at the method level. Some studies enabled integration at multiple points, linking their data collection and analysis at each point in the research. This is also noteworthy for mixed-methods studies in which integration is generally made at the level of data interpretation or discussion [
In terms of validity or legitimation strategies, little variation existed among the studies. In addition to inside-outside legitimation, these advanced designs should have also used weakness minimization and conversion legitimation methods because they are very specific to mixed-methods research.
The major limitations of the studies included in this review were related to data collection processes. Surveys were the most common data collection method used in the studies, but (as opposed to experimental studies) descriptive studies do not show causality, as they have poor control over external factors and external validity problems due to standardized question types. In addition, almost all studies acknowledged sampling problems (13/14, 93%). Small sample sizes, sampling bias, and use of convenient sampling are important barriers to the generalizability of findings in any type of study.
When various data collection techniques are used together, researchers are able to provide rich analysis and interpretations. This study shows that in addition to the common qualitative data collection techniques (focus group discussions and semistructured interviews), multiple data collection techniques (including think aloud protocols, interviews, field notes, content analysis, and evaluation forms) are used.
It is also noteworthy that these studies failed to discuss the advantages gained by using mixed-methods approaches. The majority of the studies (13/14, 93%) failed to provide a methodological discussion concerning the inconsistency or consistency between quantitative and qualitative data. Thus, this study supports Brown et al [
This study is an early exploration into the scope of mixed-methods studies in the field of mHealth-based diabetes management. This paper itself provides a demonstration of how to classify, analyze, and evaluate mixed-methods research. Regardless of field of interest, we believe many researchers could benefit from the guidelines, criteria, and examples provided in this review.
Although the database search was quite comprehensive, it was limited to the papers published in scientific journals and written in English. To increase the consistency of the coding, some established frameworks were used to identify design types [
This review provided important insights into the evaluation of mixed-methods studies focusing on diabetes management via mHealth technologies. The prominent role of qualitative methods in mixed-methods research and tailored measures in diabetes self-management studies was confirmed, and the importance of using multiple methods and techniques in this field was emphasized. Future studies could continue to use different qualitative approaches and try to integrate new tools and approaches that seem to be effective in health care settings, such as participatory video, photovoice, online communities, and chat and discussion groups [
Considering the opportunities provided by mixed methods, it is surprising to see such a small number of studies in this field. The lack of systematic approaches in establishing rigor in mixed-methods studies seems to be a major barrier to the adoption and recognition of this research by a large group of social researchers [
In the future, researchers should incorporate formal mixed-methods terminology, starting with defining a mixed-methods question separately at the beginning, aiming to answer this question, and discussing the value of using the methodology at the end. In this growing field of diabetes management, future reviews should systematically analyze rigor in mixed-methods studies and compare the results with those of single-method studies.
Cumulative Index to Nursing and Allied Health Literature
mobile health
National Institute of Diabetes and Digestive and Kidney Diseases
short message service
Trial to Examine Text Message–Based mHealth in Emergency Department Patients with Diabetes
We would like to thank our JMIR DIabetes reviewers for their valuable comments and contributions.
None declared.