Randomized Controlled Trials vs Real-World Evidence: Evaluating mHealth Outcomes

The debate around rct vs real world evidence mhealth outcomes is no longer academic. As mobile health (mHealth) technologies become more prevalent in global health programs, from contactless vitals screening in Uganda to maternal health monitoring in Nigeria, the methods for evaluating their impact are under increasing scrutiny. For decades, the Randomized Controlled Trial (RCT) has been the undisputed gold standard for clinical evidence. However, the unique challenges and opportunities of mHealth, particularly in low-resource settings, are forcing a reconsideration of this paradigm and elevating the role of Real-World Evidence (RWE).

"By 2027, the global mHealth market is projected to reach $361.6 billion, a more than four-fold increase from 2020. This growth necessitates more agile and realistic methods for evaluating what actually works in the field." - Global Market Insights, 2021.

The gold standard vs. the real world: evaluating mhealth outcomes

The core of the discussion on rct vs real world evidence mhealth outcomes lies in a trade-off between control and context. RCTs are designed to isolate a single variable and measure its effect in a tightly controlled environment. This is achieved by randomly assigning participants to an intervention group or a control group. The rigorous design of RCTs minimizes bias and provides strong evidence of causality, which is why they are traditionally favored by regulatory bodies and academic journals.

However, the very strengths of RCTs can become weaknesses when applied to the dynamic world of mHealth. An RCT of a new drug can be designed and executed over several years, but a mobile app might be updated multiple times within a single year. The cost and complexity of conducting large-scale RCTs for every new feature or deployment of an mHealth tool are often prohibitive. This is particularly true in global health, where field conditions are variable and unpredictable.

This is where Real-World Evidence (RWE) comes in. RWE is clinical evidence regarding the usage and potential benefits or risks of a medical product or intervention derived from analysis of Real-World Data (RWD). RWD is data collected from a variety of sources outside of traditional clinical trials, such as:

• Electronic health records (EHRs)
• Medical claims and billing data
• Data from mobile devices and wearables
• Patient-reported outcomes

RWE allows researchers and public health officials to see how mHealth interventions perform in the real world, among diverse populations, and in conjunction with other health services. It provides a more realistic picture of a technology's effectiveness, adoption, and usability.

Feature	Randomized Controlled Trial (RCT)	Real-World Evidence (RWE)
Environment	Controlled, experimental	Real-world, observational
Primary Goal	Establish efficacy and causality	Assess effectiveness and generalizability
Data Collection	Prospective, structured	Retrospective or prospective, varied sources
Bias	Low due to randomization and blinding	Higher risk of bias and confounding
Cost	High	Generally lower
Speed	Slow, long-term	Fast, can be near real-time
Generalizability	Limited to the study population	High, reflects diverse populations

Industry Applications

The application of RWE in mHealth is not just theoretical. The ZOE COVID Study in the UK, a collaboration between King's College London and ZOE Global Ltd., is a prime example of RWE in action. Millions of users across the UK used a mobile app to self-report their symptoms, vaccination status, and COVID-19 test results. This data provided public health officials with near real-time insights into the pandemic's trajectory, the effectiveness of vaccines, and the prevalence of different symptoms.

Community health programs

In the context of global health, RWE is being used to evaluate the impact of mHealth tools deployed by community health workers (CHWs). By analyzing data collected by CHWs on their mobile devices, organizations can:

• Track the prevalence of specific health conditions in a community.
• Identify geographic hotspots for disease outbreaks.
• Assess the effectiveness of health education campaigns.
• Monitor and improve referral pathways from the community to the clinic.

Supporting regulatory decisions

Regulatory bodies, such as the U.S. Food and Drug Administration (FDA) and the European Medicines Agency (EMA), are increasingly recognizing the value of RWE. While RCTs are still required for the approval of new drugs and medical devices, RWE is being used to:

• Monitor the post-market safety of new products.
• Support label expansions for existing products.
• Provide evidence for the effectiveness of digital health technologies.

Current research and evidence

The shift towards accepting RWE is supported by a growing body of research. A 2020 study by researchers at Harvard Medical School and the FDA highlighted the potential for RWE to complement the evidence generated by RCTs, particularly in areas where RCTs are not feasible (Franklin et al., 2020). They also noted the importance of developing rigorous methods for analyzing RWD to minimize the risk of bias.

Despite the growing acceptance of RWE, challenges remain. These include concerns about data quality, data privacy and security, and the lack of standardized methods for collecting and analyzing RWD. However, as mHealth technologies become more sophisticated and data analytics tools become more powerful, these challenges are being addressed.

The future of mhealth evaluation

The future of evaluating mHealth outcomes lies not in choosing between RCTs and RWE, but in using them together. A hybrid approach, where RCTs are used to establish the efficacy of an intervention in a controlled setting, and RWE is used to monitor its effectiveness and safety in the real world, offers the best of both worlds. This approach, sometimes called an "RCT-RWE continuum," allows for a more holistic and timely evaluation of mHealth interventions. For instance, micro-randomized trials are an innovative design where components of an mHealth intervention are randomized at various decision points to assess their causal effects over time.

Frequently asked questions

---

What is the main difference between an RCT and an RWE study?

The main difference is control. An RCT is a controlled experiment where participants are randomly assigned to an intervention or control group. An RWE study is an observational study that looks at how an intervention works in the real world, without controlling who receives it.

Are RCTs still necessary for mHealth?

Yes, RCTs are still the gold standard for establishing the efficacy of a new mHealth intervention. However, they are often not sufficient for understanding how the intervention will perform in the real world.

Can RWE replace RCTs?

It's unlikely that RWE will completely replace RCTs. Instead, they should be seen as complementary. RWE can provide valuable insights that can be used to design better RCTs and to monitor the long-term effectiveness of interventions.

The growing field of rct vs real world evidence mhealth outcomes is a critical area of focus for global health programs. As organizations like Circadify continue to develop and deploy innovative mHealth solutions, it is essential to have robust and realistic methods for evaluating their impact. The thoughtful integration of both RCTs and RWE will be key to ensuring that these technologies are Effective. Equitable and sustainable. To learn more about Circadify's research and collaborations in this space, visit our research blog at circadify.com/blog.