The power—and promise—of real-world evidence in medtech

Heart pumps. Catheters. Scalpels and syringes. Medical technologies like these are essential for helping save lives. But they also have the potential to generate information that can help change the future of healthcare.

Indeed, enormous amounts of healthcare information can be gathered from how medical technologies perform in the real world. This information, which is called real-world data (RWD), can be collected from sources such as electronic health records (EHR), insurance claims and laboratory data.

When RWD is analyzed, real-world evidence (RWE) is produced. That RWE has enormous potential for not just medical technology manufacturing and performance—think: identifying potential safety issues more quickly, accelerating the regulatory approval process, assessing effectiveness among diverse groups of patients and surgeons and more—but also for healthcare providers, who can harness RWE to inform treatment decisions and tailor therapies and procedures to specific patient needs.

Fueling that enormous potential? As Jijo James, M.D., Chief Medical Officer, MedTech & Interventional Oncology, Johnson & Johnson, puts it: “A whole gamut of available data out there.” To be sure, “there exists a collective universe of data that includes everything from clinical data that may be available in EHRs to administrative data that is available in claims records or hospital billing databases, and more,” he says.

But the key to advancing medical technology by leveraging RWE is making sense of the swaths of RWD that exist—in other words, ensuring that it’s high quality, reliable and relevant. Which can be a challenge because of limitations like data being unstructured, missing or incomplete.

Despite the challenges, the promise of RWE is undeniable, as is the momentum behind its uptake in the medtech industry. To wit: In 2023 the U.S. Food and Drug Administration (FDA) updated draft guidance on how it evaluates RWD to generate RWE—proof that health authorities are increasingly recognizing the benefits of RWE in informing regulatory decision-making for medical devices.

Johnson & Johnson sees the massive potential for RWE to advance patient health, too. And the company is working to synthesize AI, large sets of data, analytics and other technological advances to help make this potential a reality—a reality that will help make surgical procedures safer and more efficient for clinicians, in turn advancing heath for patients around the world.

It all comes down to data

Expanding the benefits that RWE can provide requires enormous quantities of data. And Johnson & Johnson is a leader in the use of large databases, according to Paul Coplan, ScD, MBA, Vice President and Head of MedTech Epidemiology and Real-World Data Sciences at Johnson & Johnson, and Vice Chair of the Executive Committee of the National Evaluation System for health Technology (NEST).

In part, that leadership arises from synergies in areas like analytical techniques between the company’s Innovative Medicine and MedTech businesses. But mining in-house information is not enough to gather sufficient data to produce fruitful RWE. That’s why Johnson & Johnson invests in outside databases, such as Premier Healthcare Databases (PHD), Optum and Merative MarketScan, “which we then can leverage for medtech applications, like generating evidence for market access, for safety assessment or for helping inform clinical trials,” Coplan says.

And yet, in general, it can be difficult to collect enough of the right data.

As Dr. James puts it, “You’ve got to be careful about what data you’re selecting. Data is ubiquitous today, but so too is the abuse of data. That’s one of the challenges around RWD. You want data that's free of bias, be it ethical bias, gender-based bias or even commercial bias.” Complete and quality data is also essential to capture—but doing so can be a challenge.

For instance, as part of an evaluation of NEST, an initiative created in 2016 by the FDA to advance the use of RWE in assessing the safety and effectiveness of medical devices, Justin Timbie, Ph.D., a Senior Health Policy Researcher at RAND, and his colleagues analyzed 18 of NEST’s test cases and interviewed nearly 50 researchers to assess the potential of using RWD to enhance decision-making by regulators.

Beyond benefiting patients, RWE in this area can also help clinicians. For example, surgeons can use RWE to evaluate a particular medical device—essentially knowing what works and what doesn’t long before starting a surgery. Plus, RWE reveals how a technology performs in different clinical settings with different kinds of patients. 

Timbie and his team found numerous challenges working with RWD, including “the lack of unique device identifiers, capturing key data elements and their appropriate meaning in structured data, limited reliability of diagnosis and procedure codes in structured data, extracting information from unstructured EHR data, limited capture of long-term study end points, missing data and data sharing.” Timbie and his colleagues also noted a large amount of incomplete data; for example, data on a specific medical device might exist, but the device wasn’t named or identified in the data.

What’s the solution? In that kind of situation, Johnson & Johnson might apply AI to fill in the blanks, Dr. James explains. “We’re using natural language processing and other techniques to correlate the data with specific devices,” Dr. James says. “The data can be really, really messy, and there are multiple fault lines along this, so it’s very important to have that depth of expertise to understand what the limitations might be, and how to address them.”

Once enough quality data has been collected, then RWE can be generated for varied purposes to advance medical technology and its applications.

Label expansions

One way RWE can be harnessed is to provide relevant and reliable data for label expansions, which can include FDA approval of additional applications of a medical technology and how to use it.

As Coplan explains, RWE can be very helpful for label expansions, especially when evidence from clinical trials is not available.

That’s the case, in part, because when it comes to conducting clinical trials for medical technologies used in, say, surgery, there are a lot of roadblocks. For example, it’s hard to enroll large numbers of surgery patients into clinical trials, which can lead to smaller sample sizes. In addition, traditional clinical trial populations may not be generalizable to the general population of patients in which the device gets used. That’s why RWE can play an important role in providing evidence of medical device safety and effectiveness for both regulators and physicians.

Another reason: Many medical technology approvals are iterative improvements on existing technologies. Take scalpels. “People have been using scalpels for surgery for the last 5,000 years. So if you come up with a new version of a scalpel, you don't necessarily have to do a randomized controlled trial, because the safety and performance of scalpels are well-characterized and one can test a new scalpel in comparison to prior scalpel versions in preclinical testing,” says Coplan.

When the FDA created NEST, the intent was to collaborate with academic, healthcare provider, informatics and industry experts across the medtech community—including Johnson & Johnson—to look at the feasibility of RWE to address the industry's research questions. In part, that requires expanding the use of “some of the data that’s collected during routine clinical practice, like EHRs and claims,” Coplan says. “Learning how to use that data for medtech evidence—that is, understanding how to unlock that potential of the use of these large databases—opens up access to a very large volume of data from use in usual clinical practice.”

To figure out how to do that, as part of NEST the FDA called for proposals from the medtech industry about using RWE for label expansions and safety assessments. One of these 21 funded test cases went to Johnson & Johnson—and it led to Johnson & Johnson in partnership with Mayo Clinic, Mercy Health and Yale becoming the first medtech company to receive an FDA-approved label expansion for a medical device (or a drug) based solely on a comparative RWE study using EHR databases.

For this test-case study, Coplan and Shumin Zhang, M.D., ScD, Senior Director, Regulatory RWE & Epidemiology at Johnson & Johnson, worked with collaborators at Mercy Health, a health system that operates in four states, Mayo Clinic, a health system that operates in five states, and the Yale New Haven Health System.  As a team they used RWE as evidence for a label expansion of a device for patients with persistent atrial fibrillation.

In a peer-reviewed publication about this study, the late Guoqian Jiang, M.D., Ph.D., of the Department of Artificial Intelligence and Informatics at the Mayo Clinic, and his coauthors, including Coplan and Zhang, concluded that data from EHRs, analyzed with an appropriate data model, “can be used for medical device studies across institutions to support regulatory decision-making, evaluating the data quality of the distributed system in the process.”

Learnings from this test case were then applied to other studies; they informed a second comparative RWE study which led to a zero-fluoroscopy workflow label expansion for 11 medical devices made by Johnson & Johnson, allowing for the use of ultrasound-based imaging and navigation systems for guiding procedures as an alternative to X-rays. 

Benefit-risk assessment

RWE can be applied to medtech in other ways, such as benefit-risk assessment, which involves weighing the positive effects of a medical device against its potential risks. This is an integral part of the regulatory approval process, as agencies rely on these analyses to determine whether devices are safe to use in the market.

“This is an emerging field,” says Coplan. He explains that with pharmaceuticals, a Phase 3 trial provides information on the benefits and the risks of a treatment, but medical devices don’t have the same requirements for approval. “So, if you really want to do a data-driven benefit-risk assessment of a medical device,” he explains, “you often have to use a lot of RWE to identify the rates of those benefits and the risks.”

Another important aspect of benefit-risk evaluation is the patient perspective in weighing whether the benefits of a procedure outweigh the risks. Patient input is an essential component of the safety-evaluation processes, as it helps medtech developers understand patient priorities and evaluate the potential benefits and risks of a device through pateints’ perspectives.

As such, RWE can also be used to evaluate how patients feel about a device. Coplan describes this patient-preference information as “the patient’s perspective on the level of benefits that is worthwhile relative to the rate of the risks.”  The FDA recently updated their guidance document on using patient preference information throughout the product lifecycle.

Beyond benefiting patients, RWE in this area can also help clinicians. For example, surgeons can use RWE to evaluate a particular medical device, essentially knowing what works and what doesn’t long before starting a surgery. Plus, RWE reveals how a technology performs in different clinical settings with different kinds of patients. For instance, RWE can be used to better understand a procedure, such as general surgery or a joint replacement. Further, a technical modification might be evaluated using RWE to understand what its effects are—for example, does it reduce bleeding or even reduce the length of a hospital stay?

What’s next for RWE?

In the near future, there’s great potential to explore more generations of RWD—and thus more applications of RWE. This could entail using data responsibly to help in the product development process to better serve patients. Wearable devices could be another opportunity.  Dr. James envisions using these types of technologies to capture everyday data related to medtech, such as a person’s mobility after a joint replacement. “Then you will be able to increasingly drive innovation that has quantifiable benefit for patients,” he says.

Additionally, the uptake of RWE as suitable for approval by regulatory bodies has far-reaching effects. For example, continued and updated regulatory guidance on RWE builds trust in the safety and effectiveness of medical devices by promoting data transparency, while supporting the adoption of cost-effective medical technologies that deliver healthcare more efficiently.  

And with increased applications of RWE, so too will there be a requirement for increasingly powerful analyses. Just one example: “There will be a continuation of using AI on digital and patient-reported data to start to train models to see what’s working,” Coplan says. Not to mention capabilities to rapidly analyze ever-growing sets of data.

The digitalization of surgery plays a role in data generation, too. “That’s because now, so much information is being measured during surgery,” says Coplan. “How much force is being used, how much power, what kind of angle was used, and so on. With all of this data, we can now start to build much more sophisticated predictive models using AI or standard statistical analytics to start to find out what types of procedures or approaches work, and do they differ by type of patient.”

Ultimately, this confluence of large sets of data and technological advances will serve to drive forward human health.

As Dr. James puts it: “At the end of the day, the belief is that if we take a science- and data-driven approach and we use an ethics- and values-based framework around that, patients will benefit. And that’s the ultimate goal.”