Shirley Wang: What does replicable ‘real world’ evidence from ‘real world’ data look like?

Regulatory and Health Technology Assessment (HTA) organizations are increasingly looking toward use of ‘real world’ evidence (RWE) from ‘real world’ data such as administrative claims and electronic health record databases to support decision-making. Ideally, decision-makers want to focus on RWE that uses valid methodology, has transparent reporting, and generates reproducible evidence.

However, published database studies frequently do not report on key implementation parameters, making it difficult to detect flaws in design or analysis. To increase confidence in use of RWE, decision-makers need the means to effectively and efficiently distinguish between studies of high versus low validity. The REPEAT Initiative has projects focused on improving transparency, reproducibility and validity of database research. These projects include large scale replication of 150 published database studies, evaluation of robustness of results to alternative study parameters, and development of a structured reporting template with design visualization to increase transparency of reporting and minimize misinterpretation.

Download Presentation (.ppt)


Shirley Wang

Assistant Professor, Harvard Medical School

Shirley Wang is an Assistant Professor of Medicine at Harvard Medical School and Associate Epidemiologist in the Division of Pharmacoepidemiology and Pharmacoeconomics at Brigham and Women’s Hospital. She is a pharmacoepidemiologist focused on developing innovative, non-traditional analytic methods to understand the safety and effectiveness of medication use in clinical care as well as facilitating appropriate use of complex methods for analyzing large observational healthcare data. To that end, she has developed enhancements to epidemiologic study designs and analytic methods as well as led efforts to guide appropriate use of complex methods for analyzing large observational healthcare data.

Shirley has been involved with the US Food and Drug Administration’s Sentinel Initiative since 2011 and her methods work has been recognized with awards from two international research societies. She recently co-led a joint task force for the International Society for Pharmacoepidemiology (ISPE) and the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) focused on improving the credibility of real world evidence for decision-makers and launched the REPEAT Initiative, a non-profit program with projects designed to improve transparency, reproducibility and ability to assess the validity of healthcare database studies. Shirley is also a writing group member for a National Academy of Medicine white paper on executing and operationalizing open science.