Sentinel Q&A:
How does external validity come into play when planning and
conducting projects in the database? A typical Sentinel analysis
includes data from multiple large national and regional commercial
health plans, a state Medicaid plan, and Medicare fee-for-service plan.
Given the demographic and geographic diversity of the source data and
the focus on internal validity, external validity is typically not an
explicit consideration when planning or conducting Sentinel projects.
That said, attention is paid to the relevance of the study population
when finalizing the analysis plan and deciding on eligibility criteria
(e.g. high risk individuals, true new users of the drug). These
considerations also shape whether specific data partners are included in
the analyses. Finally, if outlier results are observed from some data
partners, the operations center may examine how the outlier populations
differ from the other study populations.
What target population, if any, underlies most analyses? For the
most part, the FDA is focused on studying treatment effects on
individuals “treated” with drugs in the United States. The major gap
with respect to this target is the lack of data on the uninsured and
individuals with Medicaid, but there are ongoing efforts to fill this
gap, such as the inclusion of more Medicaid data.
Are there ways to generalize the findings of the nodes to the
network? Depending on the specific analysis, the operations center can
always combine the outcome and treatment data across the sites to obtain
a “network-wide” effect estimate. It is also possible to include a
specific portion of the network in an analysis, e.g., running the
analysis only in the Medicare fee-for-service data when examining the
treatment effects in the elderly.
How easily can node-specific estimates be transported between
nodes or to external populations? While it is theoretically possible to
standardize node-specific estimates to one population (e.g., the entire
US population), it has not yet been implemented within Sentinel.
However, Sentinel routinely performs pre-specified subgroup analyses to
identify treatment effect heterogeneity.
Are choices ever made to maximize target validity, rather than
internal validity or precision? Given that all the data partners are
from the US, and the general focus is on general US population treated
with the medications, requesters generally prioritize internal validity
and precision..