CNODES Q&A:
How does external validity come into play when planning and conducting projects in the database? Just as with Sentinel, there is no explicit consideration of external validity during research. Still, the query refinement process helps establish whether limiting to specific groups of people (e.g. true new users of drugs, patients at high risk of the outcome) would reduce the relevance of the analysis, and whether some specific provinces and data partners like CPRD or MarketScan differ enough from the others in covariates and follow-up distribution to ultimately limit their use of the research question. External validity also comes up indirectly when combining the results, identifying outliers, and conducting meta-analyses.
What target populations, if any, underlies most analyses?Typically, the main target population of interest is the overall Canadian population. As a result of the fact that 97% of Canadian citizens reside in the provinces contributing to CNODES, the population represented in the Canadian portion of CNODES analyses and the Canadian target population are very similar; restrictions on drug coverage can change this for some analyses, however.
Are there ways to generalize the findings of the nodes to the network? The main way that findings are generalized from site-specific estimates to the broader network is typically by random-effects meta-analysis using inverse variance weighting.30 When using exclusively the Canadian portions of the network, this means that provinces with more events (and likely more individuals) tend to contribute more to the overall effect estimates. No attempt is made to generalize the results of each site, however, and effects are generally assumed to be constant across sites unless there is substantial heterogeneity.
How easily can node-specific estimates be transported between nodes or to external populations? Differences in demographics, the services and medications covered by each province, and the calendar time intervals each data source contributes can make it difficult to directly transport effect estimates between provinces. Because all these variables are measured, however, analytic methods like inverse odds weights or G-computation may be used to obtain more precise within-province estimates.26 Similar approaches could be used for researchers interested in using CNODES to estimate treatment effects in European or US populations, provided a target population was provided in the scientific and analytic protocols.
Are choices ever made to maximize target validity, rather than internal validity or precision? Internal validity is the core focus when preparing the scientific and analytic protocols. When choosing the provinces and data that will contribute to a given study, however, attention is paid to populations that are may differ greatly from the Canadian target population.