Assessment

1. Was selection of exposed and non-exposed cohorts drawn from the same population?

Examples of low risk of bias:

• Exposed and unexposed drawn for same administrative data base of patients presenting at same points of care over the same time frame

Examples of high risk of bias:

• Exposed and unexposed presenting to different points of care over a different time frame

2. Can we be confident in the assessment of exposure?

Examples of low risk of bias:

• Secure record (e.g. surgical records, pharmacy records)
• Repeated interview or other ascertainment asking about current use/exposure

Examples of higher risk of bias:

• Structured interview at a single point in time
• Written self report
• Individuals who are asked retrospectively confirm their exposure status may be subject to recall bias – less likely to recall an exposure if they have not developed an adverse outcome, and more likely to recall an exposure (whether an exposure occurred or not) if they have developed an adverse outcome

Examples of high risk of bias:

• Uncertain how exposure information obtained

Examples of low risk of bias:

• Comprehensive matching or adjustment for all plausible prognostic variables

Examples of higher risk of bias:

• Matching or adjustment for most plausible prognostic variables

Examples of high risk of bias:

• Matching or adjustment for a minority of plausible prognostic variables
• No matching or adjustment of plausible prognostic variables
• Statements of no differences between groups
• Statements that differences were not statistically significant are not sufficient for establishing comparability

5. Can we be confident in the assessment of the presence or absence of prognostic factors?

Examples of low risk of bias:

• Interview of all participants
• Self-completed survey from all participants
• Review of charts with reproducibility demonstrated
• From data base with documentation of accuracy of abstraction of prognostic data

Examples of higher risk of bias:

• Chart review without demonstration of reproducibility
• Data base with uncertain quality of abstraction of prognostic information

Examples of high risk of bias:

• Prognostic information from data base with no available documentation of quality of abstraction of prognostic variables

6. Can we be confident in the assessment of outcome?

Examples of low risk of bias:

• Independent blind assessment
• Record linkage
• For some outcomes (e.g. fractured hip), reference to the medical record is sufficient to satisfy the requirement for confirmation of the fracture

Examples of higher risk of bias:

• Independent assessment unblinded
• Self-report
• For some outcomes (e.g. vertebral fracture where reference to x-rays would be required) reference to the medical record would not be adequate outcomes

Examples of high risk of bias:

• Uncertain (no description)

Examples of low risk of bias:

• No missing outcome data
• Reasons for missing outcome data unlikely to be related to true outcome (for survival data, censoring is unlikely to introduce bias)
• Missing outcome data balanced in numbers across intervention groups, with similar reasons for missing data across groups
• For dichotomous outcome data, the proportion of missing outcomes compared with observed event risk is not enough to have an important impact on the intervention effect estimate
• For continuous outcome data, plausible effect size (difference in means or standardized difference in means) among missing outcomes is not large enough to have an important impact on the observed effect size
• Missing data have been imputed using appropriated methods

Examples of high risk of bias:

• Reason for missing outcome data likely to be related to true outcome, with either imbalance in numbers or reasons for missing data across intervention groups
• For dichotomous outcome data, the proportion of missing outcomes compared with observed event risk is enough to induce important bias in intervention effect estimate
• For continuous outcome data, plausible effect size (difference in means or standardized difference in means) among missing outcomes is large enough to induce clinically relevant bias in the observed effect size

8. Were co-interventions similar between groups?

Examples of low risk of bias:

• Most or all relevant co-interventions that might influence the outcome of interest are documented to be similar in the exposed and unexposed

Examples of high risk of bias:

• Few or no relevant co-interventions that might influence the outcome of interest are documented to be similar in the exposed and unexposed