Assessment

1. Is the source population representative of the population of interest?

Examples of low risk of bias:

• Selection of target population (either the entire population or a random sample) from a representative population roster such as a national association database

Examples of higher risk of bias (“probably yes” or “probably no”):

• Single centre/city/region study
• Non-random sampling

Examples of high risk of bias:

• Studies where the source population cannot be defined (or enumerated), i.e. any volunteer studies using self-recruitment

2. Is the response rate adequate?

Examples of low risk of bias:

• High enough response rate to ensure that any differences would be unlikely to affect results (>75%)

Examples of higher risk (“probably yes” or “probably no”) of bias:

• > or = 25% missing data but statistical analysis shows no difference in demographic variables that are associated with variability in survey responses between respondents and non-respondents

Examples of high risk of bias:

• Response rate of <50% and no testing done to explore the differences between respondents and non-respondents, or testing indicates that important difference exist

3. Is there little missing data?

Examples of low risk of bias:

• Less than 10% missing data within questionnaires

Examples of higher risk of bias (“probably yes” or “probably no”):

• Less than 15% missing data within questionnaires

Examples of high risk of bias:

• More than 15% missing data within questionnaires

4. Is the survey clinically sensible?

Examples of low risk of bias:

• Formal assessment of the comprehensiveness, clarity, and face validity of the questionnaire in a similar population

Examples of higher risk of bias (“probably yes” or “probably no”):

• Formal assessment of comprehensiveness, clarity, and face validity of the questionnaire in a different population

Examples of high risk of bias:

• No evidence that comprehensiveness, clarity, and face validity of the questionnaire have been assessed

5. Is there any evidence for the reliability and validity of the survey instrument?

Examples of low risk of bias:

• Reliability and construct validity (i.e. convergent and discriminant validity) of the survey have been well-established in a similar population

Examples of higher risk of bias (“probably yes” or “probably no”):

• Some evidence of robust psychometric properties
• Reliability and construct validity of the survey have been well-established in a different population

Examples of high risk of bias:

• No evidence that reliability and construct validity have been established for the instrument