Consider the validity of the measurement method; consider reliability to the extent that could bias findings.
Consider the measurement methods independent of the sample issues. For example, in the Nytro et al. study, there was concern that the methods used for implementation might have created a bias in the direction of better compliance because company managers would have an incentive to appear as though they are complying with the law. If we had been told that the self-report method had been verified in a sub-sample through an external audit, we could have given a high rating for the measurement methods, even though the means of selecting the sample (quota sampling; no indication of refusals) could nevertheless have biased the results towards more compliance.
Were appropriate statistical tests conducted on the F/B data?
See Q12 above.
If no statistical tests were used, and you think they were not needed, select the answer corresponding to this. If no statistical tests were used and you think they should have been, then use one of the following three options: minor deficiencies; inappropriate statistical test, which might modify the conclusions; inappropriate statistical tests, which could substantially affect conclusions.
Consider issues of statistical power in Q43 not here.
Are you confident that there are no additional potential sources of bias in the quantitative study of F/Bs not yet captured in the previous questions?
What is your overall appraisal of the quality of evidence about facilitators and barriers provided by the quantitative study?
Consider your answers to questions 25-29 and give an overall assessment.
Should this publication to on to data extraction (Level 5)?
Is supplementary information required?
Is this the final version of the Level 4 assessment?
Institute for Work & Health