Important confounders at the extra-workplace level: other extra- workplace OHS initiatives.
Was the degree of implementation of the OHSMS demonstrated or described clearly?
The reason for including this question is to aid in our interpretation of studies looking at outcomes. We are looking for a description or demonstration of the degree of implementation that will give us confidence that any observed lack of effect on outcomes does not result from a lack of intervention implementation. on is concerned with what was actually done
e., what was implemented), as opposed to what was planned.
Are you confident that the measurement methods did not introduce bias to the corresponding implementation findings?
In the comment box following your response, note the rationale for your selection.
This question addresses measurement bias; i.e., systematic error arising from inaccurate measurement (or classification) of subjects on study variable(s).
Consider the validity of the measurement method; consider reliability to the extent that could bias findings. Some examples.
Consider whether blinding was used in the assessment of implementation. For example, was the person who carried out the management audit unaware of whether a workplace was in an intervention? Did the person have a vested interest in a certain set of results?
Consider inter-rater reliability of and how audits were distributed among raters. (i.e. could observed effect result from an audit with poor IRR and uneven distribution of auditors between intervention and control groups/)
Consider the measurement methods independent of the sample issues. For example, in the Nytro et al. study, there was concern that the methods used for implementation might have created a bias in the direction of better compliance because company managers would have an incentive to appear as though they are complying with the law. If we had been told that the self-report method had been verified in a sub-sample through an external audit, we could have given a high rating for the measurement methods, even though the means of selecting the sample (quota sampling; no indication of refusals) could nevertheless have biased the results towards more compliance.
Effectiveness of Occupational Health & Safety Management Systems: A Systematic Review