Evaluating using control effectiveness

Assessing control effectiveness helps identify the strengths and weaknesses within a bowtie model and supports the prioritisation of safety improvement activities.

By evaluating how well each control performs, organisations can gain a clearer understanding of where risks are being effectively managed and where additional attention may be required. The assessment can also support wider risk evaluation processes, including matrix-based risk assessments.

Control effectiveness ratings are often displayed using a simple colour-coded scale, such as red for poor performance through to green for effective performance. This provides an easy-to-understand visual representation of the health of the controls, enabling users to quickly identify areas of concern.

Considerations for rating the effectiveness of individual controls

Understanding the Control Effectiveness 

The effectiveness of a control is determined by two key factors:

Effectiveness = Adequacy × Reliability

Effectiveness

Effectiveness describes the ability of the Control to stop the threat materialising into the top event.

Adequacy

Adequacy considers how well a control is designed to address a specific threat or consequence.

A control may be highly effective in one scenario but much less effective in another. For example, a handheld fire extinguisher may be suitable for tackling a small galley fire but would have limited value in a large fuel-spill fire.

This is why care should be taken when reusing controls within a bowtie model or across different bowties. A control that is adequate in one context may not provide the same level of protection in another.

Reliability

A well-designed control is only effective if it functions as intended when required.

Reliability considers the level of confidence that a control will perform consistently and correctly. This may be influenced by factors such as human performance, maintenance arrangements, training, procedures, resources, or organisational culture.

Considering Escalation Factors

Before assigning an effectiveness rating, consideration should be given to any escalation factors that may reduce the performance of the control.

Ask:

• How significant are the escalation factors?
• How likely are they to affect the control?
• How effective are the escalation factor controls in managing those conditions?

By considering both adequacy and reliability, together with the influence of escalation factors, organisations can make a more informed assessment of overall control effectiveness and identify where improvement efforts should be focused.