The Importance of Control Assessment in Effective Risk Management
When it comes to the assessment of risks, one of the most important parts of the assessment is the identification of current controls and their effectiveness. All issues that impact on an organisation occur due to the absence of or ineffectiveness of controls.
It follows then that ensuring the current controls that are in place are effective is fundamental to reducing an organisation’s risk exposure. This is due to the fact that there is a direct correlation between the effectiveness of current controls and the Likelihood and/or Consequences of the identified risk i.e. the more effective the controls the lower the Likelihood of the risk occurring or the lesser the consequences if the event does occur.
Often, the assessment of control effectiveness is based on a qualitative judgment, rather than on evidence gathered from data/performance metrics. What I mean by this is that the assessment of control effectiveness resembles more a “gut feel” rather than a robust analysis.
It needs to be remembered that an absence of an incident/event is not necessarily an indicator of control effectiveness – it is simply an indicator that all of the pre-conditions required for that incident to occur have not been present. This notion is captured in the Swiss Cheese Model.
In the Swiss Cheese model, an organisation’s defences against failure are modelled as a series of barriers, represented as slices of Swiss cheese. The holes in the cheese slices represent individual weaknesses in individual parts of the system, and are continually varying in size and position in all slices. The system as a whole produces failures when all of the holes in each of the slices momentarily align, permitting “a trajectory of accident opportunity”, so that a hazard passes through all of the holes in all of the defences, leading to a failure.
To use a real example to illustrate:
A Council manages an Aquatic Centre with an outdoor and indoor pool. A risk is identified: Outbreak of Cryptosporidium or other infectious Bacteria in the Council Pool. The controls that are in place to reduce the likelihood of this risk included: ongoing preventative maintenance program of filtration system; inspection of water quality five times per day; signage regarding use of the pool facilities when ill; and policies/signage around use of watertight nappies.
The consequences were assessed as: closure of pool (loss of revenue); requirement to empty and clean pool; negative impact on Council reputation; and potential legal action taken by personnel who are ill after visiting the pool.
On review of the control evidence for inspection of water quality, it was discovered that there were multiple instances where the sheet that testers are supposed to sign on completion of each water test had not been signed. This does not necessarily indicate that the tests were not completed, however, it is a control and no assurance has been undertaken to ensure that the control was actually effective. In this case, if the tests were not being conducted, the “hole” in the Swiss Cheese just became larger.
So what do we do to improve assessment of effectiveness?
The most effective method is to identify the risks with the highest consequences within the organisation. For each of these risks, identify the controls currently in place to reduce the Likelihood of the risk occurring. Once this is completed ensure that performance indicators are in place for each of the controls, and that the performance is being monitored through the organisation’s internal audit program.
The bottom line – if controls are not being measured it is impossible to determine whether or not they are effective. If you can’t establish control effectiveness you can’t make a reasonable determination of risk level and the organisation may have a higher exposure to risk than it believes to be the case.