Of all aspects of business performance, compliance can be one of the hardest to measure. If compliance were to be managed with complete success, the result would be a series of negatives, a clean sheet: no incidents, no reports, no outcomes, no claims, no legal proceedings.
Compliance training can be as hard to measure as compliance itself. You can, of course, measure learner completion and assessment scores, but try defining the extent to which your training has impacted workplace culture, or helped individuals prevent breaches of compliance! Not easy.
A number of studies examine the effectiveness of compliance training analytics. The In Focus: Compliance Trends Survey 2013 report, a joint study by Deloitte and Compliance Week, surveyed 189 senior executives with responsibilities for compliance and ethics. The study revealed that a wide variety of metrics were being used, of which one of the most common, naturally, was the successful completion of training courses by employees. Forty-four per cent of respondents cited workforce training as the biggest operational issue around managing compliance risks.
An important trend in compliance training metrics in the use of increasingly nuanced data analysis. The accumulation of statistics on reported incidents, for example, becomes far more useful if the analysis can include the demographic location of incidents within the organization, as well as the type and seriousness of the incident, and some indication of underlying factors such as intent and knowledge. The Deloitte study found that some organizations were relying on surprisingly outdated analytical tools for analyzing compliance data; an obvious advantage to using the latest software is the ability to monitor trends in real-time rather than retrospectively and to see where a compliance training focus may have made an impact.
Because there is no single outcome measure that demonstrates effective compliance training, the most effective oversight will use a diverse approach. In addition to data analytics, qualitative measures such as free-form interviews and observation can be useful tools. Look at what the business is already measuring (e.g., speak up hotline calls, successful phishing attacks) and measure your training success against those hard numbers.
Perhaps the most useful information an organization can derive from compliance training analysis is the prediction of future risk areas. An expert may be able to mine past and current data on assessment scores to foresee problems with surprising accuracy. When compliance officers need to argue the case for their training budget allocations, it may be the force of these predictions that helps them most.
Successful companies are increasingly using metrics which can predict future behavior; for example, employee ethics surveys can measure compliance training program effectiveness, and identify areas of potential risk. An effective compliance training program can include an analysis of changes in attitude amongst participants – a far more useful measure of success then recording mere attendance and completion. Another important metric is the effect of the compliance training program on incident reports; but again, the analysis needs to recognize subtlety – an increase in reports may demonstrate raised awareness rather than reduced compliance.
Are you having difficulty measuring the effectiveness of your Compliance Training Program? Do you feel it needs an injection of modernisation? something to liven it up and get better results? Give us a call, we have been working with Global companies on compliance training for over 25 years, we will be able to help.