Learning and training metrics are a form of evaluation which looks to quantify if any learning is proving effective. They help measure whether training is achieving tangible results and there is a return on investment (ROI) in the education provided. They are also a means of looking for improvements in training content and help establish training models that have proven success and can be delivered repeatedly. This article will provide a brief overview on how to measure the effectiveness of learning and understanding learning and development metrics and analysis.
How do you measure effectiveness of learning?
The fundamental basis for measuring the effectiveness of any learning is to first understand what the desired outcomes and objectives of the particular learning are. This requires organisations to understand their overall strategic objectives and establish how training can help achieve these. This may mean identifying any weaknesses or gaps in the knowledge base of a workforce and seek to implement learning and development programs that remedy and improve this.
There are then two broad areas where the effectiveness of learning can be measured – learner outcome and process measures. Learner outcome is quantifying success for the individual in fulfilling wider objectives - for example does it lead to increased sales or higher standards of customer service. Process measures will monitor the specific training structure, looking at areas such as hours of training completed, trainer satisfaction and participant engagement. Within this overview, specific learning and training metrics can be utilised to try and achieve required outcomes.
Understanding learning and development metrics and analytics
There are many recognised learning metrics and methods of evaluating training, but underpinning all are some simple areas of measurement:
- Pass or fail rate in relation to any test, quiz or assessment involved.
- Scoring system of any assessment.
- Training experience satisfaction through a survey or feedback from participants.
- Training completion rate – the amount of learners actually finishing a course.
- Post Training Performance - identifying how effective training was in improving the learner’s workplace performance.
- Return on Investment (ROI) - is the investment in training offering value in terms of achieving overall strategic objectives.
- These are the core elements in establishing a strategy or system of metrics for analysing the effectiveness of learning and development.
What are some training metrics examples?
One of the most widely used and well known models for evaluating training effectiveness is the Kirkpatrick model. It was created by Dr. Donald Kirkpatrick, past president of the American Society for Training and Development (ASTD), in the 1950s. The Kirkpatrick model identifies four levels of evaluation: Reaction, Learning, Behaviour and Results.
- Level 1: Reaction - The objective here is understanding how learners felt about the training. The metric used following training could be a face to face interview, feedback forms or surveys. Within these, key questions could include asking whether training was engaging and well presented. Did the learner feel they gained from the experience and it will improve their performance? Are there any areas where they feel training could be improved?
- Level 2: Learning - This level measures the increase in knowledge, skills and understanding following training. This requires an assessment before and after training to measure progress. An assessment could be a written exam, online test or simulated task. It is important in this level to identify the desired outcomes of the training and monitor whether this has been achieved between the pre and post training assessments. Another key measurement here would be the pass or fail rate. For example, a high fail rate may mean training is taking too long and knowledge is being lost or there is a fundamental weakness in the training content.
- Level 3: Behaviour - This measures a learner’s behaviour following training. Specifically, it monitors whether the learner shows improved workplace performance and is actively applying their learning. Examples of measurement in this level could include customer surveys or comments, data monitoring such as sales targets reached or call resolution times, and interviews with supervisors and colleagues. It is also important in this level to assess the work environment. For example, a manager or internal system may be preventing an individual from applying new skills and knowledge.
- Level 4: Results - This level identifies whether the change in individuals behaviour is leading to improved overall results for an organisation and better fulfilment of their strategic objectives. Metrics used here may include monitoring any increase in sales, a reduction in workplace errors, positive customer feedback or overall revenue gains. At this level financial assessments can be made as to whether results indicate training is providing ROI.
The term Key Performance Indicator (KPI) is widely used across industries. Key Performance Indicators are a metric that measures how individuals, departments or whole organisations perform in terms of overall strategic goals and objectives. The terms KPI and metric are often used interchangeably but there is a distinction. Whilst all KPIs are metrics, not all metrics are KPIs. KPIs are the fundamental measurement for overall strategy, whilst metrics can be used for wider areas of an organisation that are not as critical or focused on targets. As KPIs serve the overall strategy, they are less commonly used as a measure within learning and development. However, some metric systems for training will loosely interpret the kind of components in The Kirkpatrick model as KPIs.
KPIs do directly influence training, as if they are not fulfilled, the remedy may be implementing a learning and development program. There is also some thinking in the field of training development that looks to apply a model like Kirkpatrick in reverse – understand the results wanted and the behaviour required to deliver this, and develop training and education that will meet these objectives. In this sense, KPIs are the overall results wanted so learning should be aimed towards fulfilling them.