Training has long worried about justifying its existence. How do we measure training success? What are the critical outcomes of training? How do we know if the training is any good and if it is making any impact? In the second in this series, let’s turn our attention to Kirkpatrick Level 2 Learning analysis on the actual understanding and learning by participants.
Most learning and development professionals will be familiar with The Kirkpatrick Model of evaluating the effectiveness of training. It’s a great model and serves as inspiration for the built-in reporting in the Tribal Habits platform. The model suggests four levels of training measurement.
- Reaction: The degree to which participants find the training favourable, engaging and relevant to their jobs.
- Learning: The degree to which participants acquire the intended knowledge, skills, attitude, confidence and commitment based on their participation in the training.
- Behaviour: The degree to which participants apply what they learned during training when they are back on the job.
- Results: The degree to which targeted outcomes occur as a result of the training and the support and accountability package.
Over a series of four articles, let’s examine The Kirkpatrick Model and how Tribal Habits can help any organisation with reporting on all four levels – automatically! We’ll see how organisations can select the appropriate level of reporting and utilise the information at each level to improve its training topics. And certainly, for any modern organisation, the ability to gather high-quality data which can demonstrate the learning and understanding outcomes from training is critical to supporting training budgets and initiatives.
What is Kirkpatrick Level 3 Behaviour trying to measure?
Now we get to the interesting measurements…where most training experiences fail to go! This is because most training experiences stop at the transfer of knowledge (Levels 1 and 2) and leave it to someone else to track any further outcomes. Perhaps many training experiences don’t want to know the results, as it may reflect poorly on the training!
Although to be fair, often a Level 3 Behaviour measurement is not appropriate for every topic. Training involving regulations, technical knowledge or product knowledge may only require a Level 2 Learning measurement.
As a result, Level 3 Behaviour reporting is optional within Tribal Habits and should be considered carefully for any training program. Measuring behavioural change in training can be a complex and consuming task. It requires a mechanism to capture both quantitative and qualitative feedback from participants after they have left the training program – sometimes weeks or months later.
It requires commitment from all stakeholders – training provider, training admin, participant and the participant’s manager – to ensure enough data is collected and that the data is high quality.
Once you have behavioural change data, you must then do something with it. The training provider and training admin need to review the data for trends and outcomes. Perhaps the best use of behavioural data is by managers, who can use the data for 1:1 coaching with participants or to provide timely intervention when participants are struggling.
Measuring behavioural change through reflection and intent
First and foremost, asking participants to self-reflect and identify their own areas for change is a critical measurement of likely behavioural change. It is a massive leading indicator.
When a training participant is asked to compare their thoughts to others, identify areas of strength and improvement, and reflect on training ideas, their response is very telling. Participants who struggle to reflect are unlikely to see much behavioural change. If participants provide poor quality reflections or no reflections at all, they are clearly not engaging in the training. If they cannot even identify interesting points or potential new ideas to implement, then no implementation will occur.
This is why training programs must ask participants to identify areas for change. When you simply give each participant the same ‘checklist’ of things to do, participants have not taken ownership of that list or envisioned how they may change. A better idea is to ask participants to build their own checklist of things to do or ask them to select key items from a larger list. Participants must take this first step towards behavioural change themselves.
Tribal Habits enables this process in two critical ways.
- Reflection questions. Creators can simply enable reflection questions in their topic and participants will be automatically engaged with a series of reflection opportunities through-out the topic. Not only does this boost learning retention and commitment, but participants can compare their reflections with their colleagues, providing a deeper level of self-reflection and awareness.
- Poll and Q&A answers. Creators can use polls (fixed response) and Q&A (free response) elements anywhere in their topic to ask explorers to provide a response to a question. Creators can also elect to have any of those responses copied to the participant’s journal. This allows creators to use polls and questions to ask participants to identify ideas for improvement, select from next steps, highlight uncertainties or commit to action…and that have information recorded and available offline and for managers.
- Next steps. Creators can enable an optional final reflection question for participants which asks them to write down their next steps with the topic. Once again, their answer is captured in their journal for later use.
In all cases, its easy to include reflection opportunities through-out a topic with just a few checkboxes. It challenges participants to think through the status quo and look for ways to change. It also creates a wealth of useful data for the firm on participant intentions.
Measuring behavioural change through on-the-job activities
The next step in measuring behavioural change is to move from intent, to action. In this case, through on-the-job activity to demonstrate new behaviours.
For many training programs, this level of measurement falls outside the scope of the training. And that’s a shame. Demonstrating proficiency with a new skill or tool in a training experience – a controlled environment when the new knowledge is fresh – is very different from demonstrating proficiency in unpredictable real-world conditions at a later time.
For topics were behavioural change is important, setting up on-the-job activities for participants to demonstrate their new behaviours is the ultimate test. Measuring changes in behaviour is also often a great leading indicator of future improvements in business impact.
In Tribal Habits, creators can simply enable one or more on-the-job activities in any topic. These activities are unique to each topic. They can be designed to specifically test and measure new workplace behaviours related to that topic. Explorers have a deadline to complete the activity and must then report on their success and learnings, providing both quantitative and qualitative measures.
Activities can be very versatile. An activity might require the explorer to conduct a demonstration of technique with their manager or another assessor. Or it might require them to build or implement a new tool or document in their role. Or perhaps download a case study and complete a technical assessment.
Good Kirkpatrick Level 3 Behaviour reporting promotes long-term change
Kirkpatrick Level 3 Behaviour measurement is one of the most valuable measurements for any modern organisation. Without behavioural change, obtaining new outcomes for a business is almost impossible – keep doing what you are doing and you’ll keep getting what you are getting!
Any training program which is aimed at improving firm results usually requires new behaviours in staff to produce those new business results. So consideration should be given to how behavioural change can be encouraged and measured. In Tribal Habits an Australian LMS, this is an easy process with a number of simple but powerful features which automate this process.