Training must produce its intended outcome. That is, a specified change needs to have occurred after training is completed. If you cannot validate the training, then you must seriously question why it should be done at all.
Much training is done all around the world every year. Unfortunately, much of the money spent on training may be wasted because organisations do not evaluate the training outcomes. Even with the development of the National Training System, organisations may still be wasting money on the wrong kind of training and may not bother to determine how successful it is or if there is adequate return on the money invested in training and development activities. The introduction of the AQTF Quality Indicators may help RTOs with this issue and even provide valuable data on which to base organisational decisions.
In Australia, as a general rule, the mentality has persisted that training is a massive expense to organisations and that money spent on it cannot be recovered.
Why Training?
Training is concerned with the development of knowledge and skills to be used immediately, or in the very near future. It deals with developing people who already have, or who are just about to enter, a job. Where people already have a job, the focus is on improving performance; where people do not have a job, the focus is on preparing them to perform a job within an organisational framework in a satisfactory manner.
Short-Term focus of training
Within training there is an increasing emphasis on accountability. This term describes the need for the training activity to develop the skills, knowledge and attitudes required by particular individuals for use in the workplace or to meet the learning needs as identified in a needs analysis. Accountability also means that training should contribute to the productivity and value of an organisation; that is, it should provide some benefit to the organisation and the people who work within it.
Training accountability is a process driven by governments, industry and professional bodies. It requires training to demonstrate its results, just as any other division or department of an organisation must demonstrate. When we can identify the specific changes brought about by training interventions, training moves from being a tolerated expense to being recognised as a significant management tool in organisational strategy and development.
Plan and Prepare the Evaluation
Models of training evaluation
The Kirkpatrick model is one of the best-known evaluation tools among trainers. Kirkpatrick suggests that there are four levels on which evaluation can be based. These evaluation levels are:
- Reaction level
- Learning level
- Behaviour or skills change level
- Outcome or organisational level
Many trainers use the first level, that of trainee reaction. Trainee reactions are usually recorded on what is commonly called a "happy sheet". These sheets record the feelings, thoughts and reactions of the learners about the course, the trainer and the training environment. These usually take the form of a brief questionnaire given at the conclusion of training. There are major differences of opinion about their usefulness. The argument revolves around the concept that learners can be excited about the experience they have had on the training course and complete the "happy sheet" on that basis (hance the name "happy sheet"). The sheets may not be all that reliable if they are the only tool of evaluation because the information they give is not a considered and thoughtful reflection on the worth of the learning that occurred during the course.
However, combined with other levels of evaluation, these reaction sheets can provide a total view of the worth of a course in terms of the reaction by learners. The sheets are not difficult to design. You should try to develop questions that require some thought and reflection, rather than questions like "Did you enjoy the course?" A more suitable question might be "What activities were most useful to you in developing the required skills?" More recently, the release of the Quality Indicator Learner Engagement Questionnaire has taken this level further and linked it with Kirkpatrick's second level, exploring the reaction of the learner to the content and its application to the workplace.
The second level of evaluation is the learning level. This level ascertains what the learners have learned on the course. In Competency Based Training (CBT), this is known as "assessment". However, you may also like to develop that give you feedback on the learning once learners have returned to the workplace. This may take the form of a work-based project or a specific workplace problem that learners complete after a period of time back on the job, from which they report their reactions and abilities in achieving the outcome.
The behaviour or skills change is the third level and is concerned with what changes have occurred as a result of the training. Changes in behaviour are usually monitored back on the job. Learners may be observed, or supervisors and managers may produce reports on the progress of the learner. This level is really about trying to record the training transfer back to the job. It may be undertaken by trainers, or by managers and supervisors, and it adds some realism to the evaluation. It is all very well to say that the learner has achieved competency in the training environment, but the real test is if and when the learner uses the new skills, knowledge and attributes on the job. This is an important evaluation level because it focuses on what the learner has to do in the workplace and is of primary concern in CBT. The Quality Indicator Employer Satisfaction Questionnaire has been designed to capture this data and has often been referred to as an external evaluation.
The fourth and final level of evaluation is aimed at outcomes. While it may describe specific learning outcomes, the term also has much wider meaning. Here it is concerned with the impact that training has on the organisation. This is where you see how the training fits in with the organisation's business plan and strategic objectives. This level is the basis of the Australian Quality Training Framework (AQTF) and the Quality VET Framework. Here you look to see what impact the training has had on the organisation or its parts, and especially if the problem that precipitated the training has been solved. This type of evaluation may be over the long term, as the results of training may not be noticeable immediately at the organisational level.
The workshop: "Evaluate Training Programs" teach training professional to design and implement a training evaluation program for competency-based training in line with the VET Quality Framework and the AQTF.
