Evaluating Training

Determine learners’ satisfaction with training

Tips for Evaluating Training Ensure that knowledge and skill evaluation instruments are developed and ready before the course Ensure that there is time allotted in the course schedule to administer knowledge and skill evaluations and to provide feedback to learners Plan with trainers how to use information from these evaluations to revise the course design and training materials Plan how to monitor and evaluate performance of the learners on the job after training and how to coordinate this with followup support visits

Instruments that measure learners’ satisfaction help improve training and determine whether training has met learners’ needs. Learners may discuss the course in small groups or complete a form. Most forms include a rating scale and open-ended questions that ask what the learners think about the training materials and activities, the trainer, and the training environment. Learners may also indicate what helped and hindered their learning and the relevance of the content to their jobs. Asking learners what they think helps them to have a positive attitude about the course. The evaluation process tells learners that you are committed to the training and their learning, and that the people designing, managing, and delivering the training want the course to fit the needs of future learners.

In your training evaluator role, you should:

• Use a learner satisfaction instrument to gather information on what the learners think about the training materials and activities, the trainer, and training environment
• Give learners the opportunity to comment on the relevance of the training objectives and content to their job
• Allow sufficient time for learners to complete the instrument or speak with the trainer
• Share the results of learner satisfaction instruments with training managers and designers

Determine whether learners have met the learning objectives by giving and scoring knowledge evaluations

Knowledge evaluation, when it is part of the course design, is one of the most effective approaches to determining whether learners have met the learning objectives of a training course. At the beginning of the course, the learners must know what the objectives are and how achievement of the objectives will be measured. Learners should know when the knowledge evaluation instrument will be given, the format of the instrument (e.g., multiple-choice test, oral examination), and what score they must achieve to demonstrate that they have met the learning objectives.

To measure changes in knowledge, give a knowledge evaluation (pre-test) at the beginning of the course as well as an evaluation at the end of the course (post-test), and compare the learners’ scores. Often, the pre-test contains the same test items as the post-test. 

In your training evaluator role, you should:

• Tell learners how and when evaluation will occur, including the criteria to demonstrate achievement of the learning objectives (e.g., 85% correct)
• Make sure that learners know the learning objectives at the beginning of the course
• Give a knowledge pre-test and let learners know their results in order to identify knowledge areas on which they need to focus, and provide a baseline knowledge assessment for comparison with the knowledge post-test
• Make enough copies of the knowledge evaluation instruments before the course so that each learner has a copy
• Give the knowledge evaluation in a quiet room with no interruptions, and with all required supplies available
• Remain in the room while learners complete the knowledge evaluation so that you can answer any questions they may have
• Give learners enough time to complete the knowledge evaluation
• Give learners immediate and confidential feedback regarding their performance on the knowledge evaluation
• Have a plan for learners who do not meet the criteria required to demonstrate achievement of the objectives (e.g., study and retake the test, repeat the course)

Determine whether learners have met the learning objectives by giving and scoring skill evaluations

When skill evaluations are part of the course design, they are given to determine whether learners can competently perform the skills outlined by the learning objectives. At the beginning of the course, the learners must know what the objectives are and how achievement of the objectives will be measured. Learners should know when skill evaluations will be given, have copies of the performance checklists or other evaluation instruments, and know if evaluation of competence will occur during simulations (e.g., role plays, work with anatomic models), with clients, or both. Learners should also know how well they must perform the skills to demonstrate achievement of the learning objectives (i.e., how the performance checklists will be used to measure their skill competency).

Pre-tests of skills are more time-consuming and logistically difficult to give than pre-tests of knowledge and are therefore not always used. They may be necessary, however, to determine if learners have requisite skills or to measure changes in skills as a result of training. One option in this case is to pre-test only a sample of learners and evaluate their pre-training skills using simulations.

In your training evaluator role, you should:

• Tell learners how and when evaluation will occur, including the criteria for demonstrating achievement of the learning objectives (e.g., perform a skill according to the steps in a checklist—first during a simulation and then with a client)
• Make sure that learners know the learning objectives at the beginning of the course
• Administer a skills evaluation at the beginning of the course and give learners immediate feedback on their performance in order to identify skill areas on which they should focus and provide baseline skill assessment data to compare with the end-of-course evaluation 
• Make enough copies of the skill evaluation instruments before the course so that each learner has a copy
• Make sure that all of the required supplies (e.g., gloves, instruments) are present before administering the skill evaluation
• Give learners enough time to complete the skill evaluation
• Give learners immediate and confidential feedback after the evaluation
• Have a plan for learners who do not meet the criteria required to demonstrate achievement of the objectives (e.g., practice using simulations and then be evaluated with clients, repeat the course)

Improve training using information from the knowledge and skill evaluations

Systematic evaluation and revision of training improves the quality of training and reduces “guesswork” when making changes to courses, training materials, and the overall training plan. Use information from the evaluation to determine when a course should be revised. This information will also guide the specific revisions necessary to improve the quality of the course. There are four sources of evaluation information:

• informal information from learners,
• measures of learner satisfaction,
• measures of knowledge acquired during training, and
• measures of skills acquired during training.

The opinions of the trainers who deliver the training should be taken into account, but their opinions alone are not adequate substitutes for the evaluation information listed above. 

In your training evaluator role, you should:

• Collect evaluation information using a standard and systematic approach
• Develop evaluation criteria before training to help in judging information from the evaluation 
• Meet with designers, trainers, and evaluators to review evaluation information at the end of training
• Use the evaluation information to make decisions about what part(s) of a course to revise, when to revise them, and what specific changes to make
• Develop a plan for revision of training 
• Share information from past evaluations with trainers to enable them to prepare for training

Monitor and evaluate performance on the job

A key aspect of evaluating training is to monitor and evaluate the performance of learners when they are back on their jobs to find out whether training resulted in improved job performance. This typically requires followup visits to the job sites of the learners. Ideally, the trainer who delivered the course conducts these visits. At each site, the trainer meets with the supervisor and observes and coaches the worker using the same performance checklists used during training. Observations of learners after training will provide information to determine whether training is making a difference.

In your training evaluator role, you should:

• Develop and implement a monitoring and evaluation plan for followup visits after training
• Conduct followup visits to observe learners on the job
• Collect performance information on the job after training 
• Collect information from the worker’s supervisor
• Use information about learners’ performance to determine the impact of training (i.e., the extent to which performance has changed due to training) 

Determine the effectiveness of training as an intervention to improve performance

It is sometimes necessary to document the overall value of investments in training. Evaluation information will show whether training closed the performance gap identified by the PNA. Policymakers and program planners also can use this evaluation information outside of the training program to help determine the value of training as an intervention to improve workers’ performance.

In your training evaluator role, you should:

• Have mechanisms for sharing evaluation information with stakeholders
• Write evaluation reports using words and terms used by decision makers
• Document the value of training as an intervention to improve performance
• Present high-level information about the impact of training rather than information about learners’ satisfaction or knowledge and skill gains 
• Incorporate information about training impact into ongoing PNAs
• Report on the monitoring of training accomplishments and impact of training