Chapter twenty: Program evaluation
A training program does not have a life of its own, it exists to prepare participants for their role in achieving an agency's or project's goals. As objectives are achieved and conditions shift towards new priorities, modifications in preparation are appropriate. Thus, the effectiveness of a training program must be measured in how it is addressing the contemporary needs of the agency or project. However effective a training program has been historically, it cannot afford to defend "sacred cows" but instead it must be flexible enough to change with the times, but, it cannot be so flaccid as to lose needed backbone or continuity in addressing certain basic needs for preparation of its participants.
Evaluation of a training program must be accomplished in order to examine its effectiveness in preparing the participants for their future responsibilities. Self-examination can be conscientiously done so as to suggest appropriate program modification, thus, the program evolves rather than suffering from intermittent, major overhauls. To be effective, however, the evaluation process must incorporate input from a variety of sources (i.e., trainees, Peace Corps/Washington and field staff, past participants, returned Volunteers and training program staff) over a period of time. The training program described in this manual has been influenced by such a process. The program has been sensitive and responsive to scrutiny and constructive criticism.
Just as trainees receive feedback from the staff, they must also have the opportunity to provide feedback and express their feelings about the training program and the staff. This two-way flow of feedback and ideas helps everyone involved and promotes an atmosphere of trust and mutual respect. Feedback from trainees is very helpful to the staff in learning about the effectiveness of their training techniques and methods, the strengths and weaknesses of the site and logistical support, the degree to which they are being successful in setting an example for the trainees, and obtaining suggestions for improving all aspects of the program. There are a few channels through which trainees can evaluate the training program. If the atmosphere in training is as it should be, there should be enough trust and honesty for the trainees to feel comfortable expressing themselves. Approximately every two weeks, a written evaluation instrument is distributed for the trainees to fill out and return (samples are provided in the Program Design chapters). They may put their names on these or submit them anonymously. In addition, staff should point out to the trainees that they are welcome to discuss any thoughts or concerns verbally with any staff member at any time, whether informally, in a scheduled interview, or in a special interview that can be arranged upon request. Staff should take care to respond appropriately to input from trainees. Defensiveness is counter-productive, and it is imperative that staff maintain their objectivity with all trainees regardless of input they may receive from them. The degree to which staff explains or discusses points raised by trainees about the program is up to their discretion, but all feedback should be acknowledged and appreciation should be expressed for all input offered by trainees.
In addition to the formal feedback process described above, daily classroom sessions provide a mechanism to obtain on-going information which indicates a response to training. However, this must be tempered with the trainees' perspective on the rationale behind training at that point in time. In effect, the trainees are so close to the process, it is difficult to be objective. Nevertheless, their input is valuable and training program staff should scrutinize suggestions thoroughly.
It may also prove valuable to conduct a group question and answer, and feedback session part way through training. The exact timing of the session should be based on the general atmosphere of the particular training program. If it appears that trainees are generally feeling negative about training, this session should be held as early as possible. In this way, problems will be alleviated early, thus assuring a better response to training. In past programs, this session has been facilitated by someone who is not directly connected with the training program such as a Peace Corps/Washington staff member.
It is also advisable to request the trainees complete a questionnaire on technical training at the end of ICT, after six months of service, after one year of service and at completion of service. Input from returned Volunteers is also valuable but difficult to obtain. Their perspective of training at these times may be less biased than earlier assessments and should be more useful in developing constructive program modifications. Due to changes in contractors and other considerations, the later evaluations may need to be conducted by Peace Corps rather than by the training contractor.
Training Program Staff:
This phase of assessment should be initiated as part of staff training and carries through to the end of the cycle. Returned Volunteers who participate as trainers should be asked to comment on their own training in relation to their overseas experience. This provides a basis for discussing the philosophy of the contemporary training program from their perspective as a new staff person. Trainers are an integral component for trainee feedback during the program and, therefore, must understand their new staff role. This critical aspect is accomplished by a thorough staff training.
In addition, regular staff meetings should include an on-going assessment of the program's effectiveness. The staff should meet at least weekly to discuss activities and problem resolution. Trainers should also be asked to provide an overall assessment at the end of the cycle through questionnaires and a direct request from the Program Director.
Peace Corps Staff:
Visitors to the training program should be encouraged to offer suggestions in relation to their orientation. In addition, questionnaires should be sent to field staff in countries to which trainees were assigned.
[Ukrainian] [English] [Russian]