Does Pre-Survey Training Impact Knowledge of Survey Administrators and Survey Outcomes in Developing Countries? Evaluation Findings of a Training of Trainers Workshop for National AIDS and Reproductive Health Survey-Plus in Nigeria

Introduction

Any organization determined to survive and prosper in the current challenging economy, must understand the imperative to invest in training and professional development in order to improve efficiencies in production as well as to acquire the greatest return in investment of human capital.^[1] Furthermore, several authors have suggested that training is most extensive and important in any organization or establishment^[2,3]. Training is becoming increasingly complex, and tools to determine the proficiency of training programs and trainees are needed. Although emphasis has been placed recently on expanding assessment and demonstrating outcomes in proficiencies other than knowledge, evaluation of the depth and breadth of trainings' knowledge base remains critically important, because this constitutes the foundation on which competence is built, and is an essential requisite for the development of sound reasoning skills.^[4,5] Therefore, training as a tool to acquire best practices in the proficient conduct of services cannot be over emphasized.

National AIDS and Reproductive Health Survey (NARHS) had been conducted three times in Nigeria (2003,2005, and 2007)^[6,7,8]. After 2005 NARHS survey, there was a need to include biomarker-HIV testing (biological) components into the survey and it captures ages 15-64 years, hence the nomenclature of NARHS-plus was therefore adopted in 2007^[8]. The 2012 NARHS was the fourth since its inception. In the past, NARHS and NARHS-plus data collection started with central level training of States AIDS Program Coordinator (SAPC), Reproductive Health Coordinator (RHC), State Laboratory Scientists (SLS), HIV Counselor-testers (CTs), and Lead supervisors but the impact of the trainings on the participants have not been evaluated to assess whether there is need for such trainings or not, during the survey. The training of trainers (TOTs) workshop was aimed at teaching selected survey administrators on how to conduct and supervise the NARHS plus survey based on the complexity and enormity of the study especially the ‘plus’ component. The goal was to replicate same training at the state level to other supervisors, interviewers, and counselor-testers who served as research assistants in the collection of data on the field during the survey exercise.

The training was conducted using the engagement of participants in plenary discussions, presentations, group discussions, role plays, and question and answer sessions. The various sessions exposed the participants to issues of community engagement and participation in research, basics of HIV and AIDS, challenges of ensuring informed consent, reproductive health indices in Nigeria, good clinical and laboratory practice and management, data monitoring and management in research. The group interactions afforded participants the opportunity to learn from one another's experience while the role plays simulated actual practical experiences in the field. The trainees were expected to give same training in their respective states. Based on the above, this evaluation study set out to assess the impact of five-day training on the quality of conduct of research survey, namely, NARHS-plus. The overall evaluation research questions were: does training of trainers at the central level have any impact on the level of knowledge that could predicate a successful conduct of NARHS-plus survey in Nigeria, and are there variations in the level of knowledge among participants from different states or different regions of the country?

Methods

Results

One hundred and seventy (91.9 %) trainees participated in the pre-test while 177 (95.6 %) attempted the post-test, 158 attempted both pre and post-test. Also, the number of trainees that participated in the pre-test, post-test and took both during the training for the Southern Nigeria (SN) were 72,82 & 68 and Northern Nigeria (NN) were 98,95 & 90 respectively. Combining all the respondents irrespective of their zones, the overall pre-test and post-test score were 64.0% and 77.4% respectively indicating a 13.4% gain in knowledge over the baseline knowledge level (Table 1) . The pre-test score and post-test score for the SN and NN were 64.7% and 80.3% respectively, and 63.5% and 75.3%. The knowledge gain was 15.6% and 11.8% for SN and NN respectively (Table 1).

Comparing the performances in the two sets of training, Table 2 shows that the baseline knowledge was not statistically different among trainees from SN and NN (p=0.44), however there was statistical significance in the posttest scores between the two regions (p=0.001) and in knowledge gained (p=0.017).

Figure 1 shows the mean scores of the attendees from each state in the two tests as well as the differences between the scores. The highest mean scores for the pretest, posttest and their differences were recorded in Delta (87.9%), Akwa Ibom (74.0%), and Lagos (72.4%) states respectively while the lowest mean scores were from Kwara (54.1 %) -pretest, Sokoto (66.4%) -posttest and Sokoto (3.4%) knowledge gain. In Table 3, the scores in the pretest and the posttest were compared, the analysis of variability of the scores showed that the mean scores for the states were statistically significantly different (p<0.05) in both tests but there was no significant difference in knowledge gain of the attendees across all the states in the area of knowledge of HIV epidemiology, HIV counseling, AIDS-related issues, survey administration etc.

Close

Figure 1.

Distribution of scores in the pre-test, post-test and knowledge gained across the states

Export to PPT

While the highest mean scores for the pre-test was in the Counselor Testers' group (67.2%); post-test-Supervisors' group (79.1%); and knowledge gain was recorded in SAPCs' group (17.6%). The lowest mean scores for the pre-test was SRHCs' group (60.4%), post test was the Laboratory Scientists' group (73.1%), and knowledge gain was also the Laboratory Scientists' group (8.0%). The highest knowledge gain was recorded among the SAPC (17.6%), followed by the SRHC (16.7%). The overall mean pre-test and post-test scores were 64.0% and 77.4% respectively indicating a 13.4% gain in knowledge above baseline knowledge. (See Figure 2 and Table 4.)

Close

Figure 2.

Performance of attendees by their designations and regions

Export to PPT

Although there was no statistically significant difference in the pretest and posttest mean scores of the five groups of attendees, the mean scores of knowledge gained across the five groups were statistically significantly different (p<0.05) (see Table 5).

In an attempt to explore the details of the significant knowledge gain obtained in Table 5, we carried out the dependent sample t-test on the post-test and pre-test scores across different characteristics were as shown in Table 6. The differences between post-test and pre-test scores were statistically significant within the Southern attendees, North Attendees, the two zones combined, various designation groups irrespective of their zones.

Similar to the analysis shown in Table 6, we analyzed the paired (post-test and pre-test) differences among attendees from each state. The differences between post-test and pre-test scores were statistically significant (p<0.005) among the attendees from Abia, Adamawa, Bayelsa, Edo, Ekiti, Katsina, Lagos, Nassarawa, Niger, Ogun, Ondo, Oyo, Plateau, Taraba and Yobe states only. As shown in Table 7, most participants (51.5%) believed that the training sessions were very good.

Discussion

This study was aimed at evaluating the impact of five-day training on the quality of conduct of research survey of NARHS plus in order to answer the research question; does training of trainers at the central level have any impact on the level of knowledge and successful conduct of NARHS survey in Nigeria since there has been no recent systematic analyses of the quality and comprehensiveness of training received in any national survey in Nigeria.

The study revealed that the participants have positive attitudes about training (training attitudes), since the training afforded them the opportunity to acquire additional knowledge in basic issues concerning HIV/AIDS and its counseling and testing, and build data collection skills to be able to be comported to conduct interviews as well as manage a research study of this magnitude. It also afforded them the opportunity to learn about national requirements and regulations. It is clear from the training results that recent trainees feel very well prepared (well trained and competent) in many areas, particularly in HIV survey. Pre-test level of knowledge, supported by the improved post-test scores (North and South) is reasonable and indicates a right selection of methods and training participants.

For participants, the training afforded them the opportunity to acquire new knowledge and build skills to be able to conduct interviews as well as manage a research study of this magnitude. It also afforded them the opportunity to learn about national health survey requirements and regulations. Despite the relatively small number of respondents, this training provides the only current and detailed assessment of training of HIV survey in Nigeria across a wide spectrum of learning and content areas. Nevertheless, the results have potentially important implications for HIV surveillance and education in Nigeria. The train-the-trainer program was effective in developing sustainable quality NARHS plus in Nigeria as demonstrated by the fact that there is increase in knowledge base of the participants. In addition, the central training helped build a cadre of trainers who will be able to do an effective step down training throughout the states of the federation as seen in the result.

The recent evaluation of learning strategies used by United Nations Children's Fund (UNICEF) in resource-limited settings noted that training local professionals to train their colleagues is generally less expensive than sending national or international experts to conduct trainings^[9]. In addition, the use of local trainers to train their peers has the advantages of building local capacity as well as ensuring the trainings have cultural relevance and application which will help to enhance learning. Thus, it is likely that this central training model will continue to be applied to assist in ensuring that there is uniformity in knowledge impartation at the state levels. However, efforts will be made to mitigate differences in quality through use of competency-based curricula, well-designed training programs and, when needed, implementation of performance and quality improvement methodologies.

In another study^[10] on the effect of co-presenting training items during supervised classification learning of novel relational categories, in a test phase measuring learning and transfer, the comparison group significantly outperformed a control group receiving an equivalent training session of single-item classification learning. In a similar study on Family Physician (trainers and non-trainers) and their practices to see whether there were differences in trainers and non-trainers and in how their practices were organized and their services were delivered, Trainers scored higher on all but one of the items, and significantly higher on 47 items, of which 13 remained significant after correcting for covariates. Trainers (and training practices) provided more diagnostic and therapeutic services, made better use of team skills and scored higher on practice organization, chronic care services and quality management than non-training practices^[11].

Limitations

The qualities of training at the state levels were not evaluated, to ascertain the same level of delivery as the one at the central training. The result was only based on increase knowledge base of the participants after the central training. Further evaluations needed to be done whether the people trained at the various states acquire same level of improvements as seen at the central training. The paper has demonstrated that the trainees need training as indicated by the knowledge gained noted to be significant. However, the paper did not demonstrate how the knowledge gained has improved the conduct of the survey or the quality of data collected during the survey because the survey itself is yet to be concluded. For same reason, it could not relate the knowledge acquired during the trainings to the quality of data generated on the field from the southern and northern zones. This will be assessed after the conclusion and dissemination of survey findings nationwide.

Conclusions and Public Health Implications

This is the first report on the effect of a central training of trainers on the effective conduct of NARHS plus survey in Nigeria. However, it is important to follow-up on the training at the state level to see the impact of the knowledge gained at the central training and using the skills they have acquired. This information allows the organizers to determine future training needs, either by zones or otherwise. Results on the increase in knowledge base of this central training program in developing trainers are also significant, providing a basis of comparison for future programs. This finding is comparable to similar evaluations of TOT models, such as that conducted by UNICEF which found the TOT trainees going on to provide step down training to their colleagues.^[9]. Although this was a central program, the lessons learned -in terms of factors contributing to program success and the ways in which challenges were addressed -may be applicable in the implementation of any such training program in the future.

In conclusion, our evaluation of this central training program demonstrates that a TOT-based central training program can be successfully endorsed for an effective conduct of surveys in Nigeria, with the ability to rapidly scale-up human capacity for both service delivery and training in a sustainable fashion.