Does Pre-Survey Training Impact Knowledge of Survey Administrators and Survey Outcomes in Developing Countries? Evaluation Findings of a Training of Trainers Workshop for National AIDS and Reproductive Health Survey-Plus in Nigeria
Although, Nigeria had conducted various national surveys followed by central and state level trainings for survey administrators, prior pre-survey trainings have not been systematically evaluated to assess their impact on knowledge gain and final outcome of the survey. A central training of trainers' session was organized for master trainers on the conduct of the 2012 National AIDS and Reproductive Health Survey.
To evaluate the impact of training on the quality of conduct of a national research survey in the 36 states and the Federal Capital Territory in Nigeria.
A total of 185 participants consisting of State AIDS Program Coordinators, Reproductive Health Coordinators, State Laboratory Scientists, Lead Supervisors and Counselor Testers were invited from the 36 states in Nigeria and the FCT for the central training of trainers in Abuja. The training lasted 5 days and the trainees were grouped into two on the basis of behavioral epidemiology and laboratory components. Training tools such as the developed protocol, training power point slides, practical sessions such as role plays, and usage of HIV rapid test kits were utilized during the training. The facilitators were drawn from Federal Ministry of Health (FMoH), universities and research Institutions as well as Non-Governmental Organizations (NGOs). The facilitators prepared and administered 25 structured questions for the behavioral group and 28 questions for the laboratory group at the beginning of the training to assess the participants' knowledge of HIV and the survey. The same questions answered by Trainees responded to the same questions prior to the commencement and at the end of the trainings. Scores were aggregated to 100 for each test. We conducted paired t-test to determine statistically significant differences between pre-test and post-test results at 0.05 significance level and ANOVA to determine if there were differences in knowledge level among different groups.
The overall mean pre-test and post-test scores were 64.0% and 77.4% respectively indicating a 13.4% knowledge gain above what it was at the beginning of the training. The mean pre-test score and post-test score for the Southern states (SN) were 64.7% and 80.3% while that of the Northern states (NN) were 63.5% and 75.3% representing a knowledge gain of 15.6% and 11.8% respectively. There was statistical significant difference in the post-test scores between the two regions (p=0.001) and in knowledge gained after the training (p=0.017).
Conclusions and Public Health Implications:
Comparison between the pre test and post test scores at the 5-day training showed a significant gain in knowledge of participants. The survey training contributed positively to the preparation and building of knowledge needed for the conduct of 2012 NARHS-plus.
State AIDS Program Coordinators
Reproductive Health Coordinators
State Laboratory Scientists
Lead Supervisors and Counselor Testers
Any organization determined to survive and prosper in the current challenging economy, must understand the imperative to invest in training and professional development in order to improve efficiencies in production as well as to acquire the greatest return in investment of human capital. Furthermore, several authors have suggested that training is most extensive and important in any organization or establishment[2,3]. Training is becoming increasingly complex, and tools to determine the proficiency of training programs and trainees are needed. Although emphasis has been placed recently on expanding assessment and demonstrating outcomes in proficiencies other than knowledge, evaluation of the depth and breadth of trainings' knowledge base remains critically important, because this constitutes the foundation on which competence is built, and is an essential requisite for the development of sound reasoning skills.[4,5] Therefore, training as a tool to acquire best practices in the proficient conduct of services cannot be over emphasized.
National AIDS and Reproductive Health Survey (NARHS) had been conducted three times in Nigeria (2003,2005, and 2007)[6,7,8]. After 2005 NARHS survey, there was a need to include biomarker-HIV testing (biological) components into the survey and it captures ages 15-64 years, hence the nomenclature of NARHS-plus was therefore adopted in 2007. The 2012 NARHS was the fourth since its inception. In the past, NARHS and NARHS-plus data collection started with central level training of States AIDS Program Coordinator (SAPC), Reproductive Health Coordinator (RHC), State Laboratory Scientists (SLS), HIV Counselor-testers (CTs), and Lead supervisors but the impact of the trainings on the participants have not been evaluated to assess whether there is need for such trainings or not, during the survey. The training of trainers (TOTs) workshop was aimed at teaching selected survey administrators on how to conduct and supervise the NARHS plus survey based on the complexity and enormity of the study especially the ‘plus’ component. The goal was to replicate same training at the state level to other supervisors, interviewers, and counselor-testers who served as research assistants in the collection of data on the field during the survey exercise.
The training was conducted using the engagement of participants in plenary discussions, presentations, group discussions, role plays, and question and answer sessions. The various sessions exposed the participants to issues of community engagement and participation in research, basics of HIV and AIDS, challenges of ensuring informed consent, reproductive health indices in Nigeria, good clinical and laboratory practice and management, data monitoring and management in research. The group interactions afforded participants the opportunity to learn from one another's experience while the role plays simulated actual practical experiences in the field. The trainees were expected to give same training in their respective states. Based on the above, this evaluation study set out to assess the impact of five-day training on the quality of conduct of research survey, namely, NARHS-plus. The overall evaluation research questions were: does training of trainers at the central level have any impact on the level of knowledge that could predicate a successful conduct of NARHS-plus survey in Nigeria, and are there variations in the level of knowledge among participants from different states or different regions of the country?
This study was an evaluation of training using a cross sectional design with structured questionnaire pre and post training.
The target populations were all State AIDS Program Coordinator (SAPC), State Reproductive Health Coordinator (SRH), State Laboratory Scientist (SLS) from all 36 states and the Federal Capital Territory (FCT) in Nigeria. In addition there were one Lead Supervisor (LS) and one Counselor Tester (CT) from each state and FCT.
Total sampling of the target population was done giving 185 trainees invited for the central training. They were divided into two regions-North and South for convenience. Seventeen states from southern Nigeria had their training for 5 days followed by that for the FCT and 19 states from northern Nigeria for another 5 days.
The training was conducted for 5 days. The five days training of trainers (TOTs) workshop for NARHS -Plus 2012 survey for SAPCs, RHCs, SLS, CTs and lead supervisors from the 36 states of Nigeria and FCT was conducted between 27th September to 7th October, 2012. The training sessions took place in Maraba, Nassarawa State for the Southern States and in Abuja for the Northern States.
The training methods were didactic, role plays and hands-on. Training included the use of training tools such as the training protocol, training slides, and engagement in practical sessions on the use of HIV rapid test kits. At the venue of training, the trainees were grouped into two on the basis of behavioral epidemiology (SAPC, SRH and LS) and laboratory components (SLS and CT). The facilitators were drawn from the Federal Ministry of Health (FMOH), academic institutions as well as supporting organizations. Facilitators prepared 25 and 28 structured questions on knowledge of HIV and the survey for the behavioral and laboratory tract respectively, scores were aggregated to 100. Trainees answered the same set of questions before (pre-test) and after (post-test) the trainings.
We conducted descriptive analysis, used summary statistics, paired t-test at 0.05 significance level to determine statistically significant difference between pre-test and post-test performances of the trainees while ANOVA was used to determine the differences in knowledge level among different groups.
One hundred and seventy (91.9 %) trainees participated in the pre-test while 177 (95.6 %) attempted the post-test, 158 attempted both pre and post-test. Also, the number of trainees that participated in the pre-test, post-test and took both during the training for the Southern Nigeria (SN) were 72,82 & 68 and Northern Nigeria (NN) were 98,95 & 90 respectively. Combining all the respondents irrespective of their zones, the overall pre-test and post-test score were 64.0% and 77.4% respectively indicating a 13.4% gain in knowledge over the baseline knowledge level (Table 1) . The pre-test score and post-test score for the SN and NN were 64.7% and 80.3% respectively, and 63.5% and 75.3%. The knowledge gain was 15.6% and 11.8% for SN and NN respectively (Table 1).
|Regions||Pre-test (100)||Post-test (100)||Differences (100)|
|Northern States (n)||95||98||90|
|Southern States (n)||82||72||68|
Comparing the performances in the two sets of training, Table 2 shows that the baseline knowledge was not statistically different among trainees from SN and NN (p=0.44), however there was statistical significance in the posttest scores between the two regions (p=0.001) and in knowledge gained (p=0.017).
|Sum of Squares||df||Mean Square||F||Sig.|
|Pretest & Region||Between SN & NN
Within SN & NN
|Posttest & Region||Between SN & NN
Within SN & NN
|Knowledge gain & Region||Between SN & NN
Within SN & NN
Figure 1 shows the mean scores of the attendees from each state in the two tests as well as the differences between the scores. The highest mean scores for the pretest, posttest and their differences were recorded in Delta (87.9%), Akwa Ibom (74.0%), and Lagos (72.4%) states respectively while the lowest mean scores were from Kwara (54.1 %) -pretest, Sokoto (66.4%) -posttest and Sokoto (3.4%) knowledge gain. In Table 3, the scores in the pretest and the posttest were compared, the analysis of variability of the scores showed that the mean scores for the states were statistically significantly different (p<0.05) in both tests but there was no significant difference in knowledge gain of the attendees across all the states in the area of knowledge of HIV epidemiology, HIV counseling, AIDS-related issues, survey administration etc.
|Sum of Squares||df||Mean Square||F||Sig.|
|Pretest||Between the States||6357.561||36||176.599||1.792||.009|
|* State||Within the States||13798.482||140||98.561|
|Posttest||Between the States||6602.086||36||183.391||1.699||.016|
|* State||Within the States||14354.276||133||107.927|
|Knowledge gain||Between the States||4743.494||36||131.764||.951||.555|
|* State||Within the States||16768.020||121||138.579|
While the highest mean scores for the pre-test was in the Counselor Testers' group (67.2%); post-test-Supervisors' group (79.1%); and knowledge gain was recorded in SAPCs' group (17.6%). The lowest mean scores for the pre-test was SRHCs' group (60.4%), post test was the Laboratory Scientists' group (73.1%), and knowledge gain was also the Laboratory Scientists' group (8.0%). The highest knowledge gain was recorded among the SAPC (17.6%), followed by the SRHC (16.7%). The overall mean pre-test and post-test scores were 64.0% and 77.4% respectively indicating a 13.4% gain in knowledge above baseline knowledge. (See Figure 2 and Table 4.)
Although there was no statistically significant difference in the pretest and posttest mean scores of the five groups of attendees, the mean scores of knowledge gained across the five groups were statistically significantly different (p<0.05) (see Table 5).
|Sum of Squares||df||Mean Square||F||Sig.|
|Pre-test *||Between Designations||1055.952||4||263.988||2.377||.054|
|Post-test *||Between Designations||852.966||4||213.241||1.750||.141|
|Knowledge gain *||Between Designations||2198.709||4||549.677||4.355||.002|
In an attempt to explore the details of the significant knowledge gain obtained in Table 5, we carried out the dependent sample t-test on the post-test and pre-test scores across different characteristics were as shown in Table 6. The differences between post-test and pre-test scores were statistically significant within the Southern attendees, North Attendees, the two zones combined, various designation groups irrespective of their zones.
|Categories||Paired Differences||95% CI of the Difference||t||df||Sig.|
|North||Counselor/ Testers||7.776||1.568||4.45||1 1.102||4.957||16||0.000|
Similar to the analysis shown in Table 6, we analyzed the paired (post-test and pre-test) differences among attendees from each state. The differences between post-test and pre-test scores were statistically significant (p<0.005) among the attendees from Abia, Adamawa, Bayelsa, Edo, Ekiti, Katsina, Lagos, Nassarawa, Niger, Ogun, Ondo, Oyo, Plateau, Taraba and Yobe states only. As shown in Table 7, most participants (51.5%) believed that the training sessions were very good.
|s/n.||Item||Very Poor||Poor||Good||Very Good||Excellent|
|1.||Publicity for the training||3.0%||7.3%||27.4%||45.7%||16.5%|
|2.||Communication with participants prior to arrival||3.0%||7.8%||24.0%||41.3%||24.0%|
|3.||Preparation of participants for the training prior to arrival||3.0%||7.8%||31.9%||40.4%||16.9%|
|5.||Accommodation and other logistics||8.3% 1||6.7%||23.2%||31.0%||20.8%|
|6.||Hospitality||1.8% 1||1 1.7%||31.9%||33.7%||20.9%|
|7.||Responsiveness to logistic challenges||2.4% 1||0.7%||34.9%||43.8%||8.3%|
|9.||Training Materials||1.2% 1||3.2%||40.1%||36.5%||9.0%|
|10.||Quality of the training||0.0%||2.4%||18.3%||51.8%||27.4%|
|11.||Facilitation (Conduct, Performance, Time Keeping)||0.6%||0.6%||26.2%||51.2%||21.4%|
|12.||Sessions and their contents||0.0%||1.2%||23.5%||55.9%||19.4%|
|13.||Arrangement of sessions||0.0%||2.4%||27.6%||52.9%||17.1%|
|14.||Overall, how would you rate the training?||0.6%||4.1%||24.6%||51.5%||19.3%|
This study was aimed at evaluating the impact of five-day training on the quality of conduct of research survey of NARHS plus in order to answer the research question; does training of trainers at the central level have any impact on the level of knowledge and successful conduct of NARHS survey in Nigeria since there has been no recent systematic analyses of the quality and comprehensiveness of training received in any national survey in Nigeria.
The study revealed that the participants have positive attitudes about training (training attitudes), since the training afforded them the opportunity to acquire additional knowledge in basic issues concerning HIV/AIDS and its counseling and testing, and build data collection skills to be able to be comported to conduct interviews as well as manage a research study of this magnitude. It also afforded them the opportunity to learn about national requirements and regulations. It is clear from the training results that recent trainees feel very well prepared (well trained and competent) in many areas, particularly in HIV survey. Pre-test level of knowledge, supported by the improved post-test scores (North and South) is reasonable and indicates a right selection of methods and training participants.
For participants, the training afforded them the opportunity to acquire new knowledge and build skills to be able to conduct interviews as well as manage a research study of this magnitude. It also afforded them the opportunity to learn about national health survey requirements and regulations. Despite the relatively small number of respondents, this training provides the only current and detailed assessment of training of HIV survey in Nigeria across a wide spectrum of learning and content areas. Nevertheless, the results have potentially important implications for HIV surveillance and education in Nigeria. The train-the-trainer program was effective in developing sustainable quality NARHS plus in Nigeria as demonstrated by the fact that there is increase in knowledge base of the participants. In addition, the central training helped build a cadre of trainers who will be able to do an effective step down training throughout the states of the federation as seen in the result.
The recent evaluation of learning strategies used by United Nations Children's Fund (UNICEF) in resource-limited settings noted that training local professionals to train their colleagues is generally less expensive than sending national or international experts to conduct trainings. In addition, the use of local trainers to train their peers has the advantages of building local capacity as well as ensuring the trainings have cultural relevance and application which will help to enhance learning. Thus, it is likely that this central training model will continue to be applied to assist in ensuring that there is uniformity in knowledge impartation at the state levels. However, efforts will be made to mitigate differences in quality through use of competency-based curricula, well-designed training programs and, when needed, implementation of performance and quality improvement methodologies.
In another study on the effect of co-presenting training items during supervised classification learning of novel relational categories, in a test phase measuring learning and transfer, the comparison group significantly outperformed a control group receiving an equivalent training session of single-item classification learning. In a similar study on Family Physician (trainers and non-trainers) and their practices to see whether there were differences in trainers and non-trainers and in how their practices were organized and their services were delivered, Trainers scored higher on all but one of the items, and significantly higher on 47 items, of which 13 remained significant after correcting for covariates. Trainers (and training practices) provided more diagnostic and therapeutic services, made better use of team skills and scored higher on practice organization, chronic care services and quality management than non-training practices.
The qualities of training at the state levels were not evaluated, to ascertain the same level of delivery as the one at the central training. The result was only based on increase knowledge base of the participants after the central training. Further evaluations needed to be done whether the people trained at the various states acquire same level of improvements as seen at the central training. The paper has demonstrated that the trainees need training as indicated by the knowledge gained noted to be significant. However, the paper did not demonstrate how the knowledge gained has improved the conduct of the survey or the quality of data collected during the survey because the survey itself is yet to be concluded. For same reason, it could not relate the knowledge acquired during the trainings to the quality of data generated on the field from the southern and northern zones. This will be assessed after the conclusion and dissemination of survey findings nationwide.
Conclusions and Public Health Implications
This is the first report on the effect of a central training of trainers on the effective conduct of NARHS plus survey in Nigeria. However, it is important to follow-up on the training at the state level to see the impact of the knowledge gained at the central training and using the skills they have acquired. This information allows the organizers to determine future training needs, either by zones or otherwise. Results on the increase in knowledge base of this central training program in developing trainers are also significant, providing a basis of comparison for future programs. This finding is comparable to similar evaluations of TOT models, such as that conducted by UNICEF which found the TOT trainees going on to provide step down training to their colleagues.. Although this was a central program, the lessons learned -in terms of factors contributing to program success and the ways in which challenges were addressed -may be applicable in the implementation of any such training program in the future.
In conclusion, our evaluation of this central training program demonstrates that a TOT-based central training program can be successfully endorsed for an effective conduct of surveys in Nigeria, with the ability to rapidly scale-up human capacity for both service delivery and training in a sustainable fashion.
- Accreditation Council for Graduate Medical Education: Outcome Project. (accessed )
- [Google Scholar]
- Using written examinations to assess medical knowledge and its application, in. In: Holmboe ES, Hawkins RE, eds. Practical Guide to the Evaluation of Clinical Competence 2008. p. :42-59.
- [Google Scholar]
- National HIV/ AIDS and Reproductive Health survey Technical Reports 2003. FMoH, Nigeria
- [Google Scholar]
- Federal Ministry of Health (FMoH): National HIV/ AIDS and Reproductive Health survey Technical Reports 2005, FMoH, Nigeria.
- [Google Scholar]
- Federal Ministry of Health (FMoH): National HIV/ AIDS and Reproductive Health survey Technical Reports 2007, FMoH, Nigeria.
- [Google Scholar]
- United Nations Children's Emergency Fund (UNICEF): Evaluation of UNICEF learning strategy to strengthen staff competencies for humanitarian response, 2000-2004. UNICEF 2005
- [Google Scholar]