|Year : 2019 | Volume
| Issue : 4 | Page : 208-213
Pre-test and post-test resuscitation assessment among healthcare workers at Dubai health authority
Zulfiqar Ali1, Bindhu Misbahudeen2, Mohammed Anas Mohtasham1, KT Mohammed Fasil1
1 Emergency and Trauma Center, Rashid Hospital, Dubai, UAE
2 Department Medical Education, Dubai Health Authority, Dubai, UAE
|Date of Submission||28-Apr-2019|
|Date of Acceptance||24-Jun-2019|
|Date of Web Publication||11-Nov-2019|
P O Box 4545, Emergency and Trauma Center, Rashid Hospital, Dubai
Source of Support: None, Conflict of Interest: None
Background: Understanding the concept of resuscitation is extremely important in the professional life of all healthcare workers. By achieving the appropriate resuscitation knowledge and psychomotor skills, the efficacy of performance can be enhanced in real-time situations. The level of resuscitation knowledge differs among healthcare workers, and inadequate knowledge raises a considerable concern about the effectiveness when it is performed in real life. Aims: The aim of the study is to assess the changes in pre-intervention and post-intervention knowledge (pre-test and post-test scores) once the individuals are exposed to an intervention (resuscitation training). Methods: We used a pre-test-post-test design that compares different participant groups and measures the degree of change occurring as a result of intervention. Data from all the individuals (n = 4028) were scrutinised and exposed to statistical analysis. Results: We assessed the pre-test and post-test resuscitation knowledge according to type of organisation (Dubai Health Authority [DHA] or non-DHA), type of intervention (advance cardiac life support [ACLS], pediatric advance life support [PALS] or pre-hospital trauma life support [PHTLS]) and the specialty (physicians, nurses, paramedics and others). We found that the DHA staff gained better resuscitation knowledge post-intervention (P = 0.00) as compared to non-DHA staff. ACLS course (P = 0.00) is found to be a better intervention as compared to PALS (P = 0.78) and PHTLS (P = 0.41). Physicians and nurses achieved better pre-test and post-test scores as compared to paramedics and other disciplines. The overall improvement in resuscitation knowledge (pre-test 85 ± 12 vs. post-test 87 ± 9) is marginal, but the impact of intervention (ACLS, PALS and PHTLS courses) is statistically significant (P = 0.00). Conclusions: The study revealed statistically significant overall post-training improvement in resuscitation knowledge for most of the healthcare workers involved in or expected to perform resuscitation at their workplace. Life support training raises the bar of resuscitation knowledge of many DHA staff from an average to excellent level.
Keywords: Assessment, intervention, post-test, pre-test, resuscitation
|How to cite this article:|
Ali Z, Misbahudeen B, Mohtasham MA, Mohammed Fasil K T. Pre-test and post-test resuscitation assessment among healthcare workers at Dubai health authority. Hamdan Med J 2019;12:208-13
|How to cite this URL:|
Ali Z, Misbahudeen B, Mohtasham MA, Mohammed Fasil K T. Pre-test and post-test resuscitation assessment among healthcare workers at Dubai health authority. Hamdan Med J [serial online] 2019 [cited 2020 Jan 25];12:208-13. Available from: http://www.hamdanjournal.org/text.asp?2019/12/4/208/270677
| Introduction|| |
Assessment fills the gaps between instructions and the curriculum and brings uniformity to the curriculum. The bicycle is a useful model when considering assessment and teaching. Teaching and learning are represented by front wheel, assessment by the rear wheel. Disasters may occur when the two wheels go in different directions. Therefore, teaching should be complimented by assessment to achieve competencies and learning. Assessment can be either summative or formative. Summative means that the assessment has been conducted for decision-making or certification purposes, such as deciding who is admitted, progresses or qualifies. Formative assessment relates to the feedback function of assessment or, more precisely, how the assessment informs the students about their performance. Assessment optimises the capabilities of all learners and practitioners by providing motivation and direction for future learning. Assessment protects the public by identifying incompetent physicians. It ensures that only those who have attained a minimum level of competence are entrusted with task of providing health care. It also provides feedback regarding what they know and what they do not know or assessment of ignorance. It can reinforce students' intrinsic motivation to learn and inspire them to set higher standards for themselves. At the end of the course or training, assessment is needed to certify the competency.
Cardiopulmonary resuscitation (CPR) and trauma management are critical components in the professional training of all healthcare professionals. Resuscitation competency is defined as having the cognitive knowledge and psychomotor skills that are necessary for the effective performance during cardiac arrest  and life-threatening trauma situations. The resuscitation knowledge and skills differ among healthcare across the globe. The poor knowledge of resuscitation raises considerable concern about the effectiveness when performed in real life. Ragavan et al. demonstrated that the level of knowledge and skills in basic resuscitation among medical practitioners in public hospitals were poor and that resuscitation experience without training was not found to be beneficial. A systematic review on resuscitation training in developing countries revealed that cognitive knowledge was higher among physicians than among nurses or students. Many a times, the healthcare professional fail the pre-test, but their knowledge and skills improve after training. The significantly low levels of resuscitation skills can be addressed by instituting appropriate training and regular refresher courses.
It is a standard protocol with resuscitation training courses, namely advance cardiac life support (ACLS), pediatric advance life support (PALS) and pre-hospital trauma life support (PHTLS) that the participants attempt a written pre-test (pre-intervention) at home followed by 2 days of strenuous hands-on training (intervention) and a post-test written examination (post-intervention) immediately once the training is over. We want to test whether this training or intervention is bringing any impact in the resuscitation knowledge among healthcare workers. The main purpose of the study is to look at the change from the pre-test and post-test scores by comparing the scores achieved before and after the intervention or training.
| Methods|| |
The study was conducted at Dubai Health Authority (DHA) training centre and the training sites affiliated. The DHA training centre is chosen because it is a one of the 'Reference Center' in providing quality resuscitation training in Middle East and North Africa region. An ethical approval was assured from the chairperson Rashid Hospital Research Committee Dubai, CEO of Rashid Hospital Dubai and Head of Medical Education Department DHA. Participation was voluntary, and informed written consent was taken from participants during the course evaluation process. It was also explained to all the participants that they have the right to decline or withdraw from the study at any stage without incurring any penalty. The research was derived from the hypothesis 'Does pre-test and post-test assessment improve resuscitation knowledge among healthcare workers'? The data for pre-test and post-test scores were gathered and analysed for 4028 healthcare professionals enrolled for resuscitation training from January 2010 to December 2016. All data was coded and stayed locked up, accessible only to the researchers, training centre coordinator and the statistician.
An observational, quantitative retrospective study involving pre-test, intervention and post-test time-series research design (O X O) – A before-and-after, experimental design (each 'O' represents a measurement [in this case, the test] and the 'X' represents the introduction of the intervention [resuscitation training]). The term 'Pre' refers to a measurement being made before an intervention is introduced to a group and 'Post' refers to a measurement being made after its introduction. In a pre-test-post-test design, the measurements are taken both before and after an intervention. The design helps to see the effects of some type of intervention on a group. We conducted time series design involving many pre-test and post-test observations where a group was given a written test of resuscitation knowledge before the training and the score was recorded. This was followed by an extensive 2 days of training; the same group had repeat but different written test and new score was calculated.
The objective of the study are:
- To assess the baseline resuscitation knowledge of healthcare workers and
- To identify any change in resuscitation knowledge after intervention by comparing pre-test and post-test scores.
- Healthcare professionals enrolled from January 2010 to December 2016
- Healthcare professionals enrolled for Resuscitation courses (ACLS, PALS and PHTLS).
- Unable to show pre-test scores before attending classroom training.
Several methods of assessment have been developed to estimate quantity and quality of performance. We assessed all the participants with a multiple choice pre-test questionnaire as pre-test, followed by classroom training courses, and the post-test afterwards. The assessment was in accordance with American heart association (AHA)'s guidelines  for ACLS and PALS courses and national association of emergency medical technicians (NAEMT) guidelines  for PHTLS course. The number of question asked to participants in both pre-test and post-test are ACLS (33 items), PALS (50 items) and PHTLS (50 items). There is no pass or fail marks for pre-test, but a post-test pass mark for each discipline is 84% as recommended by AHA and NAEMT. In all courses, two CPR-qualified evaluators assessed each participant's scores during the pre-test as well as during the post-test conducted immediately after the CPR training. The content validity of the questionnaire for each discipline was enhanced by following the guidelines of CPR specified by the AHA and NAEMT.
All the participants received preparatory study material for the respective training course (ACLS, PALS or PHTLS) at least 2 weeks before classroom training. A pre-test examination was also included with the study material. It was made mandatory that each participant has to answer written pre-test before coming to classroom training. The attempted pre-test score was collected and recorded by course coordinators. Then, all the participants underwent strenuous 2-day classroom training for a specific course on specific dates. Once the training was over, the participants were subjected to post-test written examination. The marks obtained in post-test were collected and recorded against the pre-test score. The pre-test and post-test written examinations were collected from all the participants enrolled in these courses. We ensured that before and after measurements are carried out using the same methodology to avoid instrumentation or reporting threats.
| Results|| |
Data were exposed to strict statistical analysis using SPSS version 21 software. The results were compiled and compared between pre-test and post-test scores among three variables that include type of organisation, type of intervention and specialty. The disposition of the entire study participant (n = 4023) is shown in [Table 1] where 83% (n = 3333) participants are from DHA and 17% (n = 695) from non-DHA facilities. The number of participants for three different types of the courses tested for pre-test and post-test include ACLS 50% (n = 2020), PALS 36% (n = 1442) and PHTLS 14% (n = 566), respectively. The healthcare workers completed the pre-test and post-test written examination that includes physicians 42% (n = 1685), nursing staff 54% (n = 2183), paramedics 2% (n = 91) and other disciplines 2% (n = 69), respectively, as shown in [Table 1].
|Table 1: Demographic data according to the type of organisation, type of intervention and specialty|
Click here to view
[Table 2] and [Figure 1] compare the mean scores between pre-test, i.e., pre-intervention (85 ± 12) and post-test, i.e., post-intervention (87 ± 9) with P = 0.00 and paired mean difference of 2. It is worth noting that all those subjected to pre-test (n = 4028) were also exposed to post-test (n = 4028) after an appropriate training intervention.
|Figure 1: Assessment of resuscitation knowledge by comparing pre-test and post-test scores|
Click here to view
[Table 3] and [Figure 2] depict the comparison between pre-intervention and post-intervention scores on a Likert scale ranging 0%–100% that are interpreted as: score 0%–20% very poor; 21%–40% poor; 41%–60% average; 61%–80% above average; and 81%–100% excellent.
|Table 3: Assessment of resuscitation knowledge by comparing pre-test and post-test scores|
Click here to view
|Figure 2: Assessment of resuscitation knowledge by comparing pre-test and post-test scores|
Click here to view
All the scores of three variables, i.e., type of organisation, type of intervention and specialty with their respective components are elaborated in [Table 4] and [Figure 3]. All three variables with their components or sub-variables are further compared for various values that include the number of participants, pre-intervention mean, post-intervention mean, paired mean difference, t value and P value.
| Discussion|| |
Assessment is an integral component and one of the pillars of learning cycle. Therefore, assessment results are not only benefiting all the stakeholders in achieving more learning but also bridge any gaps identified between intended learning outcomes and the actual achievements of the learners. Assessment results can be viewed for future improvement and should be designed prospectively along with learning outcomes. Assessment should not be viewed as an external measurement of the results of the educational process but more as an integral part of the process.
We assessed the resuscitation knowledge among various healthcare workers (physicians, nurses, paramedics and others) and organisations (DHA or non-DHA) by comparing pre-test and post-test scores with one of the interventions (ACLS, PALS or PHTLS).
Type of organisation
In this study, we assessed the pre-test and post-test scores of DHA staff n = 3333 (83%) versus non-DHA staff n = 695 (17%) as seen in [Table 1]. The difference in numbers of study participants is largely due to the fact that the DHA training centre and sites offer training to its own employees, though the facility is open to private sector as well. [Table 4] compares in detail the pre-test and post-test scores between DHA staff (86.0 ± 11.0 vs. 87.0 ± 9.0; mean difference 1; P = 0.00) and non-DHA staff (86.0 ± 11.0 82.5 ± 11; mean paired difference 0.63; P = 0.24). The results show that the DHA staff gained better resuscitation knowledge post-intervention that is statistically significant (P = 0.00). As DHA training facilities are affiliated with hospitals and government medical centres, the physicians n = 1685 (42%) and nurses n = 2183 (54%) make the bulk of participants. The paramedics and other disciplines, being non-DHA entity are therefore less in number n = 91 (2%) and n = 69 (2%), respectively.
Type of intervention
We assess three interventions (ACLS, PALS and PHTLS) in our research. The two ACLS and PALS are mostly recommended for physicians and nurses as compared to PHTLS, a common training discipline for paramedics. [Table 4] and [Figure 3] reveal that the pre-test (85.0 ± 12.0) and post-test (87.6 ± 8.0) scores for ACLS are statistically significant (P = 0.00) as compared to pre-test (88.0 ± 9.0) and post-test (88.0 ± 8.0) scores for PALS (P = 0.78). The pre-test (76.0 ± 13.0) and post-test (76.0 ± 10.0) scores for PHTLS show that the gain in knowledge is also statistically insignificant (P = 0.41). Among all three training programs, the ACLS proves a best intervention in knowledge gain (paired mean difference 2.6; t = 9.0; P = 0.00). The highest CPR knowledge scores seen for ACLS are due to the fact that (1) the ACLS is a common and mandatory course; (2) it is for adult patients; and (3) recommended for in-hospital and out-of-hospital settings and for most of the healthcare workers. On the contrary, the PHTLS course is mandatory for pre-hospital staff and paramedics but is also recommended for some of the hospital staff as well, for example, those working in emergency departments and trauma units.
There is difference in scores for different healthcare workers when categorised by the specialty as shown in [Table 4]. Physicians achieved the highest pre-test (86.0 ± 11.0) and post-test scores (88.5 ± 8.0) with mean paired difference of 2.14. The highest scores achieved by physicians may reflect their diverse clinical exposure and experience and statistically significant improvement post-training (P = 0.00). Munezero et al. assessed the post-intervention improvement in CPR knowledge and skills with similar results, but that study was limited to nurses only. Similarly, the pre-test and post-test scores for nurses (84.0 ± 12.0 vs. 85.0 ± 10.0; mean difference 0.96) are also significant (P = 0.00). However, if we look into pre-test and post-test scores of paramedics (83.0 ± 11.0 vs. 80.0 ± 12.0; mean paired difference − 3.18), this shows a reduction in the post-intervention score compared to the pre-intervention score and that the reduction was statistically significant (P = 0.02). This signifies that the intervention received by the paramedic staff was either not effective or some other administrative issues that need to be explored. The pre-test and post-test scores of other healthcare workers (81.0 ± 12.0 vs. 82.0 ± 11.0; mean difference 0.90) are also clinically insignificant (P = 0.60) and shows that the intervention did not bring any noticeable gain in knowledge. The other staff that include healthcare educators and administrators struggle to achieve better post-test that might be due to less exposure to clinical environment.
Interpretation of pre-test and post-test scores by Likert scale
The pre-test scores derived from data (n = 4028) according to Likert criteria are shown in [Table 3], and [Figure 2] includes very poor 0% (n = 1); poor 0.4% (n = 16); average 5% (n = 192); above average 22.0% (n = 883) and excellent 73.0% (n = 2936). When we applied the same Likert scale, the post-test scores derived are very poor 0% (n = 0); poor 0.1% (n = 3), average 2% (n = 79); above average 20.6% (n = 829) and excellent 77.4% (n = 3117). It is worth to note that the pre-test and post-test 'Excellent' category scores improve from 73.0% (n = 2936) to 77.4% (n = 3117) after intervention which is significant. In addition, the pre-test and post-test scores for 'Above Average' group reduced n = 22.0% (n = 883) versus 20.6% (n = 829) which indicate that there is transfer of scores from 'Above Average' group to 'Excellent' category.
Overall knowledge gain
Although the difference between overall pre-test and post-test scores (85 ± 12 vs. 87 ± 9) is marginal as shown in [Table 2], the impact of intervention (ACLS, PALS and PHTLS training courses) in improving the overall knowledge among healthcare workers is statistically significant (P = 0.00).
Our study recruited large number of individuals (n = 4028) and the findings can be generalised. Pre-test interventions are used to assess the baseline knowledge, the intervention builds on existing knowledge to bridge the gap and the post-test claims the final gain in knowledge. The improvement in resuscitation knowledge may therefore a key factor for the survival of cardiac arrest victims.
The before-and-after design is most useful in demonstrating the immediate impacts of short term programs (ACLS, PALS and PHTLS). It is less useful for evaluating longer term interventions. This is because over the course of a longer period of time, more circumstances can arise that may obscure the effects of an intervention. These circumstances are collectively called threats to internal validity. In addition, there is inherent vulnerability of a before-and-after design to internal validity threats, especially for long-term evaluation periods. A quasi-experimental design can be an alternative study design to overcome such internal validity threats.
Pre-test scores are given 2 weeks before intervention along with preparatory manual of respective course. As pre-test was completed at home or outside training center in non-examination environment, participants tend to score high. This behavior at times might a limited factor in assessing the effectiveness of intervention in the form of post-test scores.
| Conclusions|| |
The study revealed statistically significant improvement (paired mean difference 2%; P = 0.00) in the resuscitation knowledge for all healthcare workers (n = 4028) post training. Among healthcare workers, the physicians and nurses showed statistically significant improvement from pre-test to post-test scores, but there was a reduction in the post-intervention score among paramedics and other healthcare professionals. The results of our study can be generalised, and the findings can be applied to large and multiple groups of healthcare professionals involved in resuscitation and emergency care of the patients.
We acknowledge the contribution of Medical Education Department, Dubai health authority (DHA) for providing all the related data to conduct this research.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Amin Z, Sen C, Eng H. Practical Guide to Medical Student Assessment. 2nd
ed. Singapore: World Scientific Printers; 2010. p. 215-21.
Fraser SW, Greenhalgh T. Coping with complexity: Educating for capability. BMJ 2001;323:799-803.
Swanwick T. Understanding Medical Education: Evidence, Theory and Practice. 2nd
ed. Wiley-Blackwell; 2013. p. 520.
Epstein RM. Assessment in medical education. N Engl J Med 2007;356:387-96.
Norton LS, Tilley AJ, Newstead SE. Franklyn-Stokes BA. The pressures of assessment in undergraduate courses and their effect on student behaviours. Assess Eval High Educ 2001;26:269-84.
Dent JA, Harden RM. A Practical Guide for Medical Teachers. 1st
ed. Elsevier; 2001. p. 281-90.
Broomfield R. A quasi-experimental research to investigate the retention of basic cardiopulmonary resuscitation skills and knowledge by qualified nurses following a course in professional development. J Adv Nurs 1996;23:1016-23.
Einspruch EL, Lynch B, Aufderheide TP, Nichol G, Becker L. Retention of CPR skills learned in a traditional AHA heartsaver course versus 30-min video self-training: A controlled randomized study. Resuscitation 2007;74:476-86.
Botha L, Geyser MM, Engelbrecht A. Knowledge of cardiopulmonary resuscitation of clinicians at a South African tertiary hospital. S Afr Fam Pract 2012;54:447-54.
Ragavan S, Schneider H, Kloeck WG. Basic resuscitation – Knowledge and skills of full-time medical practitioners at public hospitals in Northern Province. S Afr Med J 2000;90:504-8.
Meaney PA, Sutton RM, Tsima B, Steenhoff AP, Shilkofski N, Boulet JR, et al.
Training hospital providers in basic CPR skills in Botswana: Acquisition, retention and impact of novel training techniques. Resuscitation 2012;83:1484-90.
Rajeswaran L, Ehlers VJ. Cardiopulmonary resuscitation knowledge and skills of registered nurses in Botswana. Curationis 2014;37:e1-7.
Eccles M, Grimshaw J, Campbell M, Ramsay C. Research designs for studies evaluating the effectiveness of change and improvement strategies. Qual Saf Health Care 2003;12:47-52.
Linn RL, Gronlund NE. The role of measurement and assessment in teaching. In: Measurement and Assessment in Teaching. 8th
ed. New Jersey: Prentice Hall Publications; 2000. p. 33-4.
Norcini JJ. Current perspectives in assessment: The assessment of performance at work. Med Educ 2005;39:880-9.
Munezero JB, Atuhaire C, Groves S, Cumber SN. Assessment of nurses knowledge and skills following cardiopulmonary resuscitation training at Mbarara regional referral hospital, Uganda. Pan Afr Med J 2018;30:108.
[Figure 1], [Figure 2], [Figure 3]
[Table 1], [Table 2], [Table 3], [Table 4]