• Users Online: 373
  • Print this page
  • Email this page


 
 
Table of Contents
ORIGINAL ARTICLE
Year : 2019  |  Volume : 12  |  Issue : 4  |  Page : 208-213

Pre-test and post-test resuscitation assessment among healthcare workers at Dubai health authority


1 Emergency and Trauma Center, Rashid Hospital, Dubai, UAE
2 Department Medical Education, Dubai Health Authority, Dubai, UAE

Date of Submission28-Apr-2019
Date of Acceptance24-Jun-2019
Date of Web Publication11-Nov-2019

Correspondence Address:
Zulfiqar Ali
P O Box 4545, Emergency and Trauma Center, Rashid Hospital, Dubai
UAE
Login to access the Email id

Source of Support: None, Conflict of Interest: None


DOI: 10.4103/HMJ.HMJ_34_19

Rights and Permissions
  Abstract 


Background: Understanding the concept of resuscitation is extremely important in the professional life of all healthcare workers. By achieving the appropriate resuscitation knowledge and psychomotor skills, the efficacy of performance can be enhanced in real-time situations. The level of resuscitation knowledge differs among healthcare workers, and inadequate knowledge raises a considerable concern about the effectiveness when it is performed in real life. Aims: The aim of the study is to assess the changes in pre-intervention and post-intervention knowledge (pre-test and post-test scores) once the individuals are exposed to an intervention (resuscitation training). Methods: We used a pre-test-post-test design that compares different participant groups and measures the degree of change occurring as a result of intervention. Data from all the individuals (n = 4028) were scrutinised and exposed to statistical analysis. Results: We assessed the pre-test and post-test resuscitation knowledge according to type of organisation (Dubai Health Authority [DHA] or non-DHA), type of intervention (advance cardiac life support [ACLS], pediatric advance life support [PALS] or pre-hospital trauma life support [PHTLS]) and the specialty (physicians, nurses, paramedics and others). We found that the DHA staff gained better resuscitation knowledge post-intervention (P = 0.00) as compared to non-DHA staff. ACLS course (P = 0.00) is found to be a better intervention as compared to PALS (P = 0.78) and PHTLS (P = 0.41). Physicians and nurses achieved better pre-test and post-test scores as compared to paramedics and other disciplines. The overall improvement in resuscitation knowledge (pre-test 85 ± 12 vs. post-test 87 ± 9) is marginal, but the impact of intervention (ACLS, PALS and PHTLS courses) is statistically significant (P = 0.00). Conclusions: The study revealed statistically significant overall post-training improvement in resuscitation knowledge for most of the healthcare workers involved in or expected to perform resuscitation at their workplace. Life support training raises the bar of resuscitation knowledge of many DHA staff from an average to excellent level.

Keywords: Assessment, intervention, post-test, pre-test, resuscitation


How to cite this article:
Ali Z, Misbahudeen B, Mohtasham MA, Mohammed Fasil K T. Pre-test and post-test resuscitation assessment among healthcare workers at Dubai health authority. Hamdan Med J 2019;12:208-13

How to cite this URL:
Ali Z, Misbahudeen B, Mohtasham MA, Mohammed Fasil K T. Pre-test and post-test resuscitation assessment among healthcare workers at Dubai health authority. Hamdan Med J [serial online] 2019 [cited 2019 Dec 9];12:208-13. Available from: http://www.hamdanjournal.org/text.asp?2019/12/4/208/270677




  Introduction Top


Assessment fills the gaps between instructions and the curriculum and brings uniformity to the curriculum.[1] The bicycle is a useful model when considering assessment and teaching. Teaching and learning are represented by front wheel, assessment by the rear wheel. Disasters may occur when the two wheels go in different directions. Therefore, teaching should be complimented by assessment to achieve competencies and learning.[2] Assessment can be either summative or formative. Summative means that the assessment has been conducted for decision-making or certification purposes, such as deciding who is admitted, progresses or qualifies. Formative assessment relates to the feedback function of assessment or, more precisely, how the assessment informs the students about their performance.[3] Assessment optimises the capabilities of all learners and practitioners by providing motivation and direction for future learning.[4] Assessment protects the public by identifying incompetent physicians. It ensures that only those who have attained a minimum level of competence are entrusted with task of providing health care. It also provides feedback regarding what they know and what they do not know or assessment of ignorance. It can reinforce students' intrinsic motivation to learn and inspire them to set higher standards for themselves.[5] At the end of the course or training, assessment is needed to certify the competency.[6]

Cardiopulmonary resuscitation (CPR) and trauma management are critical components in the professional training of all healthcare professionals. Resuscitation competency is defined as having the cognitive knowledge and psychomotor skills that are necessary for the effective performance during cardiac arrest [7] and life-threatening trauma situations. The resuscitation knowledge and skills differ among healthcare across the globe.[8] The poor knowledge of resuscitation raises considerable concern about the effectiveness when performed in real life.[9] Ragavan et al. demonstrated that the level of knowledge and skills in basic resuscitation among medical practitioners in public hospitals were poor and that resuscitation experience without training was not found to be beneficial.[10] A systematic review on resuscitation training in developing countries revealed that cognitive knowledge was higher among physicians than among nurses or students.[11] Many a times, the healthcare professional fail the pre-test, but their knowledge and skills improve after training. The significantly low levels of resuscitation skills can be addressed by instituting appropriate training and regular refresher courses.[12]

It is a standard protocol with resuscitation training courses, namely advance cardiac life support (ACLS), pediatric advance life support (PALS) and pre-hospital trauma life support (PHTLS) that the participants attempt a written pre-test (pre-intervention) at home followed by 2 days of strenuous hands-on training (intervention) and a post-test written examination (post-intervention) immediately once the training is over. We want to test whether this training or intervention is bringing any impact in the resuscitation knowledge among healthcare workers. The main purpose of the study is to look at the change from the pre-test and post-test scores by comparing the scores achieved before and after the intervention or training.


  Methods Top


The study was conducted at Dubai Health Authority (DHA) training centre and the training sites affiliated. The DHA training centre is chosen because it is a one of the 'Reference Center' in providing quality resuscitation training in Middle East and North Africa region. An ethical approval was assured from the chairperson Rashid Hospital Research Committee Dubai, CEO of Rashid Hospital Dubai and Head of Medical Education Department DHA. Participation was voluntary, and informed written consent was taken from participants during the course evaluation process. It was also explained to all the participants that they have the right to decline or withdraw from the study at any stage without incurring any penalty. The research was derived from the hypothesis 'Does pre-test and post-test assessment improve resuscitation knowledge among healthcare workers'? The data for pre-test and post-test scores were gathered and analysed for 4028 healthcare professionals enrolled for resuscitation training from January 2010 to December 2016. All data was coded and stayed locked up, accessible only to the researchers, training centre coordinator and the statistician.

Design

An observational, quantitative retrospective study involving pre-test, intervention and post-test time-series research design (O X O) – A before-and-after, experimental design (each 'O' represents a measurement [in this case, the test] and the 'X' represents the introduction of the intervention [resuscitation training]). The term 'Pre' refers to a measurement being made before an intervention is introduced to a group and 'Post' refers to a measurement being made after its introduction. In a pre-test-post-test design, the measurements are taken both before and after an intervention. The design helps to see the effects of some type of intervention on a group. We conducted time series design involving many pre-test and post-test observations where a group was given a written test of resuscitation knowledge before the training and the score was recorded. This was followed by an extensive 2 days of training; the same group had repeat but different written test and new score was calculated.

Objectives

The objective of the study are:

  1. To assess the baseline resuscitation knowledge of healthcare workers and
  2. To identify any change in resuscitation knowledge after intervention by comparing pre-test and post-test scores.


Eligibility

Inclusion criteria

  1. Healthcare professionals enrolled from January 2010 to December 2016
  2. Healthcare professionals enrolled for Resuscitation courses (ACLS, PALS and PHTLS).


Exclusion criteria

  1. Absentees
  2. Unable to show pre-test scores before attending classroom training.


Procedure

Several methods of assessment have been developed to estimate quantity and quality of performance.[13] We assessed all the participants with a multiple choice pre-test questionnaire as pre-test, followed by classroom training courses, and the post-test afterwards. The assessment was in accordance with American heart association (AHA)'s guidelines [14] for ACLS and PALS courses and national association of emergency medical technicians (NAEMT) guidelines [15] for PHTLS course. The number of question asked to participants in both pre-test and post-test are ACLS (33 items), PALS (50 items) and PHTLS (50 items). There is no pass or fail marks for pre-test, but a post-test pass mark for each discipline is 84% as recommended by AHA and NAEMT. In all courses, two CPR-qualified evaluators assessed each participant's scores during the pre-test as well as during the post-test conducted immediately after the CPR training. The content validity of the questionnaire for each discipline was enhanced by following the guidelines of CPR specified by the AHA and NAEMT.

Data collection

All the participants received preparatory study material for the respective training course (ACLS, PALS or PHTLS) at least 2 weeks before classroom training. A pre-test examination was also included with the study material. It was made mandatory that each participant has to answer written pre-test before coming to classroom training. The attempted pre-test score was collected and recorded by course coordinators. Then, all the participants underwent strenuous 2-day classroom training for a specific course on specific dates. Once the training was over, the participants were subjected to post-test written examination. The marks obtained in post-test were collected and recorded against the pre-test score. The pre-test and post-test written examinations were collected from all the participants enrolled in these courses. We ensured that before and after measurements are carried out using the same methodology to avoid instrumentation or reporting threats.


  Results Top


Data were exposed to strict statistical analysis using SPSS version 21 software. The results were compiled and compared between pre-test and post-test scores among three variables that include type of organisation, type of intervention and specialty. The disposition of the entire study participant (n = 4023) is shown in [Table 1] where 83% (n = 3333) participants are from DHA and 17% (n = 695) from non-DHA facilities. The number of participants for three different types of the courses tested for pre-test and post-test include ACLS 50% (n = 2020), PALS 36% (n = 1442) and PHTLS 14% (n = 566), respectively. The healthcare workers completed the pre-test and post-test written examination that includes physicians 42% (n = 1685), nursing staff 54% (n = 2183), paramedics 2% (n = 91) and other disciplines 2% (n = 69), respectively, as shown in [Table 1].
Table 1: Demographic data according to the type of organisation, type of intervention and specialty

Click here to view


[Table 2] and [Figure 1] compare the mean scores between pre-test, i.e., pre-intervention (85 ± 12) and post-test, i.e., post-intervention (87 ± 9) with P = 0.00 and paired mean difference of 2. It is worth noting that all those subjected to pre-test (n = 4028) were also exposed to post-test (n = 4028) after an appropriate training intervention.
Table 2: Pre-test and post-test scores comparison

Click here to view
Figure 1: Assessment of resuscitation knowledge by comparing pre-test and post-test scores

Click here to view


[Table 3] and [Figure 2] depict the comparison between pre-intervention and post-intervention scores on a Likert scale ranging 0%–100% that are interpreted as: score 0%–20% very poor; 21%–40% poor; 41%–60% average; 61%–80% above average; and 81%–100% excellent.
Table 3: Assessment of resuscitation knowledge by comparing pre-test and post-test scores

Click here to view
Figure 2: Assessment of resuscitation knowledge by comparing pre-test and post-test scores

Click here to view


All the scores of three variables, i.e., type of organisation, type of intervention and specialty with their respective components are elaborated in [Table 4] and [Figure 3]. All three variables with their components or sub-variables are further compared for various values that include the number of participants, pre-intervention mean, post-intervention mean, paired mean difference, t value and P value.
Table 4: Comparison of resuscitation knowledge score

Click here to view
Figure 3: Comparison of resuscitation knowledge score

Click here to view



  Discussion Top


Assessment is an integral component and one of the pillars of learning cycle. Therefore, assessment results are not only benefiting all the stakeholders in achieving more learning but also bridge any gaps identified between intended learning outcomes and the actual achievements of the learners.[16] Assessment results can be viewed for future improvement and should be designed prospectively along with learning outcomes. Assessment should not be viewed as an external measurement of the results of the educational process but more as an integral part of the process.[17]

We assessed the resuscitation knowledge among various healthcare workers (physicians, nurses, paramedics and others) and organisations (DHA or non-DHA) by comparing pre-test and post-test scores with one of the interventions (ACLS, PALS or PHTLS).

Type of organisation

In this study, we assessed the pre-test and post-test scores of DHA staff n = 3333 (83%) versus non-DHA staff n = 695 (17%) as seen in [Table 1]. The difference in numbers of study participants is largely due to the fact that the DHA training centre and sites offer training to its own employees, though the facility is open to private sector as well. [Table 4] compares in detail the pre-test and post-test scores between DHA staff (86.0 ± 11.0 vs. 87.0 ± 9.0; mean difference 1; P = 0.00) and non-DHA staff (86.0 ± 11.0 82.5 ± 11; mean paired difference 0.63; P = 0.24). The results show that the DHA staff gained better resuscitation knowledge post-intervention that is statistically significant (P = 0.00). As DHA training facilities are affiliated with hospitals and government medical centres, the physicians n = 1685 (42%) and nurses n = 2183 (54%) make the bulk of participants. The paramedics and other disciplines, being non-DHA entity are therefore less in number n = 91 (2%) and n = 69 (2%), respectively.

Type of intervention

We assess three interventions (ACLS, PALS and PHTLS) in our research. The two ACLS and PALS are mostly recommended for physicians and nurses as compared to PHTLS, a common training discipline for paramedics. [Table 4] and [Figure 3] reveal that the pre-test (85.0 ± 12.0) and post-test (87.6 ± 8.0) scores for ACLS are statistically significant (P = 0.00) as compared to pre-test (88.0 ± 9.0) and post-test (88.0 ± 8.0) scores for PALS (P = 0.78). The pre-test (76.0 ± 13.0) and post-test (76.0 ± 10.0) scores for PHTLS show that the gain in knowledge is also statistically insignificant (P = 0.41). Among all three training programs, the ACLS proves a best intervention in knowledge gain (paired mean difference 2.6; t = 9.0; P = 0.00). The highest CPR knowledge scores seen for ACLS are due to the fact that (1) the ACLS is a common and mandatory course; (2) it is for adult patients; and (3) recommended for in-hospital and out-of-hospital settings and for most of the healthcare workers. On the contrary, the PHTLS course is mandatory for pre-hospital staff and paramedics but is also recommended for some of the hospital staff as well, for example, those working in emergency departments and trauma units.

Specialty

There is difference in scores for different healthcare workers when categorised by the specialty as shown in [Table 4]. Physicians achieved the highest pre-test (86.0 ± 11.0) and post-test scores (88.5 ± 8.0) with mean paired difference of 2.14. The highest scores achieved by physicians may reflect their diverse clinical exposure and experience and statistically significant improvement post-training (P = 0.00). Munezero et al.[18] assessed the post-intervention improvement in CPR knowledge and skills with similar results, but that study was limited to nurses only. Similarly, the pre-test and post-test scores for nurses (84.0 ± 12.0 vs. 85.0 ± 10.0; mean difference 0.96) are also significant (P = 0.00). However, if we look into pre-test and post-test scores of paramedics (83.0 ± 11.0 vs. 80.0 ± 12.0; mean paired difference − 3.18), this shows a reduction in the post-intervention score compared to the pre-intervention score and that the reduction was statistically significant (P = 0.02). This signifies that the intervention received by the paramedic staff was either not effective or some other administrative issues that need to be explored. The pre-test and post-test scores of other healthcare workers (81.0 ± 12.0 vs. 82.0 ± 11.0; mean difference 0.90) are also clinically insignificant (P = 0.60) and shows that the intervention did not bring any noticeable gain in knowledge. The other staff that include healthcare educators and administrators struggle to achieve better post-test that might be due to less exposure to clinical environment.

Interpretation of pre-test and post-test scores by Likert scale

The pre-test scores derived from data (n = 4028) according to Likert criteria are shown in [Table 3], and [Figure 2] includes very poor 0% (n = 1); poor 0.4% (n = 16); average 5% (n = 192); above average 22.0% (n = 883) and excellent 73.0% (n = 2936). When we applied the same Likert scale, the post-test scores derived are very poor 0% (n = 0); poor 0.1% (n = 3), average 2% (n = 79); above average 20.6% (n = 829) and excellent 77.4% (n = 3117). It is worth to note that the pre-test and post-test 'Excellent' category scores improve from 73.0% (n = 2936) to 77.4% (n = 3117) after intervention which is significant. In addition, the pre-test and post-test scores for 'Above Average' group reduced n = 22.0% (n = 883) versus 20.6% (n = 829) which indicate that there is transfer of scores from 'Above Average' group to 'Excellent' category.

Overall knowledge gain

Although the difference between overall pre-test and post-test scores (85 ± 12 vs. 87 ± 9) is marginal as shown in [Table 2], the impact of intervention (ACLS, PALS and PHTLS training courses) in improving the overall knowledge among healthcare workers is statistically significant (P = 0.00).

Strengths

Our study recruited large number of individuals (n = 4028) and the findings can be generalised. Pre-test interventions are used to assess the baseline knowledge, the intervention builds on existing knowledge to bridge the gap and the post-test claims the final gain in knowledge. The improvement in resuscitation knowledge may therefore a key factor for the survival of cardiac arrest victims.

Limitations

The before-and-after design is most useful in demonstrating the immediate impacts of short term programs (ACLS, PALS and PHTLS). It is less useful for evaluating longer term interventions. This is because over the course of a longer period of time, more circumstances can arise that may obscure the effects of an intervention. These circumstances are collectively called threats to internal validity. In addition, there is inherent vulnerability of a before-and-after design to internal validity threats, especially for long-term evaluation periods. A quasi-experimental design can be an alternative study design to overcome such internal validity threats.

Pre-test scores are given 2 weeks before intervention along with preparatory manual of respective course. As pre-test was completed at home or outside training center in non-examination environment, participants tend to score high. This behavior at times might a limited factor in assessing the effectiveness of intervention in the form of post-test scores.


  Conclusions Top


The study revealed statistically significant improvement (paired mean difference 2%; P = 0.00) in the resuscitation knowledge for all healthcare workers (n = 4028) post training. Among healthcare workers, the physicians and nurses showed statistically significant improvement from pre-test to post-test scores, but there was a reduction in the post-intervention score among paramedics and other healthcare professionals. The results of our study can be generalised, and the findings can be applied to large and multiple groups of healthcare professionals involved in resuscitation and emergency care of the patients.

Acknowledgments

We acknowledge the contribution of Medical Education Department, Dubai health authority (DHA) for providing all the related data to conduct this research.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.



 
  References Top

1.
Amin Z, Sen C, Eng H. Practical Guide to Medical Student Assessment. 2nd ed. Singapore: World Scientific Printers; 2010. p. 215-21.  Back to cited text no. 1
    
2.
Fraser SW, Greenhalgh T. Coping with complexity: Educating for capability. BMJ 2001;323:799-803.  Back to cited text no. 2
    
3.
Swanwick T. Understanding Medical Education: Evidence, Theory and Practice. 2nd ed. Wiley-Blackwell; 2013. p. 520.  Back to cited text no. 3
    
4.
Epstein RM. Assessment in medical education. N Engl J Med 2007;356:387-96.  Back to cited text no. 4
    
5.
Norton LS, Tilley AJ, Newstead SE. Franklyn-Stokes BA. The pressures of assessment in undergraduate courses and their effect on student behaviours. Assess Eval High Educ 2001;26:269-84.  Back to cited text no. 5
    
6.
Dent JA, Harden RM. A Practical Guide for Medical Teachers. 1st ed. Elsevier; 2001. p. 281-90.  Back to cited text no. 6
    
7.
Broomfield R. A quasi-experimental research to investigate the retention of basic cardiopulmonary resuscitation skills and knowledge by qualified nurses following a course in professional development. J Adv Nurs 1996;23:1016-23.  Back to cited text no. 7
    
8.
Einspruch EL, Lynch B, Aufderheide TP, Nichol G, Becker L. Retention of CPR skills learned in a traditional AHA heartsaver course versus 30-min video self-training: A controlled randomized study. Resuscitation 2007;74:476-86.  Back to cited text no. 8
    
9.
Botha L, Geyser MM, Engelbrecht A. Knowledge of cardiopulmonary resuscitation of clinicians at a South African tertiary hospital. S Afr Fam Pract 2012;54:447-54.  Back to cited text no. 9
    
10.
Ragavan S, Schneider H, Kloeck WG. Basic resuscitation – Knowledge and skills of full-time medical practitioners at public hospitals in Northern Province. S Afr Med J 2000;90:504-8.  Back to cited text no. 10
    
11.
Meaney PA, Sutton RM, Tsima B, Steenhoff AP, Shilkofski N, Boulet JR, et al. Training hospital providers in basic CPR skills in Botswana: Acquisition, retention and impact of novel training techniques. Resuscitation 2012;83:1484-90.  Back to cited text no. 11
    
12.
Rajeswaran L, Ehlers VJ. Cardiopulmonary resuscitation knowledge and skills of registered nurses in Botswana. Curationis 2014;37:e1-7.  Back to cited text no. 12
    
13.
Eccles M, Grimshaw J, Campbell M, Ramsay C. Research designs for studies evaluating the effectiveness of change and improvement strategies. Qual Saf Health Care 2003;12:47-52.  Back to cited text no. 13
    
14.
American Heart Association. CPR & ECC Guidelines. American Heart Association. Available from: https://eccguidelines.heart.org/index.php/ circulation/cpr-ecc-guidelines-2/. [Last accessed on 2018 Nov 23].  Back to cited text no. 14
    
15.
PHTLS Courses. Available from: https://www.naemt.org/education/phtls/phtls-courses. [Last accessed on 2017 Jun 23].  Back to cited text no. 15
    
16.
Linn RL, Gronlund NE. The role of measurement and assessment in teaching. In: Measurement and Assessment in Teaching. 8th ed. New Jersey: Prentice Hall Publications; 2000. p. 33-4.  Back to cited text no. 16
    
17.
Norcini JJ. Current perspectives in assessment: The assessment of performance at work. Med Educ 2005;39:880-9.  Back to cited text no. 17
    
18.
Munezero JB, Atuhaire C, Groves S, Cumber SN. Assessment of nurses knowledge and skills following cardiopulmonary resuscitation training at Mbarara regional referral hospital, Uganda. Pan Afr Med J 2018;30:108.  Back to cited text no. 18
    


    Figures

  [Figure 1], [Figure 2], [Figure 3]
 
 
    Tables

  [Table 1], [Table 2], [Table 3], [Table 4]



 

Top
 
  Search
 
    Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
    Access Statistics
    Email Alert *
    Add to My List *
* Registration required (free)  

 
  In this article
Abstract
Introduction
Methods
Results
Discussion
Conclusions
References
Article Figures
Article Tables

 Article Access Statistics
    Viewed103    
    Printed8    
    Emailed0    
    PDF Downloaded24    
    Comments [Add]    

Recommend this journal