• Users Online: 535
  • Home
  • Print this page
  • Email this page
Home About us Editorial board Ahead of print Current issue Search Archives Submit article Instructions Contacts Login 


 
 Table of Contents  
MEDICAL EDUCATION
Year : 2019  |  Volume : 32  |  Issue : 2  |  Page : 100-102

Health research methodology workshop: Evaluation with the Kirkpatrick model


1 Department of Community Medicine, Chettinad Hospital and Research Institute, Kelambakkam, Chennai, Tamil Nadu, India
2 Saudi Board of Preventive Medicine, Ministry of Health, Riyadh, Kingdom of Saudi Arabia

Date of Web Publication7-Jan-2020

Correspondence Address:
Vinoth Gnana Chellaiyan
Department of Community Medicine, Chettinad Hospital and Research Institute, Kelambakkam, Chennai, Tamil Nadu
India
Login to access the Email id

Source of Support: None, Conflict of Interest: None


DOI: 10.4103/0970-258X.275352

  Abstract 

Background. Workshops on research methodology impart skills of research among medical students. Both qualitative and quantitative evaluation of an academic programme is essential to enhance the effectiveness and quality improvement. We assessed the gain in learning and effect of a workshop on research methodology among medical students.
Methods. We did a quasi-experimental, single-group study at a tertiary care hospital and research institute in southern India. It included 33 students enrolled in various residency positions of the institute. The Kirkpatrick 4-level model was used to assess the effectiveness of the workshop on research methodology. Paired t-test was used to compare pre- and postworkshop scores.
Results. Twenty-five students rated the academic sessions as excellent. The score before the workshop ranged from 0 to 17 with mean (SD) 9.27 (4.2). The post-workshop score had a minimum to maximum score of 10-26 with mean (SD) of 16.18 (3.7) (p<0.005). The effect size d cohen (confidence interval [CI]) was 1.743 (0.942-2.545). The mean (SD) of absolute and relative gain was 10.8 (3.8) and 1.41 (0.07), respectively. 66.7% medical students showed a 30% rise in their post-workshop scores, the cut-off for effectiveness of the workshop.
Conclusion. The evaluation of a workshop on research methodology provided insights into the outcomes and modifications required for their future improvement.


How to cite this article:
Chellaiyan VG, Suliankatchi RA. Health research methodology workshop: Evaluation with the Kirkpatrick model. Natl Med J India 2019;32:100-2

How to cite this URL:
Chellaiyan VG, Suliankatchi RA. Health research methodology workshop: Evaluation with the Kirkpatrick model. Natl Med J India [serial online] 2019 [cited 2020 Aug 12];32:100-2. Available from: http://www.nmji.in/text.asp?2019/32/2/100/275352


  Introduction Top


Evidence-based medicine (EBM) requires the use of the available evidence in making decisions about individual patients.1,2 It is important that research brings benefits to patients and to society in healthcare. However, many medical graduates have opined that research skills have not been given importance in the undergraduate curriculum.3-5 Few studies have established the effectiveness of educational interventions in research methodology and the importance of EBM curriculum based on group activities and workshops. [6],[7] Both qualitative and quantitative evaluation of an academic programme is essential to enhance its effectiveness and quality. We evaluated the outcome of an educational intervention in research methodology among undergraduate medical students.


  Methods Top


Our study was done at Chettinad Hospital and Research Institute, a tertiary care hospital and research institute in Chennai, India. We used a quasi-experimental, single group, pre- and post-test study design.

We included all 33 residents enrolled in various postgraduate courses at our institution in the academic year 2016. The workshop sessions were free and no tuition fee was charged for attending the workshop. We used the Kirkpatrick 4-level model8 to assess the workshop on research methodology. The levels included reaction, learning, behaviour and results, and the model has been validated for the evaluation of educational interventions including research methodology. [9],[10]

Data collection

Level 1.
Satisfaction levels of the participants were measured in a 4-point Likert scale--excellent, good, average and poor. Scores were given to assess the satisfaction of participants as: excellent 4; good 3; average 2; and poor 1. Five parameters were used to assess quality: quality of sessions, hands on experience (practice problems at the end of every session), ability of teaching faculty to explain concepts, teaching materials (handouts and shared power point slides) and time allocation for every session.

Level 2. Pre-workshop knowledge on research methodology was assessed using a questionnaire before the start of the academic sessions. The academic sessions were conducted by experts from the Department of Community Medicine of Chettinad Hospital and Research Institute. The workshop included 19 sessions. Each daily session lasted an hour. The materials for the academic session were prepared from the following resources: WHO health research methodology, 'Principles of epidemiology in public health practice' by the Centers for Disease Control and Prevention, Epidemiology- Lancet Series, Ethical guidelines of biomedical research on human participants by the Indian Council of Medical Research, and 'Research methods in community medicine: Surveys, epidemiological research, programme evaluation, clinical trials' by Abramsons and some online material.[11],[12],[13],[14],[15],[16],[17],[18] The topics covered were: study designs (5 sessions), sampling methods (1), basics of biostatistics (1), estimation of sample size (2), critical appraisal of research studies (3), ethical considerations in research (1), review of literature (1), tests of significance (3), questionnaire development for descriptive studies (1) and research protocol development (1).

At the end of each session, a group discussion was held and handouts relevant to that session were provided.

The pre-workshop assessment questionnaire included 30 items. The items covered various sections: epidemiology, biostatistics, ethical considerations in research and critical appraisal of research studies. The assessment was done using multiple-choice questions and a few open-ended questions. After the last academic session, a post-workshop assessment was done. The post-test questionnaire consisted of 30 items similar to those in the pre-workshop assessment.

Research knowledge acquisition (learning gain) was measured from pre- and post-workshop assessment scores. Total scores and section scores were calculated. For measuring effect size, dcohen was calculated (d=Mean scorepost – mean scorepre/standard deviation). A pre-defined target of 30% was taken as the cut-off for the workshop to be considered as effective. Absolute gain (post-workshop score– pre-workshop score) and relative gain (post-workshop score– pre-workshop score/pre-workshop score) were calculated.

Level 3. Application of gained research knowledge in practice was assessed in terms of proposal writing, sample size estimation and manuscript writing. Proposal writing assessment was done at the end of the workshop. Sample size calculation exercises for various types of study designs were administered. Research manuscript writing assessment of content was done with guidelines and checklists for critical appraisal of different study designs.

Level 4. Overall impact was measured in terms of research studies started and manuscripts made ready for publication within 3 months of the research workshop. This level was assessed by enumeration of all research studies and assessment of their quality by the research committee of the institution.

Data entry and analysis

Data were entered and analysed with SPSS IBM version 21.0. Proportions were calculated for categorical variables and mean and standard deviation (SD) used for scores. A comparison of means between pre- and post-workshop scores was made with paired t-test. A value of p<0.05 was considered statistically significant.


  Results Top


Thirty-three medical students participated in the study, 17 were men [Table 1]. Twenty-eight (85%) students attended >90% of academic sessions. None of the students had exposure to any previous health research workshop.
Table 1: Disciplines of the students who attended the research workshop (n=33)

Click here to view


Levels 1 and 2

Overall, 76% of the students rated the academic sessions as excellent, 18% said the sessions were good and 6% said the sessions were average.

Assessment scores and effect size

The performance as shown by mean scores improved after the workshop with the scores improving from 0– 17 to 10– 26 ([Figure 1] and [Table 2]). The effect size dcohen (confidence interval) was 1.74 (0.94-2.55). The mean (SD) of absolute and relative gain was 10.8 (3.8) and 1.41 (0.07). Twenty-two (66.7%) students showed a 30% rise in their post-workshop scores.
Figure 1:Box plot showing total pre- and post-research workshop scores

Click here to view
Table 2: Overall satisfaction of the workshop participants (n=33)

Click here to view


On comparing the outcome, i.e. the total scores with paired test applied, there was a statistically significant improvement in the total scores [Table 3].
Table 3: Mean (SD) scores of workshop participants before and after the academic sessions (n=33)

Click here to view


Levels 3 and 4

Twenty-six (79%) participants were able to write the proposal correctly as per research protocol guidelines. Twenty-three (70%) were able to estimate sample size for various studies. Twenty-one (64%) participants were able to write the manuscript as per the guidelines of critical appraisal.

Within 3 months of the research methodology workshop, 30 (91%) participants started research studies; 24 studies were started of which 14 were case reports, 7 were cross-sectional studies and 3 interventional studies. About 25% of study participants prepared manuscripts of research studies to be sent for publication.


  Discussion Top


The majority of participants rated the workshop as excellent. However, 6% of participants rated it as average. This Likert rating provides an insight to work further on difficult areas of understanding and enhance the quality of workshop in the future. All our students were interested in conducting research studies and practising EBM. Such a positive attitude and high level of satisfaction have been reported in previous studies. [9],[19]

The post-workshop scores improved significantly (p=0.001). The workshop on research methodology was effective in imparting knowledge to the participants as shown by a 30% rise in the post- workshop assessment scores. The majority of participants were able to write a complete research proposal and estimate the sample size correctly. Hence, our workshop was able to impart skills as was reported by similar evaluation studies. [6],[9] A few studies from India too have shown improvement in post-workshop scores. [20],[21],[22] Pawar et al.'s study showed only a moderate improvement of scores.[23]

The impact of our workshop in terms of students starting new research studies and writing manuscripts was considerable. It was higher than the 57% of participants starting research studies in another study. [9] Thus, such workshops could play an important role in acquiring knowledge and skills as well as developing a 'research culture'--an imminent need. [19],[24],[25],[26] The Medical Council of India has proposed reforms in the graduate and postgraduate medical education curriculum envisaging promotion of research.27 Similarly, to promote research among undergraduate students, the Indian Council of Medical Research conducts short-term studentship projects every year.[28] Our study emphasizes the need to include research methodology in the medical education curriculum. The restriction of levels 3 and 4 evaluation within the study duration was a limitation of the study.

Conclusion

The performance of students who attended the workshop on research methodology was significantly improved and there was evidence of application of the knowledge gained into practice.

Conflicts of interest. None declared



 
  References Top

1.
Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: What it is and what it isn't. BMJ 1996; 312: 71-2.  Back to cited text no. 1
    
2.
Sackett DL, Richardson WS, Rosenberg W, Haynes RB. Evidence-based medicine: How to practice and teach. 2nd ed. Edinburgh:Churchill-Livingstone; 2000.  Back to cited text no. 2
    
3.
Rosenthal F, Ogden F. Changes in medical education: The beliefs of medical students. Med Educ 1998; 32: 127-32.  Back to cited text no. 3
    
4.
Aslam F, Qayyum MA, Mahmud H, Qasim R, Haque IU. Attitudes and practices of postgraduate medical trainees towards research--a snapshot from Faisalabad. J Pak Med Assoc 2004; 54: 534-6.  Back to cited text no. 4
    
5.
Siemens DR, Punnen S, Wong J, Kanji N. A survey on the attitudes towards research in medical school. BMC Med Educ 2010; 10: 4.  Back to cited text no. 5
    
6.
Mostafa SR, Khashab SK, Fouaad AS, Abdel Baky MA, Waly AM. Engaging undergraduate medical students in health research: Students' perceptions and attitudes, and evaluation of a training workshop on research methodology. J Egypt Public Health Assoc 2006; 81: 99-118.  Back to cited text no. 6
    
7.
Taheri H, Mirmohamadsadeghi M, Adibi I, Ashorion V, Sadeghizade A, Adibi P. Evidence-based medicine (EBM) for undergraduate medical students. Ann Acad Med Singapore 2008; 37: 764-8.  Back to cited text no. 7
    
8.
Donald L, James D, Kirkpatrick WK. Kirkpatrick's four level evaluation model. Newnan, GA:Kirkpatick Partners; 2016. Available at www.kirkpatrickpartners.com/ OurPhilosophy/TheKirkpatrickModel/tabid/302/Default.aspx (accessed on 21 Jul 2016).  Back to cited text no. 8
    
9.
Abdulghani HM, Shaik SA, Khamis N, Al-Drees AA, Irshad M, Khalil MS, et al. Research methodology workshops evaluation using the Kirkpatrick's model: Translating theory into practice. Med Teach 2014; 36 (Suppl 1):S24-S29.  Back to cited text no. 9
    
10.
Bates R. A critical analysis of evaluation practice: The Kirkpatrick model and the principle of beneficence. Eval Program Plan 2004; 27: 341-7.  Back to cited text no. 10
    
11.
World Health Organization. Health research methodology. Geneva:WHO; 2001. Available at www.wpro.who.int/publications/docs/Health_research_edited.pdf (accessed on 21 Jul 2016).  Back to cited text no. 11
    
12.
Centers for Disease Control and Prevention. Principles of epidemiology in public health practice. Atlanta, GA:Centers for Disease Control and Prevention; 2012. Available at www.cdc.gov/ophss/csels/dsepd/ss1978/ss1978.pdf (accessed on 21 Jul 2016).  Back to cited text no. 12
    
13.
Lancet. Epidemiology. Lancet Series; 2002. Available at www.thelancet.com/series/ epidemiology-2002 (accessed on 21 Jul 2016).  Back to cited text no. 13
    
14.
Indian Council of Medical Research. Ethical guidelines for biomedical research on human participants. New Delhi:Indian Council of Medical Research; 2006. Available at www.icmr.nic.in/ethical_guidelines.pdf (accessed on 21 Jul 2016).  Back to cited text no. 14
    
15.
Abramson JH, Abramson ZH. Research methods in community medicine: Surveys, epidemiological research, programme evaluation, clinical trials. 6th ed. Hoboken:John Wiley and Sons; 2008.  Back to cited text no. 15
    
16.
Dean AG, Sullivan KM, Soe MM. OpenEpi: Open Source Epidemiologic Statistics for Public Health, Version. Updated; 6 April 2013. Available at www.OpenEpi.com (accessed on 19 Jul 2016).  Back to cited text no. 16
    
17.
Consort Checklist. Consort Group; 2010. Available at www.consort-statement.org/ Media/Default/Downloads/CENT%20checklist.pdf (accessed on 12 Jun 2016).  Back to cited text no. 17
    
18.
Strobe Checklist. Geneva:Strobe Statement; 2007. Available at www.strobe- statement.org/?id=available-checklists (accessed on 12 Jun 2016).  Back to cited text no. 18
    
19.
Nel D, Burman RJ, Hoffman R, Randera-Rees S. The attitudes of medical students to research. S Afr Med J 2013; 104: 33-6.  Back to cited text no. 19
    
20.
Prabhu GR, Rao SB, Sridhar MS, Reddy AK, Chandrasekharan PA, Shankara Reddy SD, et al. Evaluation of research methodology workshop for postgraduates in a medical college, Tirupati. J Evol Med Dent Sci 2015; 4: 15868-72.  Back to cited text no. 20
    
21.
Thakre SB, Thakre SS, Golawar SH, Ughade SN, Thakre AD. Awareness about biomedical research among postgraduate students at a tertiary care hospital, Central India: Pretest-posttest design. Prospect Med Res 2014; 2: 8-14.  Back to cited text no. 21
    
22.
Khan H, Khwaja MR. Impact of a workshop on the knowledge and attitude of medical students regarding health research. J Coll Phys Surg Pak 2007; 17: 59.  Back to cited text no. 22
    
23.
Pawar DB, Gawde SR, Marathe PA. Awareness about medical research among resident doctors in a tertiary care hospital: A cross-sectional survey. Perspect Clin Res 2012; 3: 57-61.  Back to cited text no. 23
    
24.
Scaria V. Whisking research into medical curriculum. Calicut Med J 2004; 2: e1.  Back to cited text no. 24
    
25.
What is the purpose of medical research? Lancet 2013; 381: 347.  Back to cited text no. 25
    
26.
Lansang MA, Dennis R. Building capacity in health research in the developing world. Bull World Health Organ 2004; 82: 764-70.  Back to cited text no. 26
    
27.
Medical Council of India. Reforms in Undergraduate and Postgraduate Medical Education. Vision 2015. New Delhi:The Medical Council of India; 2011:34. Available at www.mciindia.org/tools/announcement/MCI_booklet.pdf (accessed on 6 Jul 2016).  Back to cited text no. 27
    
28.
ICMR. Short Term Studentships. New Delhi:Indian Council of Medical Research; 2015. Available at www.icmr.in/short.htm (accessed on 20 Jul 2016).  Back to cited text no. 28
    


    Figures

  [Figure 1]
 
 
    Tables

  [Table 1], [Table 2], [Table 3]



 

Top
 
 
  Search
 
Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
Access Statistics
Email Alert *
Add to My List *
* Registration required (free)

 
  In this article
Abstract
Introduction
Methods
Discussion
Results
References
Article Figures
Article Tables

 Article Access Statistics
    Viewed247    
    Printed8    
    Emailed0    
    PDF Downloaded79    
    Comments [Add]    

Recommend this journal


[TAG2]
[TAG3]
[TAG4]