Evaluation of blended internet and face-to-face continuous medical education for cupping providers in Saudi Arabia

Abdullah A.Al-Mudaiheem, Nasser A.Al-Hamdan, Abdullah AlBedah, Saud Al Sanad, , Tamer S.Aboushanab , Ahmed T.Elolemy,4,Mohamed K.M.Khalil

1National Center for Complementary and Alternative Medicine,Ministry of Health,Riyadh, Saudi Arabia.2Faculty of Medicine, King Fahad Medical City, Riyadh, Saudi Arabia.3College of Medicine, Al-Imam Mohammad Ibn Saud Islamic University (IMSIU), Riyadh,Saudi Arabia.4Faculty of Medicine,Tanta University,Tanta,Egypt.

Abstract Background: In regulating cupping therapy in Saudi Arabia, there was a need to develop a system for continuous medical and professional development for cupping practitioners.The current study aimed to evaluate the effect of blended-internet continuing-medical-education (CME) on knowledge, knowledge retention, and intellectual skills in cupping providers compared with conventional, face-to-face CME. Methods:Participants were recruited from a list of registered cupping providers (physicians, nurses, and physiotherapists trained and licensed to practice cupping therapy) for a randomized, open, controlled trial.After providing informed consent, participants were allocated into three groups (classes).Classes were randomized to (1) blended-internet CME (n = 31); (2)conventional, face-to-face CME (n = 31); and (3) control group receiving no intervention (n = 32). Results:Posttest knowledge scores were increased to a greater extent by blended-internet CME training compared to conventional CME (P= 0.002)or no training (P ＜0.001).Similarly, knowledge retention was significantly higher in the blended-internet CME group compared to the conventional CME group (P = 0.011).Posttest skills scores immediately after the course were significantly higher in the blended-internet CME group compared to the conventional CME group(mean difference:1.408,95%confidence interval:0.728-2.088,P ＜0.001).Conclusion:Although the current study aimed to determine if blended-internet CME is as effective as conventional,face-to-face CME,the results showed that cupping providers who participated in blended-internet CME had significantly higher scores for posttest knowledge,knowledge retention,and posttest intellectual skills.

Keywords:Cupping therapy,Complementary medicine,Education,Internet,Saudi Arabia

Background

Methods

This was a class-randomized, open, controlled trial.Participants were recruited from a list of registered cupping providers (physicians, nurses, and physiotherapists trained and licensed to practice cupping therapy).After providing informed consent,participants were allocated into three groups (classes).Classes were randomized into 1) blended-internet CME; 2) conventional, FTF CME; and 3) control group(waitlisted),receiving no intervention.

Interventional education activities were developed as follows.

Conventional CME

Three days of conventional CME, named “Updates on Efficacy and Safety of Wet-Cupping” was prepared by two experienced consultants in complementary medicine and based on National Center for Complementary and Alternative Medicine, Ministry of Health (NCCAM/MOH) publications [17, 18].They also identified specific learning outcomes based on the knowledge and intellectual skills delivered through the training program.The training was submitted to the Saudi Commission for Health Specialties for accreditation and was accredited with CME hours(Saudi Commission for Health Specialties accreditation number:17000040322).

Developing the blended-internet CME

The educational contents of two out of the three days of the conventional CME training were converted into a video-based, e-learning format.This video content was administered over five days as test-enhanced [19]learning in which each video was followed by a quiz to be answered.The e-learning was then followed by one day of FTF training focused on intellectual skills development.

Developing the e-learning platform

A group discussion at NCCAM assessed the relative merits of purchasing a Learning Management System software license and installing it on NCCAM’s servers versus using a commercial, web-based Learning Management System service.Based on the nature of the project and the available time frame for the project,the decision was made to subscribe to a web-based Learning Management System.In selecting a service provider, several factors were considered, including the value of subscription; platform ease of use for the administrator, instructors, and trainees; ability to customize the website, add quizzes, exams, and assignments; activity tracking; and being mobile and tablet friendly.

Evaluation of knowledge and intellectual skills

Knowledge score.One hundred true or false questions were prepared to cover the primary learning outcomes,with 30 of these questions each randomly selected for the pretest, posttest, and knowledge-retention tests.The tests were scored on a binary scale (0 points for incorrect answers and 1 point for correct answers)[20].

Knowledge retention.The knowledge score measured two weeks after the intervention was used as a measure of knowledge retention.This time point was selected due to feasibility and time limitations.Thirty true or false questions were administered with each question weighted equally.

Then the poor young man again set forth2 as well as his horse could take him, begging his living as he went, and, somehow or other, he ended by reaching the home of the wind

Intellectual skills score.In the training program,fourteen evidenced-based-medicine queries were extracted from the NCCAM/MOH publication“evaluation of wet cupping therapy: a systematic review of a randomized clinical trial [17]”.The trainees were asked to (1) develop an appropriate clinical question, (2) perform a medline search for appropriate articles, (3) critically appraise a complete selected article, and (4) reach a conclusion regarding their question.Two queries were used to evaluate the skills scores before and after the training program.Each answer was evaluated by two independent trainers (consultants).The average of the scores from both trainers was used as the intellectual skills score[21].

Participants were assigned to one of the following three groups: (1) blended-internet CME consisting of five days of online instructional material using test-enhanced e-learning followed by one day of FTF training, (2) conventional CME consisting of three days of FTF training, or (3) control that received no intervention until after the study’s conclusion.The two interventional activities (blended-internet CME and FTF, conventional CME) were conducted in two different cities (Riyadh and Madinah) to prevent cross-dissemination of knowledge or a spillover effect.

Main outcome measures

Primary outcome.To compare the knowledge score from 30 true or false questions within and between the blended-internet CME group, the conventional CME group, and the control group before and immediately after the intervention.

Secondary outcomes.To compare knowledge retention within and between the blended-internet CME group and the conventional CME group two weeks after the intervention; and to compare intellectual skills between the blended-internet CME group and the conventional CME group before and immediately after the intervention using two queries,each with a maximum score of five.

A questionnaire was designed to collect background variables and outcome measures.Variables included:demographic data, occupation, nationality, and the scores of the outcome results.

Statistical Analysis

Based on the results of a Shapiro-Wilk test,knowledge scores before and after the intervention did not show a significant departure from normality.Analysis of covariance (ANCOVA) was, therefore, used to compare postintervention scores using the baseline score as a covariate.Two-tailedP-values less than 0.05 were considered statistically significant.

Sample size calculation

According to our noninferiority design, we calculated the sample size necessary to provide a 90% (1-beta)power to detect a 20% mean difference in the posttest knowledge score (non-inferiority limit: d), assuming a pooled standard deviation of 20% [20] with a significance level of alpha = 0.05.The above criteria gave a sample size of 18 in each group using a sealed-envelope site calculator [22].The sample size was doubled to include 40 in each group to allow for drop out and to end with at least 30 in each group.

Ethical consideration

The study was registered o n ClinicalTrials.gov(identifier: NCT03235895) and approved by the Institutional Review Board of King Fahad Medical City (the approval number: 17-292E, category:exempt).

Results

Sample characteristics

Of the 298 practitioners who received the invitation to attend the CME activities, 114 agreed to participate.Eighty cupping practitioners were allocated to the two CME interventions activities without knowing the intervention type (40 blended-internet CME; 40 conventional, FTF CME; and the remaining 34 as controls).Out of the 80 who started CME activities, 62 completed educational activities and completed the posttest (31 blended-internet CME and 31 conventional, FTF CME).Forty completed the knowledge-retention test two weeks after the course(23 blended-internet CME and 17 conventional, FTF CME).The participants assigned to the control group only completed the pre- and post-CME tests.More details are provided in the CONSORT chart(Figure 1).

Participants did not differ statistically with regards to age, gender, occupation, or nationality (Table 1).The mean age of the 16 males and 15 females in the blended-internet CME group was (40 ± 10.56) years;the 19 males and 12 females in the conventional, FTF CME group had a mean age of(42±13.90)years;and the mean age of the control group’s 17 males and 15 females was (38 ± 6.178) years.No statistically significant difference was found between the groups with respect to their duration of general experience since acquiring a basic degree (P= 0.45), duration of cupping experience before completing the cupping certification course (P= 0.45), or duration of cupping experience after completing the cupping certification course(P=0.16).

Baseline (pretest) mean knowledge and skills scores are reported in Table 2.No statistically significant differences were found between groups for pretest knowledge(P=0.31)or skills(P=0.46)scores.

Comparisons of knowledge scores, knowledge retention and skills score

There was a statistically significant increase in posttest knowledge compared to pretest scores in the blended-internet CME group(mean difference:10.032 points, 95% confidence interval (CI): 8.3-11.8,P＜0.001) and the conventional, FTF CME group (mean difference: 6.194 points, 95% CI: 4.467-7.920,P＜0.0011).In the control group, there was no statistically significant difference between the pretest and posttest knowledge scores.There was no statistically significant decrease in knowledge retention in either CME group.There was a statistically significant increase in the skills scores of the blended-internet CME group (mean difference: 5.787 points, 95% CI:5.221-6.353,P＜0.001) and the conventional, FTF CME group (mean difference: 4.387, 95% CI:3.77-5.1,P＜0.001).

Figure 1 CONSORT flow diagram.CME,continuing-medical-education.

Table 1 Sample characteristics

Table 2 Among-group outcome measures

Table 3 Paired posttest knowledge score comparisons between groups

A one-way, between-subjects ANCOVA was calculated to examine the effects of the interventions on posttest knowledge scores and knowledge retention while controlling for the effect of pretest knowledge scores.The adjusted means showed a statistical significant difference in posttest knowledge scores between groups (P＜ 0.001) (Table 2) and a statistically significant difference in knowledge retention(P=0.011)(Table 2).

Pairwise comparisons of posttest knowledge scores between study groups showed a statistically significant mean difference between all study groups (P＜0.05),with 41.6% of the between-subject variance accounted for by the intervention (Table 3).The blended-internet CME group posttest knowledge score was significantly higher than that of the conventional CME (P= 0.002)or control groups (P＜0.001), and the conventional CME group was significantly higher than the control group(P＜0.001).Similarly, a pairwise comparison of the knowledge retention of the blended-internet and conventional CME groups showed a statistically significant difference(mean difference:2.563,95%CI:0.625-4.501,P=0.011).

A one-way, between-subjects ANCOVA was calculated to examine the effect of the interventions on posttest skills scores while controlling for the effect of the pretest skills score, which revealed a statistically significant difference (P＜ 0.001).Pairwise comparisons found that the difference was statistically significant between all study groups (P＜0.05), with a higher posttest score in the blended-internet CME group compared to the conventional CME (mean difference: 1.408, 95% CI: 0.728-2.088,P＜0.001)and control groups (P＜ 0.001).Additionally, the conventional group was significantly higher than the control group(P＜0.001).

After controlling for pretest knowledge scores, the mean posttest knowledge score in the blended-internet CME group (mean: 21.988, 95% CI: 20.656-23.319)was significantly higher than that of the conventional CME group (mean:18.903,95%CI:17.586-20.220,P= 0.001), with nonoverlapping CIs (Table 4).The mean difference between the blended-internet CME and conventional CME groups was 3.084 (95% CI:1.209-4.960,P= 0.002) (Table 4), resulting in a nonnegative CI and a difference greater than the proposed 20%.Therefore, the null hypothesis is rejected.The alternative hypothesis is that the experimental intervention is noninferior to the standard intervention:H1:μe-μs ＞-d,21.99-18.9 ＞-3.78.

Discussion

This current, problem-based research aimed to solve the problem of providing the required CME hours to renew the registrations of the cumulative number of registered cupping providers.It was an attempt to transition complementary medicine CME to a digitalized, internet-based activity to enhance and speed CME completion.The primary objective was to compare a test-enhanced, internet-based CME to conventional, FTF CME.The current study a first in the field of Saudi Arabian complementary medicine.The results showed that traditional CME training blended with a test-enhanced, internet-based component can significantly increase knowledge,knowledge retention, and intellectual skills among cupping practitioners compared to conventional, FTF CME.The sample was homogenous, with no significant difference in participant ages, genders,occupations, nationalities, or years of cupping experience.Thus, the results of this study can be generalized to health care professionals qualified to practice complementary medicine.However, it cannot be generalized to traditional healers.

There was a significant change in knowledge within both intervention groups after the course, but with a greater improvement by participants in the blended-internet CME in posttest knowledge and skills compared to participants in the conventional CME group.This higher score can be explained in that the blended-internet CME group had more time to complete the learning material and prepare for the workshop (five days) with repeated visits to the website possible any time during the course, which reinforces their learning process[15,23].

The results of the current study are in agreement with the previously published research[13, 24].Richmond et al.conducted a meta-analysis of six trials comparing online health care professional training to conventional training, four assessing knowledge and two assessing practical skills, and found no difference between online and conventional training [13].Another meta-analysis by Cook et al.included 76 studies comparing internet-based learning with conventional methods in training pre- and postgraduates,and their findings showed no significant difference between the groups in knowledge, skills,behavior, or satisfaction [25].However,blended-internet education can benefit from the advantages of internet and FTF training to yield a greater impact on education and training outcomes[24,26].

Of the 38 practitioners who registered for the platform, only one needed assistance with signing in.Technical support was provided during the course by email and phone.The dropout rate was nearly equal in both intervention groups, which does not indicatedifficulty with e-learning.Trainee adherence to and progress with the online course were monitored, and reminder messages were sent to trainees before course assignment due dates.

Table 4 Effect of intervention after controlling for pretest knowledge score as a covariate

This study has some limits.For feasibility reasons and because participants were living and practicing all over SA, CME type was randomized by class and not by participant.However, class randomization took place after course registration, and participants did not know which method they would be using until the start of the course.This level of randomization is commonly used in interventional educational programs[27, 28].Randomization can be conducted at different levels, such as individuals, classes, or schools,according to feasibility and study design [27, 29].The two interventional activities (blended-internet CME and conventional, FTF CME) were conducted in two different cities (Riyadh and Madinah) to prevent cross-dissemination of knowledge or the spillover effect [30].The short follow-up period (two weeks) to measure knowledge retention was another limitation due to the time limits of the study.Longer follow-up periods are needed.Different weighting reflecting the most important knowledge to be retained would also be an important additon to future studies.Also,measuring the impact of intervention on clinical practice would be an additional, crucial outcome measure.

Conclusion

In conclusions, although the current study aimed to determine if blended-internet CME is as effective as conventional, FTF CME, the results showed that cupping providers who participated in blended-internet CME had significantly higher posttest scores for knowledge,knowledge retention,and intellectual skills.Blended-internet CME can provide trainers with a broad spectrum of tools to create and deliver effective,quality education and to permit more time for practical sessions for cupping practitioners.