|Year : 2022 | Volume
| Issue : 3 | Page : 608-615
Design and evaluate psychometric properties of scale to assess students' perception toward online teaching in nursing education: Sequential exploratory study
Suresh K Sharma1, Shiv K Mudgal2, Rakhi Gaur2
1 College of Nursing, All India Institute of Medical Sciences, Jodhpur, Rajasthan, India
2 College of Nursing, All India Institute of Medical Science, Deoghar, Jharkhand, India
|Date of Submission||09-Jul-2021|
|Date of Decision||12-May-2022|
|Date of Acceptance||26-May-2022|
|Date of Web Publication||2-Nov-2022|
Dr. Rakhi Gaur
College of Nursing, All India Institute of Medical Science, Deoghar, Jharkhand
Source of Support: None, Conflict of Interest: None
Background and Aims: Given the scarcity of a valid and reliable tool specially designed to measure perception to online teaching in nursing science students. This study was done with an aim to design and evaluate the psychometric properties of perception to online teaching questionnaire. Methods: The present study was conducted using mixed-method approach in two phases at the Eternal University, in 2020. In first phase, qualitative study was done to explore the students' perception on online teaching. Then, the theme and subtheme were extracted, and initial items were developed through reviewing the literature. In the second phase, total of 384 nursing students who were recruited conveniently participated in the study. Furthermore, the psychometric properties including face, content, and construct validities, test–retest reliability and internal consistency (Cronbach's alpha), were measured in the second phase. Statistical analysis was done using SPSS Analysis of Moment Structure version 23 (SPSS Inc., Chicago, IL, USA) for quantitative data. Results: Four factors were extracted through exploratory factor analysis. The confirmatory factor analysis confirmed 28 items in four factors: Ascendancy, snag, technophilia and prerequisites, and these factors explained 61.67% of variances. The Cronbach's alpha and stability of the questionnaire were 0.906 and 0.91, respectively. The discriminant validity and composite reliability of this final model were confirmed. Conclusion: This proposed questionnaire had met at desired standard of reliability and validity making it a useful measurement tool for measuring the perception to online teaching in nursing science students. It could be used for teaching–learning method improvement.
Keywords: Measures, online teaching, perception, psychometrics, students nursing
|How to cite this article:|
Sharma SK, Mudgal SK, Gaur R. Design and evaluate psychometric properties of scale to assess students' perception toward online teaching in nursing education: Sequential exploratory study. J Datta Meghe Inst Med Sci Univ 2022;17:608-15
|How to cite this URL:|
Sharma SK, Mudgal SK, Gaur R. Design and evaluate psychometric properties of scale to assess students' perception toward online teaching in nursing education: Sequential exploratory study. J Datta Meghe Inst Med Sci Univ [serial online] 2022 [cited 2023 Feb 1];17:608-15. Available from: http://www.journaldmims.com/text.asp?2022/17/3/608/360201
| Introduction|| |
The outbreak of COVID-19 has spread around the globe and its impact is observed in every sector of society. This situation has brought radical changes in the field of education around the globe. The effects of this pandemic and thereby its preventive measures, has upended the life of teachers, parents, and students. Aside from the disruption confront due to this pandemic, education around the globe, particularly in developing countries like India has experienced some major changes wherein, institutes of higher education across the world have been closed to remain their students healthy and safe at home and shift from traditional classroom teaching to online teaching.,
Online teaching is an electronically based teaching–learning method usually conducted outside conventional classroom wherein, students and teacher interact through the internet. This teaching method allows students to access educational material without entering in a traditional classroom. Furthermore, permit them to complete and submit their assignments and conduct examination through online mode.,
The success of online education depends upon learners' perception toward online teaching to a large extent. Perception toward online teaching is a hypothetical construct which may be influenced by various factors such as knowledge, readiness to learn, perceived barriers, beliefs, and environment. Therefore, it is imperative to measure the perception to online teaching in nursing science students.
A valid and reliable scale is essential to precisely measure the perception to online teaching in nursing science students so that empirical evidence can be generated on this subject. However, there are very few studies conducted to assess nurse,, and students,,, perception or attitude toward online teaching/e-learning. Furthermore, limiting the measurement of perception to online teaching only to quantitative scales provides the readers to a limited insight, which is disseminated by the available scales, while the some important factors like barriers, prerequisites, etc., remain hidden.,,
Currently, there are very few scales that measure students' perception or attitude to online learning/teaching, some of which are not applicable and specific to nursing students., Considering the importance of online teaching and paucity of valid and specific scale to measure the perception of nursing science students in a comprehensive manner toward this teaching–learning method, create an exigency to develop a validate and specially designed measurement questionnaire. Therefore, the present study aimed to design and evaluate the psychometric properties of perception to online teaching in nursing science students' questionnaire to achieve a comprehensive view on this resurging trend.
| Methods|| |
The present sequential exploratory research design with an instrumental development model was conducted in two phases. Initially, a systematic literature review and qualitative stages of the study were employed for item generation and in the next phase of this study quantitative analysis part was conducted for items reduction and developed questionnaire was field-tested. Furthermore, psychometric analysis for validity and reliability estimation was carried out using suitable statistical methods.
Phase I: Items generation
In this phase of the study, the concept of students' perception to online teaching in nursing science was explored and its aspects and elements were picked up using the hybrid concept analysis model. This model consists of three phases: Theoretical, qualitative, and analytical phase. In the theoretical phase, articles related to the students' perception to online teaching in nursing education were systematically searched using database such as PubMed (MEDLINE), EMBASE, SCOPUS, and CINHAL till June 30, 2020. Keywords for searching the articles were nursing students, perception, online teaching, e-learning, attitude, barriers, questionnaire, scale, and tool used. Inclusion criteria were article published in English language, available to the full text and studied students' perception/attitude to online teaching or e-learning in nursing education while editorial, review, and opinion articles were excluded. Of 643 articles identified in the initial search, 13 related articles were extracted. The steps were implemented as per PRISMA guideline [Figure 1].
Next, the qualitative phase was executed to get a thorough understanding of the concept of students' perception to online teaching in nursing education, 18 semi-structured interviews were conducted. The interviewees were selected from all levels of undergraduate nursing students to attain a maximum diversity of sampling methods. Participants were selected through purposive sampling technique and included those who had a minimum 2-month experience of online teaching, available online at scheduled time of interview and gave their consent to participate in the study. The duration of individual interview was approximately 30–35 min. Data were analyzed through qualitative content analysis and codes were extracted that reflect nursing students' perception to online teaching from the interviews.
In the analytical phase, findings of the systematic review of literature and qualitative phase were integrated and evaluated using qualitative content analysis method. A pool of 92 items was generated in the initial draft of the perception of nursing students to online teaching. Then, meetings were held by research team members, and items were examined with greater sensitivity. Finally, the initial draft of the nursing students' perception towards online teaching (SP-TOT) with 73 items was developed. The newly designed scale was named as SP-TOT questionnaire.
Phase II: Quantitative phase
In this phase, psychometric properties of the SP-TOT were evaluated for face, content and construct validity, and reliability.
Both qualitative and quantitative methods were used to establish the face validity of SP-TOT. For qualitative face validity, questionnaire was sent to eleven nursing students and they were asked to give comment on difficulty level, relevance, and ambiguity of each item. The suggestions that they expressed were re-examined and the items were modified, corrected, or deleted. For quantitative face validity, 10 nursing students were asked to answer the item “to what extent each item is necessary for measuring nursing students' perception to online teaching on a scale, which score from 1 (completely unnecessary) to 5 (absolutely necessary). We used item impact technique to determine quantitative face validity, where the following formula: Importance × frequency (%) was used to calculate the item impact score of each item. The items which had an impact score ≥1.5 were considered relevant and retained for further step.
To determine the content validity SP-TOT questionnaire, quantitative and qualitative ways were used. In qualitative content validity, 10 experts were asked to submit their comments on wording, grammar, item allocation, and scaling of each item. Then, based on experts' recommendations, the questionnaire's items were edited. Thereafter, the quantitative content validity was determined by assessing the content validity ratio (CVR) and content validity index (CVI). To examine the CVR, 13 experts were asked to rate the necessity of the items on a three-point Likert scale, which score from 1 (not necessary); 2 (useful but not necessary); and 3 (necessary). The CVR was determined based on Lawshe formula, which is acceptable when CVR score is ≥0.54 for 13 experts. CVI was assessed which represents the degree of relevancy of items for the proposed questionnaire. Item-CVI (I-CVI) and scale-content validity index (S-CVI) were calculated for each item and overall scale, respectively. In this phase, eleven experts were asked to rate the items according to their relevancy on a five-point Likert scale from 1 (not relevant and should be removed) to 5 (highly relevant). The item which scores of 0.79 or greater was accepted. The S-CVI score was determined by calculating mean of the I-CVI for all items in the scale. A score ≥0.9 was considered excellent and 0.8 as acceptable for the S-CVI. Kappa statistic computation needs the determination of probability of chance agreement, which is calculated using the following formula: K = (I-CVI‒Pc)/1 − Pc formula. The Kappa coefficient values greater than 0.74, 0.73 to 0.60 and 0.59 to 0.40 are classified as excellent, good and fair respectively.
A cross-sectional descriptive study was carried out in October 2020. Through a convenient sampling technique, 415 nursing students who were studying in different colleges of northern India were recruited as study participants. The sample size was estimated according to a rule of thumb, which recommends 3–10 samples for each item.
To evaluate the construct validity of the questionnaire, the principal axis factoring (PAF) with Promax Rotation was used. The sample size adequacy was determine by calculating Kaiser–Meyer–Olkin (KMO) value and a KMO value ≥0.8 was considered appropriate for factor analysis. The Bartlett's test of Sphericity was done and a significant value (P < 0.05) was considered suitable for model. The retaining of an item in the factor was determined using CV = 5.152/√(n − 2); (where n = sample size), and it was around 0.3. Therefore, any item, which had a value of correlation coefficient <0.3 with at least another item was, eliminated. In the next stage, the confirmatory factor analysis (CFA) was done using IBM SPSS (Inc., Chicago, IL, USA) Analysis of Moment Structure 23.0 to validate the results of extracted factors attained from exploratory factor analysis (EFA). The fit indexes of the model were determine by CMIN/df value <2, goodness of fit (GFI) value ≥0.95, and root mean square error of approximation (RMSEA) value <0.06, PCLOSE >0.05.
Convergent and divergent validity
The convergent and divergent validity of this proposed questionnaire were examined. The average variance extracted (AVE), maximum shared squared variance (MSV), average shared squared variance, and composite reliability (CR) were calculated. Convergent validity was established if AVE >0.5 and CR >AVE while divergent validity was confirmed if AVE > MSV.
The reliability of this questionnaire was estimated using internal consistency (Cronbach's alpha) and stability (test-retest) methods. Then, CR was measured and values more than 0.7 were considered as acceptable reliability.
The research project was approved by the Institutional Ethical Committee of Eternal University vide letter no. PMCH/IEC/21/0121. The objectives of the study were well informed to all participants and informed written consent for voluntary participation was obtained from each study participant.
To design the questionnaire, items of related concept were extracted through a systematic literature review and semi-structured individual interviews with 18 nursing students. Afterward, a pool of 92 items, which were found necessary and relevant, was prepared. The items were eliminated, edited, and merged after consultation and discussion with research team members and experts. Then, the best relevant items were chosen and a five-point Likert scale with 73 items was developed.
Findings of quantitative phase
In the phase of determining face validity, nine items were modified and edited based on the suggestions of the participating nursing students, and all items were found an impact score >1.5. Therefore, all the 73 items, further examined for content validity.
During this phase, based on the comments of experts 18 items were removed because of semantic similarity. Furthermore, some of the items were modified and revised as per experts' suggestions and 55 items were retained in the questionnaire. CVR was computed and 14 items did not meet the threshold (according to Lawshe' table CVR threshold ≥ 0.54 for 13 experts) and therefore, these 14 items were eliminated. In the phase of determining CVI of the questionnaire, the threshold of 0.79 and 0.90 was set for I-CVI and S-CVI, respectively. As 11 items did not meet the threshold and were omitted. Furthermore, S-CVI was calculated for 30 items and it was found 0.96. While kappa coefficient for all items in the scale ranged from 0.81 to 1.
The findings of the EFA revealed that the value of KMO and Bartlett's test was 0.933, and 6951.41, P < 0.001 which showed sample adequacy for performing EFA [Table 1]. Analysis was done by consideration of PAF with promax rotation and based on Eigenvalue and Kaiser Criterion (>0.5), EFA extracted four factors – ascendency, Snag, Readiness, and adoptability and prerequisites for online teaching, which had Eigenvalue 9.127%, 5.742%, 2.212%, and 1.421%, respectively. These all four factors explained 61.671% of the total variance [Table 2].
|Table 2: The results of conducting exploratory factor analysis on the perception to online teaching in nursing science students|
Click here to view
The findings of the first-order CFA, wherein all 30 items retained after EFA were included, did not show a satisfactory model fit. On the basis of modification indices and standardized residual covariance value, model was modified. After modification and drawing correlation between measurement errors, final model with 28 items was developed [Figure 2]. In this model, the factor loadings for SP-TOT were ranging from 0.51 to 0.87 and all were significant at P < 0.001, which showed that all of the items were appropriate for expounding their related factor. In addition, final model satisfied fitting indices as follows: Chi-square/degree-of-freedom ratio (CMIN/df) = 1.723, Goodness-of-Fit Index (GFI) = 0.908, normed fit index = 0.912, incremental fit index = 0.961, Tucker-Lewis index = 0.957, comparative fit index = 0.961, Root mean square error of approximation (RMSEA) = 0.043 and P value of close fit (PCLOSE) = 0.974. These results confirmed the final model perfectly. The findings of the goodness-of-fit indices are presented in [Table 3].
|Figure 2: First order confirmatory factor analysis of the 4 factors, 30 items model of SP-TOT. SP-TOT: Students' perception towards online teaching|
Click here to view
|Table 3: Goodness-of-fit indices for first- and second-order confirmatory factor analysis|
Click here to view
After the first order CFA, the second-order CFA was performed to assess the factors of perception to online teaching in nursing science students and the correlation between its factors. The second-order CFA also done to confirm the general concept of perception to online teaching in nursing science students. [Figure 3] presents the second-order CFA of the SP-TOT with the standardized factor loading coefficients.
|Figure 3: SP-TOT Construct: Second order confirmatory factor analysis. SP-TOT: Students' perception towards online teaching|
Click here to view
As presented in [Table 4], AVE and CR are greater than the desirable limit of 0.5 and 0.7, respectively, for all factors. Furthermore, the CR is more than AVE and AVE > MSV in all factors, which shows that convergent and divergent validity are adequate.
|Table 4: Internal consistency, convergent and divergent validity of students' perception toward online teaching questionnaire|
Click here to view
Twenty-five nursing students were requested to complete the questionnaire to estimate test-retest reliability. Two weeks later, the questionnaire was sent to the same 25 participants again and 19 participants completed and returned it. The reliability was computed and it was found 0.91, which indicated the stability of the questionnaire over time.
Internal consistency for the whole questionnaire with items was calculated through Cronbach's alpha coefficient and it was obtained 0.906. Furthermore, the Cronbach's alpha coefficient for Factor I (Ascendency), Factor II (Snag), Factor III (Readiness), and Factor IV (Prerequisites) was 0.926, 0.916, 0.896, and 0.720, respectively, which denoted good internal consistency for all subscales.
| Discussion|| |
The present study attempted to design and evaluate the psychometric properties of a questionnaire, for measuring the perception to online teaching in nursing science students.
In our study, both quantitative and qualitative methods, which included interview from nursing students and systematic review literature, were used to generate initial items for this questionnaire. However, a scale which appears most pertinent to our study used only quantitative approach to design attitude toward e-learning scale and had only one factor with nine items. Another scale was developed to measure student attitudes towards e-learning, used review of literature to generate initial items and had three factors with 40 items.
Findings of the qualitative phase revealed that nursing students showed ascendancy and readiness toward online teaching due to the easiness and attractions of the online teaching. Features such as quality improvement, easy learning, interesting and joyful learning and enhance communication and computer skills make students attract to adopt online teaching. These interesting aspects are main features in adopting nursing students to online teaching. Nine items in the designed questionnaire measure the ascendancy and readiness/attraction of online teaching.
Another concept was hidden barriers over the lack in interaction, privacy, peer support, control over group, and evaluate students' properly. Also increase social isolation and physical problems. These elements affected the perception to online teaching in nursing students. Some of the teachers were in favor of online teaching, while others did not want it. The findings revealed that the parents had a different opinion from students regarding online teaching and use of technology. This is an important aspect, which create a hidden barrier over subjective perception in nursing students. The factor “snag” was developed and it includes 10 items in this questionnaire.
Learners' interest and love toward technology as “techophilia” is one of the crucial factors in nursing students' perception to online teaching. Learners' interest toward new technology, feeling of its suitability and abilities to handle it properly play a vital role in online teaching. The factor “technophilia” is covered by six statements in this proposed questionnaire.
Another concept generated from the results of interviews was the basic requirements for online teaching. The concept of “prerequisites” represents the need of internet, computer/smartphone, and prior training. To determine the prerequisites is a key factor for online teaching. The proposed questionnaire covers prerequisites by three items.
It is suggested that there are at minimum four essential criteria to be satisfied by a measurement test to be applied for conducting research. These minimum criteria are as follows: One type of content validity, one type of construct validity, and two types of reliability (test-retest and internal consistency).
In a study validity was determined by only construct validity and reliability was estimated by using Cronbach's alpha. While in another study, validity was examined through content and construct validity and only Cronbach's alpha was used to evaluate reliability. Whereas in our study, the validity was determined by face (both quantitative and qualitative methods), content (CVR, CVI, and Kappa coefficient methods), construct (EFA and CFA and convergent and divergent) validity and reliability were estimated using test-retest method and Cronbach's alpha. Therefore, it can be said that the proposed questionnaire satisfied all minimum standards and can be an acceptable measurement questionnaire for assessing perception to online teaching in nursing science students.
This scale does not include a sample of other disciplines and all educational levels. Thus, this scale may not be applicable for students of other disciplines or other levels of education. However, the present study suggests that future researches should be conducted to design and validate scales to measure the perception to online teaching in other disciplines and different levels of education. We also suggest to use this scale to validate the model with other populations and settings by other researchers.
| Conclusion|| |
The validity and reliability of nursing students' perception to online scale were adequate. The scale can be used to measure the perception on this issue and this information can be used by policymakers and administrators for curriculum planning and improve education in nursing sciences students.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Gaur R, Mudgal SK, Kaur S, Sharma R. Undergraduate nursing students' attitude towards online classes during lockdown period in India: Imposed or interested? Int J Community Med Public Health 2020;7:3371-7.
Goswami K. COVID-19 lockdown: How the pandemic bringing change in Indian education system. India Today 2020;3:17-19.
Golband F, Hosseini AF, Mojtahedzadeh R, Mirhosseini F, Bigdeli S. The correlation between effective factors of e-learning and demographic variables in a post-graduate program of virtual medical education in Tehran University of Medical Sciences. Acta Med Iran 2014;52:860-4.
Gaur R, Mudgal SK, Dharni IT, Sharma R, Suyal N. Barriers encountered during online classes among undergraduate nursing students during COVID-19 pandemic in India. Int J Res Med Sci 2020;8:3687-93.
Garrison D. E-Learning in the 21st
ed. London: Taylor & Francis Group; 2011.
Van den Berg H, Manstead A, Van der Pligt J, Wigboldus D. The impact of affective and cognitive focus on attitude formation. J Exp Soc Psychol 2006;42:373-9.
Sangwan A, Sangwan A, Punia P. Development and validation of an attitude scale towards online teaching and learning for higher education teachers. TechTrends 2021;65:187-95.
Chong MC, Francis K, Cooper S, Abdullah KL, Hmwe NT, Sohod S. Access to, interest in and attitude toward e-learning for continuous education among Malaysian nurses. Nurse Educ Today 2016;36:370-4.
Xing W, Ao L, Xiao H, Cheng L, Liang Y, Wang J. Nurses' attitudes toward, and needs for online learning: Differences between rural and Urban Hospitals in Shanghai, East China. Int J Environ Res Public Health 2018;15:E1495.
Xing W, Ao L, Xiao H, Liang L. Chinese nurses' preferences for and attitudes about e-learning in continuing education: A correlational study. J Contin Educ Nurs 2020;51:87-96.
Opeyemi OZ, Adeyemi AA, Olajuwon TD, Oloruntosin ON, Oladeji BS. Perception of nursing students towards online learning: A case study of Lautech open and distance learning centre, Ogbomoso, Oyo State, Nigeria. Galore Int J Health Sci Res 2019;4:23-30
Abbasi S, Ayoob T, Malik A, Memon SI. Perceptions of students regarding E-learning during COVID-19 at a private medical college. Pak J Med Sci 2020;36:S57-61.
Ali N, Jamil B, Sethi A, Ali S. Attitude of nursing students towards e-learning. Adv Health Prof Edu 2016;2:24-9.
Williams B, Boyle M, Molloy A, Brightwell R, Munro G, Service M, et al
. Undergraduate paramedic students' attitudes to e-learning: Findings from five university programs. Res Learn Technol 2011;19:89-100.
Guillasper JN, Soriano G, Oducado RM. Psychometric properties of 'attitude towards e-learning scale' among nursing students. Int J Edu Sci 2020;30:1-3.
Al-Musawi N. Development and validation of a scale to measure student attitudes towards e-learning. J Teach Teach Educ 2014;2:1-12.
Creswell JW, Clark VL. Designing and Conducting Mixed Methods Research. California: Wiley Online Library; 2007.
Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res 2005;15:1277-88.
Sharma SK, Mudgal SK. Development and validation of a scale to measure attitude of people toward men in nursing profession. J Educ Health Promot 2021;10:54.
Polit DF, Yang F. Measurement and the Measurement of Change: A Primer for the Health Professions. Philadelphia: Wolters Kluwer; 2016.
Haynes SN, Richard D, Kubany ES. Content validity in psychological assessment: A functional approach to concepts and methods. Psychol Assess 1995;7:238-44.
Lawshe CH. A quantitative approach to content validity 1. Person Psychol 1975;28:563-75.
Mohd Suan MA, Tan WL, Soelar SA, Ali AM. The development and validation of the nurses' attitude towards conducting research questionnaire (NA2CRESQ). Ann Med Health Sci Res 2017;7:377-82.
Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health 2007;30:459-67.
Zamanzadeh V, Ghahramanian A, Rassouli M, Abbaszadeh A, Alavi-Majd H, Nikanfar AR. Design and implementation content validity study: Development of an instrument for measuring patient-centered communication. J Caring Sci 2015;4:165-78.
Plichta SB, Kelvin EA, Munro BH. Munro's Statistical Methods for Health Care Research. New York: Wolters Kluwer Health/Lippincott Williams & Wilkins; 2013.
Hair J, Anderson RE, Black B, Babin BJ. Multivariate Data Analysis.7th Edition, New York: Pearson Education; 2014.
Baghcheghi N, Koohestani HR, Karimy M. Design and psychometric properties of willingness to mobile learning scale for medical sciences students: A mixed-methods study. J Educ Health Promot 2020;9:141.
[Figure 1], [Figure 2], [Figure 3]
[Table 1], [Table 2], [Table 3], [Table 4]