Teacher performance evaluation model in COVID19 times
1st
Olger Gutiérrez Aguilar
Faculty of philosophy and humanities
Universidad
Nacional de San Agustín
Arequipa,
Perú
https://orcid.org/0000-0002-6657-7529
2nd Aleixandre Brian Duche Pérez
Faculty of Sciences and Social
Technologies and Humanities
Universidad
Católica de Santa María
Arequipa, Perú
https://orcid.org/0000-0001-9905-1489
3rd Ananí Gutiérrez Aguilar
Faculty of philosophy and humanities
Universidad Nacional de San Agustín
Arequipa, Perú
https://orcid.org/0000-0002-2916-2957
line 5: email address or
ORCID
Keywords— evaluation model, teaching performance, digital
competences, satisfaction index, Covid19, PLS-SEM.
I.
Introduction
The decisions taken by the different
national governments with respect to minimizing the
spread of Covid19 , in the face of a health emergency that several countries
have been experiencing, and on the other hand, the need to continue with the
educational service provided by universities, has led to processes of
adaptation and change of curricula from face-to-face to non-presential
modalities, in Peru the Ministry of Education and the National Superintendence
of University Higher Education (SUNEDU), have fostered in the
universities, processes of implementation of resources with digital technology,
using virtual learning environments, developing in turn, teaching skills in the
proper management of the Learning Management System (LMS), thus ensuring the
continuity of the provision of training service according to the criteria of
accessibility, adaptability, quality and other essential conditions for
learning.
On the other hand, there is consensus that the
quality of the educational service is achieved through the teachers performance, on the one hand, by adequately managing the learning process
of their students, especially those who experience processes of adaptation from
secondary education to university [1], and on the other hand the development and
implementation of appropriate and relevant strategies for the integration of
Information and Communication Technologies (ICT) into educational processes,
leading to the development and evaluation of e-learning models for higher
education [2].
Evaluating for decision making about teaching
performance is a need and a latent concern of the current universities.
Thus, the performance of the university
teacher, would be framed in the following dimensions, the Planning of the
teacher, which allows adequate management of resources and the time needed
before the start of teaching activities, proper management of educational
materials and content, and how these are being adapted to learning situations
by the Covid19 [3-5], the
Didactic Methodology, especially suitable for models of e-learning educational
management, the Relationship with the Students, in such a way that it favour
favourable environments for learning and the Evaluation and Academic Monitoring
of the student, related to aspects of tutoring and accompaniment of the student
from synchronous and asynchronous communication networks [6, 7].
In this particular context of non-face-to-face
educational experiences, the use of Web 2.0 concepts for virtual exchanges is
beginning to take hold, especially Synchronous Computer-Mediated Communication
(SCMC), which allows teachers and students to use video conference applications,
[8, 9] to
interact face-to-face, tools like Skype [10], Zoom [11], YouTube
[12], MS Teams,
are of unusual value, added to which is the proper management of the LMS -
Moodle [13] which
is based on a constructivist approach to education [14-17].
The learning experiences developed through
management in virtual learning environments are achieving significant effects,
as preceded by other research [18-20], and in
particular those related to health, which integrate the Zoom for the recording
of learning situations, distinguishing three methods. The first is self-study
which involves motivated study by each resident; the second is a pedagogical
method which requires independent interpretation by the study resident and the
third is a subsequent review with an experienced faculty mentor [21].
To measure or
control the satisfaction of the services and especially, the educational ones,
is a permanent concern for the educational institutions, especially for the
universities, the Net Promoter Score (NPS), is being used by diverse
organizations for this purpose, we have the properly business, health and
education, it is so that the organizations have incorporated this tool
inside their organizational culture and a practice of continuous improvement to
establish quality experiences. The
NPS is a contribution made by F. F. Reichheld [22, 23]. The NPS is the application of a rating scale, whose values range from
0 - 10, with a key question: "How likely are you to recommend the product
or service to a family member or friend" the same F. F. Reichheld, &
Markey, R. [24], for the book Spanish version he would title
it «La Pregunta Decisiva 2.0», thus asking the user of the service to rate it,
where 0 is «Very unlikely» and 10 is «I would definitely recommend it».
According to the results, clients are classified into promoters, passive and
detractors:
II.
Methodology
The sample for the validation of the model was made up by 358 students
from the professional school of psychology of the Universidad Católica de Santa
María de Arequipa - Peru. Table 1 presents the data. The students were randomly
selected from the 5 odd semesters, being 51% male and 49% female, whose ages
are between 17 and 23 years old. The total average was 19.86 (SD=2,015). The
instrument was implemented in June 2020 in a condition of social isolation and
quarantine, as decreed by Peru government with the aim of minimizing Covid19
contagion.
TABLE
I.
Sample studied
|
Subjects of the odd semester 2020 |
Total |
% |
|
Methodology of university work |
73 |
20.39% |
|
Child Developmental Psychology |
61 |
17.04% |
|
Behavioral Neuroscience |
56 |
15.64% |
|
Motivation and emotion |
57 |
15.92% |
|
Oral and written communication |
55 |
15.36% |
|
Introduction to Psychology |
56 |
15.64% |
|
Total |
358 |
100.00% |
For the construction of the questionnaire, five
observable variables were established, these are Planning; Educational
materials and contents; Methodology, Relationship with students and Evaluation
and academic follow-up, with 3
items, as shown in table 2:
TABLE
II.
Questionnaire structure for the teacher performance
assessment
|
Components |
Items |
|
Planning (PLA) |
3 |
|
Educational Materials and Content (MAT) |
3 |
|
Methodology (MET) |
3 |
|
Relationship with students (REL) |
3 |
|
Evaluation and academic monitoring (EVA) |
3 |
|
Total |
15 |
A scale from (1 to 5) is used to fill in
the questionnaire, with the value given based on the following criteria: 1
Never; (2) Hardly ever; (3) Sometimes; (4) Almost always and (5) Always. With
fully agreed criteria and through
an expert evaluation with background research of this nature.
III. Results and discussion
As part of the instrument analysis, the reliability statistics of the
instrument were processed with the SPSS software, resulting in a Cronbach's
alpha of 0.958, which means that it is an excellent measuring instrument. The analysis started with a
preliminary analysis of the 15 items by means of factorial analysis that aims
to condense the information into a series of original variables to smaller
series [25]. To establish the coupling level of the
components, the KMO Bartlett's Test of Sphericity were used, which is based on
the observed correlation coefficients [26, 27]. The process was carried out using the
Principal Component Analysis (ACP) technique, which allows the joint treatment
of variables formed from a combination of them, in order to analyze the
correlation matrix. The result of the KMO test is 0.958 which means an
excellent sample adequacy, as shown in table 3.
TABLE
III.
KMO test and Barlett sphericity test
|
Kaiser-Meyer-Olkin Measurement of Sampling Adequacy |
|
0.958 |
|
Bartlett's Test of Sphericity |
Aprox. Chi-square |
4069.925 |
|
|
gl |
105 |
|
|
Sig. |
0.000 |
After checking the sample adequacy, the communalities were evaluated, which are the sum of the factor weights squared by each of
the rows, these indicate the proportion of the variance explained by the common
factors for each variable. From the analysis it is observed that the exogenous
latent variables, present communalities that oscillate between 0.471 and 0.708,
which means that in the worst case the observed items would explain the model
in 47.1% and in the best case they would manage to explain the model in 70.8%.
The data are presented in the table 4.
TABLE
IV.
Communalities, first test with the SPSS
|
Items |
Initial |
Extraction |
|
In the development of the subject, a planning of
theoretical and practical activities is evident (PLA_01). |
1.000 |
0.663 |
|
The selection of contents for the learning, are the
adequate, updated and pertinent. (PLA_02). |
1.000 |
0.596 |
|
The teacher demonstrates mastery on the subject matter
(PLA_03). |
1.000 |
0.594 |
|
There is a variety of educational resources and activities
in the implementation of the subjects programmed during the semester
(MAT_01). |
1.000 |
0.643 |
|
The resources and activities used by the teacher
contribute to the achievement of the competencies (learning objectives)
(MAT_02). |
1.000 |
0.687 |
|
The teacher clearly explains the relationship between
educational resources and learning activities (MAT_03). |
1.000 |
0.708 |
|
The computer skills at the beginning of the course are
clearly established by the teacher (MET_01). |
1.000 |
0.674 |
|
The competences of the course, express results that are
measurable or achievable (MET_02). |
1.000 |
0.655 |
|
The relationship of the educational activities and
resources are clearly established in the virtual classroom (MET_03). |
1.000 |
0.641 |
|
The teacher's communication plan to interact with the
students in the course is clearly established (Teams, Chat, Whatsapp, E-mail
and others) (REL_01). |
1.000 |
0.505 |
|
The communication you establish with your teacher is
satisfactory (REL_02). |
1.000 |
0.554 |
|
The dialogue and the participation of the students are
encouraged (REL_03). |
1.000 |
0.600 |
|
The teacher, besides the exams, considers other possibilities
of evaluation of the learning (oral, practical work, portfolios, projects,
etc.) (EVA_01). |
1.000 |
0.471 |
|
The evaluations used are sequenced (of process), varied,
and adequate for the level of the subject (EVA_02). |
1.000 |
0.644 |
|
The evaluation of the teacher is objective (EVA_03). |
1.000 |
0.576 |
Extraction
method: principal component analysis
The total variance
explained must be greater than 0.50 to be considered acceptable,
therefore, percentages of 0.70 or more are desirable [28-30], being 1.00 the perfect reliability, in this sense, the results
obtained from the 5 components, would explain the model by 79.198%.
The analysis was carried out for each of
the factors, in terms of the factor: Planning, the Kaiser-Meyer-Olkin test -
KMO, determined a value of 0.721, which is equivalent to an acceptable sample
suitability. Table 5 explains the results of the component matrix.
TABLE
V.
Planning component matrix (PLA)
|
Component |
|
|
1 |
|
|
In the development of the subject, a planning of
theoretical and practical activities is evident. |
0.887 |
|
The selection of contents for learning, are adequate,
updated and relevant. |
0.869 |
|
The teacher demonstrates mastery of the subject matter. |
0.851 |
Main Component Extraction Method
The second factor,
educational materials and content, showed a KMO measure of sampling adequacy of 0.725, which is
equivalent to acceptable.
TABLE
VI.
Component matrix of educational materials and
content (MAT)
|
Component |
|
|
1 |
|
|
The teacher clearly explains the relationship between
educational resources and learning activities. |
0.903 |
|
The resources and activities used by the teacher contribute
to the achievement of competencies (learning objectives). |
0.893 |
|
There is a variety of educational resources and activities
in the implementation of the subjects programmed during the semester. |
0.856 |
Main Component Extraction Method
The third analysis factor: Relationship
to students, KMO test results, revealed a value of 0.727, which is equivalent
to an acceptable sample fit. The results of the component matrix are presented
in Table 7.
TABLE
VII. Relationship
with students component matrix (REL)
|
Component |
|
|
1 |
|
|
The communication you establish with your teacher is
satisfactory. |
0.885 |
|
Dialogue and student participation are encouraged. |
0.872 |
|
The teacher's communication plan for interacting with
students on the course is clearly established (Teams, Chat, Whatsapp, Email
and others). |
0.863 |
Main Component Extraction Method
The fourth factor,
methodology, showed a KMO measure of 0.731, which is equivalent to an acceptable sample adequacy. The results
of the component matrix are presented in Table 8.
TABLE
VIII. Methodology
component matrix (MET)
|
Component |
|
|
1 |
|
|
The competencies of the subject, express results that are
measurable or achievable |
0.876 |
|
The list of educational activities and resources are
clearly established in the virtual classroom. |
0.875 |
|
The computer skills at the beginning of the course are
clearly established by the teacher |
0.873 |
Main Component Extraction Method
The fifth factor: Evaluation and academic
monitoring, which gives a KMO measure of sampling adequacy of 0.714 which is
equivalent to an acceptable adequacy; the component matrix, is detailed in table 9.
TABLE
IX.
Evaluation and academic monitoring component
matrix (EVA)
|
Component |
|
|
1 |
|
|
The teacher, in addition to the exams, considers other
possibilities of evaluation of the learning (oral, practical work,
portfolios, projects, etc.). |
0.850 |
|
The teacher's evaluation is objective. |
0.849 |
|
The evaluations used are sequential (process), varied, and
appropriate to the level of the subject |
0.847 |
Main Component Extraction Method
The preliminary analysis, called exploratory, allowed us to obtain the
necessary inputs to proceed with the confirmatory analysis, for which we used
SmartPLS (v.3.3.2), which is a tool for the modeling of structural equations,
based on the variance that uses the least-squares path modeling method, devised
by Christian M. Ringle, Sven Wende and Jan-Michael Becker[31].
PLS-SEM (Partial Least Square Structural
Equation Modeling), is a second generation multivariate method called Partial
Least Square Structural Equation Modeling [32], which is composed by two elements, the first one proposes a
measurement model that tries to analyze the factorial loads of the variables
through their items, in such a way that the relationship with their
corresponding variables can be established. With this method, the reliability
and validity of the dimensions of the theoretically supported model are
evaluated. On the other hand, we have the structural model, which analyzes the
causal relationships between independent and dependent latent variables, this
last variable is the Satisfaction Index - Net Promoter Score NPS.
Figure 1, presents the internal
consistency of the model expressed through Cronbach's alpha to establish the
degrees of reliability of the indicators, the results range from 0.805 to
0.860, with acceptable values for this from 0.700 to more [33].
Fig. 1. Cronbach's Alpha of the
SmartPLS model
As regards the
analysis of the reliability and validity of the model, explained from R2
(Pearson's coefficient) for the dependent variable: satisfaction index - SPL, according to the data obtained R2
has a value of 0.561, which is equivalent to saying that 56.1% of the variance
is explained by the model, as can be seen in figure 2.
Fig. 2. SmartPLS model R-square
The matrix of
external loads, explains the measurement model and is composed of the
indicators and their trajectories that connect them to their factors, the
result represents the contribution of the indicators to the definition of the
dependent variable. The values of the external loads could have a variation
between 0 and 1, so the closer to the unit they are, the stronger they are to
explain the validity of the model.
In table 10, the
matrix of external loads is presented with their respective values, these are
considered acceptable, and therefore none of the items were discarded.
TABLE
X.
External loads matrix – smart PLS
|
|
(EVA) |
(MAT) |
(MET) |
(PLA) |
REL) |
NPS |
|
EVA_01 |
0.834 |
|
|
|
|
|
|
EVA_02 |
0.855 |
|
|
|
|
|
|
EVA_03 |
0.856 |
|
|
|
|
|
|
MAT_01 |
|
0.831 |
|
|
|
|
|
MAT_02 |
|
0.908 |
|
|
|
|
|
MAT_03 |
|
0.91 |
|
|
|
|
|
MET_01 |
|
|
0.868 |
|
|
|
|
MET_02 |
|
|
0.889 |
|
|
|
|
MET_03 |
|
|
0.866 |
|
|
|
|
PLA_01 |
|
|
|
0.898 |
|
|
|
PLA_02 |
|
|
|
0.875 |
|
|
|
PLA_03 |
|
|
|
0.833 |
|
|
|
REL_01 |
|
|
|
|
0.858 |
|
|
REL_02 |
|
|
|
|
0.893 |
|
|
REL_03 |
|
|
|
|
0.868 |
|
|
NPS |
|
|
|
|
|
1 |
The internal
consistency of the items of the latent variables of the instrument (Planning;
Educational materials and content; Methodology, Relationship with students and
Evaluation and academic follow-up), the Composite Reliability is within the
values ranging from 0.885 to 0.914, which is equivalent to satisfactory. Table
11.
TABLE
XI.
Construct reliability and validity
|
|
Cronbach’s alpha |
rho_A |
Composed Reliability |
Mean extracted variance (AVE) |
|
Evaluation and academic monitoring |
0.805 |
0.81 |
0.885 |
0.719 |
|
Educational materials and content |
0.86 |
0.883 |
0.914 |
0.781 |
|
Methodology |
0.846 |
0.852 |
0.907 |
0.765 |
|
Planning |
0.838 |
0.851 |
0.902 |
0.755 |
|
Relationship with students |
0.844 |
0.849 |
0.906 |
0.762 |
|
Satisfaction Index (NPS) |
1 |
1 |
1 |
1 |
Taking into account
the correlation coefficients in all its dimensions, the reliability and
validity of the construct, correlated with the satisfaction index variable, the
5 factors proposed in the model and that are manifested in the construct are
statistically significant. The values of the average variance extracted -
Average Variance Extracted - (AVE), obtained with SmartPLS, range from 0.719 to
0.781, table 11. These results exceed the minimum recommended value of 0.50 [34]. The assessment of this analysis allows us to
conclude that convergent validity is acceptable in all components of the model.
As for composite reliability, for a social science approach,
different authors have suggested its application [35,
36], Therefore, if the values are greater than 0.6 then high levels of
internal consistency reliability are demonstrated in each of the reflective
latent variables of the proposed model. However, the discussion today around
the application of PLS, it is suggested instead of using Cronbach's alpha and
composite reliability for the analysis, it is recommended to use the
coefficient (rho_A) to verify the reliability of the values obtained in the
construction and design of the PLS [37], the results obtained in the (rho_A) should have values of 0.7 or
higher to demonstrate composite reliability, the results obtained are between
0.81 and 0.883
There are 2 accepted ways to verify the
discriminant validity of the model, the first one corresponds to the one
proposed by Fornell and Larcker [38], which suggests that the square root of AVE in each variable, its
resulting values should be greater than the results of the correlation between the latent variables, the result
indicates that the discriminant validity is well established, as shown in table
12.
TABLE
XII. Fornell-Larcker
criteria analysis for verification and discriminant validity
|
EVA |
NPS |
MAT |
MET |
PLA |
REL |
|
|
EVA |
0.848 |
|||||
|
NPS |
0.648 |
1 |
||||
|
MAT |
0.776 |
0.688 |
0.884 |
|||
|
MET |
0.795 |
0.708 |
0.825 |
0.874 |
||
|
PLA |
0.727 |
0.616 |
0.86 |
0.788 |
0.869 |
|
|
REL |
0.707 |
0.658 |
0.71 |
0.757 |
0.674 |
0.873 |
The second way to
verify the discriminant validity of the model is proposed by Henseler, Ringle and Sarstedt [39] based on the heterotrait-monotrait correlation (HTMT), whose
essence is the multitrait-multimethod matrix. In an ideal model, the
heterotrait correlations should be smaller than the monotrait correlations,
therefore, the HTMT ratio should be below 1.0. Table 13.
TABLE
XIII. Heterotrait-monotrait
(HTMT) correlations
|
EVA |
NPS |
MAT |
MET |
PLA |
REL |
|
|
EVA |
|
|
|
|
|
|
|
NPS |
0.719 |
|
|
|
|
|
|
MAT |
0.931 |
0.731 |
|
|
|
|
|
MET |
0.958 |
0.766 |
0.963 |
|
|
|
|
PLA |
0.881 |
0.668 |
1.024 |
0.936 |
|
|
|
REL |
0.851 |
0.714 |
0.833 |
0.895 |
0.801 |
|
Similarly, as part
of the proposed model validation process, it was considered necessary to obtain
the variance inflation factor (VIF), which will make it possible to avoid the
collinearity problem, if the values exceed 5.0 they will indicate collinearity
within the indicator [34]. The results of the collinearity test reveal
that the rates range from 1,731 to 2,503 and therefore satisfy the criteria,
Table 14.
TABLE
XIV. Results
of the indicator collinearity test
|
|
VIF |
|
EVA_01 |
1.749 |
|
EVA_02 |
1.731 |
|
EVA_03 |
1.743 |
|
MAT_01 |
1.907 |
|
MAT_02 |
2.374 |
|
MAT_03 |
2.503 |
|
MET_01 |
2.019 |
|
MET_02 |
2.055 |
|
MET_03 |
2.038 |
|
PLA_01 |
2.163 |
|
PLA_02 |
1.991 |
|
PLA_03 |
1.831 |
|
REL_01 |
1.93 |
|
REL_02 |
2.145 |
|
REL_03 |
2.015 |
IV. Conclusions
The methodology of PLS-SEM, has allowed
to evaluate the proposed model, using for this purpose an initial exploratory
analysis and later a confirmatory analysis using the SmartPLS, as part of the
model, is the validation of the instrument for the evaluation of the teaching
performance, key aspect in the quality of the university educational services,
which consists on 5 factors, as observed latent variables, like they are
Planning (PLA); Educational Materials and Content (MAT); Methodology (MET);
Relationship with students (REL); Evaluation and academic follow-up (EVA), as a
dependent variable, the Net Promoter Score (NPS).
Statistical tests for exploratory and
confirmatory analyses have revealed acceptable results in their assessment,
therefore, it is stated that the instrument is reliable and consistent,
therefore, its relationship with the variable (NPS), validates the model in its
totality.
Finally, it is concluded that, the
statistical analysis allowed by the PLS-SEM is a technique of great interest for
social research, based on an alternative approach of structural equation
modeling with partial least
squares.
V. Future works
As future work, the
instrument will be applied to university students studying in the virtual modality; the model
will also serve to implement a quality system in terms of university teacher
performance.
References
[1] A. B. Duche-Perez,
F. M. Paredes-Quispe, and O. A. Gutierrez-Aguilar, "The Transition from
high school to university: Identifying internal and external factors for a
successful transition in peruvian students of Architecture and
Engineering," in EDUNINE 2019 - 3rd
IEEE World Engineering Education Conference: Modern Educational Paradigms for
Computer and Engineering Career, Proceedings, 2019, doi: 10.1109/EDUNINE.2019.8875751.
[Online]. Available: https://www.scopus.com/inward/record.uri?eid=2-s2.0-85074787943&doi=10.1109%2fEDUNINE.2019.8875751&partnerID=40&md5=4ddb2e09ff96cab86778bcd102f446f0
[2] A. B. Duche Perez, F. M. Paredes Quispe, O.
A. Gutierrez Aguilar, and D. Arias Chavez, "Development and evaluation of
an e-learning model of teaching-learning in multidisciplinary education
subjects in technological higher education," 2019, pp. 359-366, doi:
10.1109/LACLO49268.2019.00067. [Online]. Available: https://www.scopus.com/inward/record.uri?eid=2-s2.0-85081124307&doi=10.1109%2fLACLO49268.2019.00067&partnerID=40&md5=d8e062d5301f250565cbc6431d4a1fc5
[3] L. C. Warth, N. O. Noiseux, S. T. Duncan, S.
B. Daines, and C. R. Mahoney, "How Has COVID-19 Affected Our Orthopedic
Implant Industry Partners? Implications for the Surgeon-Industry Relationship
in 2020 and Beyond," The Journal of
Arthroplasty, vol. 35, no. 7, Supplement, pp. S56-S59.e10, 2020/07/01/
2020, doi: https://doi.org/10.1016/j.arth.2020.04.063.
[4] C. Carolan, C. L. Davies, P. Crookes, S.
McGhee, and M. Roxburgh, "COVID 19: Disruptive impacts and transformative
opportunities in undergraduate nurse education," Nurse Education in Practice, vol. 46, p. 102807, 2020/07/01/ 2020,
doi: https://doi.org/10.1016/j.nepr.2020.102807.
[5] E. E. Lewis, L. J. Taylor, J. L. Hermsen, D.
P. McCarthy, and A. G. Fiedler, "Cardiothoracic Education in the Time of
COVID-19: How I Teach It," The
Annals of Thoracic Surgery, 2020/04/10/ 2020, doi: https://doi.org/10.1016/j.athoracsur.2020.04.002.
[6] J. Brouwer, C. Downey, and C. Bokhove,
"The development of communication networks of pre-service teachers on a
school-led and university-led programme of initial teacher education in
England," International Journal of
Educational Research, vol. 100, p. 101542, 2020/01/01/ 2020, doi: https://doi.org/10.1016/j.ijer.2020.101542.
[7] S. Rubinelli, K. Myers, M. Rosenbaum, and D.
Davis, "Implications of the current COVID-19 pandemic for communication in
healthcare," Patient Education and
Counseling, vol. 103, no. 6, pp. 1067-1069, 2020/06/01/ 2020, doi: https://doi.org/10.1016/j.pec.2020.04.021.
[8] C. A. Lenkaitis, "Teacher candidate
reflection: Benefits of using a synchronous computer-mediated
communication-based virtual exchange," Teaching
and Teacher Education, vol. 92, p. 103041, 2020/06/01/ 2020, doi: https://doi.org/10.1016/j.tate.2020.103041.
[12] M. Farag, D. Bolton, and N. Lawrentschuk,
"Use of YouTube as a Resource for Surgical Education—Clarity or
Confusion," European Urology Focus, vol.
6, no. 3, pp. 445-449, 2020/05/15/ 2020, doi: https://doi.org/10.1016/j.euf.2019.09.017.
[14] L.
Vygotsky and J. P. T. Abadía, Pensamiento
y lenguaje. Grupo Planeta, 2013.
[15] J. Piaget, The
child's construction of reality. London, 1955.
[16] J.
Bruner, "Aprendizaje por descubrimiento," NYE U: Iberia, 2011.
[19] P. J. Lewis, T. M. Catanzano, L. P. Davis, and
S. G. Jordan, "Web-based Conferencing: What Radiology Educators Need to
Know," Academic Radiology, vol.
27, no. 3, pp. 447-454, 2020/03/01/ 2020, doi: https://doi.org/10.1016/j.acra.2019.05.017.
[20] J. Chiodini, "Online learning in the time
of COVID-19," Travel Medicine and
Infectious Disease, vol. 34, p. 101669, 2020/03/01/ 2020, doi: https://doi.org/10.1016/j.tmaid.2020.101669.
[21] C. H. Li
et al., "Virtual Read-Out: Radiology Education for the 21st Century
During the COVID-19 Pandemic," Academic
Radiology, vol. 27, no. 6, pp. 872-881, 2020/06/01/ 2020, doi: https://doi.org/10.1016/j.acra.2020.04.028.
[22] F. F. Reichheld, "The one number you need to grow," Harvard business review, vol. 81, no. 12, pp. 46-55, 2003.
[Online]. Available: http://marketinglowcost.typepad.com/files/the-one-number-you-need-to-grow-1.pdf.
[24] F.
F. Reichheld, & Markey, R., La
Pregunta Decisiva 2.0. Madrid, España: LID Editorial, 2012.
[28] M.
A. Cea, Análisis multivariabl: teoría y
práctica en la investigación social. Editorial síntesis, 2004.
