Printer Friendly

Analisis psicometrico de una escala de percepcion sobre la utilidad de Moodle en la universidad.


Learning Management Systems (LMS) have acquired a significant relevance as support tools for learning (Britain and Liber, 1999; Melton, 2006; Ellis, 2009) in Higher Education, due to the fact that they allow us to manage contents, to establish synchronous and an asynchronous communication and to manage student assessment (Ross, 2008). However, these organisational changes entail a modification of the tutoring processes and types of activities, while they enable a constant and continuous monitoring of the students' evolution (Antonenko, Toy and Niederhauser, 2004). Their main interest lies in the space that Information and Communication Technologies (ICT) take up in all sectors of the global society (Castells, 1999; Cohen and McCuaig, 2008). This fact opens up new fields of ICT-related research, such as the one explored in this study.

We can consider an LMS as "a software system that combines a number of different tools that are used to systematically deliver content online and facilitate the learning experience around that content" (Weller, 2007, p. 5). These environments have evolved in the past few years and they have been used as a complement in diverse learning formats and contents, from a face-to-face context to an exclusively virtual one (e-learning), including mixed or b-learning contexts (DeNeui and Dodge, 2006; Conrey and Smith, 2007; Vaughan, 2007).

Their didactic contribution lies in the combination of elements that are specific to traditional teaching (information presentation, accessibility to materials, evaluation of student work) (Yueh and Hsu, 2008), and a series of additional elements that provide multiple communication pathways (including learning-centred social media) (Ellison, Steinfield and Lampe, 2007). Thus, this supports the adjustment of "traditional tools" to the new teaching-learning scenarios conditioned by the potential provided by the use of LMS (Perez and Garcias, 2007).

LMS entail innovative elements that have different characteristics from face-to-face education and allow students to take a more active role (Silva Quiroz, 2011). Their integration in the different educational stages favours both the virtual learning and the student-teacher interaction. Consequently, the student's role is redefined as an individual who generates and transmits knowledge on the net. To this end, the student needs a series of skills aimed at self-regulating the leaning process and favouring the construction of knowledge through information searching, selection, transformation and dissemination (Barbera and Badia, 2004).

To attend to the characteristics of LMS and the need to promote an active role for the students, different free-access or commercial-access platforms have been designed and implemented (Martin-Blas and Serrano Fernandez, 2008): Atutor, Claroline, ILIAS, Chamilo, Moodle, LRN, Teleduc, FLE3, Ganesha, etc. The institutions' choice is based on criteria such as the users' needs, the cost and the potential number of users (Martin-Blas and Serrano Fernandez, 2008).

In the educational field, many universities have decided in favour of the Moodle platform. It is an open resource based upon pedagogycal principles (Cole y Foster, 2007; Goyal y Puhorit, 2010), which integrates diverse multimedia resources. For these very reasons, Moodle has become one of the most used LMS in Higher Education (Aydin and Tirkes, 2010; Saito and Ulbricht, 2012; Williams van Rooij, 2012).

Moodle is presented as a platform that provides the necessary tools for virtual education (Aydin and Tirkes, 2010; Saito and Ulbricht, 2012; Williams Van Rooij, 2012). In addition, it promotes new learning, easing the access to the material in an organised way (Peat and Franklin, 2002).

Thus, Moodle facilitates the development of the teaching-learning processes in e-learning, b-learning and face-to-face education through elements such as interaction (Swan, Shea, Fredericksen, Pickett, Pelz and Maher, 2002), usability (Kirner and Saraiva, 2007) and social presence (Richardson and Swan, 2003).

In this sense, it becomes clear that LMS improve learning outcomes (Martin-Blas and Serrano-Fernandez, 2009; Nunez et al., 2011; Escobar-Rodriguez and Monge-Lozano, 2012), and that teachers who use virtual resources increase student attention and participation, enabling a more meaningful learning process (Soyibo and Hudson, 2000). Other authors such as Steyaert (2005) prove that both LMS and the use of the internet, allow teachers to organise the contents by topics and to manage the organisation of the subject in a more efficient way, enabling a simpler visualisation of the syllabus (Peat and Franklin, 2002).

Therefore, the aim of LMS in general, and Moodle in particular, entails interaction with the information and joint work between teachers and students. However, reality seems to be far from this aim, and these environments are often used as mere document repositories.

In this context, the interest of studying the students' perception of the virtual platform's usefulness in Higher Education processes becomes apparent. With the aim of analysing and trying to collaborate in the improvement of the implementation of LMS, we have designed a scale for its real use in university contexts.

It is a fact that the usefulness perceived by the user is extremely important for the success of a technological tool such as LMS (Davis, 1993; Friedrich and Hron, 2010; Sorebo, Halvari, Gulli and Kristiansen, 2009). Several studies based on the Technology Acceptance Model (Davis, Bagozzi and Warshaw, 1989), developed from the Theory of Reasoned Action (Ajzen and Fishbein, 1980; Rus, Pina, Sanchez and Martinez, 2011), which explains the current use of technologies based on the attitudes of the user towards the very use of the technologies, the usefulness and the perceived ease of use, show that there is a positive relationship between the use of a given technology and these three variables. Thus, people's beliefs about an object will influence their attitude towards it. Therefore, the level and frequency of use of an LMS will be partly affected by the individual attitudes towards it. If we wish the LMS to be integrated in the teaching methods we need to promote adequate attitudes towards these tools.

Based on the scarcity of studies in this field (that provide valid and reliable measuring instruments) that help to design and evaluate empirical models based on the Technology Acceptance Model, it seems useful to design a scale of the student's perception of the usefulness of Moodle within the learning process. Therefore, the ultimate aim of this study is to create a tool that provides valuable information on the integration of Moodle in the university context and that allows for improvements in the teaching-learning environments that integrate LMS



From the total population of university students enrolled in Educational Sciences degrees in the academic year 2011-12 we established a non-probability accidental sample (Arnal, Rincon and Latorre, 1992) comprising 754 subjects. Thus, considering an infinite population, and supposing maximum variability (p = q = .5) and a k-sigma = 2, the error obtained for the sample is 3.64%.

Variables and instruments

The instrument consists of a scale designed to evaluate the students' perception of the usefulness of Moodle in the university teaching processes. We chose a survey study based on a quantitative instrument.

After the content validation process (with judges), we obtained an instrument composed by 40 items (see table 1). It is a Likert-type scale (Morales, Urosa and Blanco, 2003), with four response options (0=not at all, 1=not much, 2=quite a lot, 3=a lot/very much). The items are distributed in five theoretical dimensions (Moore and Iida, 2010; Palmer and Holt, 2010; Al-Busaidi and Al-Shihi, 2012):

* Contents (9 items): Level of suitability of the transference of contents to a virtual platform.

* Activities (11 items): Student perception on the real usefulness of Moodle as a work environment.

* Assessment (8 items): Assessment strategies used in the platform.

* Interaction (4 items): Level of relationship between students and teachers within the platform.

* Learning (8 items): Student opinion on the degree to which the platform facilitates learning.

Content validity is guaranteed through expert judges. The first draft, composed of 32 items, was assessed based on clarity and relevance criteria by eight experts in educational technology, three experts in research methodology and four university students. Based on these assessments we drew up the survey that was administered to the sample.

Each expert judge had to indicate whether each of the 32 items assigned to the theoretical dimensions was adequate. In the case an item was deemed inadequate, the judge specified some recommended modifications. Given that we have a qualitative and multi-judge measurement, we decided to calculate the concordance index and the Kappa index (Cohen, 1960) to check the level of interjudge agreement, considering each questionnaire item as an observed subject, in particular based on the calculation of the free-marginal multi-rather Kappa (Brennan and Perdiger, 1981). This index is recommended when the judges are unaware a priori of the number of cases they must assign to each category in the different observations, which is the case here (Brennan and Prediger, 1981). Thus, we obtained a concordance index of 83.75% and a Kappa index of .67, from which we can consider the existence of a good level of interjudge agreement (Landis and Koch, 1977).


The first version of the questionnaire was designed in November 2011. After the judges' assessment, the instrument went from 32 items to 40.

The data collection was carried out between January and June 2012 through an on-line survey procedure. The resource that integrated the questionnaire guaranteed anonymity, which avoids social desirability bias.

We tried to reduce the measurement errors which stemmed from the survey respondents by standardizing the conditions for administering the survey. Each survey taker was given precise instructions so that they would pass on systematic information to the participants.

Given the low partial non-response index, which is not over 1.5% in any item (the highest frequency amounts to 10 lost values), considering in consequence that the effect of imputation is going to be minimum, we applied classical imputation techniques. On the other hand, taking into account the multivariate techniques used below, to avoid overestimation of the correlation coefficients among the variables, we decided to substitute the lost values with the unconditional mean imputation (Medina and Galvan, 2007).

Data analysis

Even though, given the nature of the scales used in this study, it might be preferable to use alternative methods other than those based on Person's correlation matrix, (Lopez-Gonzalez, Perez-Carbonell and Ramos, 2011; Lopez-Gonzalez, 2012) or the use of the polychoric correlation matrix (Elosua Oliden and Zumbo, 2008), the vast scientific evidence developed in the '70s and '80s of the 20th century (Hofacker, 1984; Labovitz, 1967, 1970; Morales Vallejo, 2006) suggests that the use of these methods entails small biases (Nunnally, 1978).

In this study, given that "any answer codification, consistent with the conceptual order, does not distort the statistical conclusions to an acceptable degree" (Morales Vallejo, 2006, p. 39), we consider the Likert-type scale used as an interval scale, and we used Pearson's correlation for the calculation of the matrices. However, we conducted a previous analysis to confirm this, which consisted in comparing Pearson's correlation matrix and the polychoric correlation matrix obtained, confirming that the differences among the coefficients, always in favour of the polychoric correlation matrix, do not exceed in any case a 0.2 value, and in 87% of the cases they don't reach a 0.1 value.

We started by studying the inter-item correlation between the groups of items within each dimensions. When we obtained excessively low or high correlation indexes we assessed their elimination by studying their theoretical importance and/or their co-linearity with other items of the factor.

We checked the previous assumptions of univariate and multivariate normality, homoscedasticity and non multi co-linearity, with the aim of selecting the most suitable estimation method. Once checked, we studied the dimensional characteristics of each theoretical factor and the possible reduction of dimensions of their item groups through a Principal Components Analysis (Martorell, Gonzalez, Ordonez and Gomez, 2011).

After that, we measured the reliability through Cronbach's alpha, both for the whole of the scale and the groups of items that make up each principal component, and we also measured the reliability of the factors through the Composite Reliability Index (CRI). Finally, we studied both the convergent and the discriminant validity through the calculation of the Average Variance Extracted (AVE).

Once the reliability and validity of the dimensions that make up the scale had been proven, we confirmed the model's goodness-offit through a confirmatory factor analysis (CFA). We studied the normed absolute fit indexes, such as Root Mean Square Error of Approximation (RMSEA), the Root Mean Square Residual (RMR) or the Goodness of Fit

Firstly, item 06 is considered of theoretical relevance because it belongs to the factor that addresses the resources related to class presentations, so we did not consider discarding it. Item 09, apart from being vague and excessively generic in its definition, addressing generally the interest of revising any kind of resource, slightly overlaps with item 04. In addition, the perspective provided by item 04 is closer to what we desire to convey with the item, so we decided to remove item 09 from the definitive scale.

Regarding the rest of theoretical dimensions, we do not observe indexes below .4 or above .8. On the other hand, by analysing the inter-item correlations for each dimension, we obtained acceptable correlation items for the most part, and in any case above .75.

Index (GFI), and also the incremental fit indexes, including the Adjusted Goodness of Fit Index (AGFI), the Normed Fit Index (NFI) or the Relative Fit Index (RFI).

The results of this study have been obtained by using the statistical programme IBM SPSS v.21, together with its extension AMOS, Excel y Epidat 3.1.


Item analysis

The study of item-element correlations, considering as low correlations those below or close to .4 (Morales Vallejo, 2006), focuses its attention only on two items of the theoretical dimension contents (see table 2).

Previous assumptions

After checking the co-linearity multi colinearity assumption, we obtained Variance Inflation Factor values below 2.5, and condition indexes below 25 points for all items with regard to their theoretical dimension. On the other hand, the value of the correlation coefficient between the items of different dimensions does not exceed a .75 in any case.

Regarding normality and homoscedasticity, being CFA a multivariate technique, we must check both the univariate and the multivariate normality. The Kolmogorov-Smirnov test locates, in every case, the contrast statistic in the reject area of Ho (a=.05). Table 3 shows how all null hypotheses are rejected with a p value below .001. On the other hand, Mardia's Coefficient (Mardia, 1970) results in a standardized score of 67.669 points, meaning that the joint distribution of the items is very far from the normal multivariate distribution. Thus, the normality of the data is not confirmed, and we will select an estimation method that does not imply this previous assumption.

Principal Components Analysis

Since principal component analysis does not demand the assumptions of normality and homoscedasticity (Garcia Jimenez, Gil Flores, and Rodriguez Gomez 2000), we proceeded with its direct application on all items, forcing the extraction of 5 factors.

With the model, a 50.9% of the total variance is explained, from which the first factor (in the initial solution, without rotating) explains a 30.3%. On the other hand, the sedimentation graph does not show evidences of the scale's unidimensionality. Regarding the rotated solution, the configuration matrix (oblimin method) suggests that the theoretical assumptions on the dimensionality of the questionnaire can be correct, as shown in table 4.

Extraction method: Principal components analysis. Rotation method: Oblimin normalization with Kaiser.

The three last items of the dimension activities have a saturation below .4 on the dimension itself. Despite this, we decided to keep them because we deemed them to be theoretically important. The same goes for the first item of the dimension learning.

Likewise, we found that the dimensions learning and interaction contain shared information, saturating a good part of their items in the fourth dimension. After the content analysis if the items involved, we verified this similarity, taking into account that learning is, ultimately, a shared and social process (Vigostky, 1995).

If we apply the principal component analysis dimension after dimension, we obtain unidimensional structures except

Confirmatory Factors Analysis

Given the evidences of the noncompliance with the normality assumption, and that the measurement scale for all the items is identical, we chose to use the estimation method of unweighted least squares (Bollen, 1989; Byrne, 2001; Kline, 2005). Choosing this method over other non-parametric ones responds to its analogue properties of OLS estimation (Levy Mangin, 2006) and to its good behaviour towards the existence of low factorial loads (Ximenez and Garcia, 2005), which is evidenced in the study.

The implemented theoretical model is shown in figure 1. There are five first order factors, and a second order one, with the exception of the first order factor contents which, as explained before, is divided in two subdimensions. for the dimension contents, where there are 2 principal components that explain 58.31% of the variance. By analysing the items of this dimension, we observe how the first five are related to general aspects of the platform's content, and the other 3 refer to specific resources. In other cases, we obtained an explained variance of 43.4% (activities), 45.8% (assessment), 64.7% (interaction) and 56.1% (learning).

Table 5 shows the correlation between the factors obtained in the exploratory factor analysis. The value of the coefficients indicates that the factors have shared information, which makes the rotation method the most suitable for this case.


The parameters of the model, which estimation is presented in table 6, have factorial loads with scores on each factor exceeding .4 in every case, and in average they are above .6. Therefore we can conclude that every item properly contributes to its dimension. The square multiple correlations are generally low, which indicates there is a good part of the variance that is not explained by the factors. However, the CRI and Cronbach's alpha obtained seem to indicate that the instrument has an acceptable reliability level, both at global and factor level (Levy Mangin, 2006). On the other hand, the AVE values are below .5 in most of the factors, and therefore the validity (both convergent and discriminant) is not compromised (Kline, 2005). However, given the scarcity of scales that collect information on the different dimensions involved in the educational use of Moodle, this instrument represents a starting point in this research field. This way, the questions introduced in the questionnaire have a great amount of noise (non-explained variance), valuable information contained by these variables, although it is placed in acceptable percentages, it is lower than desirable.

Regarding the model's goodness of fit, we obtain indexes that show good fit (Bollen, 1989; Byrne, 2001; Kline, 2005; Levy Mangin, 2006) both global (GFI=.985; RMSEA=0.47; RMR=.022) and incremental (AGFI=981; NFI=977; RFI= .976), by being below .05 in RMSEA and RMR and above .95 in the rest of the cases. As for the global fit, the results indicate that the model makes a satisfactory prediction of the data covariance matrix. The results regarding incremental fit suggest that the proposed model is adequate in comparison with the null model, and therefore the proposed relations have substantial weights


Moodle is one of the more complete and adequate platforms for its implementation in Higher Education (Aydin and Tirkes, 2010; Saito and Ulbricht, 2012; Williams van Rooij, 2012). This fact is evidenced because Moodle provides three essential resources: the possibility of supplying contents and activities online, interactive assessment (Ross, 2008) and the flexible interaction and communication between the teacher and the students- (Ellison et al., 2007). In this sense, both the use of Moodle and on-line materials and resources enhance and/or improve the learning outcomes (Martin-Blas and Serrano-Fernandez, 2009; Nunez et al, 2011; Escobar-Rodriguez and Monge-Lozano, 2012).

The success of an LMS depends on many factors (Davis et al., 1989), among which is the user's perceived usefulness of the LMS itself. In the scientific literature, we can locate a large number of studies which determine the keys of the success of an LMS. While these studies often contain student perception scales on the usefulness of the LMS, in many cases these scales don't have the psychometric properties that would be necessary in order to ensure the reliability and validity of the collected information (Lin, 2008; Weaver, Spratt and Nair, 2008; Klobas and McGill, 2010; Al-Busaidi and Al-Shihi, 2012).

In many cases, the researchers only include ad hoc designed scales without an associated psychometric study (Ozkan and Koseler, 2009; Naveh, Tubin and Pliskin, 2010; Palmer and Holt, 2010; Rubin, Fernades and Avgerinou, 2013), or with a very simple and superficial study. In other cases, the sizes of the samples obtained in order to implement the psychometric study are small (Lin, 2008; Ozkan and Koseler, 2009; Al-Busaidi and Al-Shihi, 2012) and largely limit its results. On the other hand, the dimensions included in these scales are vague and highly varied, depending on the interest of the specific research.

This way, the present study overcomes many of these obstacles, because based on the set objectives, we designed a scale with content validity criteria, to subsequently validate the psychometric properties through the application of statistical techniques from a representative sample.

Given the importance of the perceived usefulness as predictor variable of the success in the implementation of a technological tool such as the LMS (Davis, 1993; Sorebo et al., 2009; Friedrich and Hron, 2010), the described scale constitutes an instrument that is valid and reliable on a psychometric level, which allows the organisations and teachers to reflect on the weaknesses of Moodle as a complement of face-to-face education.

On the other hand, due to the amount and robustness of the empirical studies that verify the predictive value of the perception studies in the sphere of the Social Sciences (Eastman and Marzillier, 1984; Bandura and Locke, 2003; Rottinghaus, Larson, and Borgen, 2003; Valentine, Dubois and Cooper, 2004) in the field of the Higher Education (De Barros, 2012), the scale can be used as a measure of the quality and efficacy of the use of the institutional LMS in its every dimension, and it can be useful in the diagnostic and evaluation of the current state of the use of the platform.

While the results obtained suggest that the scale adequately measures the users' perceived usefulness of the LMS Moodle in the dimensions identified as key for all LMS (Moore and Iida, 2010; Palmer and Holt, 2010; Al-Busaidi and Al-Shihi, 2012), which are the contents, activities, assessment, interaction and learning, we obtain variance indexes below the desirable ones in some cases. Given the inexistence of global scales with adequate psychometric properties, and this scale being a perception one, it is only logical to obtain these values. Using Pearson's correlation matrix to carry out the analysis includes a certain bias relative to the metric of the variables (Elosua Oliden and Zumbo, 2008; Morales Vallejo, 2006),and it contributes to the obtainment of these low indexes, because in this case the value of the coefficients is underestimated in relation to the polychoric correlation matrix. Taking into consideration the underestimation of the real value of the correlation between the pairs of variables obtained here, an improvement of the goodness of fit indexes and the explained variance is to be expected with the application of more adequate techniques for Likert-type response scales when facing future research (Lopez-Gonzalez, Perez-Carbonell and Ramos, 2011; Lopez-Gonzalez, 2012).

Despite the importance of the weaknesses present in this research, this scale entails a starting point in future investigations for the development of scales adapted to subpopulations of university students from diverse areas of knowledge, whose dimensions will explain a bigger percentage of the variance. In this sense, it is worth remembering that the present study is restricted to the population of students from the area of Social Sciences, in particular from Educational Sciences, and that despite the large size of the sample, the non-probabilistic sampling method can be a source of important biases.

On the other hand, it is worth mentioning another weakness related to the context of this study, which focuses on the use of Moodle as a complement to face-to-face education. In this way, the subjects involved in this study have been users of Moodle in face-to-face contexts, and because of that we are unable to know how the scale would behave when adapted to b-learning or e-learning contexts. There is room for the possibility of adapting the scale in future studies in order to observe its behaviour in these other educational contexts.

In conclusion, based on the evidences gathered and shown in the present research, we can conclude that this study constitutes a solid base for the development of coming investigations related with the analysis of the use of Moodle as an LMS in different educational institutions.

DOI: 10.7203/relieve.20.2.4221


Ajzen, I., & Fishbein, M. (1980). Understanding attitudes and predicting social behavior. Englewood Cliffs, New York: Prentice-Hall.

Al-Busaidi, K. A., & Al-Shihi, H. (2012). Key factors to instructors' satisfaction of learning management systems in blended learning. Journal of Computing in Higher Education, 24 (1), 18-39.

Antonenko, P., Toy, S., & Niederhauser, D. (2004). Modular Object-Oriented Dynamic Learning Environment: What Open Source Has to Offer. En Association for Educational Communications and Technology. Recuperado de ail?accno=ED485088

Arnal, J., Rincon, D. del, & Latorre, A. (1992). Investigacion educativa: fundamentos y metodologias (1a. ed.). Barcelona: Labor.

Aydin, C. C., & Tirkes, G. (2010). Open Source Learning Management Systems in Distance Learning. Turkish Online Journal of Educational Technology, 9 (2), 175-184.

Bandura, A., & Locke, E. A. (2003). Negative self-efficacy and goal effects revisited. The Journal of Applied Psychology, 55(1), 87-99.

Barbera, E. & Badia, A. (2004). Educar con aulas virtuales. Orientaciones para la innovacion en el proceso de ensenanza y aprendizaje. Madrid: Antonio Machado Libros.

Bollen, K. A. (1989). Structural equations with latent variables. New York: Wiley Interscience.

Brennan, R. L., & Prediger, D. J. (1981). Coefficient Kappa: Some uses, misuses, and alternatives. Educational and Psychological Measurement (41), 687-699.

Britain, S. & Liber, O. (1999). A Framework for Pedagogical Evaluation of Virtual Learning Environments. Recuperado de

Byrne, B. (2001). Structural equation modeling with amos: basic concepts, applications, and programming. Oxford: Routledge.

Castells, M. (1999). La Era de la informacion: economia, sociedad y cultura. La sociedad Red (Vol. 1). Madrid: Alianza Editorial.

Cohen, D., & McCuaig, W. (2008). Three lectures on post-industrial society. Cambridge: MIT Press.

Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, (20), 37-46.

Cole, J. & Foster, H. (2007). Using Moodle: Teaching with the Popular Open Source Course Management System. London: O'Reilly.

Conrey, F. R. & Smith, E. R. (2000). Attitude Representation: Attitudes as Patterns in a Distributed, Connectionist Representational System, Social Cognition, 25 (5), 718-735.

Davis, F. D. (1993). User acceptance of information technology: system characteristics, user perceptions and behavioral impacts. International Journal of Man-Machine Studies, 38 (3), 475-487.

Davis, F. D., Bagozzi, R. P., y Warshaw, P. R. (1989). User Acceptance of Computer Technology: A Comparison of Two Theoretical Models. Management Science, 35 (8), 982-1003.

De Barros, A. F. (2012). Caracteristicas Psicometricas da Adaptacao Portuguesa do Perfil de Auto-Percepcao para Estudiantes Universitarios--SPPCS. Revista Iberoamericana de Diagnostico y Evaluacion Psicologica, 1(33), 93-110.

DeNeui, D. L., & Dodge, T. L. (2006). Asynchronous Learning Networks and Student Outcomes: The Utility of Online Learning Components in Hybrid Courses. Journal of Instructional Psychology, 33 (4), 256-259.

Eastman, C., & Marzillier, J. S. (1984). Theoretical and methodological difficulties in Bandura's self-efficacy theory. Cognitive Therapy and Research, 8 (3), 213-229.

Ellis, R.K. (2009). Learning Managament Systems. Alexandria: American Society for Training & Development (ASTD).

Ellison, N.B., Steinfield, C., & Lampe, C. (2007). The benefits of facebok "friends": Social capital and college students' use of online social network sites. Journal of Computer-MediatedCommunication, 12(4), 1143-1168.

Elosua Oliden, P. & Zumbo, B.D. (2008). Coeficientes de fiabilidad para escalas de respuesta categorica ordenada. Psicothema, 20(4), 896-901.

Escobar-Rodriguez, T., & Monge-Lozano, P. (2012). The acceptance of Moodle technology by business administration students. Computers & Education, 58 (4), 1085-1093.

Friedrich, H. F., & Hron, A. (2010). Factors Influencing Pupils' Acceptance of an E-Learning System for Secondary Schools. Journal of Educational Computing Research, 42(1), 63-78.

Garcia Jimenez, E., Gil Flores, J., & Rodriguez Gomez, G. (2000). Analisis factorial. Madrid: La Muralla.

Gomez Rey, I.; Hernandez Garcia, E., & Rico Garcia, M. (2009). Moodle en la ensenanza presencial y mixta del ingles en contextos universitarios. RIED. Revista Iberoamericana de Educacion a Distancia, 12 (1), 169-194.

Goyal, E., & Purohit, S. (2010). Study of Using Learning Management System in a Management Course. SIES Journal of Management, 6(2), 11-20.

Hofacker, C.F. (1984). Categorical judgment scaling with ordinal assumptions. Multivariate Behavioral Research, 19, 91-106.

Kirner, T.G., & Saraiva, A.V. (2007). Software Usability Evaluation: an Empirical Study. Paper presented at Proceedings of the 9th International Conference on Enterprise Information Systems. Funchal, Portugal, 459-465.

Kline, R. (2005). Principles and practice of structural equation modeling. New York: Guilford Press.

Klobas, J. E., & McGill, T. J. (2010). The role of involvement in learning management system success. Journal of Computing in Higher Education, 22(2), 114-134.

Labovitz, S (1970). The assignment of numbers to Rank order categories. American Sociological Review, 35, 315-324.

Labovitz, S. (1967). Some observations on measurement and statistics. Social Forces, 46, 151-160.

Landis J.R. & Koch, G. (1977). The measurement of observer agreement for categorical data. Biometrics (33), 159-174.

Levy Mangin, J.-P. (2006). Modelizacion con estructuras de covarianzas en ciencias sociales: temas esenciales, avanzados y aportaciones especiales. Espana: Netbiblo.

Lin, Q. (2008). Student satisfactions in four mixed courses in elementary teacher education program. Internet and Higher Education, 11(1), 53-59.

Lopez-Gonzalez, E. (2012). Sugerencias para el analisis de Escalas con Metrica Delicada. Revista Iberoamericana de Evaluacion Educativa, 5(1e), 84-105.

Lopez-Gonzalez, E., Perez-Carbonell, A. & Ramos, G. (2011). Modelos complementarios al analisis factorial en la construccion de escalas ordinales: un ejemplo aplicado a la medida del Clima Social Aula. Revista de Educacion, 354, 369-397.

Mardia, K. V. (1970). Measures of multivariate skewness and kurtosis with applications. Biometrika, 57(3), 519-530.

Martin-Blas, T., & Serrano-Fernandez, A. (2009). The role of new technologies in the learning process: Moodle as a teaching tool in Physics. Computers & Education, 52, 35-44.

Martorell, C., Gonzalez, R., Ordonez, A. N. A., & Gomez, O. (2011). Estudio confirmatiorio del cuestionario de conducta antisocial (CCA) y su relacion con variables de personalidad y conducta antisocial. Revista Iberoamericana de Diagnostico y Evaluacion Psicologica, 1(31), 97-113.

Medina, F., & Galvan, M. (2007). Imputacion de datos: teoria y practica. Santiago de Chile: Naciones Unidas, CEPAL, Division de Estadistica y Proyecciones Economicas.

Melton, J. (2006). The LMS Moodle: A Usability Evaluation. Japan: Prefectural University of Kumamoto.

Moore, K., & Iida, S. (2010). Students' perception of supplementary, online activities for Japanese language learning: Groupwork, quiz and discussion tools. Australasian Journal of Educational Technology, 26(7), 966-979.

Morales Vallejo, P. (2006). Medicion de actitudes en psicologia y educacion. Construccion de escalas y problemas metodologicos. Madrid: Universidad Pontificia de Comillas.

Morales, P., Urosa, B., & Blanco, A. (2003). Construccion de escalas de actitudes tipo Likert. Madrid: La Muralla.

Naveh, G., Tubin, D., & Pliskin, N. (2010). Student LMS use and satisfaction in academic institutions: The organizational perspective. Internet and Higher Education, 13(3), 127-133.

Nunnally, J.C. (1978). Psychometric theory. New York: McGraw-Hill.

Nunez, J.C, Cerezo, R., Bernardo, A., Rosario, P., Valle, A., Fernandez, E., & Suarez, N. (2011). Implementation of training programs in self-regulated learning strategies in Moodle format: Results of an experience in higher education. Psicothema 23(2), 274-281

Ozkan, S., & Koseler, R. (2009). Multidimensional students' evaluation of elearning systems in the higher education context: An empirical investigation. Computers & Education, 53(4), 1285-1296.

Palmer, S., y Holt, D. (2010). Students' perceptions of the value of the elements of an online learning environment: looking back in moving forward. Interactive Learning Environments, 18(2), 135-151.

Peat, M., y Franklin, S. (2002). Supporting Student Learning: The Use of Computer-based 9. Formative Assesment Modules. British Journal of Educational Technology, 33(5), 515-523.

Perez i Garcias, A. (2006). Internet aplicado a la educacion: aspectos tecnicos y comunicativos. En J. Cabero (coord.) (2006). Nuevas tecnologias aplicadas a la educacion. Madrid: Mc Graw Hill.

Richardson, J., & Swan, K. (2003). Examining social presence in online courses in relation to students' perceived learning and satisfaction. Journal of Asynchronous Learning 6 (1), 21-40.

Ross, I. (2008). Moodle, la plataforma para la ensenanza y organizacion escolar. Ikastorratza, e-Revista de Didactica 2. Recuperado de alea/mood le.pdf

Rottinghaus, P. J., Larson, L. M., & Borgen, F. H. (2003). The relation of self-efficacy and interests: a meta-analysis of 60 samples. Journal of Vocational Behavior, 62(2), 221-236.

Rubin, B., Fernandes, R., y Avgerinou, M. D. (2013). The effects of technology on the Community of Inquiry and satisfaction with online courses. Internet and Higher Education, 17, 48-57.

Rus, T. I., Pina, F. H., Sanchez Y, J. J. M., & Martinez, O. L. (2011). Adaptacion y validacion de la escala de actitudes hacia el trabajo en desempleados mayores de 45 anos. Revista Iberoamericana de Diagnostico y Evaluacion Psicologica, 2(32), 105-122.

Saito, D. S., & Ulbricht, V. R. (2012). Learning Managent Systems and Face-to-Face Teaching in Bilingual Modality (Libras/Portuguese).IEEE Latin America Transactions, 10(5), 2168-2174.

Silva Quiroz, J. (2011). Diseno y moderacion de entornos virtuales de aprendizaje (EVA). Barcelona: UOC.

Sorebo, 0., Halvari, H., Gulli, V. F., & Kristiansen, R. (2009). The role of self-determination theory in explaining teachers' motivation to continue to use e-learning technology. Computers & Education, 53(4), 1177-1187.

Soyibo, K., & Hudson, A. (2000). Effects of Computer-assisted Instruction (CAI) on 11th 8. Graders' Attitudes to Biology and CAI and Understanding of Reproduction in Plants and Animals. Research in Science Technological Education, 18 (2), 191-199.

Steyaert, J. (2005). Web based higher education, the inclusion/exclusion paradox. Journal of Technology in Human Services, 23 (1/2), 67-68.

Swan, K., Shea, P., Fredericksen, E., Pickett, A., Pelz, W., & Maher, G. (2000). Building Knowledge Building Communities: Consistency, Contact and Communication in the Virtual Classroom. Journal of Educational Computing Research, 23(4), 359-83.

Valentine, J. C., Dubois, D. L., & Cooper, H. (2004). The relation between self-beliefs and academic achievement: a meta-analytic review. Educational Psychologist, 39(2), 111-133.

Vaughan, N. (2007). Perspectives on Blended Learning in Higher Education. International Journal on E-Learning, 6 (1), 81-94.

Vigostsky, L. (1995). Pensamiento y lenguaje. Buenos Aires: Paidos.

Weaver, D., Spratt, C., & Nair, C. S. (2008). Academic and student use of a learning management system: Implications for quality. Australasian Journal of Educational Technology, 24(1), 30-41.

Weller, M. (2007). Virtual Learning Environments: using, choosing and developing your VLE. New York: Routledge.

Williams van Rooij, S. (2012). Open-source learning management systems: a predictive model for higher education. Journal of Computer Assisted Learning, 28(2), 114-125.

Yueh, H., & Hsu. S. (2008). Designing a learning management system to support instruction. Communications of the ACM, 51 (4), 59- 63.


Olmos-Miguelanez, S. (

Martinez-Abad, F. (

Torrecilla-Sanchez, E. M. (

Mena-Marcos, J. J. (juanjo

Olmos-Miguelanez, Susana ( Professor in the field of Research and Diagnostic Methods in Education at the University of Salamanca (Spain). This is the author's contact information for this article. His areas of interest are in the evaluation and processes of formative assessment. Mailing address: Paseo de Canalejas, 169-37008, Salamanca (Spain).

Martinez-Abad, Fernando ( Assistant Professor in the field of Research and Diagnostic Methods in Education at the University of Salamanca (Spain). Doctor of Educational Sciences. Mailing address: Paseo de Canalejas, 169-37008, Salamanca (Spain).

Torrecilla-Sanchez, Eva Maria ( Doctor of Educational Sciences at the University of Salamanca (Spain). Member of the Group of Educational Assessment and Educational Guidance at the University of Salamanca. Mailing address: Paseo de Canalejas, 169-37008, Salamanca (Spain).

Mena-Marcos, Juanjo ( Associate Doctoral Professor in the department of Didactics, Organization and Methods of Research at the University of Salamanca (Spain). His areas of interest are the analysis of the teaching practice, especially the processes of reflection, self-regulation, mentoring, and the evaluation of the Practicum. Like the associated aspects from before, the study of the teaching practice and the TIC are resources for a bettered, overall teaching environment. He forms part of the Innovative Research in Educational Technology at the University of Salamanca (GITEUSAL). Mailing address: Paseo de Canalejas, 169, 37008--Salamanca (Spain).
Table 1: Questionnaire items about student perception on the usefulness
of Moodle


Contents_01      There is a logic organisation of the teaching units

Contents_02      The contents are appropriate to the syllabus

Contents_03      The contents are updated

Contents_04      The resources uploaded by the teacher are

Contents_05      Studium is an efficient tool to get relevant
                 information related to the subject

Contents_06      I like that the teacher provides the class with
                 presentations through Studium

Contents_07      The links to web sites selected by the teacher allow
                 us to extend the topic of study and understand it

Contents_08      The videos or the images selected allow us to learn
                 in a more intuitive and dynamic way

Contents_09      I am interested in checking all the resources listed
                 in Studium

Activities_01    Critical thinking

Activities_02    Drawing up creative and personal syntheses

Activities_03    Applying knowledge to real-life situations

Activities_04    Problem solving

Activities_05    Understanding basic concepts and ideas within the

Activities_06    Analysis and reflection on the contents

Activities_07    Memorizing and reproducing contents

Activities_08    Evaluating and giving personal value judgements
                 about the covered topics

Activities_09    Researching and/or consulting other sources and

Activities_10    Cooperative work among the students

Activities_11    Organising the study and presenting assignments on

Assessment_01    The teacher proposes self-assessment activities in
                 the platform

Assessment_02    The teacher lays out the exams in the platform

Assessment_03    There is a clear definition of the assessment
                 criteria for the activities proposed by the teacher

Assessment_04    The teacher assesses the assignments in the platform

Assessment_05    The teacher offers continuous feedback to the
                 students in the platform

Assessment_06    The teacher assesses participation in the platform

Assessment_07    All the activities proposed in the platform have an
                 influence on the final mark of the subject

Assessment_08    We have access to grades in the platform

Interaction_01   Studium allows for a more fluid communication with
                 the teacher

Interaction_02   It promotes longer and more continuous tutoring
                 sessions with the teacher (they are not limited to a
                 fixed timetable)

Interaction_03   It promotes communication among students

Interaction_04   It is the tool that I most frequently use to
                 communicate and work with other classmates

Learning_01      It complements face-to-face learning

Learning_02      It increases my involvement with content learning

Learning_03      It constitutes an environment that favours the
                 knowledge building process

Learning_04      It facilitates learning

Learning_05      It is important for my future professional practice
                 because it allows for continuous learning

Learning_06      It enables cooperative learning by allowing the
                 students to share information and opinions with
                 their classmates

Learning_07      It makes it possible to attend to the diverse
                 interests of the students

Learning_08      It is motivating to receive feedback from the
                 teacher about the learning process (through the
                 correction of tasks and exercises, interaction in
                 the forums...)

Table 2: Total-element statistics for each theoretical dimension

                     Contents   Activities   Assessment

Item 01              .525       .602         .495
Item 02              .604       .626         .491
Item 03              .526       .586         .540
Item 04              .597       .626         .613
Item 05              .550       .593         .516
Item 06              .388       .633         .587
Item 07              .564       .477         .616
Item 08              .553       .572         .568
Item 09              .406       .511         --
Item 10              --         .508         --
Item 11              --         .483         --
Cronbach's [alpha]   .818       .864         .827

                     Interaction   Learning

Item 01              .626          .577
Item 02              .672          .693
Item 03              .714          .688
Item 04              .522          .676
Item 05              --            .733
Item 06              --            .674
Item 07              --            .664
Item 08              --            .549
Item 09              --            --
Item 10              --            --
Item 11              --            --
Cronbach's [alpha]   .809          .883

Table 3: Tests for the verification of normality and homoscedasticity

                 Asymmetry   Critical Ratio   Kurtosis

Contents_01       -0.410         -4.594        1.141
Contents_02        0.128         1.431         0.801
Contents_03       -0.358         -4.010        0.527
Contents_04       -0.172         -1.925        0.657
Contents_05       -0.456         -5.115        0.212
Contents_06       -1.678        -18.811        3.187
Contents_07       -0.555         -6.216        0.881
Contents_08       -0.538         -6.027        0.521
Activities_01     -0.381         -4.267        1.753
Activities_02     -0.400         -4.485        0.762
Activities_03     -0.259         -2.908        0.486
Activities_04     -0.301         -3.371        1.281
Activities_05     -0.305         -3.415        0.942
Activities_06     -0.178         -1.993        0.174
Activities_07     -0.308         -3.453        0.891
Activities_08     -0.367         -4.112        0.550
Activities_09     -0.604         -6.776        0.173
Activities_10     -0.851         -9.541        0.945
Activities_11      0.167         1.874         -1.182
Assessment_01      0.450         5.046         -1.062
Assessment_02     -0.555         -6.226        -0.093
Assessment_03     -0.582         -6.521        -0.801
Assessment_04     -0.037         -0.420        -0.922
Assessment_05     -0.129         -1.443        -0.926
Assessment_06     -0.863         -9.671        0.052
Assessment_07     -0.665         -7.455        -0.678
Assessment_08     -0.596         -6.683        0.498
Interaction_01    -0.623         -6.985        0.381
Interaction_02    -0.379         -4.248        -0.289
Interaction_03     0.287         3.213         -0.664
Interaction_04    -0.487         -5.455        0.596
Learning_01       -0.335         -3.757        0.952
Learning_02       -0.412         -4.620        1.131
Learning_03       -0.395         -4.423        0.666
Learning_04       -0.308         -3.450        -0.065
Learning_05       -0.482         -5.405        0.091
Learning_06       -0.489         -5.480        0.463
Learning_07       -0.860         -9.637        0.717
Learning 08       -0.410         -4.594        1.141
                     Mardia's coefficient      278.72

                 Critical Ratio   Z (k-s)    p.

Contents_01          6.393         9.376    <.001
Contents_02          4.492        10.995    <.001
Contents_03          2.952         9.106    <.001
Contents_04          3.680         9.941    <.001
Contents_05          1.191         8.446    <.001
Contents_06          17.863       12.162    <.001
Contents_07           4.94         8.454    <.001
Contents_08          2.923         8.201    <.001
Activities_01        9.828        10.119    <.001
Activities_02        4.270         9.160    <.001
Activities_03        2.726         8.903    <.001
Activities_04        7.178         9.741    <.001
Activities_05        5.280         9.318    <.001
Activities_06        0.977         8.900    <.001
Activities_07        4.992         9.505    <.001
Activities_08        3.082         8.635    <.001
Activities_09        0.968         7.666    <.001
Activities_10        5.296         8.203    <.001
Activities_11        -6.626        4.794    <.001
Assessment_01        -5.955        6.710    <.001
Assessment_02        -0.523        7.656    <.001
Assessment_03        -4.492        6.271    <.001
Assessment_04        -5.168        5.705    <.001
Assessment_05        -5.191        6.018    <.001
Assessment_06        0.292         6.818    <.001
Assessment_07        -3.798        6.228    <.001
Assessment_08        2.790         7.735    <.001
Interaction_01       2.134         7.910    <.001
Interaction_02       -1.622        7.195    <.001
Interaction_03       -3.719        6.353    <.001
Interaction_04       3.338         8.072    <.001
Learning_01          5.338         9.089    <.001
Learning_02          6.338         9.045    <.001
Learning_03          3.736         8.702    <.001
Learning_04          -0.364        7.942    <.001
Learning_05          0.508         7.746    <.001
Learning_06          2.595         8.412    <.001
Learning_07          4.018         8.202    <.001
Learning 08          6.393         9.376    <.001

Mardia's             67.66

Tabla 4: Configuration matrix in the complete scale

                 Activities   Assessment   Interaction

Activities_01       .629
Activities_02       .690
Activities_03       .688
Activities_04       .696
Activities_05       .640
Activities_06       .673
Activities_07       .570
Activities_08       .592
Assessment_01                    .656
Assessment_02                    .672
Assessment_03                    .623
Assessment_04                    .708
Assessment_05                    .571
Assessment_06                    .697
Assessment_07                    .732
Assessment_08                    .665
Interaction_01                                -.588
Interaction_02                                -.686
Interaction_03                                -.802
Interaction_04                                -.680
Learning_02                                   -.410
Learning_03                                   -.450
Learning_04                                   -.426
Learning_05                                   -.537
Learning_06                                   -.749
Learning_07                                   -.674
Learning_08                                   -.453

                 Contents   Learning

Contents_01       -.498
Contents_02       -.602
Contents_03       -.577
Contents_04       -.667
Contents_05       -.488
Contents_06       -.614
Contents_07       -.574
Contents_08       -.531
Learning_02                  -.513
Learning_03                  -.522
Learning_04                  -.476

Table 5. Matrix of correlations between dimensions

              Activities   Assessment   Interaction

Activities      1.000         .231         -.394
Assessment                   1.000         -.355
Interaction                                1.000

              Contents   Learning

Activities     -.433      -.166
Assessment     -.259      -.172
Interaction     .331       .140
Contents       1.000       .187
Learning                  1.000

Table 6: Questionnaire reliability and validity indexes

                 Standard weight [L.sub.ij]   [R.sup.2]

Contents_01                 .660                .394
Contents_02                 .694                .257
Contents_03                 .588                .273
Contents_04                 .659                .341
Contents_05                 .701                .381
Contents_06                 .404                .244
Contents_07                 .756                .392
Contents_08                 .766                .353
Activities_01               .598                .316
Activities_02               .613                .406
Activities_03               .632                .285
Activities_04               .640                .536
Activities_05               .594                .609
Activities_06               .626                .573
Activities_07               .494                .491
Activities_08               .617                .434
Activities_09               .584                .345
Activities_10               .637                .481
Activities_11               .562                .436
Assessment_01               .522                .409
Assessment_02               .506                .399
Assessment_03               .628                .376
Assessment_04               .667                .357
Assessment_05               .625                .406
Assessment_06               .643                .491
Assessment_07               .682                .480
Assessment_08               .624                .484
Interaction_01              .757                .561
Interaction_02              .780                .543
Interaction_03              .732                .518
Interaction_04              .534                .417
Learning_01                 .637                .163
Learning_02                 .701                .572
Learning_03                 .693                .587
Learning_04                 .696                .390
Learning_05                 .749                .465
Learning_06                 .737                .414
Learning_07                 .719                .391
Learning 08                 .646                .444

                 CRI   AVE   [alpha] Cronbach

Contents_01      .80   .44         .82
Contents_06      .69   .44         .79
Activities_01    .86   .36         .87
Assessment_01    .80   .38         .83
Interaction_01   .88   .50         .82
Learning_01      .89   .49         .89
Learning 08
COPYRIGHT 2014 Interuniversity Association of Pedagogical Research
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2014 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Olmos-Miguelanez, S.; Martinez-Abad, F.; Torrecilla-Sanchez, E.M.; Mena-Marcos, J.J.
Publication:RELIEVE: Revista Electronica de Investigacion y Evaluacion Educativa
Geographic Code:4EUSP
Date:Jul 1, 2014
Previous Article:Escala de motivacion por el juego (EMJ): estudio del uso del juego en contextos educativos.
Next Article:Analisis longitudinal de tesis doctorales espanolas en educacion (1841-2012).

Terms of use | Copyright © 2018 Farlex, Inc. | Feedback | For webmasters