Students perception on e-learning: a case-study.

Abstract

The reflections presented in this work lead us to emphasize the existence of a problem in the new market of university online learning. The costly evaluation of the quality of new courses can in fact increase the risk for students in making the best choice. In the absence of enrollment fees, which can act as a signal of quality, market growth would seem to be severely hampered. On the basis of the answers to a question administered to 1,790 students at Italian universities, we are interested in analyzing the perception of quality of university courses available on line, in order to distinguish signals, a la Spence, which can reduce the problem of adverse selection.

Introduction

Studies concerning the way the markets work often assume that individuals are able to make a correct assessment of the quality and price of the goods which are exchanged. Unfortunately, these hypotheses are rarely borne out in the real world.

In a famous article published in 1970, "The market for Lemons: quality uncertainty and the market mechanism", George Akerlof introduces an idea, which is simple but also of deep and universal significance. Using the second hand car market as his example, Akerlof shows that when the buyers and sellers have different information about the quality of the goods (information asymmetry) this results in a problem of adverse selection on the market. In other words, the agents who have less information find themselves operating with those they would have preferred to avoid. In fact, if the quality is not apparent, goods of both good and poor quality converge in a single market and have a single price tag. The latter, determined on the basis of an expected average quality, provides ample profit for the worst sellers but not necessarily for the best who find themselves driven out of the market. Consequently the buyers are left to operate only with the worst sellers. So the term "lemons" is used to indicate poor quality goods which are found in the market in the absence of correct information. This conclusion leads Akerlof to explain the set up of many third-party institutions which offer guarantees for the real quality of the goods, so contributing to a reduction in the inefficiencies of the market.

The work we present here attempts to analyse the problem of adverse selection in an unexplored context, namely that of online university learning. In this market there is great information asymmetry which penalises the students (buyers of instruction) in their evaluation of the quality of the courses offered by the universities (producers of learning). However, unlike Akerlof's hypothesis, the mechanism of adverse selection in this case is not due to a reduction of the market price (strict, by definition) but rather to a reduction in the number of buyers. However, following the contribution of Michael Spence, the market itself can offer a way out of the problem. In 1973 in "Job Market Signaling", Spence showed that the best sellers are not necessarily forced out of the market if they 'signal' the true quality of their goods. But in order for the signals to reach their objective of contrasting the adverse selection, they need to have a production cost which is inversely correlated to the quality of the seller. In other words, the signal must be more expensive for the worst sellers.

Following the paths traced by Akerlof and Spence, there are numerous applications and empirical tests to be found in the latest literature. In the financial economy, for example, Myers and Majluf (1984) have shown how shareholders can become victims of adverse selection among firms, while John and Williams (1985) have highlighted how dividends can act as a credible signal of the profitability of the companies quoted. In the field of industrial organization, Nelson (1974), Milgrom and Roberts (1986) have shown how the highest prices and the use of advertising are generally considered by consumers to be signs of the high quality products. From a methodological point of view, Rothschild and Stiglitz (1976) among others, show how, under certain conditions, agents who are less well informed can indirectly obtain information from those who are better informed, while Cho and Kreps (1987) use the game theory to discriminate between the many equilibria which can be resolved on the market in the presence of signals. On the basis of the answers to a question administered to 1,790 students at Italian universities, we are interested in analyzing the perception of quality of university courses available on line, in order to distinguish signals, a la Spence, which can reduce the problem of adverse selection. Our approach differs from some previous work. Hsing and Chang (1996) and Bezmen and Depken (1998) analysed the market demand for distance learning and concluded that university fees are an efficient signal of the quality of courses. In contexts where the fees are fixed administratively, however, they lose their intrinsic value as a signal. Alternative signals are thus required.

The characteristics of online university courses

As Harasim (2000) emphasised, online learning must not be confused with distance learning. They share many features, such as the context not limited by space or time, but they are completely different as regards group communication. This is a phenomenon which, as Dean (1994) pointed out, is at the centre of the learning process. It is in fact through group participation that students have the chance to elaborate concepts, share experiences, acquire knowledge and socialize. Moreover, interaction between teachers and learners helps to increase student motivation and satisfaction, and consequently to produce greater benefits. From this perspective online learning appears to be more similar to traditional learning, although differing in its pedagogical approach.

Recent studies of online learning have tried to compare the results of online learning with traditional 'face to face' learning. Analyses of students' marks at the end of a course of studies by Smeaton and Keogh (1999), Wade (1999), Navarro and Shoemaker (1999), Sener and Stover (2000), Fallah and Ubell (2000) show that no substantial difference exists between the results of traditional learning and online learning. This would also confirm Russell's position (1999) in his well known book "The No Significant Difference Phenomenon" in which he underlines, however, the importance of the quality of online learning. While there are numerous contributions concerning the effectiveness of online learning, according to Fresen (2002) there is little research regarding quality. Phipps and Merisotis (1999), in their report which generated wide debate amongst US academics about what constitutes quality in learning, focused their attention on the specificity of online learning. In fact the authors noted that the way in which technology conditions the learning and teaching processes is still far from clear, so any question regarding 'specific' traits of quality becomes superfluous. In agreement with this position, Pond (2002) considers it useful to distinguish "universal" traits of quality, independent of teaching methodologies. However, while it is true that the final result of learning is important, notwithstanding the pedagogical approach, an evaluation of the quality of the different specific teaching tools makes it possible to anticipate, as well as to improve, the final assessment of the learning process. We consider this to be the correct line of approach and we are encouraged by the contribution of McGorry (2003), who identifies six 'specific' traits of quality in online learning: flexibility, responsiveness and student support, student learning, interaction, technology and technical support, and student satisfaction.

Aside from the difficulty in identifying "specific' traits, the evaluation of quality from an empirical point of view, appears to be particularly costly for potential buyers, generating a classic problem of information asymmetry. Actually a solution to this could be found in the workings of the markets, in sending signals. The fees for online courses, for example, could be interpreted as signals of quality if they were fixed for each individual course on the base of production costs and expected demand. The problem becomes more complex in contexts in which the enrollment fees are decided by the public administration on the basis of welfare state policy. In these circumstances, which are, moreover, common practice in most European countries, such a signaling function is invalid. If the market is unable to send signals and if the evaluation of quality is particularly costly, the less well informed agents are faced with the problem of adverse selection. Students who have no way of distinguishing good online courses from mediocre ones, perceive a greater risk in choosing online courses compared to traditional educational contexts. A "virtual examination" of a course which is only accessible online, is, in fact, much less conclusive than physical contact with a traditional educational institution which can easily be assessed for the number of facilities, the atmosphere of the campus and the level of satisfaction of the students.

While the risk of choosing online courses is greater than making alternative choices, it is likely that students who are averse to risk taking will opt for the latter, so causing a drop in the numbers of online enrollment. This fact is particularly relevant for the equilibrium of the online learning market. As Rumble (2001) stated, the creation of an online course imposes costs which are strictly related to the use of technology. Therefore, if there is a drop in student numbers, the faculties which intend to make the best use of technology by offering high quality online courses, may no longer find it profitable to stay on the market. Increased production costs could, in fact, exceed the benefits. The best would find themselves excluded from the market and the risk in choosing online courses of quality would increase. This leads a majority of students to opt for alternative teaching methodologies, so setting up a vicious cycle which would tend to wipe out the whole online learning market. The classic solutions to the problem have already been indicated in the introduction to this paper, but our attention is focused on the study of signals a la Spence.

Survey and results

In carrying out our survey we collected 1,790 answers from students in the Faculties of Science, Political Science and Communication Sciences at the University of Palermo. Students were randomly selected. Each one was given the following instructions: "Imagine you go back in time and have to decide which university course you want to follow. You have to choose between a traditional course and an online one. What information regarding these two types of course would you consider decisive in making a thorough evaluation?". Every student was given the chance to freely indicate different types of information. In the following list, students' answers and related scores:

career prospects                    842
quality of teachers                 725
interaction with teachers           631
adequacy and clarity of materials   461
degree of flexibility               387
ease of finding materials           358
efficiency of auxiliary services    351
technology used 281
limited number of participants      230
personalization of study programs   209
use of tests for self evaluation     86
availability of tutors               79
prestige of the faculty              21

Career relevance appears to receive most attention from the students. Over 46% of the students mentioned it, considering it to be an important signal in evaluating online courses. It should be noticed that results of Dey, Astin and Korn's studies (1991) on what quality means for students in traditional university courses apply also to on line courses. The closeness of the learning path to the world of work in fact presupposes great attention to the curriculum and the effectiveness of group communication, factors which are crucial to the process of acquiring skills and professional competence. In this direction, the university could produce statistics, certified by external institutions, about the percentage of students who have got a job (within six months, one year and three years), their average salary and whether their expectations have been fulfilled. For the purpose of our analysis, it is interesting to observe that the production cost of the highest values of the above indicators is inversely correlated to the quality of the courses: in fact, increasing career relevance proves to be, ceteris paribus, more expensive for online courses of mediocre quality. This allows us to deduce that the information regarding the career relevance of online courses could constitute real signals a la Spence.

The quality of teachers, indicated by about 40% of those interviewed, would, on the other hand, be an ambiguous indicator. How can you measure the quality of teachers? According to their teaching experience, their academic achievements or other complex parameters? And once you have assessed the quality of each individual teacher, how do you relate this to the quality of online courses? In fact, online courses require different teaching methods from traditional courses; they involve the use of multimedia teaching materials, as well as the construction and implementation of a computer-mediated communication space. These skills are not part of the usual teaching experience. The following three indications provided by the students appear to be of a more concrete nature: the level of interaction with teachers (35.2% of those interviewed), the adequacy and clarity of the materials (25.7%) and the degree of flexibility of the course (21.2%). As Arbaugh (2000) points out, these are factors which can have a direct influence on online learning and the satisfaction level of the students.

Concerning the "interaction with teachers" issue, it should be said that there is an open debate in literature regarding the different ways for measuring interaction in on-line educational systems, and many solutions have been proposed [Anderson et al. (2001), Rourke et al. (2001)]. Nowadays, the technological solutions for managing on-line education processes, such as LMS and LCMS, include tools that can measure statistical data on specific elements of interaction. Of the different proposals, the average response time of the teachers to students' questions and the number of discussion threads proposed by the teacher for a course can be considered a measurement of the interaction between teachers and students that could be collected automatically. It should be noticed that the answer to the questionnaire provided by the students concerns only interaction with teachers, and shows that students are not aware of the importance of interaction with other students.

Regarding the "adequacy and clarity of materials", some well known measurement criteria are: the full availability of on-line material, the necessity to buy printed material in addition to that available on line, the organization of learning paths, the presence of references to external sites in order to deepen the explanations, and so on. Besides, the students interviewed highlighted the importance of flexibility, considered as the opportunity of accessing the educational material and interacting with teachers without time constraints. One of the measurement criteria for flexibility is the frequency of synchronous sessions. Actually, this parameter is not an absolute value of quality, but it is important in the personal evaluation of the adequacy of the organization of the course with respect to the single student needs. The same number of weekly interactions could be, for example, too many for some students and too few for others. We think that a very useful tool to measure the level of the courses respect to indicators discussed above, is an anonymous questionnaire administered to students from previous years, because their estimation is related to the effectiveness of learning. In this way, the obtained data could enable the best online universities to send signals of quality. It is interesting to observe, also in this case, how the cost of producing quality signals concerning student satisfaction is inversely correlated to the quality of online courses.

Conclusions

The reflections presented in this work lead us to emphasize the existence of an adverse selection problem in the new market of university online learning. In fact, since the evaluation of new courses is costly and the enrollment fees cannot be used as a signal of quality, students perceive a greater risk in choosing on line courses; consequently they could be encouraged to enroll on traditional courses. In response to these concerns, the Italian Ministry for Universities and Research issued a decree on April 17 2003 about "Criteria and procedures for the accreditation of distance learning courses offered by state and non-state universities and by university institutions entitled to award academic qualifications". The aim of this decree is to ensure a quality standard for academic courses which makes use of the Internet.

The considerable limits of this decree lie, however, in the fact that it is merely an administrative act. In fact, it does not provide for a continuing evaluation procedure nor establish methods of comparison or a system for publicizing the evaluation results. This regulatory instrument seems, therefore, to be unable to send out signals regarding quality, in order to prevent problems of adverse selection. In order to avoid this risk, we propose the use of signals h la Spence. Our survey has enabled us to underline certain types of information which students in Italy see as indispensable in evaluating prospective online courses. Two of these, in particular, seem to stand out: career relevance and the level of satisfaction expressed by those who have already followed the course. In particular, the latter aspect is important for reducing doubts about the effectiveness of the new learning environment. Further research is necessary, however, in order to test the validity of these signals empirically within the market.

References

Akerlof G. (1970), "The Market for Lemons: Quality Uncertainty and the Market Mechanism", Quarterly Journal of Economics, 84, 485-500.

Arbaugh J. B. (2000), "Virtual classroom characteristics and student satisfaction in Internet-based MBA courses", Journal of Management Education, 24, 32-54.

Anderson, T., Rourke, L., Garrison, D.R. & Archer, W. (2001). Assessing teaching presence in a computer conferencing context. Journal of Asynchronous Learning Networks, 5,2.

Bedard K. (2001), "Human Capital versus Signaling Models: University Access and High School Dropouts", Journal of Political Economy, 109, 749-775.

Bezmen T. and Depken C.A. (1998), "School Characteristics and the Demand for Collage", Economics of Education Review, 17, 205-210.

Cho I. K. and D. Kreps (1987), "Signaling Games and Stable Equilibria", Quarterly Journal of Economics, 102, 179-221.

Dean L. (1994), "Telecomputer communication: The model for effective distance learning", ED Journal, 8(12), J-1-J-9.

Dey E.L., Astin A.W. and Korn W.S. (1991), "The American freshman: Twenty-five years trends, 1966-90", Higher Education Research Institute, Graduate School of Educatione, University of California.

Fallah M. H. and Ubell R. (2000), "Blind scores in a graduate test: conventional compared with web-based outcomes", ALN Magazine, 4(2).

Farber H. and Gibbons R. (1996), "Learning and Wage Dynamics", Quarterly Journal of Economics, 111, 1007-1047.

Fresen J. (2002), "Quality in Web-supported learning", Educational Technology, 42, 28-32.

Harasim L. (1993), Global networks: computers and communication, MIT Press, Cambridge.

Harasim L. (2000), "Shift happens Online education as a new paradigm in learning", The internet and Higher Education, 3, 41-61.

Hsing Y. and Chang H.S. (1996), "Testing increasing sensitivity of enrolment at private institutions to tuition and other costs", American Economist, 40, 40-45.

John K. and Williams J. (1985), "Dividends, Dilution and Taxes: A Signalling Equilibrium", Journal of Finance, 40, 1053-69.

Lang K. and Kropp D. (1986), "Human Capital versus Sorting: The Effects of Compulsory Attendance Laws", Quarterly Journal of Economics, 101, 609-624.

McGorry S. Y. (2003), "Measuring quality in on line programs", The Internet and High Education, 6, 159-177.

Milgrom P. and Roberts J. (1986), "Price and Advertising Signals of Product Quality", Journal of Political Economy, 94, 795-81.

Myers S. and Majluf N. (1984), "Corporate Financing and Investment Decisions when Firms Have Information that Investors Do Not Have", Journal of Financial Economics, 13, 187-221.

Navarro P. and Shoemaker J. (1999), "The power of cyberlearning: An empirical test", Journal of Computing in Higher Education, 11, 33.

Nelson P. (1974), "Advertising as Information", Journal of Political Economy, 82, 729-54.

Phipps R. A. and Merisotis J. P. (1999), What's the Difference? A Review of Contemporary Research on the Effectiveness of Distance Learning in Higher Education, DC: American Federation of Teachers and National Education Association, Washington.

Pond W. K. (2002), "Twenty-first century education and training Implications for quality assurance", The Internet and High Education, 4, 185-192.

Riley J. (1975), "Competitive Signalling", Journal of Economic Theory, 10, 174-186.

Riley J. (1979), "Testing the Educational Screening Hypothesis", Journal of Political Economy, 87, 227-52.

Roberts S. K. (1999), "A survey of Accrediting Agency Standards and Guidelines for Distance Education", Theological Education, volume 36.

Rothschild M. and J. Stiglitz (1976), "Equilibrium in Competitive Insurance Markets: An Essay on the Economics of Imperfect Information", Quarterly Journal of Economics, 95,629-649.

Rovai A. P. (2003), "A practical framework for evaluating on line distance education programs", The Internet and Higher Education, 6, 109-124.

Rumble G. (2001), "The costs and costing of networked learning", Journal of Asynchronous Learning Networks, 5.

Rourke, L., Anderson, T.,., Garrison, D.R. & Archer, W. (2001). Assessing social presence in asynchronous text-based computer conferencing. Journal of Distance Education, 14, 21, 50-71.

Sener J. and Stover M. (2000), "Integrating ALN into an independent study distance education program: NVCC case studies", Journal of Asynchronous Learning Networks, 4.

Smeaton A., Keogh G. (1999), "An analysis of the use of virtual delivery of undergraduate lectures", Computers and Education, 32, 83-94.

Spence M. (1973), "Job Market Signalling", Quarterly Journal of Economics, 87, 355-374.

Spence M. (1974), Market Signaling, Harvard University Press, Cambridge.

Wade W. (1999), "Assessment in distance learning", T.H.E. Journal, 27, 94-100.

Vito Pipitone, Italian National Research Council, Palermo

Giovanni Fulantelli, Italian National Research Council, Palermo

Mario Allegra, Italian National Research Council, Palermo

The authors' scientific interests concern the pedagogical, economic and social impact of the use of ICT in training and educational contexts, with special focus on higher education and small and medium sized enterprises.