Evaluating application software training methods.

Abstract

As microcomputer is used increasingly in business environments, effective training method for computer application software is indispensable. Computer software instructors have difficulty in choosing appropriate instructional approach and teaching material for their students. This study explores the learning behavior and the outcome under different instructional techniques. It is found that learners might not fully benefit from the project-based training material. The aim of the paper is to arouse further investigation on the essential elements for effective computer application software training.

Introduction

As computer application software is being adopted increasingly within business environments, employees strive for keeping their computer skills in pace with the rapid changes in computer technology (Harp, Taylor, & Satzinger, 1998). Training not only can improve user's ability but also can motivate user to use the software (Nelson & Cheney, 1987; Olfman & Bostrom, 1991). Effective software training methods are essential for teaching learners to use the software and training them to become self-learners for the new versions of the software.

There are varieties of software training material and textbook of different instructional approaches available in the market. Given that learners need to transfer their knowledge from learning setting to business setting, the project-oriented textbook becomes commonly used in educational institutions and is suggested to be one of the most effective approaches for learning software skills (Lambrecht, 2000; Orr, Allen, & Poindexter, 2001). Rather than teaching discrete functions of software, the project-based textbook integrates practical applications as projects to be accomplished. Usually, this type of textbook consists of projects simulating real world business situation with scenarios and practical problems. The instructions are presented in an easy-to-understand way. The tasks required to complete the project are clearly shown using a step-by-step and screen-by-screen approach, with the steps being highlighted and accompanying with full-color screen illustrations. In addition, the book includes detail explanation on each task with 'Other Ways' sections suggesting alternative methods for the same function. There are in-depth lab exercises at the end of each project for the learners to practice the acquired skill.

After talking with the computer software instructors who have used project-based textbook teaching application software in vocational training institution, it is noticed that the instructors face with difficulty in choosing an appropriate instructional method to teach their students. Instructors found students not fully benefit from those expensive and thoughtfully designed textbook. Moreover, seeing that computer software continues to become easier to use, it is even harder to draw students' attention in the class. Some students scorned to pay attention in learning the software since they perceived that they would be able to operate the user-friendly software. Most of the instructors adopt the behavior modeling, which consists of observation of computer skill demonstration and subsequent hands-on practice, as it is suggested to be among the most effective computer training methods (Simon & Werner, 1996; Yi & Davis, 2001). Usually, the instructor would spend two-third of the classroom time on taking students through the steps of the project by demonstrating how to perform the required tasks and having the students to do the same on their machines. After completing the project, the rest of the time would be allotted for the students to practice the skills by doing the lab exercise. Disappointedly, the instructor found that most of the students are not eager for this activity unless it was grade-motivated. Students thought doing the exercise is redundant for it is very alike the project they have completed. Indeed the students' performance in the tests indicated that with little practicing, what they have learned atrophies over time.

Aim of the study

Although a wide variety of training methods are being used to teach computer skills, there remains insufficient understanding of how training inputs are systematically related to outcomes (Nelson, Whitener, & Philcox, 1995). Less attention has been given to the learning behavior and the utilization of training material which might affect the learning effectiveness. The study aims to examine the impact of different software instructional methods on the learning behavior and the outcome of learning. From the findings, further research may be developed to examine factors that may constrain or support effective software training.

Research Methodology

Observation was employed to explore the learner's behavior in three training approaches. It is an appropriate method for understanding a particular context and the interactions there (Merriam, 1998), making it possible to record behavior as it is happening. A popular presentation software, Microsoft PowerPoint for Windows, was chosen as the application software to be learned by the research subjects. An intermediate level project "Using Outline View and Clip Art to Create a Slide Show" from a project-oriented textbook was used as the teaching object. The researcher performed the role of instructor and observer in the study.

Subjects

In order to avoid personal factors such as gender and computer literacy, purposeful sampling was used. The research sample consisted of three female working adults who have limited computer competence and possessed basic skill in using Microsoft PowerPoint. All the subjects were interested to learn the advanced features of the software which might be applicable in their work. It is expected that when learners have fundamental pre-knowledge, their interest to learn arouses for they understand that their knowledge is incomplete or can be expanded (Astleitner, Brunken, & Leutner, 2003).

Design

The observation was divided into two parts, the learning session followed by the exercise session. Each subject was randomly assigned to one of the three training methods and observed separately. All subjects had to learn the same skills and were required to do the same exercise in the second session. Like other studies, the completion time and the accuracy of the exercise were used to measure the learning outcome (Harp, et al., 1998; Kraiger, Ford, & Salas, 1993). For identification, each of the three training approaches was named and described as follow.

Instructional technique 1 (self-learning) The subject was given the textbook and required to learn the skills on her own without any help from the instructor.

Instructional technique 2 (hands-on) The researcher demonstrated the steps in the project to the subject who was required to perform the same thing on her computer simultaneously. The subject was given the textbook so that she could follow the researcher's instruction and refer to the book as well.

Instructional technique 3 (demo) The researcher demonstrated how to accomplish the project to the subject who neither had any learning material nor got the hands-on experience. The subject was allowed to raise questions and jot notes during the demonstration. Except the "self-learning" technique, the subjects in the other two techniques were allowed to ask questions in the learning session. All the subjects were not allowed to raise any questions during the exercise session where both the "self-learning" and "hands-on" subjects were allowed to use the textbook as reference while the "demo" subject could only refer to her own notes. Before the learning process, the subjects were briefed on the purpose of the two sessions. In order to give the subjects an idea on the animation effects used, the researcher had presented the slide show of both the project and the exercise to them. With the consent of the research subjects, the processes were recorded in video tape to ensure that their learning behavior and everything happened was preserved for analysis.

Results

The learning efficiency, which refers to the primarily attaining a certain learning achievement within a certain time (Astleitner et al., 2003), was more or less the same for the "self-learning" and the "hands-on" subjects. Theoretically the learning session for the "demo" technique was expected to take a shorter time since the subject was not required to do the project. In fact the researcher needed to spend more time to answer the subject's questions. Eventually all the subjects could complete the lab exercise with "self-learning" subject's output file most fulfill the requirement. Although the other two files did not follow the exact requirement of the exercise, the subjects did have the skill to finish the tasks. The subjects were just impatient at reading the instruction and following the requirements of the exercise. For instance, they chose to use the transition effect which was the one used in the project but not the one required for the exercise; and they just gave up to find the needed symbol for the bulleted list.

In the learning session, the subjects demonstrated different learning behaviors. It could be seen that the "self-learning" subject had no interest to read the wordy introduction and the scenario of the project. Following a glance at the first two pages, she jumped directly to the steps and started working on the tasks. After completing a few tasks and getting familiar with the pattern of the instruction, she began flipping through the pages and only read the required steps which can be spotted easily with highlighted numbers in bold font. The subject had skipped all the detail description about the features and the 'Other Ways' suggestion. For the "hands-on" subject, she followed the researcher's instruction most of the time at the beginning of the learning session. She has not much interest in referring to the book since the instructor had demonstrated every single step for her to imitate. But after the subject had realized that the steps shown in the textbook were so easy to follow, her curiosity to work ahead the instructor aroused. Since then she shifted to work on her own by following the book and paid attention to the instructor only when she found difficulty in understanding the book. During this session, the subject had not raised any question. On the other hand, the "demo" subject acted totally different. Throughout the learning session, she was highly concentrated on watching the demonstration and jotting notes. She had raised some questions to clarify her understanding.

In the exercise session, the "self-learning" subject referred to the textbook more frequent than the "hands-on" subject. It could be due to the fact that she had a better idea on the location of the feature. The "demo" subject referred to her notes intermittently. When facing difficulty, both the "hands-on" and the "demo" subjects preferred to use the trial-and-error method instead of referring to the learning material. Obviously the "demo" subject has less confident in using the software. The file created by the "self-learning" subject best fulfills the requirements since she has the best interpretation of the textbook's presentation. The other two subjects had no patience to read the requirements of the exercise carefully. They were just eager to make the slides resemble those illustrated in the textbook. For the animation effect, they just chose the one they have used in the project rather than the one required in the exercise. Regardless of the training approach used, all the subjects have demonstrated a common behavior. Even they have learned a more efficient way to carry out a feature, they kept using their own way to accomplish the task. For example, they preferred to take more steps by choosing the command from the menu list to complete a task instead of using a simple mouse click on the icon.

Discussions

There was no significant difference found in the knowledge gained under different training approaches. The result of the study indicated that learners have not fully utilized the well-illustrated project-based textbook. They lose patience to read the wordy material. The easy-to-follow feature of the textbook probably draws away the student's attention from the instructor. The comprehensive full-color instructional textbook is not used as intended because students used to skip the detail explanations and useful suggestions. The publishers might consider offering condensed version at a lower price. On the other hand, according to Landis (2000) the sooner the instructor can move from the presentation stage to the facilitation position, the more independently a student learns. To train the students to become independent learners, instructor has to reduce the time spend on the demonstration and expand the student practicing time. Incontrovertibly, software training with demonstration only would not be effective. It is necessary to provide students with hands-on experience which can equip the student with the competence and confidence to use the software. Without learning material students might pay more attention to the instructor's instruction. But it is easy for them to miss important content while they are busy on jotting notes. Even worse, students might feel frustrated when they cannot follow the pace. Hence, learning material is necessary to release the students from taking notes.

Although students under self-learning method might gain a better interpretation of the material and have more confidence in using the software, they tend to ignore useful information. In fact, learners prefer training with high degree of media richness which allows for face-to-face, two-way communication when learning new software programs (Harp, et al., 1998). Training which blends self-learning with demonstration and two-way communication would be desirable. Though the project-based instruction is highly recommended, incentive remains an important factor affecting student learning. As there was no incentive for the research subjects to produce high quality output, what they concerned was to finish the exercise as soon as possible. Instructor need to establish reward scheme to encourage high performance. Marks or peer pressure would be more direct and preferable than intrinsic rewards that might not apply to every student. While most of the software has built in flexibility allowing alternative ways to do the same task, learners do not like to abandon their method. The study demonstrates that subjects prefer to use their old method which is more complicated than the one learned in the project. According to the subjects' explanation, students are goal-oriented. As long as they can finish the task and produce the product, they would not bother to learn new methods. But if they are tested on alternative skills, their manners would be different. The same applies to the students who will adjust their learning focus based on what will be assessed in the test. If instructor wants to train students with knowledge on alternative ways for the same feature, the methods students use should be assessed.

Personal difference might be a factor affecting learning behavior. A personality variable applicable to learning is student's conceptual level which is the ability to differentiate and integrate information (Hunt, Butler, Noy, & Rosser, 1978). Park and Garmon stated that one person may begin learning a new computer program by experimenting with its features while another person might prefer to observe how others are using it before trying (Orr, et al., 2001). According to Hancock, Bray, and Nason (2002), matching low-conceptual-level learners with teacher-centered instruction and high-conceptual-level persons with student-centered instruction enhances student achievement. On the other hand, the training needs differ as computer users gain experience, they may be willing to forego the richness of instructor-led training in favor of the less rich training (MacKay and Lamb, 1992; Nickerson, 1981). Though it might be difficult to implement individualized instruction in big class, individualized is necessary since students of the same class usually are of different personality and level of competence.

Limitations

The research has to be criticized in the sample used. There were only three subjects being observed and their mentality might be very different from that of the students in classroom situation. In order to get the generalization, the sample size has to be much larger. Hawthorne Effect [1] might exist in the observation since the subjects knew they were being examined. Nevertheless the study demonstrates disparity in learning behavior and outcome under different training methods. Since software varies widely and training needs cannot be compared between vastly different types of software, the scope of the study was limited to specific software.

Conclusion

Different software training approaches lead to different learning behavior and have impact on the effectiveness of learning. No single method could suit all. It is a challenge for the instructors to decide on the instructional technique. Project-based approach requires the involvement in realistic activities is essential to gaining the types of understanding that will transfer outside of school settings (Lambrecht, 2000). Students might not get full benefit from the well-illustrated textbook. Motivation plays an important role in effective learning as it can encourage students to practice more and perform better. Other ingredients such as hands-on experience, learning material, communication, and individualized instruction are all necessary. It is suggested that future research with a larger sample size of student in a real classroom situation is necessary to capture the representative data. Additional research would help to more fully explore the effect of various training methods. Besides observation, other fact finding techniques such as survey or focus group can be used to understand the attitude of the students on different training methods.

Endnotes

[1] Hawthorne Effect refers to the tendency of people being observed as part of a research effort, to behave differently than they would otherwise (Buchanan & Huczynski, 1997).

References

Astleitner, H., Brunken, R., and Leutner, D. (2003), The Quality of Instructional Materials for Argumentative Knowledge Construction, Journal of Instructional Psychology; vol.30, no.l, pp.3-11.

Buchanan, D. & Huczynski, A. (1997). Organizational behavior: an introductory text, 3rd edition. London: Prentice Hall. pp. 183.

Hancock, D.R., Bray, M., & Nason, S.A. (2002), Influencing University Students' Achievement and Motivation in a Technology Course, The Journal of Educational Research; vol.95, no.6, pp.365-372.

Harp, C.G., Taylor, S.C., & Satzinger, J.W. (1998), Computer Training and Individual Differences: When Method Matters, Human Resource Development Quarterly; fall, vol.9, no.3.

Hunt, D.E., Butler, L.F., Noy, J.E., & Rosser, M.E. (1978), Assessing conceptual level by the paragraph completion method, Toronto: Ontario Institute for Studies in Education.

Kraiger, K., Ford, J.K., & Salas, E. (1993). Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation, Journal of Applied Psychology, vol.78, no.2, pp.331-328.

Lambercht, J.J. (2000), Developing End-User Technology Skills, Information Technology, Learning, and Performance Journal; vol. 18, no. 1, pp.7-19.

Landis, M. (2000), "Baste-and-stitch" multimedia instruction, Multimedia Schools, Oct 2000; vol.7, no.5, pp.48-51.

Merriam, S.B. (1998). Qualitative Research and Case Study Applications in Education. San Francisco: Jossey-Bass Publishers.

Nelson, R.R., & Gheney, P.H. (1987). Training end users: An exploratory study. MIS Quarterly, vol. 11, no.4, pp.547-559.

Orr, C., Allen, D., & Poindexter, S. (2001). The Effect of Individual Differences on Computer Attitudes: An Empirical Study, Journal of End User Computing, vol. 13, no.2.

Simon, S. & Werner, J. (1996). Computer training through behavior modeling, self-paced, and instructional approaches: A field experiment. Journal of Applied Psychology, vol.81, no.6, pp.648-659.

Yi, M.Y. & Davis, F.D. (2001), Improving Computer Training Effectiveness for Decision Technologies: Behavior Modeling and Retention Enhancement, Decision Sciences, Summer, vol.32, no.3.

Heidi Fung, Hong Kong Shue Yan College

Heidi Fung is Assistant Professor. Her research interest is in computer software training and information technology in education.