Printer Friendly

A study on development of quality standards of educational smart contents.

1. Introduction

In the current knowledge and information society, our daily life relies on the benefits of advanced ICT technologies. With aid of ICT technologies, our life style has been changed dramatically. Our life becomes more convenient and enjoyable than ever. How to use the advanced technologies becomes a part of social competativeness as well as individual competativeness.

Recently ICT technologies have been converged into smart technology. It means that the existing ICT devices become smaller, light, convenient, and portable. In this sense, the concept of "smart environment" is realized. Smart environment is defined as "a physical world that is richly and invisibly interwoven with sensors, actuators, displays, and computational elements, embedded seamlessly in the everyday objects of our lives, and connected through a continuous network." [1]. In smart environment, every aspect of our life has been affected with smart technologies. Now advanced in smart technologies have created a new concept called "smart learning" in education area.

Smart learning is a new concept so that there is no clear definition yet. However, some research works introduce the definition of smart learning [2,3]. In [2], it is argued that the principle of smart learning has the following 3 characteristics: First, rich instructional resources as learning contents, Second, participatory learning environments with interactions among teachers and learners as learning methods, Third, practical and realistic contexts as learning environments. Also, in [3], it is argued that smart learning is very essential in future education since it provides personalized contents and easy adaptation to the current education model.

The following Table 1 shows the comparison of different types of education: traditional style, e-learning, m-learning(mobile learning), u-learning(ubiquitous learning), and smart learning, respectively. Historically and chronologically our education system has been changed: traditional style, e-learning, m-learning, u-learning, and smart learning. We see that smart learning is the most extensive and comprehensive education style in the current era.

In [5], capabilities to be prepared for 21st century learners are announced. There are 3 categories: 1) learning and innovation skills, 2) life and career skills, 3) information, media, and technology skills, respectively. The following Table 2 shows those capabilities for the 21st century learners. We can see that smart learning is very best education style and provides the best environment to achieve the capability of 21st century learners.

As smart learning becomes popular, more educational smart contents have been produced and distributed. In this sense, we need some quality standards to determine how smart contents are useful or helpful for students as well as teachers. Those quality standards are very important because the future educational smart contents wil be made based on the quality standards. That is, the quality standards will guide for teachers and contents manufacturers on how to prepare and make new educational smart contents. In the literature, there are some quality standards for the existing e-learning environments [6,7,8,9,10], to our best knowledge, there is no quality standard for educational smart contents. In [11], only outlines are introduced. In the meanwhile, for smart literacy standards [12] and smart skill standards [13] for teachers and students are presented lately.

The smart contents can be classified into 2 categories for their purpose: teaching-learning and education support. The "teaching-learning" contents means any contents that can be used in class instantly. That is, those contents need not be refined for teachers to use in their class. Also, those contents can be used for students instantly. On the other hand, the "education support" contents means any contents that can be used as supplement tool for classes and students. Also, for teachers and students, those contents need to be refined for later use.

The purpose of this paper is to develop quality standards for educational smart contents. We develop the initial quality standards with various researchers and educators. The initial quality standards have 14 areas and 35 standards. In order to refine those quality standards, we ask various experts to check its validity and usefulness. Their valuable survey works are collected and analyzed. We finalize quality standards that have 14 areas and 34 standards.

This paper is organized as follows. First, in Chapter 2, we discuss literature reviews. In Chapter 3, we propose quality standards for educational smart contents. We collect responses from expert groups and do wide statistical analysis. Based on statistical analysis, we propose final quality standards. Finally, in Chapter 4, we discuss conclusions and further research works.

2. Related Work

2.1 Characteristics of Smart Learning

In [4], some characteristics of smart learning are introduced. Those characteristics are summarized in Table 3. In smart learning, a student's role is extended to be more active, more personalized, more experience-oriented. However, a teacher's role is changed to be a guide and mediator.

Also, in [4], smart learning is compared with the traditional learning in terms of categories suggested in Table 3. The traditional learning means the existing class-based learning. The following Table 4 shows comparison of smart learning and traditional learning. As we can see, smart learning environment provides more self-directed study and motivated and adaptive study for students anytime anywhere.

In [4], they argue that any smart contents must have the following 4 components. Those components are participation, sharing, cooperation, and accessibility, respectively. Each component is described as follows.

1) Participation

Students can use smart contents anytime anywhere. For this purpose, smart education contents must be accessed anytime, anywhere, anyone. Especially the various existing contents must be compatible each other with aid of cloud computing. It means that smart contents must be stored and accessed in the cloud so that students can read, update, and store anytime using networking service.

2) Sharing

Students can actualize and develop study process and results with other students and teachers using smart contents. The study contents can be developed by sharing with others. Smart contents can be shared eaisly due to its compatibility and openness.

3) Cooperation

In smart education, cooperative work is encouraged among students since smart education service provides diverse technical and social services like SNS. Cooperative works through smart education contents can support collective intelligence that ensures reliability of study results.

4) Accessibility

The great benefit of smart environment is that students can use wireless internet with various smart devices. Smat education contents can also be accessed easily without special hardware devices supporting internet connection. Also, various types of interaction can be possible in smart contents: student-contents interaction, student-student interaction, and student-teacher interaction.

2.2 Literature Review

In the literature, there is no quality standards for educational smart contents. However, there are some quality standards for the existing e-learning environments [6,7,8,9,10]. In this paper, we introduce the representative quality standards for the general educational contents in [8] since the quality standards in [8] are most extensive among the previous works. Table 5 shows those quality standards.

3. Development of Educational Smart Quality Standards

3.1 The initial Development of Smart Quality Standards

In order to develop new quality standards for educational smart contents, we develop the initial quality standards. The proposed standards are made based on the existing standards in [8] and add some features of smart learning environments. We emphasize accessibility and interactivity for smart contents. For accessibility, most smart contents must be accessed regardless of user's internet environments. Also, students with some kinds of disabilities must be considered. For interactivity, we must include the basic interaction for students: student-contents, student-student, and student-teacher. In addition, we include student-external specialist interaction.

Our proposed quality standards have 14 areas and 35 standards for those areas. The 14 areas are as follows: requirement analysis, instructional design, study contents, teaching-learning strategy, interactivity, evaluation, feedback, support system, reusability, sharing & distribution, accessibility, restructuring, ethicality, and copyright, respectively. Table 6 shows the initial smart quality standards.

3.2 Statistical Processing Procedure and Sampling Methods

The following statistical processing methods are adopted for this study. First, for each response from each standard, frequency analysis is performed. Second, descriptives such as average and standard deviation are used for check importance of each area. Third, Cronbach's a is used for checking reliability of quality standards for educational smart contents. Fourth, for empirical analysis of this study, significance level p<.05, p<.01, p<.001 are used.

For our statistical analysis, 45 experts are surveyed. Those exports are professors and researchers majoring computer education or MIS(management information system). Also, teachers are selected for this study. They are interested in smart learning and working for master degree in computer education major. For unbiased sampling, 'convenient sampling' method is adopted. Also, overcome geographic-bias, all respondants are selected from all over Seoul and suburbs of Seoul. Each respondent is required to answer every question of quality standards for both teaching-learning and education support categories. 5 scales are used for each question: 5(very important), 4(important), 3(so-so), 2(not important), 1(never important), respectively.

3.3 Statistical Analysis

3.3.1 Verification of Reliability

At first, reliability of 35 standards for both categories (teaching-learning and education support) are analyzed. The following Table 7 shows analysis results of verification of reliability for 35 standards of 2 categories. The values represent Cronbach's [alpha]. As we can see from the values of Table 7, Cronbach's [alpha] is greater than 0.6 so that most standards are sufficient to be used for quality standards.

3.3.2 Verification of Validity for Standards

Evaluation of importance for each standard of 2 categories is analyzed. Table 8 shows analysis results for evaluation of importance. In the table, the following notations are used:

A: never important

B: not important)

C: so-so

D: important

E: very important

For further analysis of importance evaluation, average and standard deviation for each standard are calculated. Also, ranking is calculated. Note that the highest score for each standard is 5(very important) and the lowest score for each standard is 1(never important). Table 9 shows results of further evaluation of importance for each standard.

After thorough analysis for importance evaluation, the following results are obtained. For teaching-learning category, the highest standard is No. 12(4.58 average) and the lowest standard is No 22(3.42 average). No. 22 is the only standard that has less than 70% of the perfect score 5. On the other hand, for education support category, No. 12 has the highest score 4.58 average while No. 22 has the lowest score 3.40 average. No. 22 is excluded since it has less than 70% of the perfect score, that is, 3.5. Table 10 shows the summary of the analysis.

In the meanwhile, we analyze importance for areas. The following Table 11 shows the importance of each area. For teaching-learning category, the most important area is "instructional design" while the least important area is "interactivity". On the other hand, for education support category, the most important area is "copyright" while the least important area is "interactivity". For both categories, "interactivity" has the least score. This can be explained as follows. The smart contents inherently include various types of interaction with wired/wireless communication tools so that "interactivity" need not be emphasized.

3.4 The Final Smart Quality Standards

Based on extensive statistical analysis in 3.3, we finally the following quality standards for educational smart contents. The final standards consist of 14 areas and 34 standards for both categories. Table 12 shows the final areas and standards.

3.5 Implication of Statistical Analysis

As we can see from analysis results in previous section, most of the initially proposed quality standards are selected as the final quality standards. The only rejected standard is No. 22 for both categories: student-external specialist interaction. In additions, the importance of interactivity is given lower priority based on the survery work. It implies that smart contents themselves have already various interaction mechanism and communication tools inherently. Thus, interactivity needs not be considered redundantly.

On the other hand, accessibility area is considered more important than interactivity. It means that smart contents must be accessed anytime anywhere anyplatform, and any network. Accessibility is very important premise for smart learning. Also, the disabled must access smart contents without any barriers as well as the non-disabled.

Analysis results also show that the quality standards of smart learning include the quality standards of the existing e-learning since the concept of smart learning includes the concept of the existing e-learning as indicated in [4].

4. Conclusion and Further Works

In the current knowledge and information age, with aid of advances in smart and ICT technologies, our life style has been changing dramatically and greatly. Regardless of age, sex, and region, everybody can enjoy the benefits of advanced technologies in every aspect of our daily life. As more people have smart devices and use them in their daily life, smart devices become the necessities. Smart technologies have affected educational area so that a new concept called smart learning is introduced.

As educational smart contents become abundant, we need quality standards for the smart contents. Those standards are very important for evaluating the smart contents and can be a milestone to guide for future smart contents production. Although there are some standards for the existing e-learning environments, to our best knowledge, there are no quality standards for educational smart contents in the literature.

The purpose of this paper is to develop and present quality standards for educational smart contents. The proposed quality standards are made based on the existing quality standards in e-learning environments and include some distinct characteristics of smart learning. For development of quality standards, 45 experts group from academy and industry are selected and surveyed. Their responses are analyzed based on thorough statistical analysis so that final quality standards for educational smart contents are developed. The final quality standards consist of 14 areas and 34 standards.

Our further research issues are as follows. First, we need to develop quality standards for different applications of smart environments such as text contents, graphic contents, sound contents, and video contents, etc. Second, we also need to develop quality standards for elementary school, middle school, and high school, even for lifelong education center, etc. Finally, our task is to develop practical guidelines for educational smart contents production. Those guidelines can be a milestone for every type of smart contents production for teachers, researchers, and manufacturer, etc.

A preliminary version of this paper was presented at ICONI 2013 and was selected as an outstanding paper.

http://dx.doi.org/10.3837/tiis.2014.06.020

References

[1] M. Weiser, R. Gold, and J. Brown, "The Origins of Ubiquitous Computing Research at PARC in the late 1980s," IBM Systems Journal, Vol. 38, No. 4, pp. 693-696, 1999. Article (CrossRef Link)

[2] K. Lim, "Research on Developing Instructional Design Models for Enhancing Smart Learning," Journal of Korean Society of Computer Education, Vol. 14, No. 2, pp. 33-45, 2011. Article (CrossRef Link)

[3] S. Kim, S. Song and Y. Yoon, "Smart Learning Services Based on Smart Cloud Computing," Sensors, Vol. 11, pp. 7835-7850, 2011. Article (CrossRef Link)

[4] H. Kim, "Issues on Smart Education Contents Quality Management and Teaching-Learning Model Development," Research Report RM-2011-20, Korea Education and Research Information Service, 2011. Article (CrossRef Link)

[5] C. Fadel and B. Trilling, 21st Century Skills: Learning for Life in Our Time, Jossey-Bass Press, San Francisco, CA, USA, 2009. Article (CrossRef Link)

[6] T. Han, Y, Kim, J, Kim, and W. Lee, "A Study on Convergence Plan of Educational Information Standardization and Quality Management," Research Report CR-2010-20, Korea Education and Research Information Service, 2010. Article (CrossRef Link)

[7] Y. Yang, M. Kang, J. Shin, Y. Kim, and E. Lee, "A Study on Vitalization Plan of E-learning Quality Certification System," Research Report CR-2008-1, Korea Education and Research Information Service, 2008. Article (CrossRef Link)

[8] S. Jung, J. Ryu, and Y. Kim, "Development of Primary and Secondary Education E-learning Quality Management Guideline," Research Report CR-2008-5, Korea Education and Research Information Service, 2008. Article (Cross Link)

[9] S. Jung, J. Ryu, Y. Kim, J. Kim, J. Kim, Y. Lee, J. Jun, E. Lee, B. Jung, and J. Um, "Development of Higher Education E-learning Quality Management Guideline," CR-2008-4, Korea Education and Research Information Service, 2008. Article (Cross Link)

[10] S. Jung, J. Ryu, Y. Kim, D. Kwak, S. Park, S. Ahn, J. Kim, J. Jun, E. Lee, B. Jung, and J. Um, "A Study on Development of Lifelong Education E-learning Quality Management Guideline," Research Report CR-2008-3, Korea Education and Research Information Service, 2008. Article (CrossRef Link)

[11] B. Lim, M. Kim, S. Choi, S. Shin, N. Cha, S. Lee, Y. Byun, J. Ryu, and Y. Lim, "An Analysis of Issues on Development of Smart Education Contents Quality Management Guidelines," Research Report RM 2011-13, Korea Education and Research Information Service, 2011. Article (CrossRef Link)

[12] W. Jun and S. Hong, "A Study on Development of Smart Literacy Standards of Teachers and Students in Smart Learning Environments," Journal of Korean Society for Internet Information, Vol. 14, No. 6, pp. 59-70, 2013. Article (CrossRef Link)

[13] W. Jun and S. Hong, "A Study on Development of Smart Skill Standards for Students and Teachers," in Proc. of Proceedings of the 5th International Conference on Internet, pp. 408-414, Dec. 12-16, 2013.

Woochun Jun (1) and Suk-ki Hong (2)

(1) Department of Computer Education, Seoul National University of Education Seoul, Korea

[e-mail: wocjun@snue.ac.kr]

(2) Department of Management, Dankook University Yongin, Korea

[e-mail: skhong017@dankook.ac.kr]

* Corresponding author: Suk-ki Hong

Received April 6, 2014; revised June 6, 2014; accepted June 10, 2014; published June 27, 2014

Woochun Jun is a professor in Dept. of Computer Education at Seoul National University of Education, Seoul, Korea. He received Ph.D. degree in Computer Science from University of Oklahoma, USA in 1997. He also received a Master's degree and BS degree in Computer Science from Sogang University, Seoul, Korea, in 1987 and 1985, respectively. His research areas include information education, information communication ethics, and gifted education in IT.

Suk-Ki Hong is a professor in the Department of Business Administration, Dankook University, Gyeonggi-do, Korea. He received Ph.D. from the University of Nebraska-Lincoln, USA, in 1996. His main research interests are e-Learning, e-Business, e-Service, Service Quality, SCM, and IT Strategies.
Table 1. Comparison of Different Types of Education [4]

                            Traditional   E-learning   M-learning
                               Style

Intelligent Personalized                                  (A)
  Study
Cooperative Activity                         (A)          (A)
Bi-direaction                                (A)          (A)
Participation Activity          (A)          (A)
Sharing Activity                             (A)
Intelligent Study
  Information Management
Time/Space Limitation                        (A)          (A)
  Conquest
Study Information
  Generation
Application of Social                        (A)
  Networking
Application of Convergent
  Education Media
Non-linear Study                             (A)          (A)

                            U-learning   Smart Learnng

Intelligent Personalized       (A)            (B)
  Study
Cooperative Activity           (B)            (B)
Bi-direaction                  (B)            (B)
Participation Activity         (A)            (B)
Sharing Activity               (B)            (B)
Intelligent Study              (B)            (B)
  Information Management
Time/Space Limitation          (B)            (B)
  Conquest
Study Information              (B)            (B)
  Generation
Application of Social          (A)            (B)
  Networking
Application of Convergent                     (B)
  Education Media
Non-linear Study               (B)            (B)

Note that the simbol (A) represents "possible" and the
simbol "(B)" represents "applicable", respectively.

Table 2. Capabilities to be Prepared for 21st Century
Learners [5]

Capability                 Description

Learning and Innovation    Critical Thinking and Problem
  Skills                   Communications and Collaboration
                           Creativity and Innovation

Life and Career Skills     Information Literacy
                           Media Literacy
                           ICT Literacy

Information, Media, and    Flexibility and Adaptability
  Technology Skills        Initiative and Self-Direction
                           Social and Cross-cultural Interaction
                           Productivity and Accountability
                           Leadership and Responsibility

Table 3. Characteristics of Smart Learning [4]

Characteristic        Description

Self-directed         (Knowledge Generator)
                      Student's role change:
                      from knowledge consumer to
                      knowledge generator

                      Teacher's role change:
                      from knowledge messenger
                      to helper

                      (Intelligent)
                      Online achievement diagnosis and
                      prescription

Motivated             (Experience-oriented)
                      Knowledge reconstruction:
                      from lecture-oriented to
                      experience-oriented

Adaptive              (Flexible)
                      Flexibility of education system

                      (Personalized) Role of
                      school: from massive
                      delivery to personalized
                      study

                      (Open Market)
                      Free sharing of online information
                      based on cloud education
                      service

Resource Free         (Social Networking)
                      Extension of cooperative
                      study using collective
                      intelligence and social
                      learning

Technology Embedded   (Globalization)
                      Anytime anywhere study and various
                      study options for students

Table 4. Comparison of Smart Learning and Traditional
Learning [4]

Characteristic               Description

Self-directed         (Extension of Time)
                      Anytime study based on
                      onlie and cloud education
                      service

Motivated             (Extension of Education
                      Method) From the existing
                      delivery-oriented study in
                      classes to cooperative
                      study, and individualized
                      study based on wireless
                      internet environment

Adaptive              (Extension of Education
                      Capability) 7C
                      capabilities(Critical
                      thinking and problem
                      solving, Creativity and
                      innovation, Collaboration
                      and leardership,
                      Cross-cultural
                      understanding,
                      Communication, ICT
                      literacy, Career and life
                      skills) as well as 3R
                      literacy in the existing
                      education(Read, wRite, and
                      aRithmetic)

Resource Free         (Extension of Education
                      Contents) Overcoming
                      limited contents in the
                      book-style textbook and
                      reference books, provison
                      of abundant information
                      using cloud education
                      service and social
                      networking

Technology Embedded   (Extension of Space)
                      Anywhere study using
                      wireless internet and
                      smart devices

Table 5. Quality Standards for E-learning Environment [8]

Area                    Sub-area                   Standards

Requirement      Study Contents Analysis   Does study activity enable
Analysis                                   study objective to be
                                           achieved?

                 Study Environment         Is it possible that study
                 Analysis                  program and application
                                           software do not depend on
                                           platform and operating
                                           system, and selective
                                           execution is provided?

                                           Are application areas of
                                           study program and
                                           application software wide
                                           and diverse?

                                           Is it possible that study
                                           program and application
                                           program can exchange data
                                           with other programs?

                                           Is installaton and
                                           updation of study program
                                           and application software
                                           easy and comfortable?

Instructional    Idea Generation           Can new study interest be
Design                                     incurred by adding
                                           creative ideas to the
                                           existing study contents

                 Presentation of Study     Is study objective
                 Objective                 presented in smart
                                           contents?

                 Study Material            Are appropriate study
                 Selection                 materials(text data,
                                           graphic data, sound data,
                                           video data, etc) used
                                           considering student's
                                           level(age and experience)?

                                           Are various technical
                                           methods used for study
                                           contents and related
                                           information considering
                                           students, study contents,
                                           study environment?

                 Screen Construction       Is screen systemically
                 and Arragement            constructed considering
                                           students(age and study
                                           experience), study
                                           contents, and study
                                           environment?

                                           Is construction of study
                                           screen is simple and
                                           consistent?

                                           Are study information
                                           elements maximized and
                                           other elements(buttons,
                                           metaphor, background
                                           image) minimized?

                                           Is important information
                                           emphasized and constructed
                                           easily? Also, is the
                                           quality good?

                                           Does screen construction
                                           and arragement include new
                                           idea?

                 Interface and Progress    Are UI components(button,
                                           menu, icon, scroll bar,
                                           etc) constructed and
                                           arranged with consistency
                                           for student's study
                                           progress?

                                           Can students identify the
                                           current position within
                                           entire contents? Is
                                           movement easy?

Study Contents   Selection of Study        Are contents selected for
                 Contents                  students to experience
                                           within their age level?

                                           Do study contents include
                                           new information and
                                           tendency?

                                           Are study contents
                                           non-repetetively and
                                           logically presented?

                 Organization of Study     Are study contents
                 Contents                  organized considering
                                           student's level(cognitive
                                           capability, etc)?

                                           Are glossary and spelling
                                           of study contents correct
                                           and error-free?

                                           Are semantic expression of
                                           study contents error-free?

                 Study Level of            Are study levels(easy,
                 Difficulty                intermediate, and
                                           difficult, etc) selected
                                           appropriately?

                                           Are study contents
                                           organized with study
                                           stages?

Teaching-        Selection of Teaching-    Are appropriate
Learning         Learning Strategy         teaching-learning
Strategy                                   strategies used in smart
                                           contents?

                                           Are appropriate
                                           teaching-learning
                                           strategies(problem-
                                           oriented learning,
                                           project-based learning,
                                           etc) selected considering
                                           online study environment
                                           and study contents
                                           analysis results?

Support System   Selection of Support      Are support contents that
                 Contents                  are necessary for students
                                           and teachers(study
                                           progress and study help,
                                           etc) included?

                 Selection Support         Are detailed and effective
                 Methods                   support contents(help
                                           menu, study guide, and
                                           FAQ, etc) included?

                                           Are appropriate and
                                           realizable support methods
                                           used?

                                           Are various support
                                           methods(checklist,
                                           worksheet, etc) applied
                                           considering
                                           characteristics of
                                           students and teachers?

                 Application of Support    Are various support
                 Tools                     functions that are
                                           available depending on
                                           study nature included?

Reusability      Restructuration           Are study process and
                                           contents constructed to be
                                           reusable for supplement
                                           purpose?

                                           Are study contents
                                           constructed to be reusable
                                           depending on study
                                           objectives, student's
                                           level, and study
                                           environment?

Ethicality       Ethical Norm              Do the contents include
                                           ethical bias such as
                                           religion, region,
                                           political belief, and
                                           violent expressions?

Copyright        Copyright Application     Do all writings follow the
                                           copyright laws and
                                           regulation?

Table 6. The Initial Smart Quality Standards

Area                Sub-area                 Serial No.

Requirement         Study Contents               1
Analysis            Analysis

                    Study Environment            2
                    Analysis

Instructional       Clarity                      3
Design
                    Leveled Learning             4

                    Study Material               5

                    Screen Construction          6
                    & Arrangement

                    Interface & Progress         7

Study Contents      Study Contents               8
                    Selection

                    Study Contents               9
                    Selection

                    Selection of Study           10
                    Contents

                    Organization of              11
                    Study Contents

                    Organization of              12
                    Study Contents

                    Organization of              13
                    Study Contents

                    Organization of              14
                    Study Contents

                    Study Level of               15
                    Difficuly

                    The Amount of                16
                    Study

Teaching-Learning   Selection of                 17
Strategy            Teaching-Learning

                    Strategy                     18
                    Motivation-Support

Interactivity       Strategy                     19
                    Student-Contents
                    Interaction

                    Student-Student              20
                    Interaction

                    Student-Teacher              21
                    Interaction

                    Student-External             22
                    Specialist Interaction

Evaluation          Evaluation Contents          23

                    Evaluation Methods           24

Feedback            Feedback                     25

Support System      Selection of Support         26
                    Contents

                    Selection of Support         27
                    Methods

Reusability         Reusability                  28

Sharing &           Sharing &                    29
Distribution        Distribution

                    Contents                     30
                    Accessibility

Accessibility       Access Restriction           31

                    Communication Tool           32

Restructuring       Restructuring                33

Ethicality          Ethical Norm                 34

Copyright           Copyright                    35
                    Application

Area                Standards

Requirement         Does study activity enable study objective to
Analysis            be achieved?

                    Is it possible that study program and
                    application software do not depend on
                    platform and operating system, and selective
                    execution is provided?

Instructional       Is study objective presented clearly?
Design
                    Is the leveled learning possible for various
                    study contents and methods depending
                    student's capability?

                    Are appropriate study materials(text data,
                    graphic data, sound data, video data, etc) used
                    considering student's level(age and
                    experience)?

                    Is screen systemically constructed
                    considering students(age and study
                    experience), study contents, and study
                    environment?

                    Is it possible that students can control study
                    process and speed?

Study Contents      Are the smart contents selected to help
                    achievement of study objectives as core
                    contents?

                    Do study contents include new information
                    and tendency?

                    Are study contents non-repetetively and
                    logically presented?

                    Are study contents organized considering
                    student' level(cognitive capability, etc)?

                    Are study contents themselves error-free?

                    Are glossary and spelling of study contents
                    correct and error-free?

                    Do study contents include prerequite contents
                    and supplement-reinforcement contents?

                    Do study contents consider study levels(easy,
                    intermediate, and difficult, etc) appropriately?

                    Are the amount of study contents appropriate
                    for each class?

Teaching-Learning   Do study contents include appropriate
Strategy            teaching-learning strategy for online study?

                    Do study contents include various motivation
                    support strategy?

Interactivity       Do study contents include interaction between
                    students and the contents?

                    Do study contents include interaction between
                    students and other students?

                    Do study contents include interaction between
                    students and teachers?

                    Do study contents include interaction between
                    students and external specialist(including
                    parents)?

Evaluation          Are evaluation contents organized to help
                    students complete the course?

                    Are various evaluation methods used?

Feedback            Are answers to student's questions and
                    evaluation results provided to students?

Support System      Are support contents that are helpful for
                    student's study progress included?

                    Are various support methods(checklist,
                    worksheet, etc) applied to students?

                    Are study contents constructed to be reusable
Reusability         depending on study objectives, student's
                    level, and study environment?

Sharing &           Is metadata provided for sharing study
Distribution        contents?

                    Are study contents accessible anytime
                    anywhere using wired/wireless internet?

Accessibility       Is there any access restriction depending on
                    student's mental or physical disabilities?

                    Are any communication tools(email, BBS,
                    etc) provided for student's questions and
                    answers to study contents?

                    Are study contents and process restuructable
Restructuring       depending on study objectives and study
                    environments?

                    Do the contents include ethical bias such as
Ethicality          religion, region, political belief and violent
                    expressions?

Copyright           Do all writings follow the copyright laws and
                    regulation?

Table 7. Verification of Reliability for Each Standard

Area                Standard No.   Teaching-Learning

Requirement              1         .965                .674
  Analysis               2         .966
Instructional            3         .965                .782
  Design                 4         .966
                         5         .966
                         6         .965
                         7         .966
Study Contents           8         .964                .895
                         9         .965
                         10        .965
                         11        .964
                         12        .965
                         13        .965
                         14        .964
                         15        .964
                         16        .963
Teaching-Learning        17        .963                .689
  Strategy               18        .964
Interactivity            19        .964                .840
                         20        .964
                         21        .964
                         22        .965
Evaluation               23        .964                .650
                         24        .964
Feedback                 25        .964                --
Support                  26        .964                .797
  System                 27        .964
Reusability              28        .964                --
Sharing &                29        .964                --
  Distribution
                         30        .964
Accessibility            31        .964                .822
                         32        .964
Restructuring            33        .964                --
Ethicality               34        .964                --
Copyright                35        .964                --

Area                Standard No.   Education Support

Requirement              1         .956                .496
  Analysis               2         .955
Instructional            3         .956                .820
  Design                 4         .956
                         5         .956
                         6         .956
                         7         .956
Study Contents           8         .956                .862
                         9         .955
                         10        .956
                         11        .955
                         12        .956
                         13        .955
                         14        .954
                         15        .954
                         16        .954
Teaching-Learning        17        .954                .657
  Strategy               18        .954
Interactivity            19        .954                .891
                         20        .955
                         21        .954
                         22        .955
Evaluation               23        .955                .553
                         24        .955
Feedback                 25        .954                --
Support                  26        .955                .633
  System                 27        .955
Reusability              28        .954                --
Sharing &                29        .954                --
  Distribution
                         30        .955
Accessibility            31        .955                .655
                         32        .955
Restructuring            33        .955                --
Ethicality               34        .955                --
Copyright                35                            --

Table 8. Evalution of Importance for Each Standard

Standard                   Teaching-Learning

             A       B       C       D       E

1                          8.9%    26.7%   64.4%
2                  2.2%    13.3%   37.8%   46.7%
3                          11.1%   31.1%   57.8%
4                          6.7%    37.8%   55.6%
5                  2.2%    8.9%    20.0%   68.9%
6                          8.9%    33.3%   57.8%
7                  4.4%    13.3%   46.7%   35.6%
8                          15.6%   17.8%   66.7%
9                  2.2%    31.1%   31.1%   35.6%
10                 4.4%    20.0%   46.7%   28.9%
11                         11.1%   22.2%   66.7%
12                         13.3%   15.6%   71.1%
13                 4.4%    13.3%   35.6%   46.7%
14                 8.9%    31.1%   33.3%   26.7%
15                 6.7%    28.9%   28.9%   35.6%
16                 2.2%    28.9%   31.1%   37.8%
17                 4.4%    15.6%   31.1%   48.9%
18                 2.2%    20.0%   35.6%   42.2%
19                 6.7%    26.7%   31.1%   35.6%
20         4.4 %   15.6%   11.1%   35.6%   33.3%
21                 8.9%    24.4%   28.9%   37.8%
22         2.2 %   20.0%   37.8%   13.3%   26.7%
23                 2.2%    11.1%   24.4%   62.2%
24                 4.4%    31.1%   31.1%   33.3%
25                 4.4%    11.1%   21.1%   53.3%
26                         33.3%   28.9%   37.8%
27                 11.1%   26.7%   31.1%   31.1%
28                 2.2%    22.2%   40.0%   35.6%
29                 11.1%   31.1%   24.4%   33.3%
30                 4.4%    17.8%   31.1%   46.7%
31                 6.7%    17.8%   35.6%   40.0%
32         2.2 %   4.4%    24.4%   33.3%   35.6%
33                 4.4%    26.7%   40.0%   28.9%
34                 8.9%    20.0%   17.8%   53.3%
35                         17.8%   22.2%   60.0%

Standard                  Education Support

            A       B       C       D       E

1                 4.4%    4.4%    37.8%   53.3%
2          2.2%           15.6%   42.2%   40.0%
3                         17.8%   33.3%   48.9%
4                         15.6%   42.2%   42.2%
5                         8.9%    40.0%   51.1%
6                         13.3%   40.0%   46.7%
7          2.2%           20.0%   46.7%   31.1%
8                         24.4%   22.2%   53.3%
9                 4.4%    20.0%   28.9%   46.7%
10                8.9%    26.7%   42.2%   46.7%
11                        17.8%   35.6%   46.7%
12                        11.1%   20.0%   68.9%
13                2.2%    22.2%   31.1%   44.4%
14         2.2%   6.7%    35.6%   24.4%   31.1%
15                8.9%    26.7%   26.7%   37.8%
16                4.4%    31.1%   33.3%   31.1%
17                2.2%    20.0%   33.3%   44.4%
18                6.7%    26.7%   31.1%   35.6%
19         2.2%   8.9%    24.4%   31.1%   33.3%
20         4.4%   15.6%   17.8%   33.3%   28.9%
21                4.4%    35.6%   24.4%   35.6%
22         4.4%   22.2%   28.9%   17.8%   26.7%
23         2.2%           17.8%   31.1%   48.9%
24                8.9%    22.2%   35.6%   33.3%
25                2.2%    17.8%   33.3%   46.7%
26         2.2%           22.2%   40.0%   35.6%
27                6.7%    24.4%   33.3%   35.6%
28                4.4%    20.0%   37.8%   37.8%
29                6.7%    40.0%   11.1%   42.2%
30                6.7%    11.1%   31.1%   51.1%
31                4.4%    17.8%   26.7%   51.1%
32                11.1%   20.0%   31.1%   37.8%
33                8.9%    20.0%   40.0%   31.1%
34                4.4%    20.0%   24.4%   51.1%
35                2.2%    15.6%   24.4%   57.8%

Table 9. Further Evaluation of Importance for Each Standard

Standard             Teaching-Learning

           Average   Standard Deviation   Ranking

1          4.56      .659                 2
2          4.29      .787                 12
3          4.47      .694                 8
4          4.49      .626                 6
5          4.56      .755                 2
6          4.49      .661                 6
7          4.13      .815                 18
8          4.51      .757                 5
9          4.00      .879                 23
10         4.00      .826                 23
11         4.56      .693                 2
12         4.58      .723                 1
13         4.24      .857                 13
14         3.78      .951                 33
15         3.93      .963                 28
16         4.04      .878                 21
17         4.24      .883                 13
18         4.18      .834                 16
19         3.96      .952                 25
20         3.78      1.204                33
21         3.96      .999                 25
22         3.42      1.158                35
23         4.47      .786                 8
24         3.93      .915                 28
25         4.33      .853                 11
26         4.04      .852                 21
27         3.82      1.007                31
28         4.09      .821                 19
29         3.80      1.036                32
30         4.20      .894                 15
31         4.09      .925                 19
32         3.96      .999                 25
33         3.93      .863                 28
34         4.16      1.043                17
35         4.42      .783                 10

Standard             Education Support

           Average   Standard Deviation   Ranking

1          4.40      .780                 3
2          4.18      .860                 16
3          4.31      .763                 6
4          4.27      .720                 9
5          4.42      .657                 2
6          4.33      .707                 5
7          4.04      .852                 21
8          4.29      .843                 7
9          4.18      .912                 16
10         3.78      .902                 32
11         4.29      .757                 7
12         4.58      .690                 1
13         4.18      .860                 16
14         3.76      1.048                33
15         3.93      1.009                25
16         3.91      .900                 28
17         4.20      .842                 15
18         3.96      .952                 23
19         3.84      1.065                31
20         3.67      1.187                34
21         3.91      .949                 28
22         3.40      1.232                35
23         4.24      .908                 11
24         3.93      .963                 25
25         4.24      .830                 11
26         4.07      .889                 20
27         3.98      .941                 22
28         4.09      .874                 19
29         3.89      1.049                30
30         4.27      .915                 9
31         4.24      .908                 11
32         3.96      1.021                23
33         3.93      .939                 25
34         4.22      .927                 14
35         4.38      .834                 4

Table 10. The Excluded Quality Standard

Standard             Teaching-Learning

           Average   Standard    Ranking
                     Deviation

22          3.42      .1.158       35

Standard             Education Support

           Average   Standard    Ranking
                     Deviation

22          3.40       1.232       35

Table 11. Evaluation of Importance for Each Area

Area                   Teaching-Learning

                       Average   Standard    Ranking
                                 Deviation

Requirement Analysis   4.42      .630        2
Instructional          4.43      .522        1
  Design
Study Contents         4.18      .620        7
Teaching-Learning      4.21      .750        5
  Strategy
Interactivity          3.78      .891        14
Evaluation             4.20      .734        6
Feedback               4.33      .853        4
Support System         3.93      .850        11
Reusability            4.09      .821        9
Sharing &              3.80      1.036       13
  Distribution
Accessibility          4.08      .808        10
Restructuring          3.93      .863        11
Ethicality             4.16      1.043       8
Copyright              4.42      .783        2

Area                   Education Support

                       Average   Standard    Ranking
                                 Deviation

Requirement Analysis   4.29      .670        2
Instructional          4.28      .566        3
  Design
Study Contents         4.10      .611        7
Teaching-Learning      4.08      .776        10
  Strategy
Interactivity          3.71      .967        14
Evaluation             4.09      .778        8
Feedback               4.24      .830        4
Support System         4.02      .783        11
Reusability            4.09      .874        8
Sharing &              3.89      .1.049      13
  Distribution
Accessibility          4.16      .730        6
Restructuring          3.93      .939        12
Ethicality             4.22      .927        5
Copyright              4.38      .834        1

Table 12. The Final Quality Standards for Educational Smart Contents

Area             Sub-area          Serial   Standards
                                   No.

Requirement      Study Contents    1        Does study activity enable
  Analysis       Analysis                     study objective to be
                                              achieved?
                                            Is it possible that study
                 Study             2          program and application
                   Environment                software do not depend
                   Analysis                   on platform and
                                              operating system, and
                                              selective execution is
                                              provided?
Instructional    Clarity           3        Is study objective
  Design                                      presented clearly?
                 Leveled                    Is the leveled learning
                   Learning        4          possible for various
                                              study contents and
                                              methods depending
                                              student's capability?
                 Study Material    5        Are appropriate study
                                              materials(text data,
                                              graphic data, sound
                                              data, video data, etc)
                                              used considering
                                              student's level(age and
                                              experience)?
                 Screen            6        Is  screen systemically
                   Construction               constructed considering
                   & Arrangement              students(age and study
                                              experience), study
                                              contents, and study
                                              environment?
                 Interface &       7        Is it possible that
                   Progress                   students can control
                                              study process and speed?
Study            Study Contents    8        Are the smart contents
  Contents         Selection                  selected to help
                                              achievement of study
                                              objectives as core
                                              contents?
                 Study Contents    9        Do study contents include
                                              new information
                 Selection                    and tendency?
                 Selection of      10       Are study contents non-
                   Study                      repetetively and
                   Contents                   logically presented?
                 Organization               Are study contents
                   of Study        11         organized considering
                   Contents                   student' level(cognitive
                                              capability, etc)?
                 Organization      12       Are study contents
                   of Study                   themselves error-free?
                   Contents
                 Organization      13       Are glossary and spelling
                   of Study                   of study contents
                   Contents                   correct and error-free?
                 Organization      14       Do study contents include
                   of Study                   prerequite contents and
                   Contents                   supplement-reinforcement
                                              contents?
                 Study Level of    15       Do study contents consider
                   Difficulty                 study levels(easy,
                                              intermediate, and
                                              difficult, etc)
                                              appropriately?
                 The Amount        16       Are the amount of study
                   of Study                   contents appropriate
                                              for each class?
Teaching-        Selection of      17       Do study contents include
  Learning         Teaching-                  appropriate teaching-
  Strategy         Learning                   learning strategy for
                   Strategy                   online study?
                 Motivation-       18       Do study contents include
                   Support                    various motivation
                   Strategy                   support strategy?
Interactivity    Student-          19       Do study contents include
                   Contents                   interaction between
                   Interaction                students and the
                                              contents?
                 Student-          20       Do study contents include
                   Student                    interaction between
                   Interaction                students and other
                                              students?
                 Student-          21       Do study contents include
                   Teacher                    interaction between
                   Interaction                students and teachers?
Evaluation       Evaluation        22       Are evaluation contents
                   Contents                   organized to help
                                              students complete the
                                              course?
                 Evaluation        23       Are various evaluation
                   Methods                    methods used?
Feedback         Feedback          24       Are answers to student's
                                              questions and evaluation
                                              results provided to
                                              students?
Support          Selection of      25       Are support contents that
  System           Support                    are helpful for
                   Contents                   student's study
                                              progress included?
                 Selection of      26       Are various support
                   Support                    methods (checklist,
                   Methods                    worksheet, etc) applied
                                              to students?
Reusability      Reusability       27       Are study contents
                                              constructed to be
                                              reusable depending on
                                              study objectives,
                                              student's level, and
                                              study environment?
Sharing &        Sharing &         28       Is metadata provided for
  Distribution     Distribution               sharing study contents?
                 Contents          29       Are study contents
Accessibility      Accessibility              accessible anytime
                                              anywhere using wired/
                                              wireless internet?
                 Access            30       Is there any access
                  Restriction                 restriction depending
                                              on student's mental or
                                              physical disabilities?
                 Communication     31       Are any communication
                  Tool                        tools(email, BBS, etc)
                                              provided for student's
                                              questions and answers
                                              to study contents?
Restructuring    Restructuring     32       Are study contents and
                                              process restructurable
                                              depending on study
                                              objectives and study
                                              environments?
                 Ethical Norm      33       Do the contents include
Ethicality                                    ethical bias such as
                                              religion, region,
                                              political belief, and
                                              violent expressions?
Copyright        Copyright         34       Do all writings follow
                   Application                the copyright laws
                                              and regulation?
COPYRIGHT 2014 KSII, the Korean Society for Internet Information
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2014 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Jun, Woochun; Hong, Suk-ki
Publication:KSII Transactions on Internet and Information Systems
Article Type:Report
Date:Jun 1, 2014
Words:6485
Previous Article:Self-adaptive testing to determine sample size for flash memory solutions.
Next Article:A subjectivity study on the promotion of Korean smart TV industry through Q methodology.
Topics:

Terms of use | Privacy policy | Copyright © 2019 Farlex, Inc. | Feedback | For webmasters