A study on development of quality standards of educational smart contents.
In the current knowledge and information society, our daily life relies on the benefits of advanced ICT technologies. With aid of ICT technologies, our life style has been changed dramatically. Our life becomes more convenient and enjoyable than ever. How to use the advanced technologies becomes a part of social competativeness as well as individual competativeness.
Recently ICT technologies have been converged into smart technology. It means that the existing ICT devices become smaller, light, convenient, and portable. In this sense, the concept of "smart environment" is realized. Smart environment is defined as "a physical world that is richly and invisibly interwoven with sensors, actuators, displays, and computational elements, embedded seamlessly in the everyday objects of our lives, and connected through a continuous network." . In smart environment, every aspect of our life has been affected with smart technologies. Now advanced in smart technologies have created a new concept called "smart learning" in education area.
Smart learning is a new concept so that there is no clear definition yet. However, some research works introduce the definition of smart learning [2,3]. In , it is argued that the principle of smart learning has the following 3 characteristics: First, rich instructional resources as learning contents, Second, participatory learning environments with interactions among teachers and learners as learning methods, Third, practical and realistic contexts as learning environments. Also, in , it is argued that smart learning is very essential in future education since it provides personalized contents and easy adaptation to the current education model.
The following Table 1 shows the comparison of different types of education: traditional style, e-learning, m-learning(mobile learning), u-learning(ubiquitous learning), and smart learning, respectively. Historically and chronologically our education system has been changed: traditional style, e-learning, m-learning, u-learning, and smart learning. We see that smart learning is the most extensive and comprehensive education style in the current era.
In , capabilities to be prepared for 21st century learners are announced. There are 3 categories: 1) learning and innovation skills, 2) life and career skills, 3) information, media, and technology skills, respectively. The following Table 2 shows those capabilities for the 21st century learners. We can see that smart learning is very best education style and provides the best environment to achieve the capability of 21st century learners.
As smart learning becomes popular, more educational smart contents have been produced and distributed. In this sense, we need some quality standards to determine how smart contents are useful or helpful for students as well as teachers. Those quality standards are very important because the future educational smart contents wil be made based on the quality standards. That is, the quality standards will guide for teachers and contents manufacturers on how to prepare and make new educational smart contents. In the literature, there are some quality standards for the existing e-learning environments [6,7,8,9,10], to our best knowledge, there is no quality standard for educational smart contents. In , only outlines are introduced. In the meanwhile, for smart literacy standards  and smart skill standards  for teachers and students are presented lately.
The smart contents can be classified into 2 categories for their purpose: teaching-learning and education support. The "teaching-learning" contents means any contents that can be used in class instantly. That is, those contents need not be refined for teachers to use in their class. Also, those contents can be used for students instantly. On the other hand, the "education support" contents means any contents that can be used as supplement tool for classes and students. Also, for teachers and students, those contents need to be refined for later use.
The purpose of this paper is to develop quality standards for educational smart contents. We develop the initial quality standards with various researchers and educators. The initial quality standards have 14 areas and 35 standards. In order to refine those quality standards, we ask various experts to check its validity and usefulness. Their valuable survey works are collected and analyzed. We finalize quality standards that have 14 areas and 34 standards.
This paper is organized as follows. First, in Chapter 2, we discuss literature reviews. In Chapter 3, we propose quality standards for educational smart contents. We collect responses from expert groups and do wide statistical analysis. Based on statistical analysis, we propose final quality standards. Finally, in Chapter 4, we discuss conclusions and further research works.
2. Related Work
2.1 Characteristics of Smart Learning
In , some characteristics of smart learning are introduced. Those characteristics are summarized in Table 3. In smart learning, a student's role is extended to be more active, more personalized, more experience-oriented. However, a teacher's role is changed to be a guide and mediator.
Also, in , smart learning is compared with the traditional learning in terms of categories suggested in Table 3. The traditional learning means the existing class-based learning. The following Table 4 shows comparison of smart learning and traditional learning. As we can see, smart learning environment provides more self-directed study and motivated and adaptive study for students anytime anywhere.
In , they argue that any smart contents must have the following 4 components. Those components are participation, sharing, cooperation, and accessibility, respectively. Each component is described as follows.
Students can use smart contents anytime anywhere. For this purpose, smart education contents must be accessed anytime, anywhere, anyone. Especially the various existing contents must be compatible each other with aid of cloud computing. It means that smart contents must be stored and accessed in the cloud so that students can read, update, and store anytime using networking service.
Students can actualize and develop study process and results with other students and teachers using smart contents. The study contents can be developed by sharing with others. Smart contents can be shared eaisly due to its compatibility and openness.
In smart education, cooperative work is encouraged among students since smart education service provides diverse technical and social services like SNS. Cooperative works through smart education contents can support collective intelligence that ensures reliability of study results.
The great benefit of smart environment is that students can use wireless internet with various smart devices. Smat education contents can also be accessed easily without special hardware devices supporting internet connection. Also, various types of interaction can be possible in smart contents: student-contents interaction, student-student interaction, and student-teacher interaction.
2.2 Literature Review
In the literature, there is no quality standards for educational smart contents. However, there are some quality standards for the existing e-learning environments [6,7,8,9,10]. In this paper, we introduce the representative quality standards for the general educational contents in  since the quality standards in  are most extensive among the previous works. Table 5 shows those quality standards.
3. Development of Educational Smart Quality Standards
3.1 The initial Development of Smart Quality Standards
In order to develop new quality standards for educational smart contents, we develop the initial quality standards. The proposed standards are made based on the existing standards in  and add some features of smart learning environments. We emphasize accessibility and interactivity for smart contents. For accessibility, most smart contents must be accessed regardless of user's internet environments. Also, students with some kinds of disabilities must be considered. For interactivity, we must include the basic interaction for students: student-contents, student-student, and student-teacher. In addition, we include student-external specialist interaction.
Our proposed quality standards have 14 areas and 35 standards for those areas. The 14 areas are as follows: requirement analysis, instructional design, study contents, teaching-learning strategy, interactivity, evaluation, feedback, support system, reusability, sharing & distribution, accessibility, restructuring, ethicality, and copyright, respectively. Table 6 shows the initial smart quality standards.
3.2 Statistical Processing Procedure and Sampling Methods
The following statistical processing methods are adopted for this study. First, for each response from each standard, frequency analysis is performed. Second, descriptives such as average and standard deviation are used for check importance of each area. Third, Cronbach's a is used for checking reliability of quality standards for educational smart contents. Fourth, for empirical analysis of this study, significance level p<.05, p<.01, p<.001 are used.
For our statistical analysis, 45 experts are surveyed. Those exports are professors and researchers majoring computer education or MIS(management information system). Also, teachers are selected for this study. They are interested in smart learning and working for master degree in computer education major. For unbiased sampling, 'convenient sampling' method is adopted. Also, overcome geographic-bias, all respondants are selected from all over Seoul and suburbs of Seoul. Each respondent is required to answer every question of quality standards for both teaching-learning and education support categories. 5 scales are used for each question: 5(very important), 4(important), 3(so-so), 2(not important), 1(never important), respectively.
3.3 Statistical Analysis
3.3.1 Verification of Reliability
At first, reliability of 35 standards for both categories (teaching-learning and education support) are analyzed. The following Table 7 shows analysis results of verification of reliability for 35 standards of 2 categories. The values represent Cronbach's [alpha]. As we can see from the values of Table 7, Cronbach's [alpha] is greater than 0.6 so that most standards are sufficient to be used for quality standards.
3.3.2 Verification of Validity for Standards
Evaluation of importance for each standard of 2 categories is analyzed. Table 8 shows analysis results for evaluation of importance. In the table, the following notations are used:
A: never important
B: not important)
E: very important
For further analysis of importance evaluation, average and standard deviation for each standard are calculated. Also, ranking is calculated. Note that the highest score for each standard is 5(very important) and the lowest score for each standard is 1(never important). Table 9 shows results of further evaluation of importance for each standard.
After thorough analysis for importance evaluation, the following results are obtained. For teaching-learning category, the highest standard is No. 12(4.58 average) and the lowest standard is No 22(3.42 average). No. 22 is the only standard that has less than 70% of the perfect score 5. On the other hand, for education support category, No. 12 has the highest score 4.58 average while No. 22 has the lowest score 3.40 average. No. 22 is excluded since it has less than 70% of the perfect score, that is, 3.5. Table 10 shows the summary of the analysis.
In the meanwhile, we analyze importance for areas. The following Table 11 shows the importance of each area. For teaching-learning category, the most important area is "instructional design" while the least important area is "interactivity". On the other hand, for education support category, the most important area is "copyright" while the least important area is "interactivity". For both categories, "interactivity" has the least score. This can be explained as follows. The smart contents inherently include various types of interaction with wired/wireless communication tools so that "interactivity" need not be emphasized.
3.4 The Final Smart Quality Standards
Based on extensive statistical analysis in 3.3, we finally the following quality standards for educational smart contents. The final standards consist of 14 areas and 34 standards for both categories. Table 12 shows the final areas and standards.
3.5 Implication of Statistical Analysis
As we can see from analysis results in previous section, most of the initially proposed quality standards are selected as the final quality standards. The only rejected standard is No. 22 for both categories: student-external specialist interaction. In additions, the importance of interactivity is given lower priority based on the survery work. It implies that smart contents themselves have already various interaction mechanism and communication tools inherently. Thus, interactivity needs not be considered redundantly.
On the other hand, accessibility area is considered more important than interactivity. It means that smart contents must be accessed anytime anywhere anyplatform, and any network. Accessibility is very important premise for smart learning. Also, the disabled must access smart contents without any barriers as well as the non-disabled.
Analysis results also show that the quality standards of smart learning include the quality standards of the existing e-learning since the concept of smart learning includes the concept of the existing e-learning as indicated in .
4. Conclusion and Further Works
In the current knowledge and information age, with aid of advances in smart and ICT technologies, our life style has been changing dramatically and greatly. Regardless of age, sex, and region, everybody can enjoy the benefits of advanced technologies in every aspect of our daily life. As more people have smart devices and use them in their daily life, smart devices become the necessities. Smart technologies have affected educational area so that a new concept called smart learning is introduced.
As educational smart contents become abundant, we need quality standards for the smart contents. Those standards are very important for evaluating the smart contents and can be a milestone to guide for future smart contents production. Although there are some standards for the existing e-learning environments, to our best knowledge, there are no quality standards for educational smart contents in the literature.
The purpose of this paper is to develop and present quality standards for educational smart contents. The proposed quality standards are made based on the existing quality standards in e-learning environments and include some distinct characteristics of smart learning. For development of quality standards, 45 experts group from academy and industry are selected and surveyed. Their responses are analyzed based on thorough statistical analysis so that final quality standards for educational smart contents are developed. The final quality standards consist of 14 areas and 34 standards.
Our further research issues are as follows. First, we need to develop quality standards for different applications of smart environments such as text contents, graphic contents, sound contents, and video contents, etc. Second, we also need to develop quality standards for elementary school, middle school, and high school, even for lifelong education center, etc. Finally, our task is to develop practical guidelines for educational smart contents production. Those guidelines can be a milestone for every type of smart contents production for teachers, researchers, and manufacturer, etc.
A preliminary version of this paper was presented at ICONI 2013 and was selected as an outstanding paper.
 M. Weiser, R. Gold, and J. Brown, "The Origins of Ubiquitous Computing Research at PARC in the late 1980s," IBM Systems Journal, Vol. 38, No. 4, pp. 693-696, 1999. Article (CrossRef Link)
 K. Lim, "Research on Developing Instructional Design Models for Enhancing Smart Learning," Journal of Korean Society of Computer Education, Vol. 14, No. 2, pp. 33-45, 2011. Article (CrossRef Link)
 S. Kim, S. Song and Y. Yoon, "Smart Learning Services Based on Smart Cloud Computing," Sensors, Vol. 11, pp. 7835-7850, 2011. Article (CrossRef Link)
 H. Kim, "Issues on Smart Education Contents Quality Management and Teaching-Learning Model Development," Research Report RM-2011-20, Korea Education and Research Information Service, 2011. Article (CrossRef Link)
 C. Fadel and B. Trilling, 21st Century Skills: Learning for Life in Our Time, Jossey-Bass Press, San Francisco, CA, USA, 2009. Article (CrossRef Link)
 T. Han, Y, Kim, J, Kim, and W. Lee, "A Study on Convergence Plan of Educational Information Standardization and Quality Management," Research Report CR-2010-20, Korea Education and Research Information Service, 2010. Article (CrossRef Link)
 Y. Yang, M. Kang, J. Shin, Y. Kim, and E. Lee, "A Study on Vitalization Plan of E-learning Quality Certification System," Research Report CR-2008-1, Korea Education and Research Information Service, 2008. Article (CrossRef Link)
 S. Jung, J. Ryu, and Y. Kim, "Development of Primary and Secondary Education E-learning Quality Management Guideline," Research Report CR-2008-5, Korea Education and Research Information Service, 2008. Article (Cross Link)
 S. Jung, J. Ryu, Y. Kim, J. Kim, J. Kim, Y. Lee, J. Jun, E. Lee, B. Jung, and J. Um, "Development of Higher Education E-learning Quality Management Guideline," CR-2008-4, Korea Education and Research Information Service, 2008. Article (Cross Link)
 S. Jung, J. Ryu, Y. Kim, D. Kwak, S. Park, S. Ahn, J. Kim, J. Jun, E. Lee, B. Jung, and J. Um, "A Study on Development of Lifelong Education E-learning Quality Management Guideline," Research Report CR-2008-3, Korea Education and Research Information Service, 2008. Article (CrossRef Link)
 B. Lim, M. Kim, S. Choi, S. Shin, N. Cha, S. Lee, Y. Byun, J. Ryu, and Y. Lim, "An Analysis of Issues on Development of Smart Education Contents Quality Management Guidelines," Research Report RM 2011-13, Korea Education and Research Information Service, 2011. Article (CrossRef Link)
 W. Jun and S. Hong, "A Study on Development of Smart Literacy Standards of Teachers and Students in Smart Learning Environments," Journal of Korean Society for Internet Information, Vol. 14, No. 6, pp. 59-70, 2013. Article (CrossRef Link)
 W. Jun and S. Hong, "A Study on Development of Smart Skill Standards for Students and Teachers," in Proc. of Proceedings of the 5th International Conference on Internet, pp. 408-414, Dec. 12-16, 2013.
Woochun Jun (1) and Suk-ki Hong (2)
(1) Department of Computer Education, Seoul National University of Education Seoul, Korea
(2) Department of Management, Dankook University Yongin, Korea
* Corresponding author: Suk-ki Hong
Received April 6, 2014; revised June 6, 2014; accepted June 10, 2014; published June 27, 2014
Woochun Jun is a professor in Dept. of Computer Education at Seoul National University of Education, Seoul, Korea. He received Ph.D. degree in Computer Science from University of Oklahoma, USA in 1997. He also received a Master's degree and BS degree in Computer Science from Sogang University, Seoul, Korea, in 1987 and 1985, respectively. His research areas include information education, information communication ethics, and gifted education in IT.
Suk-Ki Hong is a professor in the Department of Business Administration, Dankook University, Gyeonggi-do, Korea. He received Ph.D. from the University of Nebraska-Lincoln, USA, in 1996. His main research interests are e-Learning, e-Business, e-Service, Service Quality, SCM, and IT Strategies.
Table 1. Comparison of Different Types of Education  Traditional E-learning M-learning Style Intelligent Personalized (A) Study Cooperative Activity (A) (A) Bi-direaction (A) (A) Participation Activity (A) (A) Sharing Activity (A) Intelligent Study Information Management Time/Space Limitation (A) (A) Conquest Study Information Generation Application of Social (A) Networking Application of Convergent Education Media Non-linear Study (A) (A) U-learning Smart Learnng Intelligent Personalized (A) (B) Study Cooperative Activity (B) (B) Bi-direaction (B) (B) Participation Activity (A) (B) Sharing Activity (B) (B) Intelligent Study (B) (B) Information Management Time/Space Limitation (B) (B) Conquest Study Information (B) (B) Generation Application of Social (A) (B) Networking Application of Convergent (B) Education Media Non-linear Study (B) (B) Note that the simbol (A) represents "possible" and the simbol "(B)" represents "applicable", respectively. Table 2. Capabilities to be Prepared for 21st Century Learners  Capability Description Learning and Innovation Critical Thinking and Problem Skills Communications and Collaboration Creativity and Innovation Life and Career Skills Information Literacy Media Literacy ICT Literacy Information, Media, and Flexibility and Adaptability Technology Skills Initiative and Self-Direction Social and Cross-cultural Interaction Productivity and Accountability Leadership and Responsibility Table 3. Characteristics of Smart Learning  Characteristic Description Self-directed (Knowledge Generator) Student's role change: from knowledge consumer to knowledge generator Teacher's role change: from knowledge messenger to helper (Intelligent) Online achievement diagnosis and prescription Motivated (Experience-oriented) Knowledge reconstruction: from lecture-oriented to experience-oriented Adaptive (Flexible) Flexibility of education system (Personalized) Role of school: from massive delivery to personalized study (Open Market) Free sharing of online information based on cloud education service Resource Free (Social Networking) Extension of cooperative study using collective intelligence and social learning Technology Embedded (Globalization) Anytime anywhere study and various study options for students Table 4. Comparison of Smart Learning and Traditional Learning  Characteristic Description Self-directed (Extension of Time) Anytime study based on onlie and cloud education service Motivated (Extension of Education Method) From the existing delivery-oriented study in classes to cooperative study, and individualized study based on wireless internet environment Adaptive (Extension of Education Capability) 7C capabilities(Critical thinking and problem solving, Creativity and innovation, Collaboration and leardership, Cross-cultural understanding, Communication, ICT literacy, Career and life skills) as well as 3R literacy in the existing education(Read, wRite, and aRithmetic) Resource Free (Extension of Education Contents) Overcoming limited contents in the book-style textbook and reference books, provison of abundant information using cloud education service and social networking Technology Embedded (Extension of Space) Anywhere study using wireless internet and smart devices Table 5. Quality Standards for E-learning Environment  Area Sub-area Standards Requirement Study Contents Analysis Does study activity enable Analysis study objective to be achieved? Study Environment Is it possible that study Analysis program and application software do not depend on platform and operating system, and selective execution is provided? Are application areas of study program and application software wide and diverse? Is it possible that study program and application program can exchange data with other programs? Is installaton and updation of study program and application software easy and comfortable? Instructional Idea Generation Can new study interest be Design incurred by adding creative ideas to the existing study contents Presentation of Study Is study objective Objective presented in smart contents? Study Material Are appropriate study Selection materials(text data, graphic data, sound data, video data, etc) used considering student's level(age and experience)? Are various technical methods used for study contents and related information considering students, study contents, study environment? Screen Construction Is screen systemically and Arragement constructed considering students(age and study experience), study contents, and study environment? Is construction of study screen is simple and consistent? Are study information elements maximized and other elements(buttons, metaphor, background image) minimized? Is important information emphasized and constructed easily? Also, is the quality good? Does screen construction and arragement include new idea? Interface and Progress Are UI components(button, menu, icon, scroll bar, etc) constructed and arranged with consistency for student's study progress? Can students identify the current position within entire contents? Is movement easy? Study Contents Selection of Study Are contents selected for Contents students to experience within their age level? Do study contents include new information and tendency? Are study contents non-repetetively and logically presented? Organization of Study Are study contents Contents organized considering student's level(cognitive capability, etc)? Are glossary and spelling of study contents correct and error-free? Are semantic expression of study contents error-free? Study Level of Are study levels(easy, Difficulty intermediate, and difficult, etc) selected appropriately? Are study contents organized with study stages? Teaching- Selection of Teaching- Are appropriate Learning Learning Strategy teaching-learning Strategy strategies used in smart contents? Are appropriate teaching-learning strategies(problem- oriented learning, project-based learning, etc) selected considering online study environment and study contents analysis results? Support System Selection of Support Are support contents that Contents are necessary for students and teachers(study progress and study help, etc) included? Selection Support Are detailed and effective Methods support contents(help menu, study guide, and FAQ, etc) included? Are appropriate and realizable support methods used? Are various support methods(checklist, worksheet, etc) applied considering characteristics of students and teachers? Application of Support Are various support Tools functions that are available depending on study nature included? Reusability Restructuration Are study process and contents constructed to be reusable for supplement purpose? Are study contents constructed to be reusable depending on study objectives, student's level, and study environment? Ethicality Ethical Norm Do the contents include ethical bias such as religion, region, political belief, and violent expressions? Copyright Copyright Application Do all writings follow the copyright laws and regulation? Table 6. The Initial Smart Quality Standards Area Sub-area Serial No. Requirement Study Contents 1 Analysis Analysis Study Environment 2 Analysis Instructional Clarity 3 Design Leveled Learning 4 Study Material 5 Screen Construction 6 & Arrangement Interface & Progress 7 Study Contents Study Contents 8 Selection Study Contents 9 Selection Selection of Study 10 Contents Organization of 11 Study Contents Organization of 12 Study Contents Organization of 13 Study Contents Organization of 14 Study Contents Study Level of 15 Difficuly The Amount of 16 Study Teaching-Learning Selection of 17 Strategy Teaching-Learning Strategy 18 Motivation-Support Interactivity Strategy 19 Student-Contents Interaction Student-Student 20 Interaction Student-Teacher 21 Interaction Student-External 22 Specialist Interaction Evaluation Evaluation Contents 23 Evaluation Methods 24 Feedback Feedback 25 Support System Selection of Support 26 Contents Selection of Support 27 Methods Reusability Reusability 28 Sharing & Sharing & 29 Distribution Distribution Contents 30 Accessibility Accessibility Access Restriction 31 Communication Tool 32 Restructuring Restructuring 33 Ethicality Ethical Norm 34 Copyright Copyright 35 Application Area Standards Requirement Does study activity enable study objective to Analysis be achieved? Is it possible that study program and application software do not depend on platform and operating system, and selective execution is provided? Instructional Is study objective presented clearly? Design Is the leveled learning possible for various study contents and methods depending student's capability? Are appropriate study materials(text data, graphic data, sound data, video data, etc) used considering student's level(age and experience)? Is screen systemically constructed considering students(age and study experience), study contents, and study environment? Is it possible that students can control study process and speed? Study Contents Are the smart contents selected to help achievement of study objectives as core contents? Do study contents include new information and tendency? Are study contents non-repetetively and logically presented? Are study contents organized considering student' level(cognitive capability, etc)? Are study contents themselves error-free? Are glossary and spelling of study contents correct and error-free? Do study contents include prerequite contents and supplement-reinforcement contents? Do study contents consider study levels(easy, intermediate, and difficult, etc) appropriately? Are the amount of study contents appropriate for each class? Teaching-Learning Do study contents include appropriate Strategy teaching-learning strategy for online study? Do study contents include various motivation support strategy? Interactivity Do study contents include interaction between students and the contents? Do study contents include interaction between students and other students? Do study contents include interaction between students and teachers? Do study contents include interaction between students and external specialist(including parents)? Evaluation Are evaluation contents organized to help students complete the course? Are various evaluation methods used? Feedback Are answers to student's questions and evaluation results provided to students? Support System Are support contents that are helpful for student's study progress included? Are various support methods(checklist, worksheet, etc) applied to students? Are study contents constructed to be reusable Reusability depending on study objectives, student's level, and study environment? Sharing & Is metadata provided for sharing study Distribution contents? Are study contents accessible anytime anywhere using wired/wireless internet? Accessibility Is there any access restriction depending on student's mental or physical disabilities? Are any communication tools(email, BBS, etc) provided for student's questions and answers to study contents? Are study contents and process restuructable Restructuring depending on study objectives and study environments? Do the contents include ethical bias such as Ethicality religion, region, political belief and violent expressions? Copyright Do all writings follow the copyright laws and regulation? Table 7. Verification of Reliability for Each Standard Area Standard No. Teaching-Learning Requirement 1 .965 .674 Analysis 2 .966 Instructional 3 .965 .782 Design 4 .966 5 .966 6 .965 7 .966 Study Contents 8 .964 .895 9 .965 10 .965 11 .964 12 .965 13 .965 14 .964 15 .964 16 .963 Teaching-Learning 17 .963 .689 Strategy 18 .964 Interactivity 19 .964 .840 20 .964 21 .964 22 .965 Evaluation 23 .964 .650 24 .964 Feedback 25 .964 -- Support 26 .964 .797 System 27 .964 Reusability 28 .964 -- Sharing & 29 .964 -- Distribution 30 .964 Accessibility 31 .964 .822 32 .964 Restructuring 33 .964 -- Ethicality 34 .964 -- Copyright 35 .964 -- Area Standard No. Education Support Requirement 1 .956 .496 Analysis 2 .955 Instructional 3 .956 .820 Design 4 .956 5 .956 6 .956 7 .956 Study Contents 8 .956 .862 9 .955 10 .956 11 .955 12 .956 13 .955 14 .954 15 .954 16 .954 Teaching-Learning 17 .954 .657 Strategy 18 .954 Interactivity 19 .954 .891 20 .955 21 .954 22 .955 Evaluation 23 .955 .553 24 .955 Feedback 25 .954 -- Support 26 .955 .633 System 27 .955 Reusability 28 .954 -- Sharing & 29 .954 -- Distribution 30 .955 Accessibility 31 .955 .655 32 .955 Restructuring 33 .955 -- Ethicality 34 .955 -- Copyright 35 -- Table 8. Evalution of Importance for Each Standard Standard Teaching-Learning A B C D E 1 8.9% 26.7% 64.4% 2 2.2% 13.3% 37.8% 46.7% 3 11.1% 31.1% 57.8% 4 6.7% 37.8% 55.6% 5 2.2% 8.9% 20.0% 68.9% 6 8.9% 33.3% 57.8% 7 4.4% 13.3% 46.7% 35.6% 8 15.6% 17.8% 66.7% 9 2.2% 31.1% 31.1% 35.6% 10 4.4% 20.0% 46.7% 28.9% 11 11.1% 22.2% 66.7% 12 13.3% 15.6% 71.1% 13 4.4% 13.3% 35.6% 46.7% 14 8.9% 31.1% 33.3% 26.7% 15 6.7% 28.9% 28.9% 35.6% 16 2.2% 28.9% 31.1% 37.8% 17 4.4% 15.6% 31.1% 48.9% 18 2.2% 20.0% 35.6% 42.2% 19 6.7% 26.7% 31.1% 35.6% 20 4.4 % 15.6% 11.1% 35.6% 33.3% 21 8.9% 24.4% 28.9% 37.8% 22 2.2 % 20.0% 37.8% 13.3% 26.7% 23 2.2% 11.1% 24.4% 62.2% 24 4.4% 31.1% 31.1% 33.3% 25 4.4% 11.1% 21.1% 53.3% 26 33.3% 28.9% 37.8% 27 11.1% 26.7% 31.1% 31.1% 28 2.2% 22.2% 40.0% 35.6% 29 11.1% 31.1% 24.4% 33.3% 30 4.4% 17.8% 31.1% 46.7% 31 6.7% 17.8% 35.6% 40.0% 32 2.2 % 4.4% 24.4% 33.3% 35.6% 33 4.4% 26.7% 40.0% 28.9% 34 8.9% 20.0% 17.8% 53.3% 35 17.8% 22.2% 60.0% Standard Education Support A B C D E 1 4.4% 4.4% 37.8% 53.3% 2 2.2% 15.6% 42.2% 40.0% 3 17.8% 33.3% 48.9% 4 15.6% 42.2% 42.2% 5 8.9% 40.0% 51.1% 6 13.3% 40.0% 46.7% 7 2.2% 20.0% 46.7% 31.1% 8 24.4% 22.2% 53.3% 9 4.4% 20.0% 28.9% 46.7% 10 8.9% 26.7% 42.2% 46.7% 11 17.8% 35.6% 46.7% 12 11.1% 20.0% 68.9% 13 2.2% 22.2% 31.1% 44.4% 14 2.2% 6.7% 35.6% 24.4% 31.1% 15 8.9% 26.7% 26.7% 37.8% 16 4.4% 31.1% 33.3% 31.1% 17 2.2% 20.0% 33.3% 44.4% 18 6.7% 26.7% 31.1% 35.6% 19 2.2% 8.9% 24.4% 31.1% 33.3% 20 4.4% 15.6% 17.8% 33.3% 28.9% 21 4.4% 35.6% 24.4% 35.6% 22 4.4% 22.2% 28.9% 17.8% 26.7% 23 2.2% 17.8% 31.1% 48.9% 24 8.9% 22.2% 35.6% 33.3% 25 2.2% 17.8% 33.3% 46.7% 26 2.2% 22.2% 40.0% 35.6% 27 6.7% 24.4% 33.3% 35.6% 28 4.4% 20.0% 37.8% 37.8% 29 6.7% 40.0% 11.1% 42.2% 30 6.7% 11.1% 31.1% 51.1% 31 4.4% 17.8% 26.7% 51.1% 32 11.1% 20.0% 31.1% 37.8% 33 8.9% 20.0% 40.0% 31.1% 34 4.4% 20.0% 24.4% 51.1% 35 2.2% 15.6% 24.4% 57.8% Table 9. Further Evaluation of Importance for Each Standard Standard Teaching-Learning Average Standard Deviation Ranking 1 4.56 .659 2 2 4.29 .787 12 3 4.47 .694 8 4 4.49 .626 6 5 4.56 .755 2 6 4.49 .661 6 7 4.13 .815 18 8 4.51 .757 5 9 4.00 .879 23 10 4.00 .826 23 11 4.56 .693 2 12 4.58 .723 1 13 4.24 .857 13 14 3.78 .951 33 15 3.93 .963 28 16 4.04 .878 21 17 4.24 .883 13 18 4.18 .834 16 19 3.96 .952 25 20 3.78 1.204 33 21 3.96 .999 25 22 3.42 1.158 35 23 4.47 .786 8 24 3.93 .915 28 25 4.33 .853 11 26 4.04 .852 21 27 3.82 1.007 31 28 4.09 .821 19 29 3.80 1.036 32 30 4.20 .894 15 31 4.09 .925 19 32 3.96 .999 25 33 3.93 .863 28 34 4.16 1.043 17 35 4.42 .783 10 Standard Education Support Average Standard Deviation Ranking 1 4.40 .780 3 2 4.18 .860 16 3 4.31 .763 6 4 4.27 .720 9 5 4.42 .657 2 6 4.33 .707 5 7 4.04 .852 21 8 4.29 .843 7 9 4.18 .912 16 10 3.78 .902 32 11 4.29 .757 7 12 4.58 .690 1 13 4.18 .860 16 14 3.76 1.048 33 15 3.93 1.009 25 16 3.91 .900 28 17 4.20 .842 15 18 3.96 .952 23 19 3.84 1.065 31 20 3.67 1.187 34 21 3.91 .949 28 22 3.40 1.232 35 23 4.24 .908 11 24 3.93 .963 25 25 4.24 .830 11 26 4.07 .889 20 27 3.98 .941 22 28 4.09 .874 19 29 3.89 1.049 30 30 4.27 .915 9 31 4.24 .908 11 32 3.96 1.021 23 33 3.93 .939 25 34 4.22 .927 14 35 4.38 .834 4 Table 10. The Excluded Quality Standard Standard Teaching-Learning Average Standard Ranking Deviation 22 3.42 .1.158 35 Standard Education Support Average Standard Ranking Deviation 22 3.40 1.232 35 Table 11. Evaluation of Importance for Each Area Area Teaching-Learning Average Standard Ranking Deviation Requirement Analysis 4.42 .630 2 Instructional 4.43 .522 1 Design Study Contents 4.18 .620 7 Teaching-Learning 4.21 .750 5 Strategy Interactivity 3.78 .891 14 Evaluation 4.20 .734 6 Feedback 4.33 .853 4 Support System 3.93 .850 11 Reusability 4.09 .821 9 Sharing & 3.80 1.036 13 Distribution Accessibility 4.08 .808 10 Restructuring 3.93 .863 11 Ethicality 4.16 1.043 8 Copyright 4.42 .783 2 Area Education Support Average Standard Ranking Deviation Requirement Analysis 4.29 .670 2 Instructional 4.28 .566 3 Design Study Contents 4.10 .611 7 Teaching-Learning 4.08 .776 10 Strategy Interactivity 3.71 .967 14 Evaluation 4.09 .778 8 Feedback 4.24 .830 4 Support System 4.02 .783 11 Reusability 4.09 .874 8 Sharing & 3.89 .1.049 13 Distribution Accessibility 4.16 .730 6 Restructuring 3.93 .939 12 Ethicality 4.22 .927 5 Copyright 4.38 .834 1 Table 12. The Final Quality Standards for Educational Smart Contents Area Sub-area Serial Standards No. Requirement Study Contents 1 Does study activity enable Analysis Analysis study objective to be achieved? Is it possible that study Study 2 program and application Environment software do not depend Analysis on platform and operating system, and selective execution is provided? Instructional Clarity 3 Is study objective Design presented clearly? Leveled Is the leveled learning Learning 4 possible for various study contents and methods depending student's capability? Study Material 5 Are appropriate study materials(text data, graphic data, sound data, video data, etc) used considering student's level(age and experience)? Screen 6 Is screen systemically Construction constructed considering & Arrangement students(age and study experience), study contents, and study environment? Interface & 7 Is it possible that Progress students can control study process and speed? Study Study Contents 8 Are the smart contents Contents Selection selected to help achievement of study objectives as core contents? Study Contents 9 Do study contents include new information Selection and tendency? Selection of 10 Are study contents non- Study repetetively and Contents logically presented? Organization Are study contents of Study 11 organized considering Contents student' level(cognitive capability, etc)? Organization 12 Are study contents of Study themselves error-free? Contents Organization 13 Are glossary and spelling of Study of study contents Contents correct and error-free? Organization 14 Do study contents include of Study prerequite contents and Contents supplement-reinforcement contents? Study Level of 15 Do study contents consider Difficulty study levels(easy, intermediate, and difficult, etc) appropriately? The Amount 16 Are the amount of study of Study contents appropriate for each class? Teaching- Selection of 17 Do study contents include Learning Teaching- appropriate teaching- Strategy Learning learning strategy for Strategy online study? Motivation- 18 Do study contents include Support various motivation Strategy support strategy? Interactivity Student- 19 Do study contents include Contents interaction between Interaction students and the contents? Student- 20 Do study contents include Student interaction between Interaction students and other students? Student- 21 Do study contents include Teacher interaction between Interaction students and teachers? Evaluation Evaluation 22 Are evaluation contents Contents organized to help students complete the course? Evaluation 23 Are various evaluation Methods methods used? Feedback Feedback 24 Are answers to student's questions and evaluation results provided to students? Support Selection of 25 Are support contents that System Support are helpful for Contents student's study progress included? Selection of 26 Are various support Support methods (checklist, Methods worksheet, etc) applied to students? Reusability Reusability 27 Are study contents constructed to be reusable depending on study objectives, student's level, and study environment? Sharing & Sharing & 28 Is metadata provided for Distribution Distribution sharing study contents? Contents 29 Are study contents Accessibility Accessibility accessible anytime anywhere using wired/ wireless internet? Access 30 Is there any access Restriction restriction depending on student's mental or physical disabilities? Communication 31 Are any communication Tool tools(email, BBS, etc) provided for student's questions and answers to study contents? Restructuring Restructuring 32 Are study contents and process restructurable depending on study objectives and study environments? Ethical Norm 33 Do the contents include Ethicality ethical bias such as religion, region, political belief, and violent expressions? Copyright Copyright 34 Do all writings follow Application the copyright laws and regulation?
|Printer friendly Cite/link Email Feedback|
|Author:||Jun, Woochun; Hong, Suk-ki|
|Publication:||KSII Transactions on Internet and Information Systems|
|Date:||Jun 1, 2014|
|Previous Article:||Self-adaptive testing to determine sample size for flash memory solutions.|
|Next Article:||A subjectivity study on the promotion of Korean smart TV industry through Q methodology.|