Printer Friendly

ARTIFACT EVALUATION IN INFORMATION SYSTEM DESIGN SCIENCE RESEARCH - SOCIAL CONSTRUCTIVISM ENVIRONMENT.

Byline: Farib Ataie, Asadullah Shah and Mior Nasir Mior Nazir

ABSTRACT: Design Science Research Methodology in Information Systems (IS) relates to the design, development, implementation and evaluation of artifacts solves real life problems. The iterative cycles of Design -Evaluation are an important requirement in this methodology. This paper describes the creation and iterative cycles of Design -Evaluation of an innovative Web 2.0 based collaborative learning based on social constructivism theory and Crus and Carvalho framework in Islamic education. The various qualitative and quantitative research methods are employed at various stages of study. Artifact Evaluation in Information Systems and results are described. Findings show that the developed artifact could be as a good model for web 2.0 based collaborative learning, however, students should be more engaged for achieving learning goals and undoubtedly instructors' attitudes could play the main role in encouragement of students and honoring their passion and interest.

Keywords: Artifact Evaluation, Information System, Design Science Research, Social Constructivism Environment

1. INTRODUCTION

During the past few years, the world has witnessed an impressive growth in information communication technologies (ICTs). The potential of ICT in information system to solve the ill-structured or "wicked" problems have attracted many systems' attention. Such problems are characterized by a large degree of uncertainty that Introne et al [1] defined, "problems for which no single computational formulation of the problem is sufficient, in which different stakeholders do not even agree on what the problem really is, and for which there are no right or wrong answers, only answers that are better or worse from different points of view". A design science research methodology offers a unique problem-solving approach that enables researchers to connect numerous design theories with alternative solutions to the wicked problems.

Today, the IS field needs more Design Science Research Goes [2]; the education system needs more developmental and practical research methodology with iterative cycles of design-evaluation in using information technology for solving wicked problems of learning process. This practicum in research has been the key issues in information systems and education over the years, and neglecting it in research could have serious consequences. Ahmad, et. al [3] study in IIUM emphasized on the need to see the real example of integrating ICT to learning process. Thus, this study has focused on the Hevner and Chatterjee's study [4] that provide an abundance of evidence supporting Design science research methodology's ability in Information system (IS) for providing the real example of integrating ICT to learning process. This study's methodology includes three iterative cycles of design-evaluation that emphasizes on the design and evaluation of IT artifact with high priority on relevance in the application domain.

The main objective of this research is to conduct the design-evaluation of an innovative web 2.0 based collaborative learning artifact in IIUM for improving the existing solution of problems (integration of ICT to collaborative learning) and enhancing the opportunities. This study wants to address the complex and relevant learning environment, social negotiation, multiple perspectives, and multiple modes of learning, ownership in learning, construction, and self-awareness of knowledge with iterative cycles of design-evaluation. The goals of instruction based on social constructivist theory in this study's developed artifact are religious awareness, self-regulation, and use of knowledge, critical thinking, motivation and engagement.

It seems necessary to explain that based on Surah Ali 'Imran [verse 3: 200] of the Quran a collaboration could lead to consciousness, self-controlling, endurance and empathy, which in Hashim, Hassan's [5] study is mentioned as religious awareness.This study' framework based on cycles and steps of DRM could be used as a good model for artifact evaluation in Information System.

2. DESIGN SCIENCE RESEARCH METHODOLOGY

Design science research is a set of analytical techniques and perspectives for performing research in Information Systems. Researchers with DSR try to understand, explain, and improve information systems [4]. The field of information systems has recognized the importance of design science as an opportunity to increase relevance [6,7]. Thus, DSR combines a focus on the design and development of Information Technology (IT) artifact with high priority on relevance in the application domain. Zhang, Scialdone and Ku [8] in their literature review define IT artifact as "an object, or a bundle thereof, intentionally engineered to benefit certain people with certain purposes and goals in certain contexts" (p...3). We can distinguish product artifacts from process artifacts. "Product artifacts are technologies such as tools, diagrams, software, etc. that people use to accomplish some tasks. Process artifacts are methods, procedures, etc. that guide people what to do to accomplish some task.

Some artifacts are in some sense "purely" (or nearly purely) technical, in that they do not require human use once instantiated. Socio-technical artifacts are ones with which humans must interact to provide their utility" [9].

Design science research is fundamentally a problem-solving methodology with emphasis on collaboration for developing socio-technical artifacts. The theories support the artifact design and the design process to add knowledge to the knowledge base. Design science research seeks to create innovations that through which the analysis, design, implementation, and use of information systems can be effectively and efficiently accomplished to solve real life problems. Evaluation of design artifacts is a central part of Design Science Research (DSR) [10, 11].

3. A FRAMEWORK FOR ARTIFACT EVALUATION IN IS, DSR-SOCIAL CONSTRUCTIVISM ENVIRONMENT

This study's Framework is based on Hevner et al.'s [11] study in DSRM that appeared in Management Information Systems Quarterly (MISQ) in March 2004.This Framework for design-evaluation of web 2.0 based collaborative learning consists of the six steps of the DSR process [12] and Design science research cycles [13]. The Research Questions 1 for requirement analysis and Research Question 2 for design-evaluation are defined in this study. And also hypotheses are considered for testing the users' perception about developing artifact based on Technology Acceptance Model or TAM [14] proposed the TAM can be used to evaluate a DSR artifact. Table 2 indicates the relationship between this study's research questions and hypotheses with the DSRM steps and cycles. Figure 4 maps them to the appropriate research cycle and steps. Also table 2 shows the different method of data collection for evaluation of artifact. This study' framework could be as a good model for artifact evaluation in the Information System.

Six design science research methodology steps and activities

Peffers et al. [12] mentioned in their study the six steps of DSRM (Table 1) that begin with identifying the problem and justifying the value of a solution. In this study, investigating the collaborative learning problems in IIUM begins with the first step of this study's methodology. A well-defined problem can be used to develop an effective artifact and atomize the problem and justifying the value motivates the researcher to pursue the solution and accept the results. Secondly, the objectives are defined by the researcher. These objectives are inferred from the problem definition and knowledge of what is possible and feasible. The integration of ICT into collaborative learning is considered as the best solution in this study. The third step of DSRM includes, design and development. This study designed and developed an innovative web 2.0 based collaborative learning artifact using Facebook page and groups.

In the fourth step of the methodology we must demonstrate how to use the artifact to solve the problems. The fifth step of the methodology includes evaluation and the feasibility of the solution compared to the original goal [11]. The detail explanation of three iterative cycles of design -evaluation is discussed in section 3.4 of this paper.

In the final phase, communication, the contributions of the study are shared with the public and different publications. This is important because only after the new knowledge is out can others utilize it as well [4].

Table 1: Design Science Research Process Peffers et al [12]

1)###Problem identification and motivation;

2)###Definition of solution objectives;

3)###Design and development;

4)###Demonstration;

5)###Evaluation;

6)###Communication

Design Science Research Cycles

Hevner and Chatterjee [13], presented design science research with three cycles (Figure 1). The Relevance Cycle bridges the environment of the research project with the design science activities and provides the requirements for the research. In addition, this cycle in DSRM defines acceptance criteria for the ultimate evaluation of the research results. The Rigor Cycle connects the design science activities with the knowledge base of scientific foundations, experience, and expertise. The central Design Cycle iterates between the core activities of building, designing the artifact and evaluating.

Research Questions 1 (Requirement analysis, relevance cycle)

The requirement analysis of the project in this paper is conducted by Research Questions 1. The Research Questions 1 in relevance cycle is about investigating collaborative learning in IIUM and finding its opportunities and problems. The literature review found integration of ICT to this method of instruction as the best solution in this part of the study and definition of solution objectives. Literature review, participant observation, researcher experience, focus group and interview are used for collecting data. This research question analyzes the needs and provides the requirements for design cycle. The findings show the need for innovation in collaborative learning methods and developing the format for collaborative learning activities and ethics based on verses of Quran.

Research Questions 2 (Design-Evaluation), relevance, design and rigor cycle

Research Question 2 includes the design, development, demonstration and evaluation steps of DSRM. This Research Question wants after designing and developing, to demonstrate the artifact to relative audience and find how well the artifact supports a solution to the problem.

Research Question 2 is related to all cycles of DSRM; design cycle that iterates between the design and evaluation, rigor cycle that provides scientific foundation and past knowledge to the research project and the relevance cycle that bridges the DSRM to the environment and allows developing artifact be demonstrated to the application domain.

Formative evaluation of artifact and design process is conducted in the design cycle of DSRM with three loops of design-evaluation (Figure 5). According Venable et al. [12] this study planned for internal design-evaluation in naturalistic situation with a group of students and experts. This evaluation is conducted in two parts: ex ante and ex post evaluations. Ex ante evaluation is related to prior to artifact construction, such as a design or model and Ex post evaluation related to the evaluation of after artifact construction and instantiated artifact. Ex post evaluation helps to solve one or more instances of the problem before diffusing developing artifact in the environment and communication with the public.

Design-evaluation loop 1

In the first loop of design-evaluation, the collaborative learning as construct was defined based on social constructivism theory Driscoll [15] which emphasizes on learning together and constructing knowledge with negotiation and exposing to the diverse viewpoints. This definition of collaborative learning includes three items:

1- Leaning together and teamwork with the single shared goal

2- Social negotiation

3- Empathy

This study chose Facebook as Web 2.0 artifacts for collaborative learning based on review of literatures that show the great potential of Facebook for collaborative learning. Face book's basic functions such as, Chat, photos, videos, wall, groups, pages, comment area and home page are easy to use, accessible and visually well-structured [16]. A group can start interacting immediately after it has been created and receive notification for all massages. Mobile push notifications help groups' collaboration because they appear when you're not actively using Facebook.

The collaborative learning's ethics according requirement analysis in research question 1 was designed in this project based on verses of the Quran (Figure 3).

The collaborative learning activities using web 2.0 artifacts based on the definition of items in constructivist theory, verses of Quran and researcher experience was designed and developed that is shown in figure 4.

Evaluation of artifact (Uninstantiation or ex ante evaluation) Expert review

The utility, quality, and efficacy of developing artifact based on the third guideline of DSRM should rigorously be evaluated [4]. This evaluation must be conducted in the two parts: evaluation of design theories and principles (ex ante: un instantiation) and evaluation through an instantiation of a designed artifact to establish its utility and efficacy [9]

In this part of study after design and developing, (2) experts from the IIUM Education Faculty, 2 experts from IIUMKICT and 2 experts in the Quran from (WAMY: The World Assembly of Muslim Youth) training center and IIUM (KIRKHS) reviewed and confirmed the developing artifact.

Design-evaluation loop 2, instantiation, focus group interview, participant observation

This study designed and conducted some collaboration and negotiation in the comment area of Facebook group in developing artifact. This study planned instantiation and pilot user testing based on Etelaaho. The focus group that consists of 3 IIUM experienced PhD students among informal learners and researcher as learning facilitator participated in the negotiation in the comment area of Facebook in developing artifact. After some discussions the researcher found that there is a need for more explanation and briefing. The results show the great impact of training on students' intention to use it. This finding of study is in line with Mun and Hwang's [17] study about the role of self-efficacy in enjoyment of web 2.0 users in the technology acceptance model. In addition, the participation and engagement of learning facilitator in a negotiation has a positive impact on motivation of students and learners.

This study concluded that for more engagement of students in collaborative learning and negotiation, we need more interaction with instructors.

Design-evaluation loop 3, focus group interview, participant observation,

This study, for enhancing student-instructor interaction planned to demonstrate developing artifact, not only outside of class, but also inside of class. This study after the pedagogical design of an artifact based on social constructivist theory demonstrated the developing artifact to IIUM KICT students for two courses. We incorporated the use of developing artifact to the course outline. Based on findings in the second loop of DSRM the researcher as learning facilitator participated in class,conduct briefing and added students to create Facebook groups and page in developing artifact. Findings of participant observation and a focus group interview showed a good enhancement in the learning process and engagement of students; however, the attitude of the instructors in integrating ICT to learning process plays the main role in encouragement of students for more engagement.

Hypotheses

The testing hypothesis shows the great impact of users' perception about the usefulness and ease of use's artifact and their self-efficacy on intention to use it. In the sixth step of DSRM, we plan to generalize the use of this study developing artifact bed on Hevner and Chatterjee' study [4] and communicate with the public and relevant audience. This study introduced the developed artifact to some experts and organizations. The expert in the social media confirmed this study developed artifact; some organizations used the developed artifact in their web 2.0 pages and researcher was chosen as the administrators of an organization's group. The usability of the artifact and the users' perception are explored in this part of study.

Research Questions and Hypotheses mapped to three Design Science Research Cycles and Six Steps

This study's framework for design-evaluation of IT-based artifact consists of Research Questions and Hypotheses mapped to three Design Science Research Cycles and Six Steps DSRM steps and activities (Figure 4).

The relevance cycle bridges DSR with the environment.

Environment includes the application domain, People, organizational and technical system. Knowledge base, which includes scientific foundations and theories, experience, expertise and Meta-Artifacts (Design Products and Design Processes) supports the design process through rigor cycle. The Research Question 1 includes two steps of DSRM, identification of the problems and definition of solution and maps with relevance cycle. This research Question provides the requirement for research along the relevance cycle. The Research Question 2 includes design, demonstration and evaluation steps of DSRM. This research question maps with design cycle and with rigor cycle are connected to the knowledge base. This research question in demonstration step of DSRM relates to environment along relevance cycle. In addition, the hypotheses in this study map with relevance cycle and two steps of DSRM, demonstration and communication.

This means the artifact be demonstrated to the relative audience and the hypotheses be tested users' perception about the effectiveness of the artifact. The kind of artifacts plays the main role in their evaluation [9].

Table 2: Research Questions and hypotheses, DSR cycles and steps and Evaluation methods

Research###Research###Research###Hypotheses(HYP)

questions###question###question###(Testing users'

###1(RQ1)###2(RQ2)###perception)

###(###(Design of

###Requireme###web 2 based

###nt###collaborative

###Analysis)###learning

###artifact and

###formative

###evaluation)

Design###Proble###Design###Communication

Science###m###and devel###Demonstration

Research###identif###opment

methodology###icatio###Demon

steps###n and###stration

###Evaluat

Peffers et al.,###motiv###ion

[12]###ation

###Defini

###tion of

###soluti

###on

###object

###ives

Design###Relevance###Design,###Relevance Cycle

Science###Cycle###relevance

Research###and Rigor

methodology###cycle

cycles

(Hevner et

al., [4]

Evaluation###First###Third loop:TAM

methods###loop:Expert###(Generalisation)

###review

###(Un###Technology

###instantiation)###Acceptance Model

###(Davis et al. 1989)

###Second loop

###:Focus group

###Interview

###and

###participant

###observation

###(Instantiatio

###n)

4. CONCLUSION

This paper has described an effort to design-evaluation of an innovative web 2.0 based collaborative learning artifacts under the guideline of DSRM in IIUM. This artifact was developed and designed based on the social constructivist theory and verses of the Quran, and integrated inside and outside of class. Evaluation is a very significant issue in IS Design Science Research. The information technology (IT) artifact evaluation model in this study is based on three design science research cycles and six steps of the DSR process. The first Research Question in this study provides requirements for design and evaluation of an innovative web 2.0 based collaborative learning artifact and the second Research Question wants after designing to find how well the artifact supports a solution to the problem. Formative evaluation with three iterative cycles of design-evaluation allowed the researchers to measure effectiveness of artifact in achieving the assigned goals.

This study shows students and learners could practice collaborative learning and regulate their knowledge in this platform and enhance their motivation, engagement, performance, self -regulation, and critical thinking as long as instructors integrate their appropriate instructional strategies to developing artifact and enhance their interaction with students. Further enhancement of the finding is supported from the interviews and the researcher's experience as a learning facilitator, where challenges and barriers are discussed about lecturers' pedagogical strategies and attitude.

5. REFERENCES

[1] Introne J, Laubacher R, Olson G, Malone T Solving wicked social problems with socio-computational systems. Kunstl Intell 27: 45-52 (2013).

[2] Goes, P. B. Editor's comments: design science research in top information systems journals. MIS quarterly, 38(1), iii-viii (2014).

[3] Ahmad, T. B. T., Basha, K. M., Marzuki, A. M., Hisham, N. A. and Sahari, M. Faculty's acceptance of computer based technology: Cross-validation of an extended model. Australasian Journal of Educational Technology, 26(2), 268 http://www.ascilite.org.au/ajet/ajet26/ahmad.html (2010).

[4] A. Hevner, S. Chatterjee, Design Research in Information Systems, Integrated Series 1 in Information Systems 22, DOI 10.1007/978-1-4419-5653-8_1 C _ Springer Science+Business Media, LLC (2010).

[5] Hashim, C. N., and Hassan, S. S. S. A Comparative Study on Cyber Ethics, Religious Awareness and Satisfaction in Using Facebook for Social Networking. Academic Journal of Interdisciplinary Studies, 2(3), 419 (2013).

[6] Venable, J.R..: A Framework for Design Science Research Activities. In: Proceedings of the 2006 Information Resource Management Association Conference Washington, DC, USA (2006).

[7] Venable, J., Pries-Heje, J., and Baskerville, R. FEDS: a framework for evaluation in design science research. European Journal of Information Systems, 25(1), 77-89 (2016).

[8] Zhang, P., Scialdone, M., and Ku, M. C. IT artifacts and the state of IS research (2011).

[9] Venable, John, Pries-Heje, Jan, and Baskerville, Richard. A Comprehensive Framework for Evaluation in Design Science Research. In K. Peffers, M. Rothenberger and B. Kuechler (Eds.), Design Science Research in Information Systems. Advances in Theory and Practice (Vol. 7286, pp. 423-438). Berlin / Heidelberg: Springer (2012).

[10] March, S. T., and Smith, G. F. Design and natural science research on information technology. Decision support systems, 15(4), 251-266 (1995).

[11] Hevner, A.R., March, S.T., Park, J., Ram, S.: Design Science In Information Systems Research. MIS Quarterly, vol. 28, pp. 75-105 (2004).

[12] Peffers, K., Tuunanen, T., Rothenberger, M. A., and Chatterjee, S. A design science research methodology for information systems research. Journal of management information systems, 24(3), 45-77 (2007).

[13] Hevner, A. R. A three cycle view of design science research. Scandinavian journal of information systems, 19(2), 4 (2007).

[14] Purao, S. And Storey, V. C. Evaluating the adoption potential of design science efforts: The case of APSARA. Decision Support Systems. 44. 369-381 (2008).

[15] M.P. Driscoll, "Constructivism, " in Psychology of Learning for Instruction, pp.384-407 (2005).

[16] LLORENS, Francesc; CAPDEFERRO, Neus "Facebook's Potential for Collaborative e-Learning" (2011). [online article]. Revista de Universidad y Sociedad del Conocimiento (RUSC). Vol. 8, No 2, pp. 197-210. UOC. [Accessed: dd/mm/yy]. http://rusc.uoc.edu/ojs/index.php/rusc/article/view/v 8n2-llorens-capdeferro/v8n2-llorenscapdeferro-eng ISSN 1698-580X

[17] Mun, Y. Yi, and Yujong Hwang. "Predicting the use of web-based information systems: self-efficacy, enjoyment, learning goal orientation, and the technology acceptance model." International journal of human-computer studies 59, no. 4: 431-449 (2003).
COPYRIGHT 2017 Asianet-Pakistan
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2017 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Publication:Science International
Article Type:Report
Date:Oct 31, 2017
Words:3881
Previous Article:ENHANCEMENT IN CORROSION RESISTANCE BY NANO-STRUCTURE Ni-Cr COATING FOR BOILERS APPLICATION.
Next Article:A STUDY ON VIBRATION SENSORS FOR WATER PIPELINE LEAKAGE APPLICATION.
Topics:

Terms of use | Privacy policy | Copyright © 2019 Farlex, Inc. | Feedback | For webmasters