Printer Friendly

Non-technological Learning Environments in a Technological World: Flipping Comes to the Aid.


Spurred by scientific advances of the last century and the accelerated rate of innovations (Froehlich & Messmann, 2017), technology has reached an unquestionably important status in our daily personal and professional lives. Indeed, we do live in a world permeated by an abundance of digital technologies. Universities, too, more and more become reflective about the challenges and opportunities that this transition into what can be called the "digital age" imparts. However, this digitization is not always reflected in the learning environments of institutions of higher education; often necessary technological resources are not made available for the students. This raises questions about the adequacy of the instructional outcomes; after all, the learning context should be related to the context of application, as suggested by especially the transfer of training literature (Baldwin & Ford, 1988; Froehlich & Gegenfurtner, Forthcoming).

Flipped or inverted learning as a "type of technology-enhanced pedagogy" (Lo, Lie, & Hew, 2018, p. 150) has recently received a lot of attention in diverse domains such as humanities (Kong, 2014), nursing (Njie-Carr et al., 2017), statistics (Boeve et al., 2017; Peterson, 2016), mathematics (Tawfik & Lilly, 2015), languages (Chuang, Weng, & Chen, 2018), social work (Holmes, Tracy, Painter, Oestreich, & Park, 2015), or education (Al-Zahrani, 2015). Here, flipped classroom describes the instructional strategy where content is delivered before the class is taught (Reidsema, Kavanagh, Hadgraft, & Smith, 2017; Talbert & Bergmann, 2017). This is often done via videos--which I also refer to as "flipped elements" in this text. The effectiveness of flipped classrooms has been studied with respect to a host of different outcomes such as critical thinking skills (Kong, 2014), course satisfaction (Peterson, 2016), self-regulation (Sun, Wu, & Lee, 2017), creative thinking (Al-Zahrani, 2015), or how this approach lends support to other teaching activities such as problem-based learning (Tawfik & Lilly, 2015). In sum, previous research suggests many positive outcomes stemming from flipped classroom elements.

Boeve et al. (2017) examined students' study behavior during a course using flipped instruction. However, contrary to many other findings in the literature, they did not find a substantial difference in students' study behavior when contrasting regular and flipped classes. They highlight that more understanding is needed, when the flipped classroom pedagogy may be most fruitfully applied. It is this gap I seek to slightly narrow with this article. Specifically, while the flipped classroom pedagogy itself is mostly interpreted to be technology-based, research on using this methodology specifically in learning environments that are lacking technological features is scarce. In other words, we do not know much about how different leaning environments in terms of technological resourcefulness and the efficiency of flipped classroom elements are related to each other. In the context of this article, I focus on rather basic technological features, such as the availability of computers for the students in an applied statistics classroom.

I propose that flipped elements may function as a buffering mechanism in such a way that any differences between learning environments rich in technological resources and learning environments poor in technological resources are decreased. This, in turn, may also curb differences in terms of student outcomes such as student learning or student performance. In more practical terms, this means that flipped elements may come to the aid of learning environments that do not possess technical resources--which, in the case of the empirical demonstration below, means the hardware and software to perform applied statistics. The rationale for this proposition is that flipped elements transfer the demand for technology (i.e., having access to a computer with all the needed software for a specific course) from the teaching environment (e.g. a lab or a seminar room) to the student (e.g. privately owned computers or shared facilities at the university). If this proposition holds true, this has important implications for how universities (especially those that could be considered poorly equipped in terms of technology) could master the transition into the digital age in a sustainable and financially viable manner.

In order to generate primary data to test the proposition outlined above, I executed a design-based research project (Anderson & Shattuck, 2012; Euler & Sloane, 2014). Following the nature of design-based research projects (which will be described in more detail in the Material and Methods section), this investigation is embedded in my own teaching practice. The contribution that this article aims to make, therefore, cannot be a robust and generalizable test of this proposition. However, I do aim to test the general viability of the idea so that further research can investigate the proposition in greater detail, using more robust methods, and in a more informed and efficient way. Also, I do discuss the implications for higher education teachers and the universities they are teaching in.

The rest of this paper is structured as follows. First, I describe the methods and all the data sources tapped in the empirical part of this study. I briefly explain the nature of design-based research projects as well as the background in which this particular project is embedded in. Next, I provide information about the intervention that was implemented and the data sources that were tapped in order to measure the effect of the intervention. I briefly present the results and interpret and discuss them in light of universities and the opportunities and challenges the digital age brings for them.


In this section, I define and present the process of design-based research and provide the necessary background information about the project. Also, I describe the flipped classroom intervention that is in the focus of this article. Last, I explain how evaluative data was collected and analyzed.

2.1 Overview of the method

In the following pages, I report a design-based research project (Choi & Lee, 2008) to provide some preliminary empirical evidence for the previously presented proposition and the rationale behind it. Design-based research is often executed as a series of three major steps (Anderson & Shattuck, 2012): First, the research is situated in a real teaching context, where a potential point for improvement was identified. Second, within this context, an intervention is implemented that is estimated to be powerful enough to have a notable effect. Third, qualitative and quantitative data are purposefully integrated (Johnson & Onwuegbuzie, 2004; Schoonenboom, Johnson, & Froehlich, 2018) to assess the intervention. This then is usually repeated so that the proposed intervention can be refined iteratively (see Figure 1). In this particular study, we deviate from the latter, as I do not make a comparison over time (e.g. having pre- and post-intervention evaluations of one course) but over courses that were taught in parallel. Specifically, in this particular design-based research project, the overall research strategy is one of making comparisons between two courses in very different contexts and learning environments, which are described in the next section.

2.2 Background

I have chosen two courses as a basis for comparison. The chosen courses--which I from now on will refer to as "A" and "B"--lend themselves for this comparison as they are homogeneous in defining features of the course; for instance, they share the same goals (prepare students to execute a quantitative study on their own for their thesis work, decrease statistic anxiety, increase statistical self-efficacy) and the intended audience (first year Bachelor students). They are also similar in terms of the broader institutional context: in both cases, public universities located in Austria.

However, the courses are very different when it comes to the actual learning environment. In setting A, I teach the course to 25 students in a computer lab; every student has access to appropriate hardware and also commercial statistics software (also outside the course). In setting B, I teach 150-160 students in an auditorium without any technical equipment (except for a projector) that does not provide enough space for students to work on their own laptops. Table 1 gives an overview of the differences.

This project was originally part of a larger data collection effort that comprised more settings, including some that did not use flipped classroom elements. However, given that the differences in outcomes is not the goal of this particular paper and since these differences are already very well documented in the literature, the respective courses are ignored to increase clarity of presentation.

2.3 Intervention

The intervention in this design-based research study is the introduction of flipped-classroom elements (videos in the format of "slidecasts" about statistical theory--e.g. the central limit theorem or confidence intervals--and "screencasts" for more application-oriented videos--e.g. how to perform a specific statistical test and how to interpret it). The videos were produced by myself specifically for the courses to be studied. The total duration is about five hours, which amounts to roughly 25% of the total teaching time in the classroom. I deem this high percentage vital in order to make the intervention an important part of the courses, which increases the likelihood that there is a measurable impact.

The intervention was planned and executed in the same manner in both courses. Specifically, each flipped element was integrated in a teaching strategy of three steps:

1. Before each lecture, the students received videos to watch that introduced the topic of the lecture theoretically and showcased an exercise directly in a statistical software package. The students were not required to have any prior knowledge about the content being taught.

2. Then, the students were required to solve an exercise of similar difficulty using the same statistical software package. In setting A, the courses were given as extra resources with the explicit recommendation to watch them before each lecture in the classroom. In setting B, viewing the videos before the lecture was mandatory and tested using a graded quiz (cf Taylor, 2015). While these different approaches are interesting in their own right, they are not the focus of this particular paper.

3. During the lecture in class, I answered any questions on the theoretical video and showed a step-by-step solution to the assignment(s). Afterwards, a more complex exercise about the same topic was done directly in class. While the students were able to do the exercise directly in class in the computer room, this was not feasible in setting B, where this part was less interactive and rather teacher-centered instruction.

2.4 Instruments

I compare the courses based on several both quantitative and qualitative criteria:

In terms of quantitative criteria, I draw from three major sources: First, a quiz that was a formal part of both courses. In both settings, the quiz served as a preparatory exam before the real exam at the end of the course. This preparatory exam, however, was taken quite seriously and completed by all students. Second, I disseminated surveys that contained psychographic questions about statistics anxiety (Onwuegbuzie & Wilson, 2000) (23 items, sample item: "How anxious do you feel when you need to interpret a statistic in a journal article?"; answers from 1 = "no anxiety" to 5 = "strong feeling of anxiety") and statistical self-efficacy (Finney & Schraw, 2003; Larwin, 2014) (seven items, sample item: "How confident do you feel in selecting an appropriate statistical test for a given research question?"; answers from 1 = "not confident at all" to 5 = "totally confident"), as well as standardized course- and teacher-evaluation scales. The questionnaire that contained the psychographic information as well as the standardized evaluation was completed by 23 and 22 students in settings A and B, respectively. Third, the quantitative parts of the respective course evaluations are used.

In terms of qualitative criteria, I rely on field notes and reflection memos (Saldana, 2009) made throughout the whole teaching process as well as the information students provided at various evaluation points throughout and at the end of the course. Additionally, both courses applied for teaching awards at their respective institutions and, hence, received thorough review by a jury of senior higher education teachers.

2.5 Analytical strategy

For the analysis, it needs to be acknowledged that all the data collected describes a single case. The analytical aim, therefore, cannot be to deliver generalizable results. Instead, I test whether preliminary evidence in support of the proposition can be generated. Such first evidence may form an important basis for more generalizable (but also more costly) research (cf. Froehlich, Liu, & Van der Heijden, 2018).

Each data source presented can hardly be considered robust evidence on its own. However, the multitude of different perspectives associated with the different data sources allows many opportunities for triangulating the evidence and to form an overall conclusion (cf. Jick, 1979). We interpret each of the following comparisons as in support of the proposition that the introduction of flipped elements can help level the playing field between learning environments rich in technological resources and those not so, if the absence of negative differences between setting A and B is indicated. For comparison of quantitative data, t-tests are used.


In this section, I present the results of the comparison of all data collected.

Statistical self-efficacy was, on average, rated to be quite low in both courses, but the difference between setting A (M = 2.4, SD = 0.7) and setting B (M = 3.2, SD = 1.0) was statistically significant (T = -3.1, p [less than or equal to] 0.01). This indicates some support to the proposition (especially because the setting rather poorly equipped scored higher regarding this measure).

For statistic anxiety, the means of setting A (M = 2.6, SD = 0.7) and setting B (M = 2.2, SD = 0.8) do not show any statistically significant differences between the learning environments. This lends weak support to the proposition.

As concerns the evaluative scales of teaching performance (A: M = 4.5, SD = 1.1; B: M = 5.4, SD = 1.3) and the overall course quality (A: M = 4.8, SD = 1.0; B: M= 5.6, SD = 1.3), statistically significant differences (p [less than or equal to] 0.05) exist in support of the proposition (again, especially because the setting rather poorly equipped scored higher regarding these measures).

The qualitative data are somewhat more difficult to weigh against each other given the contextual differences, but they do not suggest a stark difference in any direction between the two courses. Students in both courses highlighted the videos as the most importance resource for preparing for each class and the exam. The independent juries of the applied for teaching awards, too, gave very positive evaluations of both courses. The course in setting B won a university-internal teaching award, the course in setting A was nominated for a nation-wide award by the institution. Both applications were highly competitive processes.

Table 2 depicts all these sources of evidence together with the results and their interpretation. As stipulated before, each single base of evidence can hardly be considered robust evidence in terms of testing the proposition. However, it is notable that all the results point into the same direction, which does give some confidence in stating that the proposition is supported by the mixed data presented.


In this article, I argue that flipped-classroom elements (Reidsema et al., 2017; Talbert & Bergmann, 2017) may help higher education teachers in learning environments poor in technological resources to deliver technology-based courses. Specifically, I proposed that flipped elements may help to curb negative student outcomes that are a direct result of learning environments poor in technological resources. As a rationale for this proposition I suggested that flipped elements transfer the demand for technology from the teaching environment to the student. In order to generate first evidence for this proposition, I have implemented a design-based research project (Choi & Lee, 2008) that features an intervention (the introduction of flipped elements) that was implemented in two very different contexts. Also, different sources of qualitative and quantitative data were tapped to procure information about the efficiency of that intervention. All data generated points into the same direction: The course in the less technologized learning environment of setting B is not evaluated to be more negative in any way than the more technologized learning environment of setting A. I interpret this to be in support of the proposition.

4.1 Limitations and implications for further research

The study presented here was not intended to produce fully conclusive results. However, the aim was to generate early-stage evidence that gives some more security when venturing into more elaborated research designs to investigate this topic. Arguably, the most important limitation concerns the question about generalizability. All the data were used to contrast two single cases; given that a plethora of factors were not part of this research (most prominently, the teacher), no generalization is possible. Still, I did use a host of very different indicators to test the proposition and they all point into the same direction. Also, although not the primary goal of this research project, the findings are in line with the positive outcomes of flipped classroom designs found elsewhere in the literature (e.g. Peterson, 2016; Sun et al., 2017). This gives some confidence in the robustness of the results generated in this article.

This study hopes to facilitate the conduct of further investigations that may help to bridge this gap and that are able to produce more generalizable outcomes.

4.2 Implications for practice

To the best of my knowledge, the mechanism of buffering potentially negative effects of learning environments poor in technological resources has neither been proposed nor tested before. These first results, however, indeed suggest that flipped elements may help to alleviate this issue. This has a few important implications for practice.

For universities, this buffering effect may be an important ingredient during the transition into the digital age. This is especially true for larger institutions (such as setting B), where the financial burden of providing the technological resources for the students may put a lot of strain on the financial balance and the flexibility as an institution. If, however, flipped elements may already be enough to limit negative effects as suggested by the proposition, this may not be necessary.

For higher education teachers, the buffering effect described here may represent another advantage of the flipped-classroom pedagogy to be added to the list of commonly defined advantages (e.g. Taylor, 2015, or see the list of example outcomes given in the introduction). This may also increase the motivation to subscribe to this model of teaching. Since the described effect requires an investment on the side of the students--they need access to a computer with the required software--, it is important to recognize the requirements when planning the course and the flipped elements. Specifically, it is recommended to only use software that is either freely available or that is accessible via the shared workstations at the university. While teaching in an environment where students have different computer setups, operating systems, etc. definitely becomes more complex and challenging for the teacher, it also furthers technological competencies on the side of the students more than a "ready-to-use" computer lab at the university does. On the side of the teacher's input, it is important to highlight the workload associated with creating flipped elements (see for instance Taylor, 2015). This stresses the need for finding more sustainable ways of creating and sharing open educational resources (Caswell, Henson, Jensen, & Wiley, 2008; Downes, 2007), as universities aim to leverage the opportunities and master the challenges of the digital age.


Al-Zahrani, A. M. (2015). From passive to active: The impact of the flipped classroom through social learning platforms on higher education students' creative thinking: From passive to active. British Journal of Educational Technology, 46(6), 1133-1148. doi:10.1111/bjet.12353

Anderson, T. & Shattuck, J. (2012). Design-Based Research: A Decade of Progress in Education Research? Educational Researcher, 41(1), 16-25. doi:10.3102/0013189X11428813

Baldwin, T. T. & Ford, J. K. (1988). Transfer of training: A revire and directions for future research. Personnel Psychology, 41(1), 63--105. doi:10.1111/j.1744-6570.1988.tb00632.x

Boeve, A. J., Meijer, R. R., Bosker, R. J., Vugteveen, J., Hoekstra, R., & Albers, C. J. (2017). Implementing the flipped classroom: an exploration of study behaviour and student performance. Higher Education, 74(6), 1015-1032. doi:10.1007/s10734-016-0104-y

Caswell, T., Henson, S., Jensen, M., & Wiley, D. (2008). Open Content and Open Educational Resources: Enabling universal education. The International Review of Research in Open and Distributed Learning, 9(1). doi:10.19173/irrodl.v9i1.469

Choi, I., & Lee, K. (2008). Designing and implementing a case-based learning environment for enhancing ill-structured problem solving: classroom management problems for prospective teachers. Educational Technology Research and Development, 57(1), 99-129. doi:10.1007/s11423-008-9089-2

Chuang, H.-H., Weng, C.-Y., & Chen, C.-H. (2018). Which students benefit most from a flipped classroom approach to language learning?: Flipped classroom does not fit all students. British Journal of Educational Technology, 49(1), 56-68. doi:10.1111/bjet.l2530

Downes, S. (2007). Models for Sustainable Open Educational Resources. Interdisciplinary Journal of E-Learning and Learning Objects, 5(1), 29-44.

Euler, D., & Sloane, P. F. E. (Eds.). (2014). Design-based research. Stuttgart: Franz Steiner Verlag.

Finney, S. J., & Schraw, G. (2003). Self-efficacy beliefs in college statistics courses. Contemporary Educational Psychology, 28(2), 161-186. doi:10.1016/S0361476X(02)00015-2

Froehlich, D. E., & Gegenfurtner, A. (Forthcoming). Social support in transitioning from training to the workplace: A social network perspective. In H. Fasching (Ed.), Beziehungen in padagogischen Arbeitsfeldern [Relations in pedagogical work]. Bad Heilbrunn: Klinkhardt.

Froehlich, D. E., Liu, M., & Van der Heijden, B. I. J. M. (2018). Work in Progress: The Progression of Competence-Based Employability. Career Development International.

Froehlich, D. E., & Messmann, G. (2017). The Social Side of Innovative Work Behavior: Determinants of Social Interaction during Organizational Innovation Processes. Business Creativity and the Creative Economy, 3(1), 31-41.

Holmes, M. R., Tracy, E. M., Painter, L. L., Oestreich, T., & Park, H. (2015). Moving from Flipcharts to the Flipped Classroom: Using Technology Driven Teaching Methods to Promote Active Learning in Foundation and Advanced Masters Social Work Courses. Clinical Social Work Journal, 43(2), 215-224. doi:10.1007/s10615-015-0521-x

Jick, T. D. (1979). Mixing Qualitative and Quantitative Methods: Triangulation in Action. Administrative Science Quarterly, 24(4), 602-611.

Johnson, R. B., & Onwuegbuzie, A. J. (2004). Mixed Methods Research: A Research Paradigm Whose Time Has Come. Educational Researcher, 33(7), 14-26.

Kong, S. C. (2014). Developing information literacy and critical thinking skills through domain knowledge learning in digital classrooms: An experience of practicing flipped classroom strategy. Computers & Education, 78, 160-173. doi:10.1016/j.compedu.2014.05.009

Larwin, K. (2014). Statistics Related Self-Efficacy A Confirmatory Factor Analysis Demonstrating a Significant Link to Prior Mathematics Experiences for Graduate Level Students. Mathematics Education Trends and Research, 2014. Retrieved from

Lo, C. K., Lie, C. W., & Hew, K. F. (2018). Applying "First Principles of Instruction" as a design theory of the flipped classroom: Findings from a collective study of four secondary school subjects. Computers & Education, 118, 150-165. doi:10.1016/j.compedu.2017.12.003

Njie-Carr, V. P. S., Ludeman, E., Lee, M. C., Dordunoo, D., Trocky, N. M., & Jenkins, L. S. (2017). An Integrative Review of Flipped Classroom Teaching Models in Nursing Education. Journal of Professional Nursing, 33(2), 133-144. doi:10.1016/j.profnurs.2016.07.001

Onwuegbuzie, A. J., & Wilson, V. A. (2000, May). Statistics Anxiety: Nature, Etiology, Antecedents, Effects, and Treatments: A Comprehensive Review of the Literature. Paper presented at the Annual meeting of the Mid-South Educational Research Association, Lexington, Kentucky. Retrieved from

Peterson, D. J. (2016). The Flipped Classroom Improves Student Achievement and Course Satisfaction in a Statistics Course: A Quasi-Experimental Study. Teaching of Psychology, 43(1), 10-15. doi:10.1177/0098628315620063

Reidsema, C., Kavanagh, L., Hadgraft, R., & Smith, N. (Eds.). (2017). The Flipped Classroom: Practice and Practices in Higher Education (1st ed. 2017 edition). New York, NY: Springer.

Saldana, J. (2009). The Coding Manual for Qualitative Researchers. Los Angeles, CA: Sage.

Schoonenboom, J., Johnson, R. B., & Froehlich, D. E. (2018). Combining Multiple Purposes of Mixing Within a Mixed Methods Research Design. International Journal of Multiple Research Approaches.

Sun, J. C.-Y., Wu, Y.-T., & Lee, W.-I. (2017). The effect of the flipped classroom approach to OpenCourseWare instruction on students' self-regulation: Flipped classroom approach and OpenCourseWare. British Journal of Educational Technology, 48(3), 713-729. doi:10.1111/bjet.l2444

Talbert, R., & Bergmann, J. (2017). Flipped Learning: A Guide for Higher Education Faculty. Sterling, Virginia: Stylus Publishing.

Tawfik, A. A., & Lilly, C. (2015). Using a Flipped Classroom Approach to Support Problem-Based Learning. Technology, Knowledge and Learning, 20(3), 299-315. doi:10.1007/s10758-015-9262-8

Taylor, A. (2015). Flipping Great or Flipping Useless? A review of the flipped classroom experiment at Coventry University London Campus. Journal of Pedagogic Development, 5(3). Retrieved from

How to cite this article: Froehlich, D. E. (2018). Non-technological Learning Environments in a Technological World: Flipping Comes to the Aid. Journal of New Approaches in Educational Research, 7(2), PP.88-92. doi: 10.7821/naer.2018.7.304

Dominik E. Froehlich (*)[iD]

Department of Education, University of Vienna, Austria {}

Received on 30 April 2018; revised on 4 May 2018; accepted on 9 May 2018; published on 15 July 2018

(*) To whom correspondence should be addressed: Sensengasse 3a 1090 Vienna Austria

DOI: 10.7821/naer.2018.7.304
Table 1. Overview of the different settings

Setting           A                 B
Institution       University of     University
                  applied science
Level             Bachelor          Bachelor, but also Master/PhD
                                    students present

Class size        25                151
Room              computer lab      lecture room
Prior knowledge   homogeneous       heterogeneous

Table 2. Overview of the research results

Data                             A                   B

Psychographic measurement of     M = 2.4, SD = 0.7   M = 3.2, 1.0
statistical self-efficacy
Psychographic measurement of     M = 2.6, SD = 0.7   M = 2.2, SD = 0.8
statistic anxiety
Evaluative scales of teaching    M = 4.5, SD= 1.1    M=5.4, SD = 1.3
Evaluative scales of overall     M = 4.8, SD = 1.0   M = 5.6, SD = 1.3
course quality
Student qualitative assessment   highly rated        highly rated
Jury qualitative assessment      highly rated        highly rated

Data                             Analysis and Interpretation

Psychographic measurement of     Statistically significant difference;
statistical self-efficacy        setting B scores higher
Psychographic measurement of     No statistically significant
statistic anxiety                difference
Evaluative scales of teaching    Statistically significant difference;
performance                      setting B scores higher
Evaluative scales of overall     Statistically significant difference;
course quality                   setting B scores higher
Student qualitative assessment   No important difference found
Jury qualitative assessment      No important difference found

Data                             Conclusion

Psychographic measurement of     Supports the proposition
statistical self-efficacy
Psychographic measurement of     Weakly supports the proposition
statistic anxiety
Evaluative scales of teaching    Supports the proposition
Evaluative scales of overall     Supports the proposition
course quality
Student qualitative assessment   Weakly supports the proposition
Jury qualitative assessment      Weakly supports the proposition
COPYRIGHT 2018 Universidad de Alicante
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2018 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Title Annotation:ORIGINAL
Author:Froehlich, Dominik E.
Publication:NAER - Journal of New Approaches in Educational Research
Date:Jul 1, 2018
Previous Article:Lifelong Learning at Universities: Future Perspectives for Teaching and Learning.
Next Article:E-Teaching in Higher Education: An Essential Prerequisite for E-Learning.

Terms of use | Privacy policy | Copyright © 2021 Farlex, Inc. | Feedback | For webmasters |