Big data in the service of educator learning: what should be done with collected online professional learning information?
The concern over big data and the ramifications of its use permeates many, if not all, aspects of life in the 21st century. With the advent of online learning, a new area of direct concern to educators has been added, that of data collection during the online learning process, as well as the collection and use of large collected data sets ("big data") that result from interactions within the online environment. This article focuses specifically on data collection during professional development venues for in-service educators as they further strengthen their educational skills.
The work in professional development parallels the efforts in higher education in providing learning opportunities for undergraduate and graduate students through e-learning. Online professional development is also related to the provision of online learning for students in the K-12 education system. While holding some of these common features, professional development e-learning holds particular challenges surrounding data collection. Professional development activities, for example, are voluntary. Engaging with professional learning through an online environment is but one option for educators within a menu of multiple available options. Additionally, the immediate relevance to job responsibilities is targeted in most professional development systems. Educators often seek professional development to meet the immediate challenges they are experiencing in their classrooms, rather than the preparation task that preservice teachers still within the higher education setting undertake. Examining educators who voluntarily choose to participate in an online professional development opportunity therefore involves a slightly different context than a situation wherein the expectation to participate in e-learning is a requirement, such as graduation from high school or a university. The expectation for data collection of those involved in voluntary, self-directed online professional development may well be different than the student who expects data to be collected as part of a traditional institutional context. The voluntary aspect of this context drives our analysis in this paper in a slightly different direction when considering the data collection and analysis that may occur in this unique setting. Students, whether in high school or in college, expect to have their engagement and their work monitored in order to assign grades to determine an overall outcome of graduation. In-service educators who attend professional development activities in the more traditional face-to-face context do not expect their engagement to be monitored or recorded in any formal sense, nor do they typically expect a "work product" to be evaluated as a result of attending a professional development event.
It is imperative that the educational field recognize the unique nature of the online professional development environment as regards the nature and type of data collected, as well as identify both data that is and is not typically collected today. We shall commence by establishing a definition of online professional development, examine the context of K-12 education as it relates to educators' continued professional learning, review some of the current efforts to support ongoing professional development in K-12 education, and then, in line with the focus of this special QRDE issue, address the collection and analysis of data for participants in professional development e-learning, concluding with some follow-on suggestions for online professional development data collection.
DEFINITION OF ONLINE + PROFESSIONAL DEVELOPMENT
Addressing the definition of the term online professional development is necessary to cogently discuss the data issues posed for online professional development. To do so, we must look at the two components in this term: "online" and "professional development."
There are many different definitions of online learning or e-learning (Januszewski & Molenda, 2008; Moore, Dickson-Deane, & Gaylen, 2011). For purposes of this article a simple definition is sufficient. Allen and Seaman's (2016) typology outlining four categories of instruction provides a clear delineation of online learning that readily applies to professional development. These authors defined "traditional instruction" as learning that incorporates NO online technology. A second category, "web-facilitated instruction," includes situations with up to 29% of the learning content online. A third category, "blended/hybrid instruction," involves a blending of face-to-face instruction with "substantial portion of content online 30-79%" (p. 7). The final category in Allen and Seaman's framework is "online instruction" (where 80% to 100% of the learning content resides online). The examples cited in this manuscript and the discussion surrounding data exclusively center on Allen and Seaman's last category, "online instruction." The professional learning for educators examined here includes all interactions that are captured within a given learning management system (LMS), and it is important that the reader note that no offline interactions are included within this analysis. The effect of face-to-face interactions on the questions of big data are not fully known, but the well-documented lack of human connection in online interactions leads one to believe that there is some effect on data collection (Gardner & Davis, 2013; Turkle, 2015). Do educators who engage with an online discussion board react and interact in the same manner as they would in a face-to-face workshop? The interactions of participants are a key data source, as we will discuss later, and if, in fact, the online environment impacts the data collected, we should be well aware of this effect.
The second portion of the equation, professional development, has a long history in the area of K-12 education, with long-standing and agreed-upon definitions. Quality indicators for professional learning in the educational context have been developed by multiple organizations (e.g., Council of Chief State School Officers, 2013; Learning Forward, 2014). Learning Forward promotes high-quality professional learning and developed, and subsequently revised, a set of standards by which professional learning systems should be judged. The Learning Forward organization outlines the purpose of professional learning as the activities are designed to enable "educators to develop the knowledge, skills, practices, and dispositions they need to help students perform at higher levels" (Learning Forward, 2015). Standards outlined by Learning Forward include: learning communities, leadership, resources, data, learning designs, implementation, and outcomes.
As a result of systematic professional learning, an impact on student learning must be demonstrated. Professional learning must have some connection to the outcome that is the heart of education, an increase in students' knowledge, skills, and dispositions. According to Learning Forward and other similar organizations, if professional development undertaken by educators does not ultimately result in higher levels of student achievement, then the effort is wasted.
The combination of online learning, or e-learning, with professional development results in a program of engagement wherein educators use an online delivery method to develop their knowledge and skills that result in a change for students' achievement. Online professional development is situated in the larger K-12 educational context, which impacts the implementation and results of these programs. The availability of high-quality online learning is a very significant benefit for the education system overall. Digital delivery of important concepts and practices will enable educators to secure the professional development necessary to educate all students at to the highest levels of achievement.
CONTEXT OF K-12 EDUCATION AND PROFESSIONAL DEVELOPMENT
As with most professions, there are requirements for ongoing professional development in order to maintain licensure or certification. Ongoing certification requirements also apply to educators, as well, with specific requirements dictated by each state's laws and statutes. The specified parameters necessary to fulfill the certification requirements are broad and allow educators to engage in many types of professional learning activities. Options such as in-person workshops, reading professional journals, or taking university coursework are recognized by the current certification system as valid means of engaging in professional development.
Data regarding educator perceptions of effective professional learning have been collected in previous studies, such as those from the Bill & Melinda Gates Foundation (Gates Foundation, 2014). Teachers in this study reported that current professional learning opportunities are not meeting their needs in serving the diverse student populations in their classrooms. Educators want professional development that is relevant to their unique teaching situations. Teachers want to engage with their learning; they therefore want professional development to be interactive in nature. Educators feel that professional learning should be provided by other professionals who have a deep knowledge of teaching and the varied situations that teachers face (Gates Foundation, 2014).
In this time of budgetary constraints, increased demands on time, and desires for effective professional learning opportunities, educators have to carefully choose how and when to engage in professional learning activities. School districts determine with much care what opportunities will be supported within their limited budgets. The cost of professional development includes not only the personnel who manage the activities and the materials required for the participants, but also travel expenses to training locations. Additionally, in the case of classroom teachers, the expense of substitute teachers must be factored into the overall cost of professional development. Beyond the inherent flexibility and convenience of e-learning, given the relative expense of face-to-face professional development opportunities, it is therefore not surprising that many educators have embraced online professional development, mirroring the general increase in online enrollment within higher education (Allen & Seaman, 2016). While the cost effectiveness of e-learning has been researched with a few professional development programs (Cavalluzzo, Lopez, Ross, Larson, & Martinez, 2005), a definitive analysis of true professional development cost has not yet been conducted.
What Does the Current e-Learning Setting Look Like Today?
There are numerous providers in the space of online professional learning for educators. The options range from small for-profit companies to large professional organizations. For the purposes of facility, we in this article restrict our discussion to four large representative projects that focus on e-learning for K-12 educators: PBS TeacherLine, ASCD PDonline, EdTech Leaders Online, and Learning Forward. These organizations are not primarily for-profit entities, and provide online professional development to a national audience.
PBS TeacherLine is a national program offered by the Public Broadcasting System providing more than 100 courses to educators for more than 15 years (ALTA Solutions Group, 2010). The program offers facilitated courses (i.e., those with a mentor to guide online discussions) and self-directed courses (i.e., those in which content is reviewed individually with no interaction between or among participants). PBS TeacherLine provides professional learning opportunities in multiple areas of curriculum (e.g., mathematics and science) as well as courses in leadership and technology.
ASCD PDonline is another national effort to provide access to online professional learning for educators. The program offers self-paced instructional courses in many areas of education such as mentoring, bullying, English/language arts, and others.
EdTech Leaders Online also provides online professional learning for in-service educators. The program offers facilitated courses in many areas of education such as mathematics, science and technology integration.
Learning Forward, as previously mentioned, is a leader in the space of professional learning for K-12 educators. This organization has recently developed some online professional development opportunities with a focus on implementation of the professional learning standards. Learning Forward's courses are self-paced in nature.
These representative programs provide a snapshot of the current efforts to give K-12 educators access to professional learning in an effective and efficient manner. Both the EdTech Leaders Online and the PBS TeacherLine programs utilize Moodle (an open source LMS) to deliver their online courses. A review of every potential LMS is beyond the scope of this essay. The discussion here focuses on the data provided by the Moodle LMS, but the issues raised by examining data collected within a typical Moodle environment may be generalized to most other LMS's. Similarities in data collection across LMS's exist by virtue of the nature of usage demands, such as, participants have to log into the system, which creates a data set. Determination of time users spend reviewing information are also built into nearly all LMS's. Other features that similarly serve the purpose of a learning platform are included in most or all of these LMS's.
What Data Are Currently Collected in Online Professional Development Systems?
Prior to accessing an online professional development course, participants must first register to enable access to the instruction. The mechanism to accomplish registration by various providers is dependent on multiple factors. If the course is free and open to the public, registration may be as simple as providing an email address and setting a password. If the professional learning is fee based, some means to electronically collect fees must be established. A fee-based program necessitates collection of payment information, which will likely include credit card information, and sometimes, additional proof of identity (e.g., date of birth, home street address, etc.). While it is important to note that such data is often collected within online professional development settings, the issues of online payment information is not the focus of this article.
Beyond basic identification of users and collection of fee information, the Moodle LMS collects a myriad of data on every user of the system (cf. Moodle, n.d.). Logs provide the date and time when a participant accesses the course content, the portion of the content accessed, what was accomplished during that access time (e.g., viewed course or posted to discussion board), and from where the participant accessed the course (IP address). These data are important in establishing some sense of the participant's engagement with the learning material. As Baer and Campbell (2012) noted, these data are a step in the provision of interventions for participants who are not engaging at an appropriate level to gain knowledge and skills, the hallmark of educator professional learning.
Additionally, data are collected on actual participant response to learning content and learning activities. If discussion forums /discussion boards are used within a given course, and most facilitated courses do include these features, participants' responses to specified questions or prompts are also collected. These data can be analyzed for understanding of the content. These data can also help determine the level of integration of the relevant concepts into the participants' practice, although in an indirect way, through self-report. A summary of the current data collected and the relationship between these data and student achievement are provided in Table 1.
What Data Should Be Collected?
The amount of data collected on educators taking online professional learning activities is immense. These data cover a range of participant behavior, including when and from where they log into the site, the web pages they view, the activities in which they engage, and so on. These data can provide some important insights into educators' professional learning, not simply from a "Big Brother'/Oversight viewpoint, but from an evaluation of actual learning and analysis of the learning content itself. Certainly, there is information that can be used to intervene if a participant in online professional learning accessed the learning content only two or three times throughout an 8-week course. There is knowledge to be gained from the data associated with the time spent on a given webpage, say an average of 30 minutes to view a 6-minute video. If learners are not accessing the professional learning content in a manner consistent with its intended use, an analysis of the learning content should be undertaken so that the online instruction may be revised, providing improved learning opportunities for K-12 educators, not penalization. As Joe Freidhoff, executive director of the Michigan Virtual Learning Research Institute, indicated in the quote at the beginning of this article, the idea of recording and analyzing the not only the quantitative data but to complete the picture of online learning with rich qualitative data.
A great deal of research has been conducted within the higher education context around discussion boards and threaded discussions (Huss, Sela, & Eastep, 2015; Russell, Carey, Kleiman, & Venable, 2009). These data are not easily reduced to quantitative representations of participants' thinking. A more qualitative approach to the analysis of these data is needed to yield useful information for those providing professional learning opportunities. One study that utilized a qualitative approach, (Jarosewich et al., 2010) reviewed discussion board posts for teachers participating in the e-Read Ohio professional development program. Their analysis provides direction for other researchers and online professional development providers to examine online interactions.
Teachers' perceptions, as reported in the Gates Foundation report (2014), should also be used to direct efforts at data collection. The relevance of the professional learning experience to teachers' individual work situations should be analyzed. While the data on time spent engaging with online learning activities is readily available, often, information relating to educators' perceptions about the applicability of the professional development experience is not collected. While perceptual data has limitations, they can also yield revelations about the utility of the professional development experience. Survey data can be collected relating to how useful the information was for participants. Questions that ascertain whether or not the knowledge and skills contained within the professional development event will be applied to the educator's work context would provide beneficial information. Data can be collected regarding the application of the learning experiences, and, therefore, relevance, to teachers' work.
Baer and Campbell (2012) indicated that for higher education there is a growing call for accountability with regard to student outcomes, including outcomes of online education. This is no less true for the K-12 education system and the professional development system supporting in-service educators. Outcome data have been a centerpiece of K-12 education for more than 2 decades and while the accountability of schools in regard to K-12 student achievement has been hijacked by standardized assessment, we should not allow the accountability of professional development to be hijacked by big data from our LMS. (One major provision of The No Child Left Behind Act (2002) was administration of standardized tests at every Grade 3-8 and once in high school. The results of this one type of data were used to determine whether or not a school would be labeled as failing. The reductive nature of using one measure of effectiveness has been decried by many educators as well as the general public (Mulholland, 2015; Strauss, 2014).
At this time, we do not possess systems that connect student achievement to educator engagement in professional development. The standards that address high-quality professional learning for educators indicate that the ultimate measure of efficacy for professional development is a change in student achievement. Yet, the literature on professional development for educators and the standards proposed by the largest organization of professional development for educators, Learning Forward, deems it necessary to include information about increases in student achievement as a result of the professional development of educators (Learning Forward, n.d.). As Freidhoff noted, such a connection requires rich analysis. The current system in many states requires "clock hours" to fulfill the certification /licensure mandates. While clock hours are easily reported and tracked, rich analysis is not possible. Big data from LMS's appear to be similar in that much of it is easily tracked and reported, but this data does not provide the complete picture surrounding learning. By what mechanism can we attribute changes in student achievement to a given professional learning opportunity? Can we capture data to determine the relationship between engagement with online professional development and student outcomes? It seems an analysis of this magnitude and a solid conclusion would be difficult using our current state of data collection in e-learning professional development settings.
While difficult, conducting such an analysis is not impossible. EdTech Leaders Online, for example, recently conducted just such an analysis (O'Dwyer et al., 2010). The study examined the effects of online professional development for K-12 teachers in mathematics and English/language arts. This study found positive effects of the online professional development on student achievement. Shaha and Ellsworth (2013), too, conducted a multistate analysis of the effects of online professional development on student achievement. Their conclusions also supported an increase in student achievement for students whose teachers engaged more with online professional learning. More research in this area is needed to confirm these effects on student achievement. In addition to this necessary research, professional development providers should integrate some efforts into their courses to obtain data on student achievement or implementation efforts corresponding to the goals and outcomes of the given courses.
As we have considered online professional development and how data are collected within the context of an LMS, questions have been raised about the use of these data and the lack of information about student achievement tied to teacher professional development. A mechanism by which teachers' engagement with professional learning, data that are currently available both in a numeric/quantitative form and in a textual/qualitative form should be tied to student achievement data. Many states require student achievement to be considered as a part of the teacher evaluation system; some mechanism must be possible, therefore, for the linking of teacher identification information to student information. This is one approach in which we can begin to collect information on the effectiveness of online professional development. Research on the hypothesis that online professional development engagement strengthens student achievement is crucial, if for no other reason that so much professional development now occurs through the online modality. Furthermore, research addressing the specific components of online professional development that may be more effect would benefit the field. The ability of professional development providers to undertake this work would be questionable, at best. The data sets for these two endeavors, professional development engagement and teacher effectiveness, are separate. The "keepers" of the two data sets have different roles in the system: the schools/districts/state are charged with evaluation of educator performance while professional development providers supply the opportunity to enhance professional skills. Certainly there is an intersection, and at times a complete overlap, in these two organizations (professional development providers and educational systems), but the role and outcome of the two are different.
There are unquestionably substantial issues surrounding the type of data pairing suggested here. Students in the K-12 system have different data privacy issues at play than do teachers who voluntarily participate in online professional learning. Students under the age of 18 are not able to provide the same level of informed consent as adults and are a part of a larger system that is designed to protect their privacy interests. Laws protecting student information of all types, (i.e., FERPA) are applied to online learning in the same manner that they are applied to other nondigital data for students. Statutes pertaining to student records also protect data about students. The security of these data is of utmost importance. The expectation of students is the collection of data within the educational system but also that these data will be used in a specific manner and protected from abuse. These same protections are not in place for adult learners. The use of student data is, and should be, more restricted.
The security and privacy issues related to the use of student data should not overshadow the issues of security and privacy of the professionals involved in this imagined professional development system. Issues of data storage for assessment information, for discussion content, and other individual information that is generated as a result of engagement with the professional learning activities will need to be addressed.
Access is another area that has to be carefully considered within such a system. Who can access such data? Can educators' evaluators review the learners' online assessments? Their online posts? Further, the long-term effects surrounding these data have to be considered. We must think about who owns these data (is it the participant or the provider?), how long these data are maintained, where these data are physically located, and other aspects of the data resulting from online professional development.
As with the O'Dwyer et al. (2010) study, data should also be collected offline regarding the actual change in teaching practice as a result of engagement with online professional development. While direct observation of teaching practice would provide the strongest connection, self-report data collected after the conclusion of a professional development course could also provide the field with insight into student outcomes and effects of online professional development. These data could be collected within the LMS as a follow-up survey. Establishing a relationship between different types of online activities, discussion forums versus interactive video, say, and implementation of particular teaching practices would be of great use to the professional development community. Survey data that address educators' perceived application of the knowledge and skills provided in the professional development activity could also be collected and analyzed to help further illuminate the effectiveness of the professional development event.
In order to effectively analyze the relationship of online professional development and student outcomes, organizations should partner with institutions of higher education. The expertise of faculty in higher education for designing, conducting, and analyzing research of this magnitude is essential. Additionally, other organizations involved in professional development for educators could and should be involved in these studies. Most states have regional agencies that are involved with professional learning and the connection of those agencies with educators would provide a rich source of data.
Partnerships are helpful in many endeavors and certainly education is a case for collaboration of many stakeholders. Partnerships do, however, raise more issues surrounding the collection of data. Within the partnership, how is identity protected? How are data securely stored for multiple agency use? Who within each partner organization has access to the data? What are appropriate uses of these data? For what purpose would these data be used, and would these uses be identified for participants prior to engaging in the professional development? Would there be a mechanism to opt out of any data usage?
Online professional development for in-service educators will potentially mirror the growth of online learning in other contexts (e.g., higher education and K-12 education). There are some data to support an increase in participation relative to online professional development (Blackboard K-12, 2013). By addressing the concerns of big data and data use now, we can direct the development of systems that benefit all in the educational system. Currently, the data generated by online professional learning in an LMS system are useful to begin an analysis of the system. These data can provide information about the learning materials, the activities, and the structure of the learning opportunity on a quantitative level. These data can point to something that may require further investigation either with the course itself or with individual participants who are engaged (or not) with the course content.
However, a very significant portion of the professional development puzzle is missing. We cannot, with any certainty, claim to have impact on student achievement as a result of the given professional learning opportunity. Research in this area is needed, but more importantly, a systematic way to gather data on student achievement related to the professional development setting is necessary if we are to continue to strengthen and enrich the value of professional development efforts for all learners.
Allen, I. E., & Seaman, J. (2016). Online report card: Tracking online education in the United States. Newburyport, MA: Babson Survey Research Group and Quahog Research Group, Online Learning Consortium.
ALTA Solutions Group. (2010). A study of PBS TeacherLine online course facilitation. Pittsburgh, NY: Author.
Baer, L., & Campbell, J. (2012). From metrics to analytics, reporting to action: Analytics' role in changing the learning environment. In D. G. Oblinger (Ed.), Game changers: Education and information technologies. Retrieved from http:// net.educause.edu/ir/library/pdf/pub7203.pdf
Blackboard K-12. (2013). 2013 trends in online learning: Virtual, blended and flipped classrooms. Retrieved from: http://www.tomorrow.org/speakup /2013_OnlineLearningReport.html
Cavalluzzo, L., Lopez, D., Ross, J., Larson, M., & Martinez, M. (2005). A study of the effectiveness and cost of AEL 's online professional development program in reading in Tennessee. Retrieved from http://files.eric.ed.gov/fulltext /ED489124.pdf
Council of Chief State School Officers. (2013, April). Interstate Teacher Assessment and Support Consortium InTASC model core teaching standards and learning progressions for teachers 1.0: A resource for ongoing teacher development. Washington, DC: Author.
Gardner, H., & Davis, K. (2013). The app generation: How today's youth navigate identity, intimacy, and imagination in a digital world. New Haven, CT: Yale University Press.
Gates Foundation. (2014). Teachers know best: Teachers' views on professional development. Retrieved from: http://k12education.gates -foundation.org/wp-content/uploads/2015/ 04/Gate s-PDMarketResearch-Dec5.pdf
Huss, J. A., Sela, O., & Eastep, S. (2015). A case study of online instructors and their quest for greater interactivity in their courses: Overcoming the distance in distance education. Australian Journal of Teacher Education, 40(4), 1286. Retrieved from http://ro.ecu.edu.au/ajte/ vol40/iss4/5
Januszewski, A., & Molenda, M. (2008). Educational technology: A definition with commentary. New York, NY: Erlbaum.
Jarosewich, T., Vargo, L., Salzman, J., Lenhart, L., Krosnick, L., Vance, K., & Roskos, K. (2010). Say what? The quality of discussion board postings in online professional development. New Horizons in Education, 53(3), 118-132. Retrieved from http://files.eric.ed.gov/fulltext /EJ966665.pdf
Learning Forward. (2015). Retrieved from http:// learningforward.org/standards/#VuhAgq_mr2A
Mulholland, Q. (2015). The case against standardized testing. Harvard Political Review. Retrieved from http://harvardpolitics.com/ united-state s/case-standardized-te sting/
Moodle. (n.d.) Documentation: Logs. Retrieved from https://docs.moodle.org/30/en/Logs
Moore, J. L., Dickson-Deane, C., & Gaylen, K. (2011). e-Learning, online learning, and distance learning environments: Are they the same? Internet and Higher Education. Retrieved from https://scholar.vt.edu/access/content/group/ 5deb92b5-10f3-49db-adeb-7294847flebc/ e-Learning%20Scott%20Midkiff.pdf
O'Dwyer, L. M., Masters, J., Dash, S., De Kramer, R. M., Humez, A., & Russell, M. (2010). E-learning for educators: Effects of on-line professional development on teachers and their students: Findings from four randomized trials. Chestnut Hill, MA: Technology and Assessment Study Collaborative, Lynch School of Education, Boston College. Retrieved from www.bc.edu/research/intasc/PDF/EFE_ Findings2010_Report.pdf
Pourreau, L. (2015). Interview with Joe Freidhoff: A bird's-eye view of K-12 online learning. Online Learning, 19(5), 12-17.
Russell, M., Carey, R., Kleiman, G., & Venable, J. D. (2009). Face-to-face and online professional development for mathematics teachers: A comparative study. Journal of Asynchronous Learning Networks, 13(2), 71-87.
Shaha, S. H., & Ellsworth, H. (2013). Predictors of success for professional development: Linking student achievement to school and educator successes through on-demand, online professional learning. Journal of Instructional Psychology, 40(1), 19-26.
Strauss, V. (2014). 11 problems created by the standardized testing obsession. The Washington Post. Retrieved from https://www.washingtonpost.com /news/answer-sheet/wp/2014/04/22/11-problems-created -by-thestandardized-testing-obsession/
Turkle, S. (2015). Reclaiming conversation: The power of talk in a digital age. New York, NY: Penguin.
Mary M. O'Brian
Illinois State University
* Mary O'Brian, Illinois State University. E-mail: email@example.com
TABLE 1 Summary of Data Collected Within Moodle Direct Data Element Format Relationship to Student Achievement Participant name Text No Participant mailing Text No address Participant email Text No address Log in/log out time Numeric No IP address Numeric No Length of view time Numeric No on page Type of activity Text No (e.g., post to forum, upload assignment) Assessment scores Numeric/text No Individual forum Text No posts/responses Data Element Secondary Relationship to Student Achievement Participant name Yes; Establishes the relationship of individual teacher to individual student for analysis Participant mailing No address Participant email No address Log in/log out time Yes; Data may be related to student achievement IP address No Length of view time Yes; Data may be related to on page student achievement Type of activity Yes; Data may be related to (e.g., post to student achievement forum, upload assignment) Assessment scores Yes; Data may be related to student achievement Individual forum Yes; Data may be related to posts/responses student achievement
|Printer friendly Cite/link Email Feedback|
|Author:||O'Brian, Mary M.|
|Publication:||Quarterly Review of Distance Education|
|Date:||Dec 22, 2016|
|Previous Article:||Emerging evidence on the use of big data and analytics in workplace learning: a systematic literature review.|
|Next Article:||Supporting online student retention in community colleges: what data is most relevant?|