Metacomprehension skills depend on the type of text: An analysis from Differential Item Functioning.
Metacomprehension has been extensively researched in children and adults in order to examine the accuracy of people's judgements of their comprehension because of its importance in learning and comprehension. Metacomprehension can be important in monitoring and optimizing learning (Dunlosky & Lipko, 2007; Dunlosky & Metcalfe, 2008), in determining cognitive abilities in children (Diamond, 2012), in improving academic skills (Duckworth & Seligman, 2005), and in intervention programs to improve reading comprehension using metacomprehension strategies (Madariaga & Martinez, 2010). Although the effects of different variables on metacomprehension have been widely studied, there is relatively little research relating it to different types of texts.
Several studies have examined how the characteristics of a text influence students' comprehension, but most have focused on general text discourse processing (e.g., Kintsch, 1998), in complete expository texts (e.g., Adesope, Trevisan, & Sundararajan, 2017; Leon, Martinez-Huertas, & Jastrzebska, 2018; Martinez-Huertas et al., 2018), in single-paragraph expository texts (e.g., Maki & Berry, 1982; Wiley, Griffin, & Thiede, 2005), or in mixed sets of texts (e.g., Leon, Olmos, Escudero, Canas, & Salmeron, 2006; Maki, Shields, Wheeler, & Zacchilli, 2005). These studies focused on discourse processing across different types of text looking at different levels of representation, but the most important split is between shallow and deep mental representations. Shallow representation are explicit ideas in a text and they can include lists of concepts, a handful of simple facts or properties of each concept, or simple definitions of key terms. These representations are called text-base levels (Kinstch, 1988). The text-base is an interconnected network of the explicit propositions contained in the text. These may correspond to exact phrases but also consist of abstract representations and paraphrases as well as a number of inferences necessary to establish coherence at the local level. It preserves the meaning of the source text but not necessarily in the original words or syntax. The text-base level is useful in many comprehension tasks that require reference to explicit information in a text such as searching for or identifying specific details or connecting information across different sections of the text. At the deepest level, there is the situational-model of what the text is about. The situational-model is more complex as it requires the reader to integrate propositions in the text with his or her own background knowledge and to generate elaborate inferences (Kinstch, 1988). Thus, evaluation of this level of comprehension is fundamental because the reader must supply a great deal of implicit information in order to achieve a good understanding of the text. This is achieved through making inferences, deductions, abstractions, associating ideas, predicting, and so on--all of which can influence processes beyond comprehension, such as interpreting or judging.
There are also differences in reading processes related to type of texts. Narrative texts, for example, typically convey information about familiar events and situations in a predictable manner, usually reflect problems of daily life and actions taken with reasons for them and are heavily influenced by temporary relations that regulate the attainment of different facts or actions (Leon et al., 2006). In contrast, expository texts frequently feature the conceptualization of ideas, explicitly specified rhetorical organization, context bound terminology, and technical uses of terms (Leon et al., 2006). Finally, discontinuous texts are organized differently than narratives and expository texts, and require a different kind of reading approach. Discontinuous texts include pictures, diagrams, graphs, tables, maps, catalogues, budgets, indexes and forms. Wiley et al. (2005) summarized some differences among narrative, expository texts and discontinuous texts. Expository and discontinuous texts generally are: (1) more difficult to read than narratives; (2) more technical and less familiar in content, which result in processing in a more item-specific manner; (3) less coherent than narratives; and (4) they assume more background knowledge that requires more effort to generate inferences. Since the structure of expository texts has a great impact on metacomprehension, most of the research in metacomprehension has been based on reading expository texts, thus more research is needed to analyze metacomprehension across different types of texts, such as narrative and discontinuous texts, because of the educational implications. Different metacomprehension patterns would be expected when different types of texts (with their own structural idiosyncrasies) are processed.
Leon, Escudero & Olmos (2012) proposed a standardized reading comprehension task (ECOMPLEC) inspired by international assessments such as the PISA (OECD, 2010), and they conceptualized reading as an activity to find relevant information among various sources of noise of the environmental context (Olmos et al., 2016). The authors based their reading comprehension task (using narrative, expository and discontinuous texts) on the situational-model and text-base representations (Kintsch, 1988; 1998). In general, reading comprehension can be defined as the generation of an appropriate situational-model related to the information contained in the processed texts. The interaction between both explicit and implicit knowledge (combined with mental or psychological processes, such as inference making) generates the basis of the reading comprehension process. Under this assumption, reading comprehension is strictly related to both text characteristics and individual differences in terms of knowledge and other relevant variables (such as working memory capacity, semantic context, and metacomprehension skills). Then, although the interaction of psychological processes involved in reading comprehension will depend on characteristics of the individual texts, general patterns should be observable in different text types due to their structures and contents and also the readers' metacomprehension skills.
In this way, researchers have proposed specific models to discriminate among various hypothetical processes involved in reading comprehension, and some useful insights have been reached (e.g., Graesser, Singer, & Trabasso, 1994; Cook & O'Brien, 2014). Then, when different psychological processes are related to the processing of different types of texts, it is plausible to consider that metacomprehension skills can be influenced by text characteristics. From this point of view, metacomprehension could be divided into higher or lower skills (Leon et al., 2012). Higher metacomprehension skills (Higher MC) refer to narrow adjustments readers make based on their own judgments of comprehension and their performance (easier task judgment and better performance in comprehension, or more difficult task judgment and worse performance in comprehension). Lower metacomprehension skills (Lower MC) refer to less adjustments readers make based on their own judgments of comprehension and their performance (easier task perception judgment and worse performance, or more difficult task judgment and good performance). Being a late-developing skill, metacomprehension reflects different maturation processes during individual development, and an appreciable improvement of metacomprehension skills is found in 10-12 year-old students, as some reading processes gradually become more automatic (Kolic-Vehovec & Bajsanski, 2006). Some authors have argued that developmental effects in reading skills do not necessarily imply an improvement of metacomprehension skills (i.e., Walczyk, 1990), and that the misguided emphasis on metacognitive judgments can be related to an overestimation of changes in reading strategies (i.e., Kolic-Vehovec & Bajsanski, 2006). Then, given that there are some varieties of types of texts used in assessing educational goals, some divergence would be expected when evaluating the effects of text types on metacomprehension skills and reading comprehension levels.
A first objective of the present study was to analyze whether different types of text (narrative, expository and discontinuous) have a differential impact on metacomprehension skills. A second objective was to analyze whether this impact on metacomprehension skills is equivalent in two different levels of primary schools; i.e., 4th and 6th grades (from 9 to 11 years old). A third objective was to analyze whether metacomprehension is stable across the texts in two groups divided by their metacomprehension skills (Higher and Lower metacomprehension). A final objective was to analyze whether metacomprehension skills are more related to text-base or situational-model mental representations of the text.
ECOMPLEC.Pri (Leon et al., 2012) was applied to ensure reliability and validity levels across the tasks because of it controls for text complexity. Different methodological approaches to the study of metacomprehension have been conducted over recent decades, but in the present study a novel and accurate methodological perspective was developed in order to test the differences in the students' metacomprehension skills. No differences are to be expected when the abilities of Higher MC and Lower MC groups are compared. Thus, students were classified by their metacomprehension skills, and a Differential Item Functioning (DIF) analysis approach was conducted in order to determine whether the underlying processes involved in reading comprehension and metacomprehension are different from one type of text to each other.
A sample of 823 Primary School students (447 females, age range from 9 to 11 years) completed the ECOMPLEC.Pri test. The sample was compound of 275 9-year-old students (166 females) and 548 11-year-old students (281 females). Students were recruited from 11 public and private schools in Madrid, and all of them were native Spanish speakers who gave their consent plus parental consent before participating in the test.
ECOMPLEC.Pri assesses comprehension of narrative, expository and discontinuous texts using multiple-choice tasks. ECOMPLEC.Pri has 22 comprehension questions for the narrative text ("The all-knowing little man"), 21 comprehension questions for the expository text ("The red blood cells") and 24 comprehension questions for the discontinuous text ("The toy museum" ). In addition, each text had two metacomprehension questions that measured the perceived text difficulty based on the individual self-perceived ability. These items were related to both, text difficulty (i.e., The text difficulty was... a. Hard for your ability, b. Appropriate for your ability, c. Easy for your ability), and the questions difficulty (i.e., The text questions were... a. Hard to understand, b. Medium because the answer is not clear, c. Easy to answer).
Students answered the ECOMPLEC.Pri test. The instructional text order was always the same: they read narrative, expository and discontinuous texts. No application time was established to answer ECOMPEC.Pri tasks (although one hour was always sufficient). Texts were available to students whenever they wanted to re-read them in order to answer the questions.
Metacomprehension accuracy was computed for each individual by correlating judgments of comprehension (metacomprehension questions) and test performance (comprehension questions) across the texts. The scores obtained in ECOMPLEC.Pri were distributed in two groups depending on their metacomprehension skills (Higher MC and Lower MC). Students' classification was established using an arbitrary criterion (an area of .30 around the perfect adjustment between metacomprehension skills and reading comprehension performance). This criterion was used to split the sample into those students that had a good adjustment between their performance and their predicted performance in the reading task (Higher MC) and those that did not (Lower MC). This classification was established on the basis of classical studies that operationalized metacomprehension accuracy as the correlation between predicted performance and actual performance (Nelson & Dunlosky, 1991).
The data analysis was conducted as followed. First, general metacomprehension was calculated to validate the classification criterion applied in this study. Second, different descriptive analyses, tests for the difference between mean text performances and Confirmatory Factor Analysis (CFA) were performed to validate current ECOMPLEC. Pri assessments. Third, different analyses were made to compare the relative performance of Higher MC and Lower MC groups in narrative, expository and discontinuous texts. Fourth, Differential Item Functioning (DIF) analyses were performed using generalized logistic regression method (Magis, Raiche, Beland, & Gerard, 2010), a method that detects both uniform and non-uniform DIF using a Likelihood Ratio Test between groups. Fifth, Higher MC and Lower MC groups were compared in each text for both text-base and situational-model questions. Sixth, age and sex effects in the performance were tested. All the statistical analyses were performed with R. Specifically, descriptive analyses were performed with basic R's functions, CFA was conducted with R's lavaan package (Rosseel, 2011), and DIF analyses were performed using the difR package (Magis, Beland, & Raiche, 2013).
In order to validate the classification criterion applied in this study, an analysis of general metacomprehension skills was conducted. General metacomprehension skills were calculated by relating a total score for each participant (i.e., the mean of narrative, expository and discontinuous text scores) with his or her answer to the metacomprehension questions (i.e., the mean answer for those questions). The polychoric correlation coefficient between both measures showed low metacomprehension (r=.21, p<.01). Mean proportion of correctly answered items was .71 (sd=.16). Using the classification criterion described above, the sample was subdivided by adjustment between their performance and their predicted performance in the task (i.e., whether they were judged to be Higher MC or Lower MC). This classification was satisfactory because the Higher MC group obtained a high polychoric correlation coefficient (r=.77, p<.01) while the Lower MC group obtained a low one (r=-.24, p<.01). On the contrary, the Higher MC group performance (.68, sd=.19), was similar to Lower MC group (.73, sd=.13).
Text descriptive analyses and factorial structure
To ensure that the analyzed texts were comparable in terms of their difficulty, total sample descriptive analyses and factorial structure for each text were calculated. Mean proportion of correctly answered items showed, approximately, the same difficulty levels for each text (see Table 1). Thus, they differed only in their contents and structural characteristics. Due to data non-normality, a DWLS estimator was used in the CFA analyses to test the unidimensional factorial structure for its non-probability density assumption. A good fit was observed for the narrative ([chi square](209) = 378.10, p < .01, CF I= .96, TLI = .95, RMSEA = .03 [.03-.04]), the expository ([chi square](189) = 306.40, p <. 01, CFI = .96, TLI = .96, RMSEA = .03 [.02-.03]) and the discontinuous texts ([chi square](252) = 210.80, p < 01, CFI = 1.00, TL I= 1.00, RMSEA = .00 [.00-.00]).
Metacomprehension measures and students' classification
In order to analyze the relationship between metacomprehension skills and comprehension performance, the total sample was divided into two groups (Higher MC and Lower groups). As it can be seen in Table 1, the total sample obtained low-to-medium metacomprehension skills in all the texts. Their answers to metacomprehension questions were not highly-related to comprehension performance (i.e., low-medium polychoric correlation coefficients were observed) within narrative-expository (r=.21, p<.05), narrative-discontinuous (r=.16, p<.05), and expository-discontinuous (r=.21, p<. 05) texts. The total sample did not show a good level of metacomprehension in the texts.
Then, students were classified by metacomprehension skills in each text using the criterion described above. As expected, the Higher MC group obtained high polychoric correlation coefficients for narrative, expository and discontinuous texts, while the Lower MC group obtained low polychoric correlation coefficients for narrative, expository and discontinuous texts. On the contrary, the mean performance of Higher MC and Lower MC groups presented some divergences. The higher MC group showed a higher level of performance than the Lower MC group in expository (t = -6.04, df = 818.95, p < .01) and discontinuous (t = -10.7, df = 708.21, p < .01) texts. On the contrary, the Higher MC group showed lower performance than the Lower MC group in narrative text (t = 19.5, df = 664.44, p < .01; see Table 1).
Results showed different patterns in the performance in reading tasks (i.e., proportion of correct answers) when metacomprehension skills were compared (specially comparing narrative with both expository and discontinuous texts). An important question in this study is to ask whether the students are stable in their level of metacomprehension in relation to performance across the texts. In other words, if students who score high or low in metacomprehension skills (Higher MC or Lower MC) in one text, also obtain the same score in the other texts. The answer that emerges from these data is that the stability of metacomprehension skills is only partial. Only 177 students (27.4 %) were classified in the same group (Higher MC or Lower MC) across all three types of text. When outcome data were paired by the different texts, results showed that 45.0% of the students in narrative-expository, 43.3% in narrative-discontinuous, and 55.8% in expository-discontinuous of the students were classified in the same category. These results suggest a strong dependence of type of text in metacomprehension skills, that is, metacomprehension skills are influenced by the type of text. Another plausible conclusion is that metacomprehension abilities would be unstable because they are yet immature in children and therefore they are not constant across types of text. Both conclusions could be interrelated.
Differential Item Functioning (DIF) analysis
Given that no differences were expected between Higher MC and Lower MC groups when their reading comprehension abilities were equated, a DIF analysis was conducted to analyze the hypothetical differences between the behaviors of students who are reading the texts. The generalized logistic regression method (Magis et al., 2010) detects both uniform and non-uniform DIF without requiring Item Response Theory models. Specifically, the DIF test was conducted using a Likelihood Ratio Test, and item purification and multiple comparisons were made with a Bonferroni correction. In the narrative text, 10 items presented uniform DIF. In the narrative text, only two items presented non-uniform DIF, and in the expository text no DIF was detected in discontinuous text. See Table 2 to see Chi-square (df=1) results for generalized logistic regression method to detect DIF in narrative, expository and discontinuous texts.
A final objective was to analyze whether metacomprehension skills are more closely related to text-base or situational-model mental representations of the text. The mean proportion of correct answers can be observed in Table 3 for the different questions of ECOMPLEC.Pri (i.e., those related to text-base and those related to the situational-model of the text). Different t-tests showed that the differences in metacomprehension skills between Higher MC and Lower MC groups were statistically significant because of the great number of participants of the present study. As it can be observed, expository and discontinuous texts showed an expected result: text-base questions were easier than situational-model ones, and Higher MC group obtained a higher performance in the comprehension task than the Lower MC group. On the contrary, the narrative text did not show differences between text-base and situational-model questions, and the Lower MC group obtained a higher performance in the comprehension task than the Higher MC group. There are differences among the type of text. Narrative resulted less complex to understand than expository and discontinuous texts in text-based as well as in situational model.
Analyzing items with and without DIF
In order to determine whether differences between the proportions of correct answers for items with and without DIF in each type of text were related to age or sex, different total scores for each combination were calculated and different multiple linear regressions were conducted (see Table 4). For narrative items with and without DIF, statistically significant effects were found only for age, but not for sex. No interaction effect was found. Similar results were found for expository items with DIF and without DIF, and for discontinuous items without DIF.
These results suggest that age and academic level are determinant regarding the degree of metacomprehension abilities. Children who were 11 years old obtained a higher level of metacomprehension skills than the younger children. Metacomprehension skills probably show a range of developments across children aged 9 to 11 years, and at lower levels of maturity, the metacomprehension skills are not as constant across types of text as they are for older children.
Metacomprehension skills determine an individual reader's ability to judge his or her degree of learning and text comprehension. Metacomprehension judgments usually are determined by asking children or adults to make a global assessment of their text comprehension, such as a general evaluation of their overall comprehension about a specific text. This measurement is limited because it is subjective and concerns a specific text-reader interaction. These are some reasons to anticipate the low correlations observed in some previous studies relating metacomprehension judgments and global assessment of text comprehension (i.e., Glenberg & Epstein, 1985). For this reason, the aim of the present study was to analyze whether three different types of texts (narrative, expository, and discontinuous) have a differential impacts on metacomprehension skills and on performance in text comprehension, and also whether metacomprehension skills are more related to text-base or situational-model mental representations of the text. Furthermore, we analyzed whether this impact on metacomprehension skills are equivalent in two different levels of primary school (4th and 6 (th,) from 9 to 11 years old), and how metacomprehension skills are consistent across the texts regarding Higher and Lower metacomprehension groups.
A first conclusion is that different types of text have a differential impact on metacomprehension skills and reading comprehension. Narrative texts produced better metacomprehension thanexpository and discontinuous. Expository texts were the most difficult. Several authors (e.g., Graesser et al., 1994; Leon et al., 2006; McNamara, Graesser, & Louwerse, 2012) have proposed that different mental processes are required in the processing of narrative texts. The comprehension of narrative texts is guided by both expectations and explanations given that higher-order knowledge structures are activated during reading. These text characteristics influence metacomprehension, and metacomprehension measures have been shown to be much more dependent on materials and individual skills than previously expected (Maki, Shields, Wheeler, & Zaccilli, 2005). In addition, the narrative text, unlike the other two texts, did not show the expected differences between text-base and situational-model mental representations. It could mean that the narrative text was less complex to understand than expository and discontinuous texts in text-based as well as in situational-model representations. Perhaps their content, very familiar events and situations occurring in a predictable manner, usually reflects daily life patterns which would more easily activate previous knowledge. In fact, the Lower MC group obtained higher performance in the comprehension task for the narrative text than the Higher MC group. Given the complexity of metacomprehension, it is probable that students evaluated their comprehension of the text-base while they were not aware of the quality of their text situation-model. A more familiar text-base, such as for the narrative text should bring differential estimations for students about their own self-performance because they would not be aware of the quality of their situation-model representation. These results are not obtained in the other texts in which the situational model was much more difficult to understand than the text-base, and thehigher metacomprehension group obtained better scores than did the lower metacomprehension group, especially for discontinuous text. Expository text was the most difficult to understand. These data support the idea that text difficulty affects metacomprehension accuracy and comprehension test performance in text learning (Vossing, StamovRo[sz]nagel, & Heinitz, 2017).
A second conclusion is that childrens' ages differentially affects the metacomprehension skills across the different levels of primary schools analyzed here (4th and 6th grades, from 9 to 11 years old). As expected, a clear effect of age and academic level was found for within-subject tests when metacomprehension skills were compared across types of text. 6th grade children scored better than 4th graders. This statement reveals the importance of the development in early ages in metacomprehension skills. Being a late-developing skill, metacomprehension reflects different maturation processes during individual development, and an appreciable improvement of metacomprehension skills is found in 10-12 years old students as some reading processes gradually become more automatic (Kolic-Vehovec & Bajsanski, 2006). These results are consistent with other studies that analyzed data from students between 8 and 12 years of age and found that older students knew more about text structures and that a progressive awareness of text characteristics was under development across that age range (i.e., Myers & Paris, 1978). But this level of metacomprehension skill is not consistent or stable across text types; the stability in metacomprehension skills is only partial. Only 27.4% of children were consistent in their metacomprehension skills across the types of text studied here. It could mean that children from 9-11 years are still immature in their level of metacomprehension skills (Destan, Hembacher, Ghetti, & Roebers, 2014). Developmental longitudinal studies are needed in order to analyze how metacomprehension skills change with age and, what differences could be observed across types of text due to educational training or personal preferences and abilities.
In conclusion, the present study demonstrates that the type of text can have a great impact on metacomprehension skills and, consequently, on comprehension and learning. Since these results have shown significant differences in terms of the type of text that is read, future research should be focused on the determination of variables that differentiate narrative texts from expository and discontinuous texts. For example, some variables from text or knowledge type (e.g., goal oriented or empathetic knowledge) can influence mental processes involved in reading comprehension and text metacomprehension. Once these variables are understood, different learning proposals could be developed to improve the acquisition of metacomprehension skills (Dunlosky & Lipko, 2007). The majority of metacomprehension research has been focused on expository texts for educational implications (Wiley et al., 2005), but further research on metacomprehension of narrative and discontinuous texts is needed and could be useful in wider educational applications.
Although researchers such as Weaver (1990) have demonstrated that metacomprehension accuracy improves when multiple comprehension items are given at test, other strategies are required in order to improve metacomprehension skills, especially in children. For example, Anderson & Thiede (2008) proposed that deeper comprehension tasks, such as text summarization, evoke more accurate metacomprehension judgments. Usually, summarization is considered to be a productive strategy for improving comprehension, at least due to its greater demands on the reader (Leon & Escudero, 2015). From a constructivist perspective, building explanations or producing written summary materials gives rise to and improves comprehension of texts more than other strategies. The combination of appropriate learning strategies and the study of text characteristics effects in metacomprehension could provide a fruitful context to improve students' metacomprehension skills.
This study was supported by Grant PSI2013-47219-P from the Ministry of Economic and Competitive (MINECO) of Spain, and European Union. We are thankful to Professor James Juola for his insightful comments and support.
Adesope, O.O., Trevisan, D.A., & Sundararajan, N. (2017). Rethinking the use of tests: A meta-analysis of practice testing. Review of Educational Research, 87(3), 659-701. https://dx.doi.org/10.3102/0034654316689306
Anderson, M.C., & Thiede, K.W. (2008). Why do delayed summaries improve metacomprehension accuracy? Acta Psychologica, 128(1), 110-118. https://dx.doi.org/10.1016/j.actpsy.2007.10.006
Carnevale, J. J., & Fujita, K. (2016). What does ego-depletion research reveal about self-control? A conceptual analysis. In E.R. Hirt, J. Clakson, & L. Jia (Eds.), Self-regulation and ego control (pp. 87-108). San Diego, CA: Academic Press.
Cook, A.E., & O'Brien, E.J. (2014). Knowledge activation, integration, and validation during narrative text comprehension. Discourse Processes, 51(1-2), 26-49. https://dx.doi.org/10.1080/0163853X.2013.855107
Diamond, A. (2012). Activities and programs that improve children's executive functions. Current Directions in Psychological Science, 21(5), 335-341. https://doi:10.1177/0963721412453722
Destan, N., Hembacher, E., Ghetti, S., & Roebers, C.M. (2014). Early metacognitive abilities: The interplay of monitoring and control processes in 5-to 7-year-old children. Journal of Experimental Child Psychology, 126, 213-228. https://dx.doi.org/10.1016/j.jecp.2014.04.001
Duckworth, A.L., & Seligman, M.E. (2005). Self-discipline outdoes IQ in predicting academic performance of adolescents. Psychological Science, 16(12), 939-944. https://dx.doi.org/10.1111/j.1467-9280.2005.01641.x
Dunlosky, J., & Lipko, A.R. (2007). Metacomprehension: A brief history and how to improve its accuracy. Current Directions in Psychological Science, 16(4), 228-232. https://dx.doi.org/10.1111/j.1467-8721.2007.00509.x
Dunlosky, J., & Metcalfe, J. (2008). Metacognition. London: Sage Publications.
Glenberg, A.M., & Epstein, W. (1985). Calibration of comprehension. Journal of Experimental Psychology: Learning, Memory, and Cognition, 11(4), 702-718. http://dx.doi.org/10.1037/0278-7393.11.1-4.702
Graesser, A.C., Singer, M., & Trabasso, T. (1994). Constructing inferences during narrative text comprehension. Psychological Review, 101(3), 371-395. https://dx.doi.org/10.1037/0033-295X.101.3.371
Kintsch, W. (1988). The use of knowledge in discourse processing: A construction-integration model. Psychological Review, 95, 163-182.
Kintsch, W. (1998). Comprehension: A paradigm for cognition. New York: Cambridge University Press.
Kolic-Vehovec, S., & Bajsanski, I. (2006). Metacognitive strategies and reading comprehension in elementary-school students. European Journal of Psychology of Education, 21(4), 439. https://doi.org/10.1007/BF03173513
Leon, J.A., & Escudero, I. (2015). Understanding causality in science discourse for middle and high school students. Summary task as a strategy for improving comprehension. In K.L. Santi & D. Reed (Eds), Improving comprehension for middle and high school students (pp. 75-98). Springer International Publishing: Switzerland. https://dx.doi.org/10.1007/978-3-319-14735-2_4
Leon, J.A., Escudero, I., & Olmos, R. (2012). ECOMPLEC. Evaluacion de la comprension lectora [ECOMPLEC. Reading Comprehension Assessment]. Madrid: TEA Ediciones.
Leon, J.A., Olmos, R., Escudero, I. Canas, J., & Salmeron, L. (2006). Assessing short summaries with human judgments procedure and latent semantic analysis in narrative and expository texts. Behavior Research Methods, 38(4), 616-627. https://doi.org/10.3758/BF03193894
Leon, J.A., Martinez-Huertas, J.A., & Jastrzebska, O. (2018). Un estudio sobre la competencia lectora en adultos con discapacidad intelectual y del desarrollo ante textos con contenidos de clinica y salud [A study on reading competence in adults with intellectual and developmental disability when they read clinical and health-related texts]. Clinica y Salud, 29(3), 115-123. https://doi.org/10.5093/clysa2018a17
McNamara, D.S., Graesser, A.C., & Louwerse, M.M. (2012). Sources of text difficulty: Across genres and grades. In J. Sabatini, E. Albro, & T. O'Reilly (Eds.), Measuring up: Advances in how we assess reading ability (pp. 89-116). Lanham, MD: R&L Education.
Magis, D., Beland, S., & Raiche, G. (2013). difR: Collection of methods to detect dichotomous differential item functioning (DIF) in psychometrics. R package version 4.5.
Magis, D., Raiche, G., Beland, S., & Gerard, P. (2010). A logistic regression procedure to detect differential item functioning among multiple groups. International Journal of Testing, 11(4), 365-386. https://dx.doi.org/10.1080/15305058.2011.602810
Maki, R.H., & Berry, S.L. (1984). Metacomprehension of text material. Journal of Experimental Psychology: Learning, Memory, and Cognition, 10(4), 663-679. http://dx.doi.org/10.1037/0278-7393.10.4663
Maki, R.H., Shields, M., Wheeler, A.E., & Zacchilli, T.L. (2005).
Individual differences in absolute and relative metacomprehension accuracy. Journal of Educational Psychology, 97(4), 723-731. https://dx.doi.org/10.1037/0022-0618.104.22.1683
Madariaga, J.M., & Martinez, E. (2010). The teaching of reading comprehension and metacomprehension strategies. A program implemented by teaching staff. Annals of Psychology, 26(1), 112-122.
Martinez-Huertas, J.A., Jastrzebska, O., Mencu, A., Moraleda, J., Olmos, R., & Leon, J.A. (2018). Analyzing two automatic assessment LSA methods (Inbuilt Rubric vs. Golden Summary) in summaries extracted from expository texts. Psicologia Educativa, 24(2), 85-92. https://doi. org/10.5093/psed2048a9
Myers, M., & Paris, S.G. (1978). Children's metacognitive knowledge about reading. Journal of Educational Psychology, 70(5), 680-690. https://dx.doi.org/10.1037/0022-0622.214.171.1240
Nelson, T.O., & Dunlosky, J. (1991). When people's judgments of learning (JOLs) are extremely accurate at predicting subsequent recall: The "delayed-JOL effect". Psychological Science, 2(4), 267-271. https://dx.doi.org/10.1111/j.1467-9280.1991.tb00147.x
Nigg, J.T. (2017). Annual Research Review: On the relations among self- regulation, self- control, executive functioning, effortful control, cognitive control, impulsivity, risk- taking, and inhibition for developmental psychopathology. Journal of Child Psychology and Psychiatry, 58(4), 361-383. https://dx.doi.org/10.1111/jcpp.12675
Olmos, R., Leon, J.A., Martin, L.A., Moreno, J.D., Escudero, I., & Sanchez, F. (2016). Psychometric properties of the reading comprehension test ECOMPLEC. Sec. Psicothema, 28(1), 89-95. https://dx.doi.org/10.7334/psicothema2015.92
Organization for Economic Cooperation and Development - OECD (2010). PISA 2009 Results: What Students Know and Can Do - Performance in Reading, Mathematics and Science (Volume I). Paris: OECD.
Rosseel, Y. (2011). Lavaan: An R Package for Structural Equation Modeling. Journal of Statistical Software, 48(2), 1-36. https://dx.doi.org/10.18637/jss.v048.i02
Vossing, J., Stamov-Ro[sz]nagel, C., & Heinitz, K. (2017). Text difficulty affects metacomprehension accuracy and knowledge test performance in text learning. Journal of Computer Assisted Learning, 33(3), 282-291. https://dx.doi.org/10.1111/jcal.12179
Walczyk, J.J. (1990). Relation among error detection, sentence verification, and low-level reading skills of fourth graders. Journal of Educational Psychology, 82(3), 491-497. https://dx.doi.org/10.1037/0022-06126.96.36.1991
Weaver, C.A. (1990). Constraining factors in calibration of comprehension. Journal of Experimental Psychology: Learning, Memory, and Cognition, 16(2), 214-222
Wiley, J., Griffin, T.D., & Thiede, K.W. (2005). Putting the comprehension in metacomprehension. The Journal of General Psychology, 132(4), 408-428. https://dx.doi.org/10.3200/GENP.132.4408-428
Jose Antonio Leon (1), Jose Angel Martinez-Huertas (1), Ricardo Olmos (1), Jose David Moreno (1), and Inmaculada Escudero (2)
(1) Universidad Autonoma de Madrid and (2) UNED
Received: May 30, 2018 * Accepted: November 13, 2018
Corresponding author: Jose Antonio Leon
Facultad de Psicologia
Universidad Autonoma de Madrid
28049 Madrid (Spain)
Table 1 Metacomprehension and reading performance mean in narrative, expository, and discontinuous texts in total sample, and both, Higher MC and Lower MC groups Narrative text Expository text MC N Mean SD MC N Mean SD Total sample .00 823 .74 .18 .18 (*) 823 .63 .18 Lower MC -.02 579 .80 .17 -.26 (*) 420 .59 .19 Higher MC .83 (**) 244 .60 .11 .88 (**) 403 .66 .17 Discontinuous text MC N Mean SD Total sample .32 (*) 823 .74 .20 Lower MC -.06 378 .67 .21 Higher MC .97 (**) 445 .81 .16 Note: MC = Metacomprehension. Lower MC = Lower metacomprehension skills group. Higher MC = Higher metacomprehension skills group. N = Number of subjects. SD = Standard deviation. (**) = p<.01. (*) = p<.05 Table 2 Chi-square (df=1) results for generalized logistic regression method Text Item DIF [chi square] N.[R.sup.2] Narrative 3 Uniform 23.0 (**) .024 Narrative 8 Uniform 18.6 (**) .024 Narrative 11 Uniform 22.3 (**) .028 Narrative 13 Uniform 15.0 (**) .018 Narrative 14 Uniform 26.2 (**) .035 Narrative 15 Uniform 37.2 (**) .048 Narrative 19 Uniform 21.1 (**) .023 Narrative 20 Uniform 15.3 (**) .017 Narrative 21 Uniform 16.4 (**) .030 Narrative 22 Uniform 13.4 (**) .027 Expository 9 Non-uni 19.4 (**) .027 Expository 21 Non-uni 12.7 (*) .018 Note: (*) = p <. 05. (**) = p < .01. N.[R.sup.2] = Nagelkerke's [R.sup.2]. Non-uni = Non-uniform DIF. No DIF was detected in the discontinuous text Table 3 Mean proportion of correct answers of Higher MC and Lower MC groups in text-base (TB) and situational-model (SM) questions for each text Lower MC Higher MC Text Mental Items Items Items representation with DIF without DIF with DIF Narrative TB .82 (N=2) .81 (N=9) .50 (N=2) SM .83 (N=8) .72 (N=3) .60 (N=8) Expository TB .72 (N=1) .64 (N=10) .84 (N=1) SM .41 (N=1) .53 (N=10) .41 (N=1) Discontinuous TB -- .69 (N=10) -- SM -- .65 (N=14) -- Higher MC Text Items without DIF Narrative .64 (N=9) .56 (N=3) Expository .71 (N=10) .61 (N=10) Discontinuous .77 (N=10) .73 (N=14) Note: Lower MC = Lower metacomprehension skills group. Higher MC = Higher metacomprehension skills group. TB= Text-based. SM= Situational Model. N = Number of items. Empty cells = No item had DIF Table 4 Multiple linear regression [beta] coefficients for predicting mean performance (mean proportions of correct answers) Type of Items Age main effect texts Narrative With DIF [beta] = 1.54, t(819) = 7.32 (*) Narrative Without DIF [beta] = 1.45, t(819) = 7.46 (*) Expository With DIF [beta] = .27, t(819) = 4.11 (*) Expository Without DIF [beta] = 2.28, t(819) = 7.36 (*) Discontinuous Without DIF [beta] = 3.07, t(819) = 6.85 (*) Type of Sex main effect Interaction effect texts Narrative [beta] = -.09, t(819) = -.35 [beta] =.14, t(819) = .43 Narrative [beta] = -.31, t(819) = -1.27 [beta] =.19, t(819) = .64 Expository [beta] = .11, t(819) = 1.34 [beta] =-.01, t(819) = -.11 Expository [beta] = -.14, t(819) = -.37 [beta] =.37, t(819) = .78 Discontinuous [beta] = -.26, t(819) = -.47 [beta] =.18, t(819) = .26 Note: (*) = p < .01. Non-standardized [beta] coefficients. DIF
|Printer friendly Cite/link Email Feedback|
|Author:||Leon, Jose Antonio; Martinez-Huertas, Jose Angel; Olmos, Ricardo; Moreno, Jose David; Escudero, Inma|
|Date:||Jan 1, 2019|
|Previous Article:||Type of memory and emotional valence in healthy aging, mild cognitive impairment, and Alzheimer's disease.|
|Next Article:||Cognitive processes of emotional regulation, burnout and work engagement.|