A summary test: criterion for majoring in the English language and literature at the University of Botswana.
English Second Language students who have just entered the world of academia are faced with a daunting task of learning in another language. For the majority of students at the Colleges of Education and the University of Botswana, English is the second language. However, for some, such as the San peoples, it is a third language (Ketsitlile, 2009). Current research on summarising strategies has shifted the focus on the native English speakers to those used by ESL/EFL students at tertiary level. 'The focus on the latter is a result of the realisation that such students are not only linguistically 'under-prepared', but are also faced with the task of decoding the meaning of texts written in a specialised register' (Johns & Mayes, 1990). All first year students at colleges of education and the University of Botswana are required to go through a Communication and Study Skills course which focuses on speaking, listening, writing, reading and academic literacy in general. This is done in order to prepare students for the rigours of academic learning and scholarship by sharpening their skills. However, faculties at the University of Botswana continue to complain that students are lacking in study skills, notably, in writing and reading. Students should be able to read with full understanding at college or university level. This is crucial for their academic success. They need to read print and electronic texts widely in order to do assignments, tests and projects. Also, the skill of summary writing is important during lectures. This is particularly important as the preferred mode of teaching at most universities in developing countries is the lecture method. The ability to summarise is therefore crucial for academic learning. Hence, ESL learners need to be scaffolded in their language skills, even at college and university levels (Oxford, 1996; Chamot, 1996).
Current research on summarising strategies has shifted attention from native speakers of the English language to ESL/EFL learners in colleges and universities (Chimbganda, 2003). Chimbganda posits that "The focus on the latter is a result of the realisation that such students are not only linguistically 'under-prepared', but are also faced with the daunting task of decoding the meaning of texts written in a specialised register' (p. 30). Johns and Mayes (1990) are of the view that students who struggle with decoding and making meaning from texts grapple with condensing and synthesising ideas when they read and write.
As teachers and researchers, we most often judge students on whether they can summarise or not, without looking at the very nature of summarising itself: what does it involve? In teaching and in testing, we should be aware of the steps or processes involved in summarisation. The basic understanding of a summary is captured very well by Robinson and Zhu (2004) Http://www.york. ac.uk/media/educationalstudies/documents/research/EnglishGrammar(syntax) pdf Retrieved, June 20th, 2012, when they say that 'students are taught to find the big ideas in paragraphs and in longer passages by deleting trivial information. They go on to say that good readers should be able to infer the following: 'referents of pronouns, meanings of unknown vocabulary, subtle connotations in text, explanations for events described in text, how ideas in a text relate to one's own opinions and theories, the author's purposes in writing the text, the author's assumptions about the world, the text characters' intentions and characteristics...conclusions suggested by the text.'
Also of interest are the mental processes a learner engenders when summarising (O'Malley & Chamot 1983, 1990; Johnson, 1996). Chamot and Kupper (1989) have developed the idea of an 'adaptive control of thought' and concluded that there are three thought processes associated with 'adaptive control': cognitive smeta-cognitive and social affective. It is important to note that cognitive and meta-cognitive processes are not that easy to distinguish, but O'Malley and Chamot (1990: 150) give useful definitions, which help with clarity as far as these terms are concerned. They define 'cognitive processes' as 'the steps or preparations used in learning or problem solving that require direct analysis, transformation, or synthesis of learning materials; and meta-cognitive processes as 'the knowledge about cognitive processes and the regulation of cognition or executive control, or self-management through processes such as planning, monitoring and evaluation". O' Malley and Chamot (1983:6) conclude that students without meta-cognitive approaches are essentially learners without direction to review their progress, accomplishments and future learning directions.
For a summary to be effective and useful for a student, it must be written in the writer's own words and not transported verbatim from the text to the page (Nist & Simpson, 2000). When students use their own words and thinking when summarising, this helps to improve not only their summary writing strategies but also their understanding. In addition, '...students. form connections across the concepts and relate the concepts to their own prior knowledge and experiences. Such a definition of summarisation implies that it is not a strategy quickly mastered' (Pressley et al., 1997). Research points to the fact that when students are taught to summarise, their performance and summary writing ability generally improves (Harris, 1991; Strode, 1991).
In writing this paper, we were guided by cognitive and meta-cognitive processes. Recently, researchers in the field of reading have moved away from solely focusing their research on a particular strategy, but more on processes (Pintrich, Smith, Garcia, & McKeachie, 1993; Nist & Simpson, 2000). The latter go on to posit that cognitive and meta-cognitive processes are important in students' reading and writing. Students, especially in colleges and university, should be able to select and transform ideas, organise, elaborate, monitor, plan, and evaluate (Hadwin & Winne, 1996, cited in Nist & Simpson, 2000, p. 657). Students should focus not only on the strategies, but also on the processes, as all these result in better understanding of a text and, consequently, improved writing.
Purpose of the test
Our test, which requires students to summarise from a passage, is intended to be used for promotion from first year to post first year for students who wish to major in the English Language and English Literature at the University of Botswana. It is a major concern of the English Department that first year students enrolled in English Language courses are 'limping' academically. One of us is a former coordinator of Communication and Study Skills in the Faculty of Humanities and used to sit in meetings where members brainstormed on the problem of students' academic inadequacies, especially those pertaining to reading and writing. There is still concern that these shortcomings affected their performance, especially those majoring in the English Language and Literature. In such meetings, it was evident that the problems were not peculiar to first year students only, but also post first year students too.
Interviews with selected lecturers
The informal interview protocol was employed to get ample information from lecturers in the Faculty of Humanities and in the Communication and Study Skills Unit (Chilisa & Preece, 2005). Lecturers were interviewed on students' writing and reading abilities (including cognitive and meta-cognitive abilities), their performance in post year 1 English Language and Literature courses and their views on testing them as a criterion for majoring in the English Language and Literature courses at the University of Botswana. A total of five lecturers were interviewed. The interviews were transcribed.
The lecturers interviewed had the following to say:
Lecturer 1: It is important that students go through an entry test before majoring in the English Language and Literature. Students need to be able to understand texts and transfer what they have read onto paper, using their own words as much as possible. This shows critical reading and understanding, which is important for college and university learning, especially for those who want to major in the English Language and Literature. Other universities have the entry test. The same should be done at U.B.
Lecturer 2: It is important because many students do not do literature in secondary schools. Testing them will reveal their cognitive and meta-cognitive skills, level of interest and readiness to major in the English Language and Literature. Currently, there is pressure from the Botswana Government for students to pass.
Lecturer 3: The proposed entry test is very important, not only in the English Department, but also in secondary schools. After administering the test, teachers and lecturers can know the general weakness of the students and this will assist greatly in improving students' shortcomings. The current problems that university students in general face in their academic work stem from secondary school. Students do not read: there is no reading culture. Hence, this impacts negatively on their reading, vocabulary and writing. Clearly, the Ministry of Education should do something about this problem.
Lecturer 4: I And this proposed test to be extremely important. It will determine whether post year 1 students will be able to read and understand texts or not. It will indeed test their competence in the English Language: written and spoken language including vocabulary.
Lecturer 5: The test is a must. Currently, we are having some students who are majoring in the English Language and Literature and are finding it very difficult to keep up with the amount of reading and writing. I think these students' problems are inherited from secondary school. I suggest the Ministry of Education relook at the English syllabus carefully.
Characteristics of test takers
The proposed test takers are young adults of ages ranging from 18-20 (these are the ages when students in Botswana enrol for tertiary education). Almost all of them are fluent in the national language, Setswana. Setswana is the lingua franca in Botswana (Nyati-Ramahobo, 2004) and all students in Botswana learn Setswana from Standard 1 (Grade 1). English is used as the official language and the language of school, business and prestige. All students need to pass English with a minimum pass of 'C' to be admitted at university. Students come from a secondary school culture where they were told almost everything they needed to know by the teacher (Prophet & Rowell, 1990). Studies have shown that such a mode of teaching does not develop critical thinking skills, creativity and strategy building (Ketsitlile, 2009; Semali & Kincheloe, 1999; Shiva, 1993). Hence, there is a general concern at the University of Botswana that students do not read as widely as expected, because the very necessary culture of reading was not reinforced at the primary and secondary levels (Arua & Lederer, 2003; Ketsitlile & Galegane, 2010).
Usefulness is an essential consideration in all stages of test development. In developing the test, we plan to be very careful and address issues of reliability, validity, authenticity, impact, practicality and interactiveness.
Reliability and validity: 'Reliability is often defined as consistency of measurement. A reliable test score will be consistent across different characteristics of the testing situation' (Bachman & Palmer, 2006: 19). Thus, in order to cater for reliability, we will pilot the test among two groups of students with similar characteristics as our test takers in terms of age, gender, education, languages spoken and others. Each group will consist of 50 students chosen randomly from about two hundred college or university students. Students should be able to take the test in either Group A or B. Our intention is that it should make no difference to the student in terms of scoring. This also calls for consistency in marking by the lecturers, for the test to be deemed reliable. The reliability coefficient of 0.88 is what the test developer is hoping for. The marking key used will be a standard uniform one, so that there are no huge discrepancies in marking. We concur with Shadish and Luellen (2002) that in order to correctly measure the cause and effect relationship, it is important to be able to compare one group to the group that received treatment. Hence, control groups can be used that are from the same population and match perfectly, as in our groups for test development: the students are of similar ages, demographics, there is the same number of subjects in each group and they are of similar or related cognitive development.
Before the actual test, we plan to conduct a pre-test on the two groups to come up with an accepted statistical baseline. We thought it would be best to employ random sampling to the groups so that the students do not know which group they are in. This, in our view, will eliminate bias. In the study carried out by Kuhn (2006) in twenty-four second grade classrooms in New Jersey and Georgia on children's' fluency, the researchers used a range of control classrooms at the two sites they chose for the study. They employed grouping formats by the use of whole-class and small group instruction, to match proposed instructions: shared reading, reading workshops and guided reading. We found this to be commendable and an important characteristic of quasi-experiments. To cater for threats to validity and reliability, remedial instruction of 45 minutes per day was adopted. We feel this was a welcome idea, as it increased the chance of reliability in the findings. The same is true for our summary test. The quasi-experimental research (use of control groups) minimises extensively the threats of validity and reliability by not allowing any spurious variables. According to Vellutino and Schatschneider (2004. p. 119).
"The validity of inferences about cause-effect relationships is dependent on the degree to which the researcher demonstrates that: (1) A treatment preceded an observed effect, (2) the treatment is related to the effect, and (3) there are no plausible explanations for the effect. However, a lot depends on the researcher(s) taking care that the necessary treatments are administered before even measuring the effects of the treatment".
Vellutino and Schatschneider (2004. p. 121) go on to say that low power results when the following is the norm:
a. Violation of assumptions of statistical tests
b. Unreliability of measurement
c. Restriction of range
d. Fidelity of treatment problems.
Construct and content validity: 'In order to justify score interpretation, we need to provide evidence that the test score reflects the area(s) of language ability we want to measure and very little else' (in our case, summarising strategies) (Bachman & Palmer, 2006).
We plan to ask for funding from the Office of Research and Development at the University of Botswana to enable all markers and test administrators to converge for a two day workshop. The workshop will discuss logistics such as pre-test preparation (test rooms, stationery, administration of the test, behaviour of administrators towards test takers during the test and other pertinent issues in test administration). After administering the test, we will meet with the markers for standardisation. The dummy marking based on the pilot test will iron out a lot of problems and inconsistencies before the actual test. This meeting will also help with other forms of feedback such as results from students rating scales, future tests, and issues affecting test takers, markers and test administrators.
Test task: The test task is on a passage of reading. It is about Capital punishment: for and against. The students are required to read the passage carefully and summarise using their own words as much as possible, the reasons for and against capital punishment in not more than 200 words.
Instructions: The instructions are going to be specific and communicate to the test taker exactly what needs to be done: to read the passage carefully and summarise using their own words as much as possible, the reasons for and against capital punishment in not more than 200 words.
Time: A very important element in test taking. The students are given 40 minutes to read the passage. The first reading is to get the general idea of what the passage is about and the second one is for understanding. Then, they have a further 40 minutes to plan, organise and produce the final product: the summary.
This test is very relevant for students at college or university. It is linguistically appropriate, cognitively challenging and also relevant content-wise. This is a kind of test and text (language comparable) the learners will come across in content books. As a result, it is very authentic.
Scoring: Researchers such as Carrell (1989) and others classify the production of main ideas at three levels: correct, partly correct and incorrect idea units. The level of copying and paraphrasing should be considered too. For scoring cognitive and meta-cognitive strategies, a taxonomy that was adapted from Li and Munby (1996) and Chimbganda (2003) is very useful. The strategies markers should be looking for are:
a. Note-making: picking out main points
b. Grouping of main points
c. Contextualisation of information appropriate.
a. Planning of task
b. Self-checking and evaluation: does the final product make sense? Are all relevant points included?
All these will result in a more interactive assessment.
Impact: This test will have a positive or negative impact on the learners. Those who pass can major in English or English Literature, or both. At the macro level, it will have an influence on how teachers structure the curriculum: it will inform teachers in which areas of reading students are experiencing difficulties: Is it in understanding a text, picking out main points or in the actual writing?
At curriculum planning level, the test will have an impact on policy makers' decisions about the English curriculum. Reading education should be taught with earnest in all grades and at college and the university, not only in Botswana, but the rest of Sub-Saharan Africa.
Practicality: We plan to be as frugal as possible in the use of available resources. Test markers are already on the university payroll. The test administrators will be paid a small honorarium by the Office of Research and Development. Also, rooms used for testing are those for lecturing. Students will be told to bring their own stationery.
Students at college and university need to be able to summarise efficiently and effectively. Such knowledge is important not only for understanding, inferring and decoding of a text, but, also in writing. Hence, the summary test is a move in the right direction for those students who wish to major in the English Language and Literature at the university. In order to develop a test successfully, one should consider the purpose for the test foremost. The target population (age, language, needs and affective schemata) is important too. It is also important to think very carefully about the format of the test, time for writing the test, scoring, and issues of reliability, validity, authenticity, impact and practicality. Pre and post-test dialogue amongst students, markers and test administrators is a must to pave the way for future testing. This test is very important in ensuring that capable students progress to post 1 year courses in the English Language and Literature at the University of Botswana. This will, without doubt, save students, their parents and lecturers the trauma of learners who have to retake courses or worse, drop out of university, because of low grades in the English Language and Literature courses at the university.
From the interviews with selected lecturers in the English Department and Communication and Study Skills Unit, it has come out clearly that there is a need for an entry test to be administered to students wanting to major in the English Language and Literature at the University of Botswana. We found the summary test to be ideal, as it will engage the students cognitively and meta-cognitively. The following recommendations are proposed:
* Students are to be tested before they can be allowed to major in the English Language and Literature at the University of Botswana.
* The English Department at the University of Botswana and the Communication and Study Skills Unit lecturers need to work together in adequately preparing students for the rigours of academic learning, not only those intending to major in the English Language and Literature, but all students.
* It is imperative that the Ministry of Education should closely relook at the English Language and Literature syllabus with a view on improving students' general competency in the English language. In addition, the culture of reading needs to be reinforced from pre-primary in Botswana schools and homes. We recommend a policy to this effect.
* The Office of Research and Development at the University of Botswana is to avail funds for pre and entry tests, not only in the English Department, but across departments and faculties.
Arua, E. A. & Lederer, M. (2003). What are students in Botswana's high schools reading?
Arua (Ed.) Reading for All in Africa: Building Communities Where Literacy Thrives (pp. 26-29). Newark, DE: International Reading Association.
Bachman, L.F. & Palmer, A.S. (2006). Language testing in practice: Designing and developing useful language tests. Oxford: Oxford University Press.
Carrell, P.L. (1989). Metacognitive awareness and second language reading. The Modern Language Journal, 73: 121-134.
Chamot, A.U. (1996). In R.L. Oxford (ed.), Language learning strategies around the world: Cross cultural perspectives, 167-73.
Chamot, A.U. & Kupper, L. (1989). Learning strategies in foreign language instruction. Foreign Language Annals, 22, 13-24.
Chilisa, B., & Preece, J. (2005). Research methods for adult educators in Africa. Cape Town: Pearson Educational South Africa.
Chimbganda, A.B. (2003). Summarizing strategies used by low and high proficiency ESL first year science students of the University of Botswana: A cognitive and meta-cognitive analysis. Botswana Journal of Applied Linguistics. Vol. 1, 28-47.
Harris, D. (1990). The use of "organising sentences" in the structure of paragraphs in science textbooks. In U. Connor and A. Johns (eds.), Coherence: Research and Pedagogical Perspectives. Washington, DC: TESOL.
Johnson, K.L. (1996). Language teaching and skill learning. Oxford : Blackwell.
Johns and Mayes (1990). 'An analysis of summary protocols of university students.' Applied Linguistics 11/3: 254-271.
Ketsitlile, L.E. (2009). San junior secondary school understandings of literacy in school and at home: A case study. Doctoral Dissertation. University of Georgia, USA.
Ketsitlile, L. E. & Galegane, G. (2010). A study of the extent to which first year humanities students at the University of Botswana read for leisure. LWATI: A Journal of Contemporary Research, 7(3), 44-57.
Kuhn, M.R. (2006). "Teaching children to become fluent and automatic readers." In Journal of Literacy Research, 38(4), pp. 357-387.
Li, S. & Munby, H. (1996). Metacognitive strategies in second language academic reading: A qualitative investigation. English for Specific Purposes 15/3 199-216.
Nist, S.L. & Simpson, M.L. (2000). College Studying. In Kamil, M.L., Mosenthal, P.B., Pearson, & R. Barr (Eds.), Handbook of Reading Research (Vol. 3). LEA: New Jersey.
Nyati-Ramahobo, (2004). Language planning and policy in Africa. In Richard B. Baldauf, Jr. & Robert, B. Kaplan (Eds.), Language planning and policy in Africa, 1. (21-78). Clevedon: Multilingual Matters LTD.
O' Malley, J.M. & Chamot, A.U. (1983). A study of learning strategies for acquisition skills in speaking and understanding English as a second language. Rosslyn, VA: Inter America Research Associates.
O'Malley, J.M. & Chamot, A.U. (1990). Learning strategies in second language acquisition. NY: Cambridge University Press.
Oxford, R.L. (ed.). (1996). Language learning strategies around the world: cross cultural perspectives. University of Hawaii at Manoa: Second language teaching and curriculum center teaching report 13. Rosslyn, VA: Inter America Research Associates.
Semali, L., & Kincheloe, L. (1999). Introduction: What is indigenous knowledge and why should we study it? In L. Semali, J.C. Kincheloe (Eds.), What is indigenous knowledge? Voices from the academy. New York: Falmer Press.
Shadish, W.R. & Luellen, J.K. (2002). Experimental quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin Company.
Shiva, J. (1993). Monocultures of the mind: Perspectives in biodiversity & biotechnology. London: Zed Books.
Strode, (1991). Teaching annotation writing to college students. Forum for Reaching, 23, 33-44.
Pintrich, P.R., Smith, D.A., Garcia, T., & McKeachie, W.J. (1993). Reliability and predictive validity of the Motivation Strategies for Learning Questionnaire (VMSLQ). Educational and Psychological Measurement, 53, 801-813.
Pressley et al., (1997). Some of the reasons preparing for exams is so hard: What can be done to make it easier? Educational Psychology Review, 9, 1-38.
Prophet, R. B., & Rowell, P. M. (1990). The curriculum observed. In C. W. Snynder and P. T. Ramatsui (Eds.), Curriculum in the classroom: Context of change in Botswana's Junior Secondary School Instructional Program, 1-56. Gaborone: Macmillan.
Robinson, A. & Zhu, D. (2004). Http://www.york.ac.uk/media/educationalstudies/ documents/research/EnglishGrammar(syntax)pdf Retrieved, June 20th, 2012.
Vellutino, F.R. & Schatschneider, C. (2004). "Experiental and Quasi Experimental Design in Literacy Research." In Literacy Research Methodologies, (Ed.), N.K. Duke & Mallette, M.H. New York: Guildford Press.
Lone Elizabeth Ketsitlile is an Associate Professor in the Communication and Study Skills Unit, University of Botswana. In 2009, Prof. Ketsitlile graduated with a PhD in Reading Education from the University of Georgia, US, with the assistance of a Fulbright Scholarship. Her research areas of interest are in reading and writing education, indigenous literacy and research of the San peoples of Botswana and transformative research methodologies in researching (with) the 'Other.'
Philip Bulawa is currently a lecturer in Educational Management at the University of Botswana. He received his PhD in Educational Management with a focus on the implementation of the Performance Management System (PMS) in senior secondary schools from James Cook University in Australia. His research areas of interest include exploring issues of policy development, analysis, implementation & evaluation, as well as professional development in education.
|Printer friendly Cite/link Email Feedback|
|Author:||Ketsitlile, Lone Elizabeth; Bulawa, Philip|
|Publication:||Nawa: journal of language and communication|
|Date:||Dec 1, 2012|
|Previous Article:||Derivation and adaptation of acronyms in Ndebele.|
|Next Article:||Is "English-centric bilingualism" suffocating Namibian national and indigenous languages?|