Philosophy and measurement of school achievement.

There are diverse schools of thought on how pupils should be measured or evaluated to show academic achievement in the school setting. Teacher written tests have been used for a least 150 years. There appears, in the past, to have been a lack of knowledge on basic techniques to use in measuring/evaluating learner achievement and progress. Presently, there are student university level textbooks containing guidelines for teachers to use in writing test items. Thus, classroom teacher may receive much assistance then in writing diverse kinds of test items.

Teacher Written Tests

High quality teacher written tests might well possess the best validity of all test writers since they are right in the classroom and might well write test items pertaining to what was taught whereas test writers for state mandated tests are rather far removed from the local classroom. The state writers of test items do not know the children for whom the test was written and thus, cannot provide for individual differences among learners who will be taking the test. Validity is a very salient concept in testing pupils in that learners need to have opportunities to achieve what is in the state mandated objectives. The state mandated objectives can become more valid if the state standards or objective directly relate to the taught curriculum. The classroom teacher may then align the written test items with the stated objectives, in a unit of study.

Test items written by the teacher to cover what was taught might consist of true-false, or multiple choice items. If the test items covered by the teacher in teaching pupils are of high quality, meaning they are very clearly written and no guesswork is involved in their interpretation, high reliability should then also be in the offing. This means that pupils should receive a similar score the second time the same test is taken. With quality reliability then, the test measures consistently when pupils take the same test twice (test/retest reliability), unless learner fatigue sets in.

There is also split half reliability when the test is taken only once and the odd numbered items are compared with the even numbered test items in terms of correct responses from pupils taking the test. The resulting statistic is called split half correlation.

A reliability figure can come about also when a comparison between alternative forms of a test taken by a set of pupil is made. The alternative forms reliability, if high, indicates the two tests are of comparable difficulty and cover similar subject matter from the same unit taught. No matter on what level, local or state, the test items are written, they should possess quality validity and reliability. Again, validity pertains to subject matter taught and measured in terms of learner achievement. Validity then emphasizes what was taught, pupils had opportunities to learn. What is taught then needs to relate to the identified objectives of instruction. Hopefully, very important objectives were emphasized in teaching pupils. Reliability stresses obtaining consistency of pupil results from testing. Inconsistent test results means that a pupil was, for example, on the 25th percentile the first time a test was taken and the eightieth percentile the second time the same test was taken. This does not tell us anything about a pupil's performance or achievement. If a pupil is on the fiftieth percentile both items the same test was taken, then out of every 100 pupils who took the same test, fifty were above and fifty below the median (or average) percentile. The mean is the average score for all pupils taking the test whereas the median is the middle most score. The mode is the most frequent score for those who took the test.

The early years of the twentieth century were conspicuous in the application of science to all phases of business and industry. It was applied not merely to the invention of new products and processes but to the details of organization and management designed to promote economy and efficiency. Experts trained in "scientific management"' studied carefully the performances of workers on the job with results so fruitful in economy and efficiency that many can be seen in "job analysis" possibilities of application not only to vocational education but to the reform of other aspects of education as well. All that was needed, it seemed, was to identify the specific outcomes by assuring that pupils engage in the activities certain to eventuate in the proper habits and skills, information, attitudes, and the like. The new emphasis upon the selection and organization of subject matter received definite formulation in 1918 with the National Commission for the Reorganization of Secondary Education. The commission stated the areas of concern or the functions of the secondary school, in the form of the Cardinal Principles of Education. Its preliminary statement read: In order to determine the main objectives that should guide education in a democracy, it is necessary to analyze the activities of the individual. Normally he is a member of a family, of a vocational group, and of various civic groups, and by virtue of these relationships, he is called upon to engage in activities that enrich family life. To render important vocational services to fellows, and to promote the common welfare." (Thayer, V. T. (1970).

Commercial Tests Used to Measure Pupil Achievement

Commercial companies develop standardized achievement tests, also called norm referenced tests (NRT). These are used rather frequently to measure pupil achievement. There are different elements that go into the development of standardized achievement tests since the conditions for test taking are to be the same for all pupils. The following factors are then to be the same:

* time limits enforced for taking each test.

* subject matter tested on for each set of pupils taking the designated test.

* directions given to pupils for test taking.

Professional writers write the test items for a specific curriculum area or areas. These test items then are responded to by pupils in a pilot study. Feedback from the pilot study provides knowledge, to the test writers, of the quality of each test item. Computerized scoring using the same key further standardizes the test in that conditions are the same for all involved in taking the test. When writers of the test items see the computerized printout from pilot study results, they notice which test item was too easy and all responded correctly to that test item. Or the other extreme whereby all responded incorrectly to a test item. Where all responded correctly or incorrectly to the same test item makes it either too easy or too difficult. Then too, pupils results are not spread out adequately. Generally, the lowest and the highest 5th percentile scores in a pilot study are eliminated to take care of extreme scores. When an adequate number of pilot studies have been run, then a spread of scores will result from the first to the 99th percentile. Standard tests are designed to spread out pupils' scores from high to low. Each academic or curriculum area will have data on pupils' scores and their equivalency in the Manual of the standardized test.

Quality validity and reliability will be discussed in the manual and how the were determined. The test writers are quite far removed from the local classroom in ascertaining which test items should be written and included on the test. The test will be administrated in various parts of the United States to pupils from diverse levels of achievement. Local pupils who have taken the standardized test will have their scores compared with those in the Manual (who were in the final pilot study) to notice the related percentile level.

Criterion Referenced Tests

To take care of the predetermined methods of spreading scores out from high to low, as did advocates of standardized testing, criterion referenced testing (CRT) was invented and brought into the measurement movement. The CRTs have objectives available which teachers may use as guidelines for instruction. CRTs are pilot tested to take out vaguely written test items. Clearly written, meaningful test items should be in evidence. As is true of standardized tests, CRTs also tend to use multiple choice test items Together with the stem, each distractor of the four in a multiple choice test item needs toe plausible. The four responses should be of similar length. No, clues should be given as to which is the correct response.

Learning opportunities selected by the teacher should be aligned with the objectives of instruction. After an instructional period of time, the CRT is used at a designated interval to measure pupil achievement. The state generally determines when the tests are to be administered. These tests are to be administered on grades three through eight. There might be cutoff points for a pupil passing CRT tests for each grade level in order to be promoted to the next higher grade level. Exit exams of the CRT may need to be passed in order to receive a high school diploma. Sometimes, there have been computer glitches which provide wrong information to the pupil and his/her parents as to having passed a CRT. The pupil is then held back one school year unless someone challenges the test results and enough pressure is put onto the school system or testing company who scored the tests. Generally, these are highly unusual cases in which a pupil did excellent daily class work, but "failed" the CRT exit test due to computer glitches.

Differences between the standardized test and the criterion referenced test are the following:

* the standardized test has no objectives for the teacher to use in guiding the selection of subject matter for pupils to study.

* the standardized test spreads pupils from high (99th) to low (1st percentile). As many pupils as can achieve well can reach the highest percentile in a CRT.

* multiple choice test items, only, are used on both

* standardized testing and CRTs. Machine scoring does not assess essay test results.

* standardized tests are quite mechanistic in their writing whereas CRTs are more open-ended.

The Portfolio

There are a plethora of educators who state that testing and measuring of pupil achievement is a rather narrow approach to evaluation of pupil achievement and progress. What about evaluating the every day products and processes revealed by pupils of daily school work? Here the portfolio comes into being. Each pupil may then prepare a portfolio with teacher guidance. The portfolio contents may be dated and the name of the involved pupil placed on the outside. Portfolio contents might well include the following:

* written work of the pupil such as summaries, outlines, poems, plays, essays, diary entries, logs, book reports, and reviews.

* art work consisting of murals, water coloring, pencil sketching, making of dioramas and models, as well as drawing individual pictures. Snapshots may be taken of the product if it is too large to place inside a portfolio.

* video tapes of personal committee work and dramatizations; cassette recordings of book reports and oral reading.

* teacher written test results and self evaluation using appropriate criteria.

The portfolio should not become too voluminous, but should pinpoint salient, representative work of the learner. The portfolio should be read by two or three competent individuals using standards which truly do state the worth of this evaluative device. Evaluators need to use rubrics in the assessment to make for increased reliability among evaluators. The rubric, in broad terms, needs to:

* show pupil growth, or lack thereof, in each academic area of the curriculum.

* pinpoint strengths and weaknesses of pupil achievement.

* assess qualities of neatness, effort., and sequence.

* show ownership and pride of the preparer.

* assist the learner to appraise the self with increased skill.

* help the learner to analyze, think, and create.

* guide the pupil in desiring to increase achievement.

* motivate the pupil to develop increased skill in individual and committee endeavors.

* emphasize that the pupil reflect upon what is and what can be achieved with a vision toward higher accomplishments.

* empower the learner to grow, achieve, and develop in each and all intelligences.

The portfolio can be an excellent way to report pupil progress to parents. It needs to be well organized so that a responsible viewer might receive the information needed. A quality Table of Contents might well assist viewers such as parents to locate information needed. The parent, for example, may wish to view the appropriate portfolio section to notice how well the son or daughter is doing in mathematics. The parent may then ask the teacher questions pertaining to the following, involving subtraction:

* which problems does the child reveal in "take away" problems?

* is the child writing numerals and words more legibly than what is in the present portfolio'?

* how well does the child understand the concepts of regrouping and renaming?

* how well is the child able to work together with classmates in the mathematical arena?

* how effectively does the child reason in mathematics? Logical thinking is a major goal in mathematics.

* is the child improving in using problem solving skills?

* how well is the learner doing in thinking critically?

* to what degree does the child reveal creativity in finding unique solutions to problems?

* how well does the learner reveal intrinsic interests in doing mathematics? The pupil then uses mathematics on his/her own as well as completes classroom assignments satisfactorily.

* how are portfolio results used to improve the mathematics curriculum?

Issues in Measurement and Evaluation

There are a plethora of issues in the measurement and evaluation of pupil achievement. Selected approaches are highly informal and yet they do have tremendous merit. One of these approaches is teacher observation of learner progress. Teacher observation is done rather continuously of pupil progress. As the classroom teacher is teaching, he/she can observe if a pupil is diligent, prompt, exacting, and responsible. These observations need to be recorded later. If possible They may be recorded in dated, journal entries written by the observer. Brief observations may also be recorded in anecdotal record form and dated. With a succession of observations made, a pattern of pupil behavior will be formulated by the observer. The observations made are for teacher reference, alone, and may be used in planning the curriculum to assist each learner to achieve as optimally as possible. Observations made indicate, for example, which child needs more assistance in phonics, syllabication skills, use of context clues, in creative and critical thinking, as well as in problem solving. Teacher observations may be helpful to use on a daily basis to guide optimal learner achievement.

Formative and Summative Evaluation

Formative evaluation is done by the teacher to assess what pupils have learned and what is left to learn in an ongoing unit of study. The ongoing unit of study has not been completed as yet. The teacher wants to find out how well pupils are achieving the objectives of the science unit being taught. Are there corrections which need to be made before the unit of study has ended? The teacher does not wish to wait until the end of the unit of study to ascertain how well pupils have done while corrections can still be made. Testing and teacher observation may be used as assessment procedures.

Toward the other end of the continuum. Summative evaluation emphasizes that the unit of study has been completed. Decisions need to be made now as to which modifications need to be made the next time the unit is taught. The following considerations must be considered:

* are there additional objectives which should be stressed in the instructional arena?

* should selected objectives be modified or culled?

* might pupils achieve more if the sequence of learning opportunities were changed?

* are there an adequate number of attitudinal or skills objectives being emphasized within the unit of study?

* should more objectives reflect higher levels of cognition?

* are pupils engaged in adequate committee work?

* do pupils show initiative in working by the self?

* are pupils evaluating the self adequately in achieving objectives?

* what can be done to assist pupils to appraise the self more adequately?

* which methods of instruction should be used to reflect needs of handicapped pupils?

Formative evaluate occurs during the time a unit is taught. Changes may then still be made in the present unit being taught. There are identified deficiencies whereby instruction might still occur to remedy inadequacies of pupil achievement in the unit being taught. Summative evaluation assesses learner achievement in the complete unit of instruction. Here, teachers evaluate what should be done differently when the unit is taught again. Deficiencies need to he pinpointed and remedied.

Teacher Accountability and Evaluation

Along with the testing and measurement movement has come teacher accountability concepts. With state mandated objectives, teachers are to be held a accountable for pupils achieving these identified ends. Through pupil testing and measuring, it is to he noticed if the teacher has done a good job of teaching. If the teacher has taught well, it is expected that pupil achievement of objectives will show up in test results. There is a certain logic involved in equating teaching well with pupil achievement. However, the teacher is not the only being who influences pupils. The home, the community, religious institutions, among others, do affect the pupil's values and standards. Then too, a single test is not adequate to show pupil achievement. Multiple institutions in society. too, need to be held responsible for learner achievement. A plethora of evaluative techniques need to be used to notice pupil progress and achievement (Ediger and Rao, 2003).

References

Ediger, M. & Rao, D, B. (2003), Philosophy and Curriculum. New Delhi, India: Discovery Publishing House.

Thayer, V. T. (1970), Formative Ideas in American Education. New York: Dodd, Mead and Company, Inc., p. 224.

Dr. Marlow Ediger, Professor Emeritus, Truman State University.

Correspondence concerning this article should be addressed to Dr. Marlow Ediger, 201 West 22nd Street, Box 417, North Newton, Kansas 67117