Printer Friendly

An evaluation study of the teaching of hands-on investigative biology in high-schools "on a shoestring".

In this project -Biology On A Shoestring,the National Association of Biology Teachers recruited excellent, experienced high school biology teachers to develop and field test investigative laboratory activities that would require only limited resources to implement. The goal of the project was to increase the teaching of biology in high schools with an emphasis on how knowledge is generated rather than just a collection of what is known. There were two objectives for the project:

1) Facilitate hands-on investigative biology instruction by providing the teacher with effective instructional options; and

2) Enable this hands-on investigative biology to be implemented by reducing the cost of materials needed for its application.

Thus, or in concert with the objectives of the Division of Elementary, Secondary and Informal Education of the National Science Foundation, this project involved the development of new and improved instructional pre-college biology materials that involve students actively in scientific investigation. It further provided ways for teachers to involve all students, including women, minorities and those with physical disabilities in learning biology

As noted in Project 2061, Science For All Americans (Rutherford & Ahlgren 1989) and The Science Report Card: Elements of Risk and Recovery, (Mullis & Jenkins, 1988), there is an urgent call for teaching the processes of how science is generated using hands-on investigative learning experiences:

... teaching related to scientific literacy needs to be consistent with the spirit and character of scientific inquiry ... This suggests ... starting with questions about phenomena rather than with answers to be learned; engaging students actively in the use of hypotheses, the collection and use of evidence, and the design of investigations and processes ... (Rutherford & Ahlgren 1989, p. 5)

Merely calling for change does not always transform into altered experiences for students. Teaching investigative or hands-on science requires more than just a commitment from teachers. It requires resources both of time and of equipment. Asking teachers to upgrade their students' laboratory experiences at a time when there is not enough money to purchase sufficient equipment and materials to implement these changes may indeed be a key reason for the lack of change in the classroom. While there are better and longer - term solutions for the lack of money for science instruction, an immediate and critical concern is to provide teachers with hands-on science activities that require little or no money. Such experiences would enable them to teach new biology labs on a shoestring". If this innovation could be accomplished, then it is believed that students from all segments of our school populations would benefit.

The Shoestring Biology Laboratory Investigations were developed by outstanding high school biology teachers and extensively reviewed by a panel of biologists, science educators and other high school biology teachers. Each laboratory investigation included information for the teachers, such as objectives, synopsis of the lab, preparation time required, materials needed, essential teacher background, teaching tips, instructional procedures including safety notes, sample hypotheses, procedures, data analysis and references for further readings. For the student, there were sections to help students design their own experiments, safety notes, questions for data analysis and opportunities for them to generate subsequent questions and design experiments to secure relevant data.

The final NABT's Shoestring Biology Laboratory Investigations consists of fifteen laboratories that have been field tested and found to meet the following four criteria, which are both teacher and student user-friendly.

1) They require minimum money to implement.

2) They are process-oriented and stress active student involvement in scientific laboratory investigations.

3) They were developed by exemplary high school biology teachers based on their successful teaching experiences and reviewed by biologists for accuracy of content.

4) They have been field tested with diverse populations of students and include modifications for students with disabilities.

Evaluation Design

Questions:

In order to evaluate the success of this project, three questions comprised the basis of the evaluation design:

Does involvement in Shoestring Biology Laboratories make a difference in student problem solving behavior?

What were the student problem solving skills at the end of individual laboratory topics?

What were the teacher perceptions of the usefulness of the Shoestring Biology Laboratories?

To answer these three questions, two data sources were used. Teachers provided their descriptions of the usefulness of the laboratories after using them in the classroom with their students. This information incorporated comments they had secured from their students. A second source was the problem solving performance of students which had been pre-tested in the fall of the school year, after the completion of each Shoestring Laboratory, and then post-tested in May.

Sample:

Students in three contrasting urban/suburban settings were selected to provide ethnic diversity. Thirty-six teachers in three school districts were invited to participate in the study. In District A, nine teachers in its single high school of that small community participated. The students in their classes reflected the ethnic and socioeconomic diversities of a rural agricultural community. In District B, the nine teachers were in five of the six high schools of that suburban community. The students in their classes reflected the diversities characteristic of a rapidly growing community. In District C, the fifteen teachers were in nine high schools. In this mostly densely populated area of their state, these students illustrated both the diversities seen in urban populations and the influx of non-English speaking students. In all three school districts, there was about a 30% change in student populations on a given teacher's class roll during the school year. Appropriate permissions were secured from each school district for the participation of the teachers and students in this study. Part of this agreement included the understanding that all data would be reported in ways that there would be no identification of specific teachers or students.

Each teacher was asked to select at least three labs that would fit their curriculum expectations. While all teacher made their selections, their actual use of the labs ranged from none to six.

Instruments:

Two instruments were used in this study. One was a generic measure of process skills which are part of problem solving behavior, the Test of Integrated Process Skills (TIPS) (Dillashaw & Okey, 1980). It includes five components of problem solving: a) interpreting data, b) identifying hypotheses, c) identifying variables, d) defining operationally and e) designing experiments. Each items in this test is in a multiple choice format and is an independent sampling of the student's process skill performance. As reported by Dillashaw and Okey (1980) the Test of Integrated Process Skills has a reliability of .89 and an estimated readability of 9.2 grade level. The original instrument has two forms with 36 questions each. For this study, the test was shortened to 20 questions. As seen in Table 1, the five components of problem solving or process skills under study were equally represented on both forms of the test. Based on split half analysis, the reliability of this test was recalculated to be .78 for TIPS Form A (the pre-test) and .80 for TIPS Form B (the post-test).

Table 1 Table of Specifications for the Test of Integrated Process Skills (as modified) (TIPS)
Process Skill            Items from         Items from
                         TIPS Form A        TIPS Form B

Identifying Variables    14, 15, 19, 20     1, 11, 12, 13
Defining Operationally   1, 2, 3, 10        2, 5, 15, 16
Identifying Hypotheses   5, 9, 16, 17, 18   3, 4, 6, 9, 20
Interpreting Data        4, 7, 8, 13        7, 10, 18, 19
Designing Experiments    6, 11, 12          8, 14, 17




A second measure of the student problem solving behavior was a contextual test, the Shoestring Problem Solving Assessment (SPSA). This test measures student problem solving behavior with questions related to a single experimental context. It provides students with a set of information about an experiment, a data table of the results, and a graph of the data. In an open response format, students were required to describe the pattern they observed in those data and identify a possible hypothesis that was being studied. The student was provided with another hypothesis and an opportunity to describe the manipulated or independent variables and responding or dependent variables of a study that they could design if they were investigating that hypothesis and an opportunity to describe the manipulated or independent variables and responding or dependent variables that they could design if they were investigating that hypothesis. Finally, in an open response format, students described how they would test the hypothesis. The instrument consists of fourteen items built around two problem contexts. As seen in Table 2, student responses were linked to the five components of problem solving or the process skills of interpreting data, identifying hypotheses, identifying variables, defining operationally and designing experiments. Split half reliability for this instrument was determined to be .82.

Table 2 Table of Specifications for Shoestring Problem Solving Assessment (SPSA)
Process Skill            Items from     Items from
                         SPSA Form A    SPSA Form B

Identifying Variables    3, 5, 10, 12   3, 5, 10, 12
Defining Operationally   4, 6, 11, 13   4, 6, 11, 13
Identifying Hypotheses   2, 9           2, 9
Interpreting Data        1, 8           1, 8
Designing Experiments    7, 14          7, 14




Data Collection Tasks

Two tasks constituted the data collection phases of this evaluation study:

1) Initial base line data for both problem-solving tests (TIPS and SPSA) were collected from classes during the 1992-93 school year and were used to determine the reliability of the tests.

2) Data collection in 1993-94 included

a. Pre test data collection for both problem-solving tests (TIPS and SPSA) in September before instruction: N = 1613;

b. Quiz data after the instruction for each laboratory: N = 2729;

c. Post test data collected for both problem-solving tests (TIPS and SPSA) in May, 1994 N = 1240

Of the data collected, matched pre and post problem solving test data were obtained for 770 students and it is from these students that the majority of the findings of this study are based.

Data Analysis

To answer the first question of the evaluation of the Shoestring Biology Laboratory Investigations,

Does involvement in Shoestring Biology

Laboratories make a difference in

student problem solving behavior?

The pre-test and post-test scores of the TIPS and SPSA for 770 students were analyzed by computing score averages for each of the five integrated process skills. Their post-test scores Question 1: were then compared using analysis of covariance with student pre-test scores and number of laboratories in which they participated as the covariants.

To answer the second question

What were the student problem solving

skills at the end of individual laboratory

topics?

A laboratory quiz for each topic was developed using a format similar to that of the SPSA. Since the investigations described in the quiz were related in content to that of the laboratory but were not taken directly from the lab activities, the lab quiz was not a test of factual recall. After the students had completed a specific Shoestring Biology laboratory Investigation, they completed the LAB QUIZ for that investigation. A total of 2729 lab quizzes from all students who participated in the labs was used for this analysis. Thus, for each of the topics, the students' success with problem involving or process skills is seen in both the lab quiz scores as well as the comparison with the averages for all the lab quiz scores which reflects changes in problem solving which occurred as the year progressed.

To answer the third question of the study,

What were the teacher perceptions of the

usefulness of the Shoestring Biology

Laboratory Investigations?

Focus group sessions were held in April and May in 1994 with the teachers participating in the evaluation study. At these sessions, all participating teachers had opportunity to share their reflections from the classroom experiences of using the labs. As described in the PACE evaluation model (Koballa, Butts & Riley, 1995) these open responses were then categorized based on the four goals of the project as another indicator of the success of the project.

Results

Questions 1:

Does involvement in Shoestring Biology

Laboratories make a difference in

student problem solving behavior?

The students scores were grouped based on the number of laboratories that they had completed. The control group (N = 113) did not have opportunity to do any of the Shoestring Laboratories. The minimum Practice group (N = 247) were whose students who had completed one or two of the Shoestring Laboratories. The Multi-Practice group (N = 441) were whose students who had completed between 3 to 6 of the Shoestring Laboratories. By grouping the students in this way, it was possible to retain the anonymity of the teachers involved. The average post-test scores for the five process skills as they relate to the number of laboratories that students completed are seen in Table 3 for the generic test of problem solving (TIPs) and Table 4 for the context measure of problem solving (SPSA). The percent of correct student responses show how successful students were in using the integrated process skills. It is clear that at the end of the school year, the pattern of the problem solving behavior of students who had no experience with Shoestring labs (control group students) was consistently lower than those who had opportunities for instruction with the Shoestring Labs. Students who had an opportunity to practice with the Shoestring labs had better post test performance than the control group students in each of the integrated process skills in both the generic (TIPS) or contextual (SPSA) measurement of these skills. Those students in the Multi-Practice group had higher end of the year scores than the scores in the Minimum-Practice group. Thus the conclusion is that participation in the Shoestring Laboratories does indeed strengthen students' problem solving performance.

Table 3 Student Problem Solving Post Test Scores on Tips
                                     Average Scores
                         Control Group   Minimum    Multi-Lab
                         Practice        Practice
                         N = 113         N = 247    N = 441

Problem Solving Skill
with Possible Score
Interpreting Data          2.30            2.73       2.97
(4)
Identifying Hypothesis     2.33            3.05       3.50
(5)
Identifying Variables      1.76            2.16       2.28
(4)
Defining Operationally     1.91            2.26       2.52
(4)
Designing Experiments      1.56            1.93       2.11
(3)




Table 4 Student Problem Solving Post Test Scores on SPSA
                                     Average Scores
                         Control Group   Minimum    Multi-Lab
                         Practice        Practice
                         N = 113         N = 247    N = 441
Problem Solving Skill
with Possible Score
Interpreting Data          1.12            1.39       1.59
(2)
Identifying Hypothesis     1.35            1.56       1.64
(2)
Identifying Variables      2.14            2.77       3.18
(4)
Defining Operationally     1.55            2.34       2.85
(4)
Designing Experiments       .73            1.19       1.43
(2)




While this finding could have been a function of experience with Shoestring Lab instruction, it could also have been a reflection of lower pre-test performance of students in some groups. Using the pre-test scores as the concomitant variable and the number of labs in which they had participated as the treatment, the results of an analysis of covariance are seen in Table 5 for TIPS and Table 6 for SPSA. The F ratios provide evidence for the conclusion that the greater the number of laboratories students experience, the higher their post test scores regardless of their pre test scores. Since the R indicates how much of the variance is accounted for by the treatment, the higher R for the SPSA indicates that this test is more sensitive indicator of the changes that occurred as a result of the treatment. The more experience they had with the Shoestring Biology Laboratory Investigations, the higher their post test scores.

Table 5 Analysis of Covariance for Student Problem Performance on TIPS
Problem Solving Skill      R       F      p=

Interpreting Data        .154    69.76   .0001
Identifying Hypothesis   .221   109.17   .0001
Identifying Variables    .061    25.06   .0001
Defining Operationally   .099    42.25   .0001
Designing Experiments    .116    50.52   .0001




Table 6 Analysis of Covariance for Student Problem Performance on SPSA
Problem Solving Skill      R       F       p=

Interpreting Data        .293   158.91   .0001
Identifying Hypothesis   .046    18.46   .0001
Identifying Variables    .202    97.04   .0001
Defining Operationally   .211   102.58   .0001
Designing Experiments    .20     95.98   .0001




Question 2:

What were the student problem solving

skills at the end of individual laboratory

topics?

LAB QUIZ data from 2917 students are shown in Table 8. After the field testing of the labs, the decision was made by NABT to exclude some of the topics based on the review of the science committee and based on feedback from the field test teachers. These decisions were made independent of the test data. The Shoestring Biology Investigation Laboratories that have been published in the final product are indicated by (#) in Table 8.

[TABULAR DATA 8 NOT REPRODUCIBLE IN ASCII]

The student quiz performance on each lab were collected and the averages computed for all students who participated in the specific Shoestring lab. Table 8 shows the average student performance of each of the integrated process skills from these lab quiz scores. While individual lab quiz scores were not matched to the pre-test and post-test scores of all students, it is interesting to note that the pattern of student performance of each of the integrated skills showed remarkable gain for three of the process skills, interpreting data, defining operationally and designing experiments. Each of these skills required students to write in an open response format rather than a multiple choice format. These results provide further evidence that student problem solving skills were enhanced by their opportunity to practice them as part of the Shoestring Biology Investigation Laboratory experiences.

Question 3:

What were the teacher perceptions of the

usefulness of the Shoestring Biology

Laboratory Investigations?

In focus group sessions at the three school district sites completed in April and May 1994, the teachers were asked to reflect on their students reactions to the Shoestring Biology Laboratory Investigations. Their reactions were then related to The four goals of the project.

For the first goal, "develop laboratory experiences that require a minimum of money to implement", teachers noted that

Students like the flexibility in the way

they could make the lab fit their interests,

time and resources.

Students really got into preparing, securing

materials and teaching parts of the

labs to the rest of the class.

The second goal was for "the labs to have a process emphasis and actively involve students". Most teachers clearly saw the labs having a process emphasis as they involved students i the design of experiments.

Students liked asking questions such as

what else? what happens now? what

happens next = and knowing that they did

not have the answer on the next

page.

Students designed their own experiments

but frequently did not have time for them

to do what they had designed. Students

had a high interest in doing the experiment

they had created.

Teachers also described the labs as

providing experiences not normally in their

courses.

Teachers also described the labs as

providing experiences not normally in their

courses.

Students liked the good practice in

constructing graphs.

Students liked the graph interpretations

- it was good practice with a skill not

usually done in class.

Involvement of students in their own learning was frequently identified by teachers as a positive outcome of using the labs.

Students really enjoyed preparing and

teaching parts of the labs to the rest of

the class.

Students took turns in sharing the results

of their experiments.

Students liked the group work once they

were organized with assignment of tasks and

responsibilities for work to be done.

However, involvement of students in their work together was not always seen as a positive outcome. Some students did not enjoy cooperative groups and teachers noted that this was probably because they did not know how to work together.

The third goal of the project was to "produce materials that reflected practices that experienced high school biology teachers know work in the classroom and include worthwhile science understandings". Related to this goal, teachers noted that

Students liked the example/non examples

of the concepts.

The fourth goal was to "include students feedback in the develop f the labs". Many teachers noted that students really enjoyed the labs but time limitations was a real concern.

Student designed their own experiments

but on occasions did not have time for

them to do what they had designed.

Students enjoyed the labs but sometimes

they did not want to do the experiments

they had designed because they had

become bored with the topic or wanted to

move on to another topic.

Other teachers noted that the involvement of the students as peer teachers was viewed very positively by students.

Students found that they liked having

those who had the skill to have

opportunities to teach those who did not

yet have the skill.

Students enjoyed a new strategy for getting

a lab grade - "Everyone in here will

get a lab grade and all will get the lowest

grade earned by anyone in the class."

But all students did not want to be involved in the creative phase of designing and conducting experiments.

Students resented having to figure out

what they were to do - they wanted to be

given the steps and then they would do it.

Students wanted more directions in the

labs and only one person assigned to do

each task.

Thus, more teachers believed that the labs were most successful in achieving the goals of the project. It should be noted that the teachers participating in the evaluation study were not part of the filed testing of the Shoestring Biology Laboratory Investigations and thus their comments related to desired changes were much more limited. On several occasions, they did described ways they found it useful to modify the student directions sheets or to use other organisms to accomplish the goals of the lab.

Conclusions

An evaluation study of this project, the Shoestring Biology Laboratory investigations, was designed to answer the question, Did these investigations make a difference and is for, in what way? Related to the two main goals of the project - to facilitate hands-on investigations in biology and to do it without the need for added expenses - it is clear that the project demonstrated its success. When given the option to use as many of the laboratory investigations as they desired, some teachers chose to use up to six of them. In co cases were additional funds required for the labs to be used in classrooms. The fact that the laboratory investigations are easily conducted with commonly found materials in the students' neighborhoods further enhances the overall usefulness of the laboratories - and practically guarantees the widespread use of them if the teacher is willing.

In addition to the usefulness of the topics of the laboratory investigations, the students clearly enjoyed the opportunity to ask questions and to design their own experiments. The quiz results show that the more that students had practice finding evidence for their answers to questions they had posed, the better their problem solving skills became. As seen in the results from the control group students, lacking these kinds of laboratory experiences, exposure to biology does not itself help students to solve problems. If problem solving is a valued outcome of high school biology, then the Shoestring Biological Laboratory Investigations provide evidence that learning how to think or problem solving can be achieved by high school students.

The results of this study show that students can be helped to learn how to solve problems using process skills, an outcome of biology instruction that is valued by teachers since success is thus accessible to all their students. The teacher is the key for the use of the labs. In this study, all teachers initially chose to participate by selecting labs that they believed fit their curriculum. Some teachers did use the labs and some did not. Was this choice influenced by their belief or concern about the appropriateness of the labs for their students or their desire to incorporate more laboratory instruction into their biology courses? Knowing the teacher's perception of the need for lab instruction and the need for inclusion of more lab instruction in their courses clearly are two issues that will influence if the labs will be used. If they are used, students will show gains in their performance of problem solving or process skills.

The Shoestring Biology Laboratory Investigations provided the teachers with a tool currently not available. This tool helped their students achieve what for many years has been described as the primary goal for teaching science that is, learning how to solve problems and think critically. Reading about what science is and what scientists have learned differs greatly from doing science - asking questions, designing experiments, interpreting results and finding ways to design experiments could be improved to provide more credibility to their answers to scientific questions.

It is in the doing of the Shoestring Biology Laboratory Investigations, that possibly for the first time, students experience the joy, the excitement and the intellectual power of science.

Table 7 Student Problem Solving Performance After Instruction as Measured by Lab Quiz Compared with End of Year SPSA Post Test
                         Lab  Quiz   SPSA  Post-test
                         N = 2729    N = 1240

Interpreting Data          70%           74%
Identifying Hypothese      75%           70%
Identifying Variables      78%           61%
Defining Operationally     63%           75%
Designing Experiments      70%           80%
(*) Results shown as average percentage of correct responses.




References

Dillashaw, F. & Okey, J (1980). Test of the integrated science process skills for secondary science students. Science Education, 64 (5) p 601-608.

Koballa, T; Butts, D & Riley, J (1995). An evaluation model with participant constructed components. National Association for Research in Science Teaching, Abstracts, 1995 Annual Meeting.

Mullis, J & Jenkins, S. (1988). The science Report Card: Elements of Risk and Recovery, NJ. Educational Testing Service.

Rutherford, J. & Ahlgren, A. (1989). Science for All Americans, New York, Oxford University Press.
COPYRIGHT 1997 Project Innovation (Alabama)
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 1997 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Butts, David P.; Jackson, David; Oliver, J. Steve; Butts, Douglas P.
Publication:Education
Date:Sep 22, 1997
Words:4267
Previous Article:The dangers of the self-esteem rhetoric in educating children with disabilities.
Next Article:Newspaper coverage of learning disabilities.
Topics:

Terms of use | Privacy policy | Copyright © 2018 Farlex, Inc. | Feedback | For webmasters