Evaluating the level of critical thinking in introductory investments courses.
Holly Dolezalek (2003), referencing Roger Shank the chief education officer at Carnegie Mellon West, writes "education by lecture in a classroom was an idea from the Middle Ages, when not everyone could read and there weren't enough textbooks to go around anyway.--not only are the conditions that produced the lecture environment gone, cognitive research indicates that people don't retain information minutes after they hear it. He endorsed--small discussion groups and project work, but he believed--the usual undergraduate degree with emphasis on lectures, large classes, and multiple choice tests--was useless" (p. 60).
The learning process has to be more than just the sharing of knowledge. Students in finance classes often fail to "integrate knowledge, see the connections among textbook chapters, or apply course concepts to real world questions" (Robertson, Peterson, & Bean, 2004, p. 16). The development and implementation of teaching and/or learning strategies appropriate for the investment course curriculum is a critical factor for the success of finance students. Some areas of study in investments are more complex or advanced than others requiring materials to be presented at different learning levels depending on course objectives. Testing should cover all levels of material presented to determine what students have mastered. It appears that most testing activities focus on remembering and understanding definitions and concepts. Learning is limited to a shallow level and critical thinking is not emphasized or evaluated. Bloom's taxonomy is a useful tool that can assist the professor in testing and instructional evaluation.
The purpose of this paper is twofold. First, it attempts to stimulate debate and thinking about the limitations of the traditional focus of testing on lower levels of learning related to investments and the value created for investments students by testing of higher level process dimensions. Second, it shows the application of the updated version of Bloom's (1956) taxonomy of the cognitive domain as presented by Anderson and Krathwohl (2001) in financial education in general and specifically in investments education testing. Anderson and Krathwohl's new version of Benjamin Bloom's (1956) taxonomy proposes six hierarchal and cumulative levels of testing to measure different levels of learning and is the latest revision to Bloom's taxonomy. This paper will review and provide example test questions for each of the levels of the new version of Bloom's taxonomy. The methodology for constructing the sample questions consists of constructing investments assessment questions and then classifying the questions according the highest possible level of the Anderson and Krathwohl update to Bloom's taxonomy. The examples provided will assist the investments instructor in developing tests to measure the level of student mastery of the subject. Through these assessment tools, professors will be better able to insure that students acquire the skills and competences required in today's financial markets.
Many university faculty believe that critical thinking should be a primary objective of a college education (Yuretich, 2004). Yet most faculty believe that critical thinking cannot be assessed or they have no method for doing so (Beyer, 1984; Cromwell, 1992; and Aviles, 1999). Consider a 1995 study from the Commission on Teacher Credentialing in California and the Center for Critical Thinking at Sonoma State University (Paul, Elder, & Bartell, 1997). These groups initiated a study of college and university faculty throughout California to assess current teaching practices and knowledge of critical thinking. They found that although 89 percent of the faculty surveyed claimed that critical thinking is a primary objective in their courses, only 19 percent could explain what critical thinking is, and only 9 percent of these faculty were teaching critical thinking in any apparent way (Paul et al.).
Previous studies have suggested a number of teaching/learning strategies for teaching finance (Robertson et al., 2004; Moore, 1999; Thoma, 1993; Dudley, Davis, and McGrady, 2001; Ulrich, 2005; and Schadler, 2007). A major challenge is how to measure the level of student success in acquiring skills and competencies required in the financial marketplace. Studies show that most college testing involves recalling memorized facts (Crooks, 1998). It is relatively straightforward to assess students' knowledge of content; however, many faculty lack the time and resources to design assessment that accurately measure critical-thinking ability (Faciaone, 1990; Paul et al., 1997; Aviles, 1999).
Additional testing should be done to evaluate the students' ability to analyze, evaluate and formulate new material. A large body of literature already exists showing that critical thinking can be assessed (Cromwell, 1992; Fisher & Scriven, 1997). Other studies have demonstrated a positive correlation between the outcomes of critical thinking assessment tests and student performance in a course or on a task (e.g., Onwuegbuzie, 2001). Such studies serve to illustrate that critical thinking per se is worth assessing, or at least that it has some relationship to students' understanding of the material. Still, generalized assessments of critical-thinking ability are almost never used in a typical classroom setting (Haas & Keeley, 1998).
Schools accredited by the AACSB International are required to focus on assessment of assurance of learning (AACSB, 2007). The college or school must specify program-level learning goals for each separate degree program. Investments faculty need to focus on how to assess student performance in ways which contribute to program-level goals.
QUESTIONS AND TASKS FOR ASSESSING COGNITIVE OBJECTIVES
Investments questions may be categorized according to Anderson and Krathwohl's (2001) hierarchy of the objectives of the cognitive domain. There are six process dimensions or objectives in the hierarchy, moving from the lowest order process to the most complex: remembering, understanding, applying, analyzing, evaluating, and creating.
The remembering objective involves the retrieval or recognition of knowledge from long-term memory. The student is challenged to recognize or recall information from presented material (Anderson & Krathwohl, 2001). Some key verbs often associated with this process are: choose, define, list, name, recite, select, state and tell. Two tasks illustrating the remembering process dimension are presented next.
Q1. Which of the following gives its holder the right to sell an asset for a specified exercise price on or before a specified expiration date?
A. Call option
B. Put option *
C. Futures contract
D. Preferred stock
E. Commercial paper
This question requires students to recall definitions of various financial securities.
Q2. List the steps required in a discounted cash flow approach to valuing a stock.
This task requires the student to remember specific information concerning stock valuation as the question itself does not provide the answer choice.
The understanding objective involves constructing meaning from different types of messages whether oral, written, or graphic. The student is challenged to integrate instructional material with prior knowledge (Anderson & Krathwohl, 2001). Some key verbs often associated with this process are: summarize, interpret, contrast, discuss, illustrate, summarize and translate.
Q3. Which of the following is an example of a highly cyclical industry?
A. The tobacco industry
B. The pharmaceutical industry
C. The utility industry
D. The automobile industry *
To answer the question correctly, the student must know the definition of cyclical industries. In addition, the student must understand the general nature of a number of industries and their relationships to the overall economy.
Q4. XYZ Company has a beta of 1.4. The industry average beta is 1.1. If the concept of regression toward the mean holds, would it be expected that the beta of XYZ will rise or fall?
The student must estimate the future consequences implied by the regression toward the mean concept and the beta values given. The question assumes that the student is familiar with regression toward the mean and that beta is not stationary.
The applying objective involves using learned procedures in exercises or problems. Students are challenged to determine the appropriate procedure to execute or implement (Anderson & Krathwohl, 2001). Some key verbs often used to test process of applying are: apply, calculate, complete, produce and relate.
Q5. A bond pays an annual coupon of 9%. Its value at maturity is $1,000. It matures in 6years. Its current market value is $1,121. Calculate the duration of this bond.
The student should understand bond mechanics, valuation techniques and the duration formula. The student must be able to apply this knowledge to this specific decision-making task.
Q6. Assume that both puts and calls trade on ABC stock. You have discovered that ABC always rises in price over the three trading days prior to the 15th of the month and falls in price over the three trading days following the 15th of the month. Form a trading rule incorporating the use of options in order to take advantage of the timing anomaly that you have discovered.
The student must apply the option pricing principles pertaining to increases and decreases in the underlying asset to a new problem situation.
Analyzing involves breaking material into parts, determining relationships, and overall structure or purpose. The process incorporates differentiating, organizing, and attributing (Anderson & Krathwohl, 2001). Key verbs pertaining to the process of analyzing are: summarize, contrast, separate, interpret and select.
Q7. We have discussed a wide variety of ratios used to analyze corporate financial statements. Given the following financial statements and industry averages, calculate the appropriate ratios and give an analysis of the strengths and weaknesses of this company.
To complete this task, the student must understand financial statements, be able to calculate ratios and be able to analyze those ratios relative to industry norms.
Q8. Using DuPont decomposition of ROE and comparison with Axel Company's peers, for each component of ROE, identify whether the component impacted Axel Company's ROE favorably or unfavorably.
The student must have prior knowledge of the components of DuPont ROE, be able to break ROE into its parts and make comparisons of the components to industry averages. The student is not asked to make any value judgments.
Evaluating is judging someone or something using criteria and standards. The sub-processes of evaluating involve checking for internal consistency and critiquing based upon external criteria (Anderson & Krathwohl, 2001). Evaluating precedes creating because it is often prerequisite to creating something.
Verbs used at the evaluating level are: argue, assess, support, explain, rate, grade, appraise, rank, judge and evaluate.
Q9. Three of the methods used to calculate the intrinsic value of a stock are the Dividend Discount model, PEG, and the Discounted Free Cash Flow model. Use these methods to determine if a stock is undervalued or overvalued. Why do the methods differ in their answers? Is one method better than the others? Explain.
The student will have to analyze the data and evaluate the resulting intrinsic stock values relative to the market value and make a recommendation. This question also requires students to evaluate the three methods used. Grading these types of questions is challenging; often there are multiple answers. In grading, the instructor should use a rubric that places value on the logic and consistency of the answer.
Q10. In analyzing a balance sheet for the purpose of valuing a stock, which is more important, analyzing assets or claims against assets, i.e., which side of the balance sheet is the greater determinant of stock value? Why?
The student must have prior understanding balance sheet accounts and the concepts of liquidity, profitability, and operating and financial leverage. The student must develop opinions concerning the interconnected risk/return tradeoffs involving the accounts and the relative importance of the tradeoffs. There is no absolute right answer and the choice must be evaluated based upon its logical appeal.
Creating is putting elements together to form a coherent pattern or functional structure that did not exist before. The process involves generating solutions, planning action, and producing the solution. Creating goes beyond previous categories in that student must produce a structure or pattern that is novel relative to his or her prior experience (Anderson, Krathwohl, 2001). Creating can be appraised using verbs such as: produce, combine, prepare, assemble, create, develop, design and compose.
Q11. Create a new financial ratio that can be used to evaluate a corporation. Discuss the information content of your ratio and any limitations of its use.
These tasks will require the student to integrate their knowledge of financial statements and existing ratios into their analysis. The student is expected to design and describe a new financial analysis tool. The grading process should make use of a rubric that places value on the creativity, uniqueness, logic and consistency of the answer.
Q12. Suppose Imelda and Ahmed discover an analyst who has outperformed the S & P 500 for the last ten years. Imelda believes the analyst's performance refutes the efficient market hypothesis (EMH). Ahmed believes the analyst's performance does not provide evidence against the EMH. Write a dramatic scene wherein the two opposing characters, Imelda and Ahmed, debate their positions.
The student must voice opinions on whether the observed analyst's performance is attributable to chance and skill. The student must have prior knowledge of the EMH, an understanding of sample size and the nature of anomalies. The student must integrate those knowledge areas in order to produce a reasoned case backing Jane's or John's opinion.
In finance, we are concerned with value creation. Can we, as investment instructors, claim to be creating value for our students if we test only at the lower levels of learning? The investment world is growing in complexity. More than ever, students need to integrate knowledge, recognize the connections among the topics covered, apply concepts, and develop new approaches. Once instructors establish higher level objectives for their investment courses, unambiguous multiple-choice questions need to be augmented with testing devices accessing student achievement in terms of the upper cognitive levels. Bloom's (1956) taxonomy, as updated by Anderson and Krathwohl (2001), provides a basis for accessing students on more complex levels of thinking. For schools accredited by the AACSB International, this type of assessment will help to enable coursework to support program-level learning goals. Constructing exams based upon the taxonomy will give instructors feedback on whether they are creating value for students in terms of the skills required in today's investment environment.
Association to Advance Collegiate Schools of Business (AACSB) International. (2007). Eligibility procedures and accreditation standards for business accreditation. [Electronic version]. Retrieved August 26, 2009, from http://www.aacsb.edu/accreditation/process/documents/ AACSB_STANDARDS_Revised_Jan07.pdf
Anderson, L. W. & Krathwohl, D. R. (Eds.) (2001) A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives. Allyn & Bacon.
Aviles, C. B. (1999). Understanding and testing for critical thinking with Bloom's taxonomy of educational objectives. Washington, DC: ERIC Clearinghouse. Report SO032219.
Beyer, B. K. (1984). Improving thinking skills--Defining the problem. Phi Delta Kappan, 65, 486-490.
Bloom, B. S., ed. (1956). Taxonomy of educational objectives. New York, NY: David McKay Co., Inc.
Cromwell, L. S. (1992). Assessing critical thinking. New Directions for Community Colleges, 77, 37-50.
Crooks, T. J. (1998). Impact of classroom evaluation on student. Review of Educational Research, (Winter), 438-481.
Dolexalek, H. (2003, April). Outdated education. Training, 40(4), 60.
Dudley, L. W., Davis, H. H., & McGrady, D. G. (2001). Using an investment project to develop professional competencies in introduction to finance. Journal of Education for Business, 76(January/February), 125-131.
Facione, P. A., Facione, N.C., & Giancarlo, C. A. (2000). The disposition toward critical thinking: Its character, measurement, and relationship to critical thinking skill. Informal Logic, 20, 61-84.
Fisher, A. & Scriven, M. (1997). Critical thinking: Its definition and assessment. Point Reyes, CA: Edgepress.
Haas, P. F. & Keeley, S. M. (1998). Coping with faculty resistance to teaching critical thinking. College Teaching, 46, 63-67.
Moore, S. (1999). Cases and lectures: A comparison of learning outcomes in undergraduate principles of finance. Journal of Financial Education, 25(Fall), 37-49.
Onwuegbuzie, A. J. (2001). Critical thinking skills: A comparison of doctoral and master's level students. College Student Journal, 35, 477-481.
Paul, R. W., Elder, L., & Bartell, T. (1997). California teacher preparation for instruction in critical thinking: Research findings and policy recommendations. Santa Rosa, CA: Foundation for Critical Thinking.
Robertson, F., Peterson, D., & Bean, J. C. (2004). Using federal reserve publications in institutions and markets courses: An approach to teaching critical thinking. Advances in Financial Education, 2(Fall), 15-25.
Schadler, F. P. (2007). A pedagogy that elevates the learnig of introductory investments. Journal of Financial Education, 33(Summer), 49-63.
Thoma, G. (1993). The Perry framework and tactics for teaching critical thinking In economics. Journal of Economic Education, 23(Spring), 128-136.
Ulrich, T. A. (2005). The relationship of business major to pedagogical strategies. Journal of Education for Business, 80(5), 269-274.
Yuretich, R. E. (2004). Encouraging critical thinking: Measuring skills in large introductory science classes. Journal of College Science Teaching, 33, 40-46.
Stephen M. Black, University of Central Oklahoma
R. Barry Ellis, University of Central Oklahoma
|Printer friendly Cite/link Email Feedback|
|Author:||Black, Stephen M.; Ellis, R. Barry|
|Publication:||Academy of Educational Leadership Journal|
|Date:||Nov 13, 2010|
|Previous Article:||An exploratory study of the effect of rewards and deadlines on academic procrastination in web-based classes.|
|Next Article:||Improving ethical education in the accounting program: a conceptual course.|