Barriers to creating a more culturally diverse nursing profession: linguistic bias in multiple-choice nursing exams.
In recent years an increasing number of culturally diverse students, including non-native speakers of English, have enrolled in nursing programs throughout the United States (1-11). Once they enter a nursing program, many of these students have difficulty succeeding academically (9,12,13). Language difficulties are often cited as a factor that contributes to student attrition (7,14).
Multiple-choice tests comprise an important means of assessing the mastery of content in nursing courses. In many courses, these tests also constitute a large percentage of the final grade. ESL (English as a second language) students, or non-native speakers of English, often do not perform well on multiple-choice exams (15). Commitment to diversity in the nursing profession, and a concomitant commitment to improving the success rate of multicultural and international students in nursing programs, should include "the reduction of biased items in nursing examinations, with the goal of making the evaluation process fair for all students" (15, p. 35).
What is bias in tests and how can it be determined? E. Julian, psychometrician at the National Council of State Boards of Nursing, states that "an item is ... free of bias if the probability of responding correctly, given total score, is the same for all sub-populations" (15, p. 36). The term given total score refers to statistical procedures that assess for the overall ability of groups of majority and minority examinees. If it is determined that "students of equal ability but [from] different ethnic groups do not have the same probability of answering the item correctly, the item may be biased" (15, p. 36).
Item bias occurs at two different levels: linguistic/structural and cultural. Linguistic bias refers to overly complex items that are not easily understood (16). Klisch defines structural bias as items that are "long, unclear, or contain awkward or misused grammatical construction[s]" (15, p. 36). Such items "contribute noise, rather than information, to the testing process" (E. Julian, 15, p. 36) and can interfere with the reader's understanding of the writer's intent.
Cultural bias refers to content in test items that is not equally available to all cultural groups. Mohan (17) defines cultural bias as the presence of culturally specific information, such that special cultural knowledge is necessary to answer certain test items correctly. Klisch defines a culturally biased item as one that "contains references to a particular culture and which, given total score, is answered incorrectly more frequently by examinees from another culture." She claims, however, that some culturally biased items should be retained if it is determined that the information is "essential to safe and effective nursing practice" (15, p. 37).
One common source of cultural bias in test items is humor. Haladyna (18) cautions against humor because it detracts from the purpose of the test and does little good. According to Klisch, "Many ESL students may not understand the attempted humor and could become confused further as to the meaning of the item" (15, p. 37). Furthermore, humor that depends on shared cultural knowledge may be culturally biased.
Background of Study The nursing program at the College of St. Catherine, St. Paul, consists of eight courses: six four-credit first-year courses and two eight-credit second-year courses. In seven of these courses, multiple-choice tests comprise an important part of the evaluation of students. In fall 1999, the College was awarded a three-year grant from the U.S. Department of Health and Human Services to recruit and retain multicultural and economically disadvantaged nursing students. As part of the nursing department's efforts to improve the success rate of students from diverse cultural and linguistic backgrounds, funding was approved for a study analyzing multiple-choice nursing course exams for linguistic and cultural bias. Copies of multiple-choice exams from the seven nursing courses that used them were requested; complete exams from six nursing courses were provided. Nineteen exams, for a total of 673 multiple-choice test items, were analyzed for this study.
Analysis of Test Items Four broad categories of analysis were used to identify flaws in the test items: testwise flaws, irrelevant difficulty, linguistic/structural bias, and cultural bias. Flaws related to testwiseness make it easier for "testwise" students to answer the question correctly (19), which gives them an unfair advantage over students who are not "testwise." Flaws related to irrelevant difficulty make a question difficult to understand for reasons unrelated to the content or focus of the assessment (19).
Linguistic/structural bias refers to unnecessary linguistic complexity in the stem or options, grammatical errors, and lack of clarity or consistency in the wording (15,16). Cultural bias refers to the use of culturally specific information that is not equally available to all cultural groups (17). Although there is some overlap, distinctions between the categories were maintained throughout the study, as the researcher felt it was useful to look at linguistic and cultural issues separately from logical and organizational issues. There were 52 subcategories of analysis (see Sidebar 1), a partial list of which was developed after a review of the literature (1820). Other subcategories of analysis emerged from the data. Each test was read by the researcher at least three times. First, the researcher circled and noted flaws in the test items. Second, flaws were analyzed for their type and categorized. Flaws not specifically represented in the literature were then added to the list. The tests were then read a third and final time, to ensure consistency and accuracy in the categorizing of flaws. The draft of the report was reviewed by four nurse faculty members at the College.
Results of Analysis Twenty-eight types of flaws from six different categories occurred at least 10 times in the reviewed exams: 1) testwise flaws (three types), 2) irrelevant difficulty in stem (five), 3) irrelevant difficulty in options (five), 4) flaws of linguistic complexity (six), 5) grammatical errors (seven), and 6) of flaws in wording (two). Flaws from each of the six categories are presented and discussed in order of frequency of occurrence within each category. In the discussion, the focus is on correcting the specified flaw; however, other changes have been made in the revisions and are discussed as needed. (The numbers in brackets indicate the exact number of occurrences.)
Testwise Flaws Flaws related to testwiseness provide clues to the correct answer to testwise students, which gives them an unfair advantage over students who are not testwise (19).
1. Grammatical cues: One or more distractors do not follow grammatically from the stem .
A person with pheochromocytoma will exhibit which of the following signs and symptoms?
A. Hypoglycemia, lethargy B. Hypertension, anxiety
C. Hypotension D. Diarrhea, abdominal cramping
Test writers pay more attention to the correct answer and less attention to distractors (19). As a result, grammatical errors tend to occur more frequently in distractors. In the example, the stem asks for "signs and symptoms" in plural form. Option C, a distractor, is in singular form. Testwise students will eliminate Option C as it does not follow grammatically from the stem.
In the revision, a second sign or symptom has been added to Option C to make it consistent with the other options. In addition, "which of the following" has been placed at the beginning of the question (see No. 16 for discussion of nonfronted WH-questions): "Which of the following signs and symptoms will a person with pheochromocytoma exhibit?"
2. Long correct answer: Correct answer is longer, more specific, or more complete than other options .
Tim asks the nurse about how to care for the umbilical cord of his son, Mark. The nurse explains that cord care should be done:
A. By cleaning the cord and skin around the base of the cord with an alcohol swab or cotton ball or cotton-tipped swab wet with 70 percent isophropyl alcohol at least three times a day (or with every diaper change)
B. After bathing the baby in a tub, gently patting the cord and the base of the cord dry
C. Only if there is odor, oozing of yellow puslike material, or reddened areas around the cord in order to prevent infection
D. Usually during the first week of life, until the cord "dries up" and looks dark in appearance
Because item writers focus more on the correct answer than on distractors (19), information included in the key increases its length and specificity, providing a clue for testwise students. In the example, Option A is longer, more specific, and more complete than Options B, C, and D. In the revision, Option A has been shortened so that it is comparable to the others in terms of length and degree of specificity: "By cleaning the cord and the skin surrounding the cord's base with isophropyl alcohol at least three times per day."
3. Word Repeats: A word or phrase from the stem is repeated in the correct answer .
The care manager role is demonstrated when the nurse:
A. Helps a diabetic client learn to give own injection
B. Meets with the client's family prior to discharge
C. Organizes and manages a client's plan of care
D. Changes a client's wound dressing
Another clue to the correct answer is the repetition of a word or phrase from the stem in one of the options (18,19). In the example, the stem contains the word "manager"; Option C uses the word "manage." Testwise students will select this option because of the repeated word. In the revision, Option C uses the synonym "coordinates" instead of "manages": "Organizes and coordinates a client's plan of care."
Irrelevant Difficulty in Stem Flaws of irrelevant difficulty make questions difficult to understand for reasons unrelated to the content or focus of the assessment (19).
4. The lead-in is in completion format, not in question format [3.5 7]. You are administering enteric-coated aspirin for treatment of your patient's arthritis. The enteric coating will:
A. Augment lipid solubility and absorption of the drug
B. Diminish permeability of the GI mucosal epithelium
C. Resist breakdown and absorption of the drug in the stomach
D. Increase hepatic "first-pass" effects
The question format is considered "a simpler and more direct tactic for eliciting a response from a test taker" (18, p. 70) than the completion format. The completion format requires the test-taker to either hold the stem in short-term memory or go back and forth between the stem and each option to evaluate its truthfulness. The time required for the additional processing may add to test anxiety. While this issue affects all test-takers, students for whom English is not their native language are at a disadvantage as their processing time is already slower than that of their native-speaking peers.
In addition, directions for many exams ask students to choose the best response to each question. Test items should be consistent with the task students are asked to accomplish, that is, items should be in question format when the directions specify questions. In the revision, the stem has been placed in question format: "You are administering enteric-coated aspirin for treatment of your patient's arthritis. What will the enteric coating do? It will ... " Although a lead-in completion format remains, it is shorter and, therefore, less demanding on the test-taker's short-term memory.
5. Negative phrasing is used in the stem or options .
Which description of the breast examination is true:
A. Postmenopausal women do not need to do a breast exam.
B. Palpate the breast tissue systematically in a clockwise motion.
C. Male breasts are not examined because males can't develop breast cancer.
D. Percussion is used to further assess any palpable breast masses.
Stems and options should be phrased positively as "students have difficulty understanding the meaning of negatively phrased items.... Students often read through the not and forget to reverse the logic of the relationship being tested" (18, p. 73). In the example, the stem asks students to determine the one true statement. Options A and C both contain the negative "not." Option C also contains a double negative. Even though, from a grammatical point of view, there is no problem when negatives occur in two different clauses in the same sentence (21), use of a double negative may prevent a clear reading of the test item (15).
If an item cannot be worded positively, negative words such as "not" or "false," as well as "true," should be bold-faced, underlined, and capitalized, to highlight the nature of the task. In the revision, "false" has been formatted in this way and the lead-in has been put into question format: "Which of the following statements about breast examinations is FALSE? A. Postmenopausal women need to do a breast exam. B. The breast tissue is palpated systematically in a clockwise motion. C. A lump in a man's breast could be breast cancer. D. Percussion is used to further assess any palpable breast masses."
6. Best-answer wording is not bold-faced, underlined, and capitalized .
The nurse is monitoring a client with adult respiratory distress syndrome (ARDS) in the respiratory intensive care unit. What is the most serious consequence of ARDS that the nurse must monitor for?
A. Collapsed airways
B. Car pulmonale
D. Pulmonary edema
In "best-answer" questions, in contrast to "correct-answer" questions, "all options are correct, but one is the best correct answer in terms of degrees of correctness" (18, p. 62). Although the best-answer format seems well suited for testing a student's ability to prioritize information, research has shown that best-answer items are more difficult for ESL students to process than correct-answer items (22,23). To compensate for the increased difficulty, words such as "most," "first," and "primary" should be bold-faced, underlined, and capitalized, to make sure readers understand the nature of the task. In the revision, the word "most" has been formatted to stand out: "The nurse is monitoring a client with adult respiratory distress syndrome (ARDS) in the respiratory intensive care unit. What is the MOST serious consequence of ARDS that the nurse must monitor for?"
7. The task/content of the stem is tricky or unnecessarily complicated .
All of the following are correct statements about the American Nurses Association EXCEPT:
A. Is a professional organization whose membership consists of physicians, nurses, and citizens interested in improving health care
B. Works to improve the quality of nursing practice
C. Identifies the appropriate academic credentials for entry into nursing practice
D. Fosters the development of nursing theory by promoting nursing research
The most frequent type of "tricky stem" is the reverse multiple-choice (reverse MC) item, which asks students to identify the one incorrect answer rather than the one correct answer. It is more difficult to process than the "correct-answer" format as the negative is often missed (23). Furthermore, such a format is often used when it is easier for the test-maker to create one incorrect alternative rather than three plausible distractors (24). Reverse MC items should be used only when they are based on the need to know incorrect nursing practices, not on the convenience of the test-maker (22).
When it is necessary to use a reverse MC, a more direct statement of the negative, such as "not," is preferable, and it should be boldfaced, underlined, and capitalized. In the revision, "not" replaces "except," and the stem has been put into question format: "Which of the following statements is FALSE about the American Nurses Association? It.... "
8. The stem is unclear or ambiguous .
Culture is best defined as:
A. A worldview and set of traditions used and transmitted from generation to generation
B. Classification of people according to shared biologic characteristics
C. A set of rules that provide the individual with a means for behaving and interpreting the behavior of others
D. Taking on the attitudes, values, and beliefs of the dominant society
1. A,D 2. C,D 3. B,C 4. A,C
Directions in the stem should be clear and unambiguous; the test-taker should have no doubt about the nature of the task (18). In the example, the superlative "best" in the stem implies one correct definition for the answer, yet the options all contain two definitions of culture. In the revision, the stem has been reworded to be consistent with the options, and key words for identifying the nature of the task ("two" and "most") have been boldfaced, underlined, and capitalized: "Which TWO of the following definitions of culture are the MOST accurate?"
Irrelevant Difficulty in Options Like flaws of irrelevant difficulty in the stem, irrelevant difficulty in the options makes test items difficult to understand for reasons that are unrelated to the content or focus of the assessment (19).
9. "None of the above" or "all of the above" is used as an option . Which of the following characterizes a wound that heals by secondary intention (as compared to one that heals by primary intention)?
A. There is formation of more granulation tissue.
B. Healing time is quicker.
C. Less scar tissue results.
D. None of the above
Haladyna argues that there is no reason for using "none of the above" as an option as "a correct answer obviously exists and should be used in the item. No advantage exists for omitting the right answer from the list of choices" (18, p. 78). Haladyna also advises against using "all of the above" as an option, as it may help testwise test takers (18). Test-takers who recognize that two of the three options are correct will choose "all of the above" without needing to know about the third option. In sum, it is preferable to avoid using both these forms as options.
10. Options are not grammatically consistent or parallel in form . Which of the following is true of nursing interventions with physically abused women?
A. Recognize that the woman may believe she is the cause of the abuse.
B. Make sure the woman has family that she can go to if there is an occurrence of abuse.
C. Keep questions of the woman indirect so the woman does not feel discovered.
D. Offering personal opinions of the situation is useful to initially develop the nurse/client relationship.
Several references mention the need for options to be parallel or grammatically consistent in form (15,19,20). Option D, which begins with the gerund, or "-ing" form of the verb, is not grammatically consistent with Options A, B, and C, which begin with the base form of the verb. To be consistent, the verb form in Option D has been revised: "Offer the nurse's personal opinions of the woman's situation to strengthen the nurse/client relationship." The grammatical subject "they" has also been added to the question: "Which of the following statements is TRUE of nursing interventions with physically abused women? They: ..."
11. There is repetitious wording in the options . An important nursing intervention for someone experiencing tuberculosis includes:
A. Teach client to correct hypoxemia by increasing respiratory rate above 30/min.
B. Teach client the importance of taking all of her medications and reporting to the clinic for regular check-ups.
C. Teach client to wear an oxygen mask at all times.
D. Teach client the most effective breathing pattern for "floppy" lung disease.
Repetitious wording, a common error in completion items (18), can be avoided by placing the repeated words in the stem. In the revision, "teach client" has been removed from the options and placed in the stem, and the stem has been put into question format: "What is an important nursing intervention for someone experiencing tuberculosis? To teach the client to:" Although the revision includes a lead-in in completion format, greater conciseness in the options will require less time for test-takers to read through and make a selection. Options B and D have also been reworded to ensure parallelism in the options:
A. Correct hypoxemia by increasing respiratory rate above 30/min.
B. Take all of her medications and report to the clinic for regular check-ups.
C. Wear an oxygen mask at all times.
D. Practice the most effective breathing pattern for "floppy" lung disease.
12. Complex multiple-choice format is used . Which of the following are prerequisites for starting a blood transfusion on your client?
1. Discontinue saline solution and hang dextrose.
2. Identify proper type blood and correct client with another RN.
3. Use a central IV line since a peripheral line cannot be used.
4.Assess vital signs and skin integrity of face, chest, and back.
5. If refrigerated, allow blood to warm for several hours before starting infusion.
A. 1,2,4 B. 2,3,4 C. 2,4,5 D. 2,4
The complex multiple-choice format lists different combinations of right and wrong answers. It is frequently used when more than one choice is correct. Despite its widespread use, such items are more difficult to process than single correct-answer items, less discriminating, and less reliable as the testwise test-taker with only partial knowledge can identify the correct answer by eliminating distractors (18).
A viable alternative to the complex multiple-choice format is the multiple true/false format. Although there is considerable research supporting its use, item writers and students are not very familiar with this format and may perceive it to be more difficult than it actually is (18). It is, therefore, especially important that clear directions be provided for the task. In addition, each statement needs to be numbered to correspond with a machine-scored answer key: Directions For 12-16 For each of the following items, mark A on your answer sheet if the statement is true or B if the statement is false. Which of the following is/are prerequisite(s) for starting a blood transfusion on your client?
13. Option(s) are not logically compatible with the stem . In which of the following situations should the nurse have a high index of suspicion for water intoxication?
A. Persons experiencing SIADH
B. Persons who have experienced head trauma
C. Persons with a diagnosis of lung cancer
D. All of the above
Options that are not logically compatible with the stem can make the test item unnecessarily difficult to understand. In the example, the stem identifies "situations," but Options A, B, and C begin with "persons." In the revision, "conditions" has been used in the stem, followed by options stating four conditions. Other changes have been made in the wording to ensure clarity and directness in style. For example, "high index of suspicion" is not wording that students are likely to have encountered in their textbooks or lectures. (See discussion of unclear wording in No. 22.) In addition, since all of the options were correct to some extent, "all of the above" has been replaced with a fourth option and best-answer wording has been used. "Clients with which of the following conditions are MOST likely to experience water intoxification? A. SIADH, B. Head trauma, C. Lung cancer, D. Hypothyroidism."
Linguistic Complexity This type of flaw, which refers to unnecessary linguistic complexity in the wording of the stem or options, is one source of linguistic or structural bias in a test item.
14. Reduced or implied conditional . According to the Roy Adaptation Model, making a judgment about whether to have a particular treatment would be considered:
A. An innate coping mechanism and a part of the cognator subsystem.
B. An innate coping mechanism and a part of the regulator subsystem.
C. An acquired coping mechanism and a part of the cognator subsystem.
D. An acquired coping mechanism and a part of the regulator subsystem.
Use of the modal "would" suggests a conditional situation, specifically a situation in the present that is not real (16). In one sense, all multiple-choice test items are hypothetical, as they refer to situations that are not actually real. But they are realistic. Indeed, the rhetorical function of multiple-choice tests is, in part, to describe clinical situations that are plausible and then to elicit responses from the examinee that require application of previously learned information (22). To select the correct option, successful test-takers must be able to imagine themselves in the situation recreated in the stem.
In the example, the use of "would" indicates a present unreal situation, which is contrary to the rhetorical function of test items, that is, to create plausible contexts for purposes of assessment. In addition, there is no clear agent in the stem. If it is the client who is making the judgment, that agency should be made clear. In the revision, "would" has been replaced by the verb "is" and "the client" has been added as the subject: "The client is making a judgment about whether to have a particular treatment. According to the Roy Adaptation Model, which of the following statements accurately describes such a judgment? It is ..."
15. Embedded and/or reduced clause . Nursing interventions Shannon may use with clients in the clinical which reflect a focus on primary prevention include:
Reduced and embedded clauses contribute to the complexity of a text. In the example, two clauses--"Shannon may use with clients in the clinical" and "which reflect a focus on primary prevention"--are embedded in the main clause: "Nursing interventions include." One embedded clause--"Shannon may use with clients in the clinical"--is also reduced, from "that Shannon may use with clients in the clinical.
Sentences with reduced and embedded clauses "often need to be reread and reinterpreted before being properly decoded" (22, p. 58). It is preferable to break down complex sentences into shorter, simpler ones. In the revision, both embedded clauses have been unpacked and two separate sentences created. In addition, the modal "may," which could mean either "possibility" or "permission," has been replaced by the direct form of the verb. (See discussion of modals in No. 17.) The revision reads: "Shannon is planning nursing interventions with clients in the clinical setting. Which of the following interventions reflects a focus on primary prevention?"
16. Word order/embedded question . The major cause of death for children in the U.S. over one year old is which of the following?
In the example, "which of the following" appears at the end of the question. WH questions, or questions that begin with a WH-word, such as "which" or "what," are more difficult to process when they are nonfronted than when they are fronted. In addition, nonfronted WH questions tend to become dense with information, which contributes to the complexity of the structure and the decoding task (22). The revision reads: "Which of the following is the major cause of death in the U.S. for children over one year old?"
17. Unnecessary or inconsistent use of modal . Which of the following would not be a characteristic of an adult who may have potential for abusing children?
A. The person is challenged by chronic stress.
B. The person is socially isolated.
C. The person is in a stable environment with good support.
D. The person was treated abusively as a child.
Modals such as "may," " would," and "should" affect the meaning of the verb in various ways and can be used to convey different meanings. For example, "may" can be used for permission and possibility, with the context determining the meaning. The use of modals in tests may give "the impression that certain practices are based on discretionary principles, not on systematic ... procedures" (22, p. 59).
In addition, ESL students may be more familiar with the social interactional meanings of modals, that is, the use of "should" to give advice or "would" to suggest probability. Students may not be familiar with their more formal usage in testing situations--as more polite forms of the commands "must" and "will." Direct forms of verbs should be used to avoid confusion.
In the example, "would" suggests an unnecessary conditional. Both "may" and "would" are eliminated in the revision, in addition, the repeated words "the person" have been moved to the stem and the stem has been positively worded: "Which of the following is characteristic of an adult who has the potential for abusing children? A. Experiences low levels of stress, B. Is socially isolated, C. Lives in a stable environment with good support, D. Had a trouble-free childhood."
18. Unnecessary use of passive voice . Mr. R. is scheduled for lithotripsy. The nurse develops a teaching plan in which the procedure is described as the:
A. Surgical removal of stones
B. Capture of stones via scope
C. Fragmentation of stones by electrical charge
D. Dissolution of stones with medication
Unless the agent is unknown or is unimportant, it is preferable to use the active rather than the passive voice of verbs. In the example, both "is scheduled" and "is described" in the stem are in the passive voice. With "is scheduled," the agent is unimportant and the passive is appropriate. With "is described," however, the nurse is the agent and should be easily recognized as such. Overall, the active voice is more straightforward and easier to process than the passive voice. In the revision, use of the active voice has also eliminated the embedded clause in the lead-in: "Mr. R. is scheduled for lithotripsy. The nurse develops a teaching plan for Mr. R. How should the nurse describe this procedure in the nurse's teaching plan?"
Grammatical Errors Grammatical errors are another source of linguistic or structural bias in a test item. Seven categories of this type of flaw occurred more than 10 times in the reviewed items: punctuation , singular/plural nouns , ambiguous pronoun referent , spelling and typos , unclear agent or recipient , dangling participle , and subject/verb agreement . A miscellaneous category of grammatical errors included such errors as article usage , missing words , capitalization , possession , sentence boundaries , and relative pronouns .
Errors in grammar, punctuation, and spelling may confuse or distract the test-taker, "causing some students to lose concentration and fail to perform as they should" (18, p. 64). While such errors are confusing for all students, they are even more problematic for ESL students. In addition, errors "reflect an impression that the test was casually or improperly prepared" (18, p. 73) and, thus, may affect its face validity.
Examples of three types of grammatical errors are provided: ambiguous referent, unclear agent or recipient, and dangling participle. These errors were not necessarily the most frequent, but they are potentially the most serious in terms of affecting comprehension.
19. Ambiguous referent . Before her patient goes to surgery, the nurse obtains and records the patient's vital signs. This is important because it provides:
A. Routine information needed from all hospitalized patients
B. Information the doctor will use when deciding where to place the patient after completion of the surgical procedure
C. A time for the nurse to get acquainted with the patient before he/she goes to surgery
D. Baseline data for comparison during and after surgery
The referent to "this" in the lead-in is ambiguous: Does "this" refer to "the vital signs" or to "the nurse obtains and records the patient's vital signs"? Options A, B, and D suggest that the correct interpretation is "the vital signs"; C suggests that the correct interpretation refers to the act of "obtaining and recording." The confusion is compounded because "this" and "it" are both singular; yet, the only singular option is "the nurse obtains and records...." In the revision, the noun phrase "the vital signs" has been repeated to ensure clarity. In addition, best-answer wording has been used since all responses are somewhat correct. "Before the patient goes to surgery, the nurse obtains and records the patient's vital signs. What is the PRIMARY reason why vital signs are important? Because they provide...."
20. Unclear agent or recipient . It is important to take care of your back. When transferring your patient you should:
The agent has not been specified at the beginning of the stem; therefore, it is not immediately clear who should take care of his or her back and in what context. In the revision, the agent has been stated and more specific wording has been used to clarify exactly what is meant by "take care of your back." In addition, the lead-in has been put into question format. "It is important for nurses to avoid strain on their backs. What is the correct procedure for nurses to use when transferring a patient?"
21. Dangling participle .
In teaching cancer prevention, one of the following should be identified as the best documented carcinogen. Select the number one culprit.
The implied subject of the introductory participial phrase "in teaching cancer prevention" is the nurse; the stated subject of the main clause, "one of the following," refers to carcinogen. If the understood subject of the participial phrase is not the same as the subject of the main clause (25), the phrase dangles. In the revision, the participial phrase is lengthened to a complete sentence, and the nurse is stated as the subject of the sentence. In addition, there have been changes in wording; for example, "culprit" may not be a widely understood word for ESL students, and "best documented" is ambiguous. "The nurse is teaching cancer prevention. Which of the following should the nurse identify as the MOST documented carcinogen?"
Wording This type of flaw refers to wording that is problematic in some way. Wording that is not consistent, clear, concise, or common is another source of linguistic/structural bias in test items.
22. Wording is not clear or concise . The utilization of meticulous aseptic technique during treatments and dressing changes following cancer surgery and chemotherapy is based upon your understanding of neoplasia and cancer therapy side effects. Which of the following reflects this understanding?
The stem contains an unnecessary nominalization: "utilization of technique"; the verb "utilize" or "use" is more direct and concise. In addition, there is no clear agent in the sentence, resulting in the abstraction of an otherwise concrete idea. In the revision, "use" has replaced "utilization" and "the nurse" has been provided as subject. Other changes in wording were made to ensure clarity. To avoid teaching in the test item, "because of your understanding of neoplasia and cancer therapy side effects" has been eliminated from the stem. "Following a patient's cancer surgery and chemotherapy, the nurse is meticulous in using aseptic techniques during treatments and dressing changes. Which of the following statements reflects the nurse's understanding of the reason for using these techniques?"
23. Wording is not consistent . A medication order reads:"Digoxin, 0.125 mg PO qod." The nurse correctly gives this drug:
A. Daily before bedtime
B. By mouth every other day
C. Twice a day by way of the oral route
D. Once a week after recording an apical rate
To avoid confusion, wording should be consistent between the stem and the options and among the options. "By mouth," used in Option B, and "by way of the oral route," used in Option C, refer to the same way of taking the drug. In the revision, "by mouth" is used in both options. In addition, frequency words should be placed in the same order in all of the options; in the revision, they have been placed after the route. Finally, all options should include the same type of information--both the route as well as the frequency. In the revision, routes have been added for Options A and D: "A medication order reads: `Digoxin, 0.125 mg PO qod.' What is the correct way for the nurse to give the drug? A. Rectally daily before bedtime, B. By mouth every other day, C. By mouth twice a day, D. Intravenously once a week after recording an apical rate."
Cultural Bias The analysis of the 19 tests for this study revealed few examples of culturally biased test items. Because cultural bias continues to be a problem in many nursing tests, four examples are presented from the 1998 Mosby AssessTest (26).
1. The physician has prescribed digoxin and furosemide for a 75-year-old client who has congestive heart failure and resides in a long-term care facility. The nurse finds the client chewing on Tums, which the client states were brought by visitors for indigestion. The most appropriate action for the nurse to take at this time would be to:
A. Tell the client to continue using the Tums if they help.
B. Ask the client if the Tums have helped relieve the heartburn.
C. Ask the client to delay using the Tums until it can be discussed with the physician.
D. Tell the client the cause of the heartburn must be found before any more Tums are used.
To answer this test item correctly, students must know that Tums contain calcium. They also need to know that calcium can "both precipitate electrolyte unbalance and increase the effects of digoxin, leading to digitalis toxicity" (27, p. 14), but this is information they learn in their nursing program. Immigrant and international students are consistently unable to provide complete information when asked to write down all they know about Tums (16). A typical response from these students is, "I have no idea." Immigrants who have spent most of their lives in the United States often give partial replies, such as, "I think it is for stomach purposes"; when asked, however, they cannot explain the exact purpose of the product or that it contains calcium. Nearly all European-American students know precisely what the product is used for and that it contains calcium. One student stated, "Tums are an antacid that relieve stomach discomfort. They are calcium rich and work within an hour. My ex-boyfriend had an ulcer and used to eat them by the boxful." In sum, this item is culturally biased because it contains culturally specific information that is not equally available to all cultural groups.
2. A six-year-old is scheduled for surgery to repair a ventricular septal defect. The child is placed on a low sodium diet. The nurse teaches the mother that the menu containing the lowest sodium content is:
A. Hot dog and baked beans
B. Beef patty and baked potato
C. Tomato soup and tossed salad
D. Bologna sandwich and french fries
Most ESL students who answer this item incorrectly select Option C rather than the correct response, Option B. Option C seems to be the healthiest choice, given that it includes a salad and is the only vegetarian entree, but the answer key explains that C is incorrect because "tomato soup, unless specially prepared without salt, is high in sodium" (27, p. 6). The culturally biased assumption here is that tomato soup is prepared in just one way--with a good deal of salt--or is taken from a can. Can we assume that all students are equally familiar with how tomato soup is prepared or processed in mainstream American culture? Following a discussion of this item, a Vietnamese student expressed concern that she would fail the nursing board exams because she was unfamiliar with many aspects of mainstream American culture, including food items (16).
Some culturally biased items should be retained in nursing tests if it is determined that the information is "essential to safe and effective nursing practice" (15, p. 37) and is equally available to all cultural groups. Cultural information should be representative of the diverse clientele that is now served by the nursing profession. Essential information about local immigrant, indigenous, and minority cultures should be included in the curriculum, in addition to essential information about mainstream American culture.
3. When Sotheby's auctioned off items from the Jackie Kennedy Onassis estate, those who paid "top dollar" for items were most likely using the behavioral mechanism of:
A. Projection B. Identification C. Rationalization D. Reaction formation
This item contains several examples of culturally specific information: a celebrity figure, a tightly scripted cultural event, and the idiomatic expression "top dollar." Students from diverse cultural backgrounds are not necessarily familiar with the information necessary to understand and correctly answer the item. Furthermore, the psychological response of identification with a celebrity figure, "whereby an individual endeavors to pattern the self after another person" (27, p. 34), may not be universal.
4. Following the evacuation of a hydatidiform mole, the nurse explains to the woman that:
A. Chemotherapy will probably be required
B. Pregnancy must be prevented for one year
C. Coitus should be avoided for at least three months
D. Sterilization should be discussed and considered ESL students are often distracted by Option C because of the word "coitus." If otherwise unsure of the correct answer, they often select a distractor that contains an unfamiliar word. The problem here is that "coitus" is a relatively uncommon word that is part of most native speakers' receptive vocabulary. It is rarely used, even in print. According to students, nursing textbooks use the term "sexual intercourse" (16). Nursing tests should use the same terminology that students have encountered in their course work, as well as common words for everyday items (e.g., "toilet" rather than "commode"). Otherwise, items are biased against students whose native language is not English.
Discussion and Conclusion The purpose of this study was to identify linguistic and cultural bias in multiple-choice nursing course exams that might have a negative impact on the performance of test-takers whose native language is not English. Of the original 52 types of flaws, 28 types occurred at least 10 times in the total dataset of 673 items. Testwise flaws comprised 3 percent of the total number of flaws; flaws of irrelevant difficulty, 61 percent; errors of linguistic/structural bias, 35 percent; and cultural bias, less than 1 percent. In sum, flaws of irrelevant difficulty and linguistic/structural bias were of much greater importance than testwise flaws and cultural bias.
This study was unique because it identified the specific flaws that were the most frequently occurring in the nursing test items in a particular program. Without identifying those flaws, efforts to address problems in a program's multiple-choice tests may miss the mark. Much of the literature focuses on basic principles of item construction (18) and flaws related to testwiseness and unnecessary difficulty (19). Faculty may not be aware of less obvious sources of flaws, such as linguistic and cultural bias. Or, they may already have had training in avoiding certain types of flaws, such as cultural bias, and no longer have difficulty avoiding culturally specific information in their test items. In short, the results of the study suggest the importance of first analyzing the local situation.
Since this study was released in November 2000, it has been instrumental in raising faculty awareness and concern about linguistic bias in test items at the College. In January 2001, the nursing department held a two-day workshop on evaluation. The faculty reviewed their own test questions using a criteria sheet that was developed from the final report of this study. (See Sidebar 2.) According to the department chair, faculty comments about the workshop were overwhelmingly positive.
In conclusion, applying the findings of this study to test item construction and to the review of existing items is both possible and rewarding. Furthermore, such efforts contribute in important ways to the nursing profession's commitment to "accelerating the inclusion of ... cultures ... that traditionally have been underrepresented [in nursing programs]" (28). Nursing's commitment to diversity requires that educators examine not only the content of what they teach, but also how they teach and evaluate students, including the use and construction of multiple-choice tests.
Key Words Assessment and Evaluation-English as a Second Language-Cultural Diversity-Linguistic Bias-Multiple-Choice Exams
(1.) Abriam-Yago, K., & Kataoka-Yahiro, M. (1997). Culturally competent teaching strategies for Asian nursing students for whom English is a second language. Journal of Cultural Diversity, 4, 83-87.
(2.) Abriam-Yago, K., Kataoka-Yahiro, M., & Yoder, M. (1999).The Cummins Model:A framework for teaching nursing students for whom English is a second language. Journal of Transcultural Nursing, 10, 143-149.
(3.) Davidhizar, R., Dowd, S. B., & Giger, J. N. (1998). Educating the culturally diverse healthcare student. Nurse Educator, 23, 38-42.
(4.) Furuta, B. S., & Lipson, J. G. (1994). Cultural diversity in the student body revisited. In J. C. McCloskey & H. K. Grace (Eds.), Current issues in nursing (pp. 665-670). St. Louis, MO: Mosby.
(5.) Keane, M. (1993). Preferred learning styles and study strategies in a linguistically diverse baccalaureate nursing student population. Journal of Nursing Education, 32, 214-221.
(6.) Kurz, J. (1993).The adult ESL baccalaureate nursing student Journal of Nursing Education, 32, 227-229.
(7.) Malu, K. F, & Figlear, M. R. (1998). Enhancing the language development of immigrant ESL nursing students. Nurse Educator, 23, 43-46.
(8.) Memmer, M. K., & Worth, C. (1991). Retention of English-as-a-second-language (ESL) students: Approaches used by California's 21 generic baccalaureate nursing programs. Journal of Nursing Education, 30, 389-396.
(9.) Phillips, S., & Hartley, J.T. (1990).Teaching students for whom English is a second language. Nurse Educator, 15, 29-32.
(10.) Shearer, R.A. (1989).Teaching foreign students. Journal of Nursing Education, 28, 427-428.
(11.) Templeton Gay, J., Estes Edgil, A., & Stullenbarger, E. N. (1993). Graduate education for nursing students who have English as a second language. Journal of Professional Nursing, 9, 104-109.
(12.) Smukler, B., & Kramer, D. (1996). Beyond basic skills: A collaboration between a resource center and a department of nursing for high-risk nursing students. Research and Teaching in Developmental English, 13, 7583.
(13.) Wong, S., & Wong, J. (1982). Problems in teaching ethnic minority nursing students. Journal of Advanced Nursing, 7, 255-259.
(14.) Swiggum, P., & Slette, E. (1997). Final report Grant to recruit and retain persons of color in nursing programs (State of Minnesota Appropriation to the Minnesota Higher Education Coordinating Board.) Inver Grove Heights: Inver Hills Community College, and White Bear Lake: Century College.
(15.) Klisch, M. L. (1994). Guidelines for reducing bias in nursing examinations. Nurse Educator, 19, 35-39.
(16.) Bosher, S. (March, 1999). ESL students and the Nursing Boards. American Association for Applied Linguistics (AAAL) Annual Conference presentation, Stamford, Connecticut.
(17.) Mohan, B.A. (1986). Language and content. Reading, MA: Addison-Wesley.
(18.) Haladyna, T. M. (1994). Developing and validating multi. pie-choice test items. Hillsdale, NJ: Lawrence Erlbaum.
(19.) Case, S. M., & Swanson, D. B. (1985). Constructing written test questions for the basic and clinical sciences. Philadelphia, PA: National Board of Medical Examiners.
(20.) TOEIC Test Developers. (March, 2000). Classroom tests that work (TESOL 2000 presentation).Vancouver, BC: Author.
(21.) Azar, B. S. (1989). Understanding and using English grammar (2nd ed.) Englewood Cliffs, NJ: Prentice Hall Regents.
(22.) Dean, B. (1988). Going by the boards: A needs assessment of refugee practical nursing students preparing for the Board exam. Unpublished master's thesis, University of Minnesota.
(23.) Gronlund, N. E. (1976). Measurement and evaluation in teaching. New York: Macmillan.
(24.) Hanges, K. H. (1982). The academic multiple-choice test: Its rhetorical, grammatical and logical structures. Unpublished paper, University of Minnesota.
(25.) Kolln, M. (2002). Rhetorical grammar (4th ed.). Boston: Allyn and Bacon.
(26.) Saxton, D. F., Pelikan, P. K., & Green, J. S. (1998). Mosby's 1998 Assess Test- St. Louis, MO: Mosby.
(27.) Saxton, D. E, Pelikan, P. K., & Green,J. S. (1998). Answers and rationales for Mosby's 1998 Assess Test. St. Louis, MO: Mosby.
(28.) American Association of Colleges of Nursing (October 27, 1997). Position statement on diversity and equality of opportunity. Washington, DC: Author.
RELATED ARTICLE: Sidebar 1. categories of analysis.
A. TESTWISE FLAWS
1. Grammatical cues One or more distractors do not follow grammatically from the stem.*
2. Logical cues A subset of the options is collectively exhaustive.
3. Absolute terms A term such as always, never, and all is used.
4. Long correct answer Correct answer is longer, more specific, or more complete than other options.*
5. Word repeats A word or phrase from the stem is repeated in the correct answer.*
6. Convergence strategy The correct answer includes the most elements in common with the other options.
7. General knowledge An item can be answered with general or common knowledge.
8. Cueing within an item or test The key is contained within the item itself or within another item on the test.
B. IRRELEVANT DIFFICULTY
1. The item has more than one correct answer; the key to one item is "hinged" to key of a related item.*
2. There are more/less than four options per item.
3. The stem is unclear or ambiguous.*
4. The central idea is not included in the stem, but is in the options.
5. Most of the phrasing is not included in the stem, but is in the options.
6. Superfluous information or unnecessary redundancy is in the stem.
7. The lead-in is in completion format, not in question format.*
8. The task/content of the stem is tricky or unnecessarily complicated.*
9. Negative phrasing is used in the stem or options; negative phrasing is not bold-faced, underlined, and capitalized.*
10. Best-answer or other wording is not bold-faced, underlined, and capitalized.*
11. Options are long and/or double; task/content of options is unnecessarily complicated.
12. Not all options are plausible; one option can be eliminated by logic.
13. Options are overlapping, rather than independent; key is subsumed in one of the distractors.
14. Two options contradict each other.
15. Options are not homogeneous in content.
16. Options are not grammatically consistent or parallel in form.*
17. There is repetitious wording in the options.*
18. Inconsistent wording in the options or between stem and options.
19. Option(s) are not logically compatible with the stem.*
20. Options are not listed in logical, numerical, alphabetical order, or in order of length, from short to long.
21. Numeric data are not stated consistently.
22. Frequency terms in the options are imprecise (e.g., rarely, usually, frequently).
23. None of the above or all of the above is used as an option.*
24. Complex multiple-choice format is used.*
C. LINGUISTIC/STRUCTURAL BIAS
I. Unnecessary linguistic complexity in wording of stem and options
a. embedded or reduced clause*
b. reduced and embedded clause*
c. unnecessary or inconsistent use of modal*
d. reduced or implied conditional*
e. word order/embedded question*
f. unnecessary use of passive*
2. Grammatical errors
a. dangling participle*
b. ambiguous referent*
c. unclear agent or recipient*
d. singular/plural error*
e. subject/verb agreement*
h. misplaced modifier
i. sexist pronoun usage
a. not consistent*
b. not clear or concise*
c. uncommon ] d. illogical
D. CULTURAL BIAS - use of culturally specific information
* Indicates flaws that occurred more than 10 times in the reviewed exams.
RELATED ARTICLE: Sidebar 2. criteria for test questions from the Nursing Faculty Workshop on Evaluation, College of St. Catherine.
1. Options follow grammatically from the stem.
2. Options are of equal length; key should not be longer, more specific, or complete.
3. Same word is not repeated in stem and key.
4. Lead-in is in question format, not completion format.
5. Negative phrasing is avoided.
6. Best-answer wording is bolded, underlined, and capitalized.
7. Task/content of the stem is not tricky or unnecessarily complicated.
8. Stem is clear and unambiguous.
9. "None of the above" and "all of the above" are not used as options.
10. Options are grammatically consistent or parallel in form.
11. Repetitious wording in the options is avoided.
12. Multiple true/false format is used instead of complex multiple-choice format.
13. Options are logically compatible with the stem.
14. There are no reduced or implied conditionals in the stem.
15. There are no embedded or reduced clauses.
16. Questions are placed at the beginning of the sentence.
17. Unnecessary or inconsistent use of modals is avoided.
18. Active voice is used rather than passive voice.
19. There are no grammatical errors, spelling mistakes, or typos. The use of singular/plural word forms is consistent, there is agreement between the subject and verb of the sentence, and punctuation is used appropriately.
20. There are no ambiguous referents.
21.There are no unclear agents or recipients.
22. There are no dangling participles.
23.The wording is clear and concise.
24.The wording is consistent between the stem and options and among the options. Frequency parameters are placed in the same position in the options.
25. The use of culturally specific information is avoided.
|Printer friendly Cite/link Email Feedback|
|Publication:||Nursing Education Perspectives|
|Date:||Jan 1, 2003|
|Previous Article:||Handling anger in the teacher-student relationship.|
|Next Article:||Predicting NCLEX-RN success: can it be simplified?|