Standards of proof in civil litigation: an experiment from patent law.
TABLE OF CONTENTS I. Introduction II. Standards of Proof--An Overview A. The Burden of Proof B. The Role and Types of Standards of Proof C. Previous Empirical Studies 1. Survey Evidence 2. Experimental Studies III. Microsoft v. i4i and the Presumption of Validity in Patent Law IV. Methodology and Study Design A. Hypotheses About i4i's Impact B. Why an Experiment? C. Study Design V. Discussion A. Results B. Implications C. Directions for Future Research VI. Conclusion Appendix A Appendix B
Our litigation system is based upon the assumption that standards of proof matter. (1) They serve "to instruct the factfinder concerning the degree of confidence our society thinks he should have in the correctness of factual conclusions." (2) The various standards of proof reflect the legal system's judgment about the proper allocation of risk between litigants, as well as the relative importance of the issues at stake. (3) For example, in criminal cases where the defendant's liberty may be at stake, the prosecution carries the burden of proving every element of the criminal charge "beyond a reasonable doubt." (4) In contrast, "preponderance of the evidence," a much less stringent standard, is most common in civil cases. (5) The third main standard, "clear and convincing evidence," is an intermediate standard employed in civil litigation when "the individual interests at stake ... are both 'particularly important' and 'more substantial than mere loss of money.'" (6)
But despite the perceived importance of standards of proof, few empirical studies have tested lay jurors' understanding and application of standards of proof, particularly in civil litigation. (7) Specifically, to our knowledge, there has not been a large-scale study of a demographically representative population comparing jurors' decisions when confronted with the two standards of proof used in civil litigation: (1) preponderance of the evidence, and (2) clear and convincing evidence. (8)
Patent law recently presented an opportunity to assess the impact of varying the standard of proof in civil litigation. Under Section 282 of the Patent Act, every claim in a patent issued by the U.S. Patent and Trademark Office ("USPTO") is presumed to be valid. (9) This same statute also provides that "[t]he burden of establishing invalidity of a patent or any claim thereof shall rest on the party asserting such invalidity." (10) However, the statute fails to specify the standard of proof necessary to overcome this presumption of validity.
In a recent case, Microsoft Corp. v. i4i Ltd. Partnership ("i4i"), the U.S. Supreme Court heard competing arguments regarding the proper standard of proof for finding a patent invalid. (11) Microsoft argued that the preponderance of the evidence standard should apply, at least when the USPTO had not considered the prior art that allegedly invalidated the patent. (12) But the Court unanimously affirmed the U.S. Court of Appeals for the Federal Circuit's ("Federal Circuit") longstanding interpretation that invalidity must be proven "by clear and convincing evidence." (13)
However, the Court also held that if the party challenging a patent's validity could introduce new evidence in litigation that had not previously been considered by the USPTO during the patent's examination, then "the challenger's burden to persuade the jury of its invalidity defense by clear and convincing evidence may be easier to sus-sustain." (14) In such situations, the jury should be instructed that "it has heard evidence that the [US]PTO had no opportunity to evaluate before granting the patent" and to "consider that fact when determining whether an invalidity defense has been proved by clear and convincing evidence." (15) But at the same time, the Court declined "to endorse any particular formulation" for such an instruction. (16)
Both the parties in i4i and the Court apparently assumed that the standard of proof would affect lay jurors' decisions regarding invalidity, at least on the margins. (17) In i4i's wake, we tested this assumption by conducting an experiment involving the validity of a hypothetical patent. In our experiment, each subject received a short fact pattern simulating the information considered by a juror in a patent infringement lawsuit. (18) In this fact pattern, a key piece of prior art introduced at trial by the party challenging the patent's validity had not been considered by the USPTO. After reading the fact pattern and a summary of the parties' arguments, each subject received a randomly assigned jury instruction regarding the standard of proof for establishing invalidity: (1) clear and convincing evidence, (2) clear and convincing evidence with an i4i-type instruction regarding the new evidence not considered by the USPTO, and (3) preponderance of the evidence.
Our results suggest that i4i's impact--and the role of burdens of proof in civil litigation more generally--may be more complex than is generally assumed. Following conventional wisdom, we anticipated that a clear and convincing evidence standard with an i4i-type instruction would result in an intermediate burden between clear and convincing evidence (highest burden) and preponderance of the evidence (lowest burden), and that the subjects' decisions regarding the invalidity challenge would reflect these differences. (19) This expectation was partially supported by our experiment. Subjects who received the preponderance of the evidence standard found the patent invalid more often than those who received the clear and convincing evidence standard. (20) Similarly, subjects who received a clear and convincing evidence standard with an i4i-type instruction (intermediate burden) found the patent invalid more often than those who received the plain clear and convincing evidence standard (i.e., without an i4i-type instruction). (21) Both of these differences were statistically significant. (22)
Subjects who received the clear and convincing standard with an i4i-type instruction (intermediate burden) unexpectedly found the patent invalid at rates statistically indistinguishable from those who received the lower preponderance of the evidence standard. (23) As discussed in more detail below, this finding may have important ramifications for invalidity challenges in patent litigation. It appears that the addition of an i4i-type instruction to the clear and convincing standard may effectively result in a standard of proof that, at least in jurors' views, is equivalent to the preponderance standard explicitly rejected by the Court in i4i. (24) This surprising result suggests that Microsoft may have actually achieved its desired outcome in i4i by making it easier for juries to invalidate questionable patents, even though Microsoft lost the case.
Part II of this Article provides an overview of burdens of proof in litigation. It also analyzes the handful of existing empirical studies regarding the standard of proof. Part III summarizes the development of the presumption of validity in patent law, including the Supreme Court's decision in i4i. Part IV explains the methodology and experimental design employed in our experimental study. Part V recounts and analyzes the study's results, and then it concludes with some implications and potential directions for future research in this area.
II. Standards of Proof--An Overview
First, this Part distinguishes the standard of proof, which is the degree of certainty required for a jury or judge to find for a party on an issue, from the broader "burden of proof." Second, it explains the purposes of standards of proof in litigation and the three main standards that are used. Third, it summarizes the literature on previous empirical studies regarding standards of proof.
A. The Burden of Proof
The burden of proof, described as one of "the slipperiest member[s] of the family of legal terms," (25) has "vexed [both] courts and commentators for decades." (26) This is at least partially because the burden of proof encompasses several different components, but courts and litigants have not always taken care to distinguish between them. (27)
First, it can mean the "burden of production." Initially described by James Bradley Thayer in his famous treatise on evidence, (28) the burden of production identifies which party must offer evidence in order to raise an issue in litigation. (29) In other words, "[t]he burden of production is the obligation to make a prima facie case." (30) This burden is relatively light; it is satisfied when a litigant has "produce[d] sufficient evidence so that a rational jury or other factfinder could find that each of the elements of the claim has been proven." (31)
Second, the burden of proof can mean the "burden of persuasion"--i.e., the ultimate obligation on a party to persuade the decision maker that the party should prevail on a contested issue. (32) The latter concept was famously explained by John Henry Wigmore as the "risk of nonpersuasion." (33) In civil litigation, the burden of persuasion for all elements of a claim typically rests with the plaintiff, while the defendant has the burden of persuasion for most affirmative defenses. (34)
Third, the burden of proof has been used to mean the "standard of proof." (35) This concept "refers to the quantum of proof required to sustain the burden of persuasion." (36) In i4i, the Supreme Court explained the "standard of proof as:
[T]he degree of certainty by which the factfinder must be persuaded ... to find in favor of the party bearing the burden of persuasion. In other words, the term "standard of proof" specifies how difficult it will be for the party bearing the burden of persuasion to convince the jury of the facts in its favor. (37)
In this Article, we are concerned exclusively with the standard of proof, as opposed to the other aspects of the burden of proof.
B. The Role and Types of Standards of Proof
The standard of proof serves multiple purposes. First, it determines how to allocate the risk of error between litigants. (38) In litigation, there is always some risk the decision maker will reach an erroneous conclusion. (39) As Justice Harlan explained in In re Winship, "in a judicial proceeding in which there is a dispute about the facts of some earlier event, the factfinder cannot acquire unassailably accurate knowledge of what happened. Instead, all the factfinder can acquire is a belief of what probably happened." (40) The standard of proof thus attempts to balance the risk between so-called Type I errors (i.e., false positives), such as an erroneous finding of liability in a civil case or the conviction of an innocent person, and Type II errors (i.e., false negatives), such as the denial of a meritorious claim in a civil case or an erroneous acquittal of a criminal defendant. (41)
Second, the standard of proof is used to "indicate the relative importance attached to the ultimate decision." (42) In particular, the Supreme Court has recognized that a heightened standard of proof--i.e., something greater than a mere preponderance of the evidence--is required when fundamental rights or important liberty interests are at issue. (43)
Because the costs of error and consequences of the outcome obviously vary in different settings, three primary standards of proof exist at common law. From lowest to highest degree of certainty required, they are: (1) preponderance of the evidence, (2) clear and convincing evidence, and (3) beyond a reasonable doubt. (44)
Preponderance of the evidence is the standard of proof for most issues in civil litigation, reflecting the legal system's assumptions that the standard "results in a roughly equal allocation of the risk of error between litigants" (45) and the costs of a "mistaken judgment for the plaintiff is no worse than a mistaken judgment for the defendant." (46) In other words, as Chief Justice Rehnquist explained, "[t]he preponderance-of-the-evidence standard ... is employed when the social disutility of error in either direction is roughly equal." (47)
In contrast, proof beyond a reasonable doubt is constitutionally required for conviction in criminal cases. (48) This standard has been described by the Court as the "bedrock ... principle whose 'enforcement lies at the foundation of the administration of our criminal law.'" (49) In terms of allocation of risk, the beyond a reasonable doubt standard places almost all risk upon the government rather than the criminal defendant. (50) In other words, it strongly prefers Type II errors (erroneous acquittals) instead of Type I errors (erroneous convictions). This is exemplified by Blackstone's famous formulation that "it is better that ten guilty persons escape, than that one innocent suffer." (51) The preference for Type II errors embodied in the beyond a reasonable doubt standard is due to the severe consequences of a conviction--the loss of liberty as well as the stigma and collateral consequences that accompany a criminal record. (52)
Clear and convincing evidence represents an "intermediate standard" between preponderance of the evidence and reasonable doubt. (53) Although the exact terminology used to refer to this heightened standard of proof has varied over time, (54) jury instructions often explain that it is higher than a preponderance of the evidence standard, but it does not require proof beyond a reasonable doubt. (55) Less commonly employed than the preponderance standard used for most civil issues, clear and convincing evidence is often required when "[t]he interests at stake ... are deemed to be more substantial than mere loss of money." (56) For example, the Court has held that clear and convincing evidence is required for involuntary civil commitment due to mental illness, (57) deportation for violation of immigration law, (58) and the termination of parental rights. (59) In patent law, a variety of issues must be proven by clear and convincing evidence, including willful infringement, (60) inequitable conduct, (61) correction of inventorship, (62) and overcoming the presumption of priority in an interference proceeding with an issued patent. (63)
C. Previous Empirical Studies
1. Survey Evidence
Several previous studies have attempted to empirically measure standards of proof by asking potential decision makers to assign percentage probabilities to the various standards.
In the early 1970s, Rita James Simon and Linda Mahan asked judges and jurors to quantify the beyond a reasonable doubt standard and the preponderance of the evidence standard as percentage probabilities. (64) Both judges and jurors believed that the beyond a reasonable doubt standard demanded a high likelihood in order to be satisfied. Specifically, the median judge quantified this standard as requiring 88% probability (i.e., 8.8 on a 10-point scale), with almost a third of judges responding that it required 100% probability (i.e., 10.0) and another third that it required 90% or 95% probability (i.e., 9.0 or 9.5). (65) Similarly, the median juror (66) quantified the beyond a reasonable doubt standard as requiring 86% probability (i.e., 8.6 on a 10-point scale), with over a third articulating it as requiring 100% probability (i.e., 10.0) and another third between 80% and 95% probability (i.e., 8.0 to 9.5). (67)
In the same study, however, judges and jurors substantially disagreed about the percentage likelihood needed to satisfy the preponderance standard. For the median judge, this standard was satisfied with a 54% probability (i.e., 5.4 on a 10-point scale). (68) In contrast, the median juror believed that 75% probability was required (i.e., 7.5 on a 10-point scale) to satisfy the preponderance standard. (69)
In the early 1980s, C.M.A. McCauliff conducted a survey of all active, senior, and retired federal judges regarding the level of certainty, on a scale of 0%-100%, required by nine phrases treated as standards of proof. (70) Her results generally paralleled the judges' responses in the Simon & Mahan study. For preponderance of the evidence, the overwhelming majority of judges (154 of 175) equated this standard with a probability of 50% or 60%, with an average probability of 55.3%. (71) For beyond a reasonable doubt, nearly all judges (160 of 171) rated this standard between 80% to 100% probability, with an average probability of 90.3%. (72) Finally, for the clear and convincing evidence standard, the majority of judges (111 of 170) rated this standard as 70% to 80% probability, with an average probability of 75.0%. (73)
More recently, Bradley Saxton surveyed actual jurors in civil and criminal cases after trial to determine their comprehension of various jury instructions, including instructions regarding the standard of proof. (74) In criminal cases, jurors were asked whether the state only needed to convince the jury "that it is more likely than not" that the defendant committed the accused crime. (75) The vast majority of jurors--almost 85%--correctly responded that they were "very sure" or "pretty sure" this statement was false, while almost 10% were "very sure" or "pretty sure" it was true and 6% did not know. (76) In civil cases, jurors were asked the opposite question--whether the jury "has to be convinced beyond a reasonable doubt that the plaintiff's claims are correct." (77) The results for this question suggested that a significant number of jurors misunderstood the preponderance standard: 38% incorrectly responded that they were "very sure" or "pretty sure" it was true, while an additional 3% did not know. (78)
In sum, these studies suggest that judges generally understand the preponderance standard to be satisfied by any probability exceeding 50%, clear and convincing evidence to be in the range of 70% to 80% probability, and beyond a reasonable doubt to require at least 80% probability. (79) It is unclear whether lay jurors have a similar understanding. The Simon & Mahan and Saxton studies surveying actual jurors had sample sizes that were too small to draw definitive conclusions, failed to ask for a percentage probability for all three standards of proof, or both. However, these studies suggest that some jurors perceive the preponderance standard to be somewhat more rigorous than do judges. Furthermore, these surveys do not test whether jurors or judges actually apply these standards consistently in litigation.
2. Experimental Studies
Another area of empirical research has been the use of experiments to test the impact of standards of proof as articulated in jury instructions, primarily in criminal cases. In several experiments, "the wording used to convey the standard of proof" was found to have "a substantial impact" on mock jurors' verdicts. (80)
In a study conducted by the Jury Project at the London School of Economics in the early 1970s, panels of mock jurors in the United Kingdom heard a tape-recorded trial of a rape case. (81) The only variation was the jury instruction on the standard of proof. (82) Two groups of mock juries received differing instructions regarding the beyond a reasonable doubt standard, (83) while a third group received a preponderance of the evidence standard. (84) The juries' conviction rates were highest for the preponderance instruction, slightly lower for the first reasonable doubt instruction, and much lower for the second reasonable doubt instruction. (85) This study "suggests that jurors can distinguish between" preponderance of the evidence and beyond a reasonable doubt standards, although one jury scholar noted that the specific wording of jury instructions regarding the latter standard could result in substantially different outcomes. (86)
In a 1985 article, Dorothy K. Kagehiro and W. Clark Stanton reported the results of several experiments that tested all three standards of proof using jury instructions that explained the relevant standard either as a percentage probability ("quantified definition") or with a conventional model jury instruction that did not explicitly quantify each standard ("legal definition"). (87) The subjects, undergraduate psychology students, were divided into an experimental and a control group. (88) All subjects were presented with a summary of the evidence in a civil trial. (89) Then the subjects in the experimental group, but not the subjects in the control group, were instructed regarding the standard of proof. (90) Finally, all subjects were asked to individually reach a decision regarding the plaintiff's claim. (91) The authors reported that although "verdicts favoring the plaintiffs decreased as the standard became stricter" in both experiments, (92) these differences were only statistically significant for the quantified definitions. (93) In contrast, "[f]or the legal definitions, the multivariate effect [for the] standard of proof was not significant." (94)
In 1996, Irwin A. Horowitz and Laird C. Kirkpatrick tested the impact of various definitions of the reasonable doubt standard. (95) The subjects--jury-eligible adults--heard one of two versions of a murder trial, one that was calibrated to be a "weak" case likely to result in a not guilty verdict, while the other was a "strong" case with evidence strongly suggestive of guilt. (96) Each juror then received one of five definitions of reasonable doubt and was asked to reach a preliminary decision. (97) Subsequently, panels of mock juries were assembled and deliberated with no time limitation. (98) The authors reported statistically significant differences between mock juries that received varying definitions of reasonable doubt. (99)
In general, although some of these experiments have limitations that impact their external validity--such as the use of undergraduate psychology students rather than a more broadly representative population--and the evidence is not clear cut, (100) they cumulatively suggest that jurors' decisions may be swayed by the standard of proof. In addition, they imply that the particular wording of jury instructions can play a significant role in jurors' decision making. However, none of the previous empirical studies directly answers the question we study here--whether altering the burden of proof in civil litigation is likely to result in different outcomes by a diverse, broadly representative group of jury-eligible citizens. This unresolved question is the subject of our experiment.
III. Microsoft v. i4i and the Presumption of Validity in Patent Law
Section 282 of the Patent Act, first adopted in 1952, expressly provides that patents are to be presumed valid, but does not specify the standard of proof. (101) Before the creation of the Federal Circuit in 1982, some of the regional circuits held that, in certain circumstances, the standard for overcoming the presumption of validity was preponderance of the evidence. (102) However, the Federal Circuit, in a decision not long after its creation, held that the proper standard of proof was clear and convincing evidence. (103) According to the Federal Circuit, the clear and convincing standard is "constant and never changes." (104)
In 2011, the U.S. Supreme Court directly confronted the question of how high the standard of proof should be for proving patent invalidity in litigation. (105) More specifically, the Court in i4i examined whether Section 282 of the Patent Act required clear and convincing evidence to overcome a patent's presumption of validity.
In the lower courts, Microsoft unsuccessfully challenged i4i's patent on the basis that i4i had previously sold software that embodied the invention more than one year before it applied for a patent, which if true would render it invalid under Section 102(b) of the Patent Act. (106) Because the prior art sale of software had not been presented by i4i to the Patent Examiner when the USPTO was considering i4i's patent application, Microsoft argued that it should be required to prove invalidity by only a preponderance of the evidence. (107) The district court judge rejected Microsoft's contention, ruled that Microsoft had to prove invalidity by clear and convincing evidence, and denied Microsoft's request for an alternative instruction. (108) Thereafter, the jury found Microsoft to have willfully infringed i4i's patent, rejected Microsoft's invalidity defense, and awarded over $200 million in damages. (109) The Federal Circuit affirmed the finding of infringement, validity, and the damages award. (110) The Supreme Court granted Microsoft's petition for certiorari on the issue of the proper standard of proof for Microsoft's invalidity defense. (111)
At the Supreme Court, Microsoft advanced two arguments in support of reversal. First, it asserted that an accused infringer need only prove a patent claim invalid by a preponderance of the evidence, rather than by clear and convincing evidence. (112) Because [section] 282 was silent regarding the standard for carrying the burden of proving invalidity, Microsoft contended that "the default standard of proof in civil cases"--preponderance of the evidence--should apply. (113) In addition, it argued that the private property rights granted by a patent fell outside the "narrow category of cases implicating uniquely important individual liberty interests," which "warrant a heightened standard of proof." (114)
Second, Microsoft argued in the alternative that the preponderance of the evidence standard was required at least when the party challenging the patent's validity relied on prior art references not considered by the USPTO. (115) Even if a heightened standard of proof was warranted out of deference to the USPTO's decision to grant a patent, this deference was appropriate only when the Patent Examiner had actually considered the relevant evidence regarding patentability. (116) If the examiner had not considered some of the relevant prior art, then, as the Court stated in KSR v. Teleflex, "the rationale underlying the presumption of ... validity--that the [US]PTO, in its expertise, has approved the claim--seems much diminished ..." (117)
In response, i4i argued that prior to the adoption of the 1952 Patent Act, the Supreme Court had "repeatedly and consistently held that the presumption of patent validity imposed a heightened standard to prove invalidity." (118) Section 282, i4i contended, codified this common-law precedent. (119) In addition, i4i argued that the Federal Circuit had correctly required a heightened standard for proving invalidity because such a standard both promoted innovation by "foster[ing] strong, stable patent rights" and "properly incorporate[d] deference to the [US]PTO." (120)
The Supreme Court rejected both of Microsoft's proposals. (121) It first analyzed the statutory text of Section 282 of the Patent Act. The Court concluded that although the statute clearly provided that the party challenging a patent claim's validity bore the burden of proof, the statute was silent on the standard of proof. (122) The Supreme Court was clear that the statute's use of "burden of proof referred to the "burden of persuasion." (123) The statute also specified that patents are "presumed valid." (124) The Court further agreed with i4i's argument that the phrase "presumed valid" had a settled meaning at the time of Section 282's adoption that incorporated a heightened standard of proof. (125) The Supreme Court relied heavily upon its 1934 decision in Radio Corporation of America v. Radio Engineering Laboratories, Inc. ("RCA") (126) In RCA, Justice Cardozo's majority opinion held that a patent challenger "bears a heavy burden of persuasion, and fails unless his evidence has more than a dubious preponderance." (127) As a result, the "dubious" preponderance standard was insufficient to find a patent invalid. (128)
Older Supreme Court decisions used similarly unequivocal language to describe the standard of proof for finding a patent invalid. For instance, in 1874, the Court described the burden of proof relating to prior inventorship as "rest[ing] upon [the defendant], and every reasonable doubt should be resolved against him." (129) Relying upon these cases, the Supreme Court held in i4i that the term "presumption of validity" had a settled meaning when the Patent Act was adopted in 1952 that required the patent challenger to satisfy this heightened standard of proof by clear and convincing evidence. (130)
The Court also addressed Microsoft's alternative argument that the preponderance standard should apply at least where evidence at trial was not provided to the USPTO during examination of the patent. The Court was reluctant to adopt a fluctuating standard that might require different burdens of proof within a single lawsuit. (131) However, the Court acknowledged that numerous courts of appeals before the 1952 Act stated that the presumption of validity was "weakened" or "dissipated" when the evidence had not previously been considered by the USPTO. (132) Despite this acknowledgement, the Court rejected the view that these statements meant that the standard of proof should be anything but clear and convincing. (133) Thus, the Court held that a patent claim must be proven invalid by clear and convincing evidence in all circumstances.
However, and important for our study, the Court decided that new evidence in invalidity challenges was entitled to different treatment in jury instructions. (134) The Court noted that previously unconsidered evidence may "carry more weight" than evidence previously reviewed by the USPTO. (135) While the burden of proof always remains clear and convincing, the Court noted that a special instruction may be provided to the jury in these circumstances:
[I]f the [US]PTO did not have all material facts before it, its considered judgment may lose significant force. And, concomitantly, the challenger's burden to persuade the jury of its invalidity defense by clear and convincing evidence may be easier to sustain. In this respect, although we have no occasion to endorse any particular formulation, we note that a jury instruction on the effect of new evidence can, and when requested, most often should be given. When warranted, the jury may be instructed to consider that it has heard evidence that the [US]PTO had no opportunity to evaluate before granting the patent. (136)
Thus, the Court urged district court judges to provide a special jury instruction when new evidence regarding invalidity was first presented in litigation. (137) While the Court apparently felt that the burden of proof always remained clear and convincing, this language suggested the burden might be somewhat easier to satisfy in these circumstances.
IV. Methodology and Study Design
A. Hypotheses About i4i's Impact
The impact of changing the standard of proof on jurors' decisions in civil cases is uncertain. More specifically, in the area of patent law, intellectual property scholars and others have disagreed about the effect of changing the standard for proving a patent invalid. For example, Doug Lichtman and Mark Lemley argue that "[w]hile we can't prove that presumptions matter, we believe that they likely do, at least at the margins." (138) However, they admit that "this is an area of uncertainty," and the legal community "know[s] far less than [it] should about how presumptions affect litigation decisions." (139)
Judge William Alsup, who has presided over numerous patent cases in the Northern District of California, has called the clear and convincing standard a "legal earthwork fortified by a protective moat" that creates "a huge advantage for the patent holder" in validity challenges. (140) Similarly, in a brief filed in the i4i case, Apple and Intel--which both own large patent portfolios and appear as frequent defendants in infringement litigation--argued that the clear and convincing evidence standard "has a compelling effect on jurors" and "insulate[s] weak patents from invalidity verdicts." (141)
Mark Janis has asserted that "the precise verbal formulation that we use for the standard of evidence for overcoming the presumption" of patent validity may be at least as important as the standard itself. (142) As a result, he argues that "one possible outcome of ... the preponderance standard for overcoming the presumption of patent validity is that [it] will cause little difference in the outcomes of cases." (143)
Based on the assumption that standards of proof matter, we developed three hypotheses for our experiment:
1. A clear and convincing standard will result in fewer findings of invalidity than a preponderance standard.
2. A clear and convincing standard will result in fewer findings of invalidity than the same standard combined with an i4i-type instruction.
3. A clear and convincing standard combined with an i4i-type instruction will result in fewer findings of invalidity than a preponderance standard.
As explained in the following section, we used an experimental approach to test each of these hypotheses.
B. Why an Experiment?
Using an experimental methodology to measure the effect of varying the standard of proof has many important advantages. There is burgeoning literature about experiments conducted by legal scholars. (144) Experiments permit the manipulation of a single variable to determine if changes in that variable affect other variables. (145) Like a classic double-blind medical experiment, this method permits us to randomly assign the change, in our case the standard of proof, while holding everything else constant. Any differences in results between conditions thus can be attributed to the randomly assigned variable, the standard of proof. (146)
We decided against a common approach used by empirical legal scholars: studying observational data. (147) Studying standards of proof by relying on observational data is extremely difficult. Unlike experiments, observational data is not controlled by the researcher. (148) Instead, the researcher records the data after it occurs naturally. (149) The fact that the studied conditions are not randomly assigned makes it more difficult to draw causal inferences from observational data than from experiments. (150)
For our particular research question, we believe that an experimental approach offers several advantages compared to observational data generated in litigation. First, the courts do not frequently change the standard of proof. Consequently, almost all cases in a given area of law are decided using the same standard. In particular, if we wanted to rely upon observational data to analyze the effect, if any, of the Supreme Court's decision regarding the standard for proving a patent invalid, we would need to wait for a sufficient number of trials to occur, which would likely take years. (151) Furthermore, observational data from litigation would not be available for one of the possible standards of proof--preponderance of the evidence--because this standard was explicitly rejected by the Court.
Second, and perhaps more importantly, if the standard of proof is changed, then other features of the cases may change as well. For instance, different cases may settle before trial if the ultimate question for the jury is based upon a clear and convincing standard of proof as opposed to a mere preponderance standard. (152) In other words, changing the standard of proof may also result in different cases being selected for resolution by juries. Priest and Klein famously argue that regardless of the law, cases where there is a high degree of certainty regarding the outcome will settle, so those cases that are tried are always the closest cases, the 50-50 cases. (153) Regardless of whether selection effects actually result in a 50-50 split for the resolution of individual issues like patent invalidity, (154) any significant change in the types of cases that reach a jury can make it more difficult to unpack causal relationships. An experimental approach overcomes these problems by permitting the researcher to control for all variables other than the one being studied. (155)
C. Study Design
After deciding upon an experimental methodology, we turned to study design. We desired a patent litigation scenario that was understandable to subjects, so we shied away from describing a dispute involving complex technology, such as biotechnology, pharmaceuticals, or computer software. Thus, we created a one-and-a-third page fact pattern based upon a real case (not i4i) involving a relatively simple technology--golf ball design. (156) We also wanted a fact pattern without a clear answer in law. In other words, we wanted the subjects to have some discretion on which way they decided the case. The case selected as the basis for our fact pattern provided this, as it involved two separate jury verdicts that reached opposite conclusions regarding invalidity, (157) and the Federal Circuit on appeal stated that "the evidence at trial was such that the jury could have rationally reached either verdict with regard to [the invalidity of] the asserted claims." (158)
The same fact pattern and summary of the arguments were provided to all subjects using a between-subjects design. (159) We limited the fact pattern to a single issue: whether the patent was invalid because it was obvious. Under the law, a patent is obvious if the differences between the prior art and the invention would have been apparent to a person in the field at the time the invention was made. (160) To focus our experiment on the effect of jury instructions, we randomly assigned the subjects one of three different jury instructions, as described in more detail below. A copy of the fact pattern, summary of the parties' arguments, and the jury instructions used in the experiment are reproduced in Appendix A. The sequence of steps in the experiment is reproduced in Figure 1 below.
Our fact pattern involved a dispute regarding patent infringement between two fictitious competitors, Acme Golf, Inc. ("Acme") and Bravo Sporting Equipment Co. ("Bravo"). (161) Acme's invention related to a new "three-piece" golf ball which combined the features of prior art "two-piece" golf balls. (162) The fact pattern explained that when the USPTO examined and allowed the application for a patent, it had only been aware of several two-piece golf balls but no three-piece golf balls. However, during the course of litigation, these fictitious parties discovered a prior patent (issued to an inventor named "Charles") that disclosed a three-piece golf ball. This prior patent was different from the Acme invention (it used different materials for some of the layers), but it was closer than any of the prior art considered by the USPTO. Thus, as in the i4i case, a key piece of prior art had not been considered by the USPTO during examination and was offered as evidence in litigation by the party challenging the patent's validity.
The scenario concluded with a paragraph describing each of Acme's and Bravo's arguments regarding the issue of patent obviousness. (163) These arguments are ones commonly made by litigants in patent trials. For the patent owner, the summary stated that no individual piece of prior art taught all of the elements of the claimed invention, that the invention had benefits over the prior art, that the Charles patent was directed to a very different problem than the Acme patent, and that the jury should defer to the "trained Patent Examiner, who was in the best position to determine whether Acme's claimed invention was obvious." (164) For the patent challenger, the summary argued that the Acme invention was merely a combination of preexisting items, that the invention's modification of the prior art was trivial, and that the jury should not defer to the Patent Examiner because the patent would not have been granted if the Examiner had known of the Charles patent. (165)
After the fact pattern and summary of the parties' arguments, each subject received one of three randomly assigned jury instructions regarding the legal standard for determining whether the patent was invalid. The one-page jury instructions varied only regarding the standard of proof; they contained identical descriptions regarding obviousness in patent law. (166) For the standard of proof, three different instructions were given: (1) clear and convincing evidence; (2) clear and convincing evidence with an additional instruction based upon the Court's decision in i4i; and (3) preponderance of the evidence. We adapted language from model patent jury instructions promulgated by the American Intellectual Property Law Association ("AIPLA") (167) and the National Jury Instruction Project to explain these standards. (168) Specifically, the instructions regarding these conditions stated:
Clear and Convincing Evidence:
Clear and convincing evidence is evidence that shows it is highly probable that the patent was obvious. This is a higher standard of proof than a preponderance of the evidence, which means more probable than not. However, clear and convincing evidence is lower than the beyond a reasonable doubt standard used in criminal cases.
Clear and Convincing Evidence with i4i-type Instruction:
Clear and convincing evidence is evidence that shows it is highly probable that the patent was obvious. This is a higher standard of proof than a preponderance of the evidence, which means more probable than not. However, clear and convincing evidence is lower than the beyond a reasonable doubt standard used in criminal cases. The burden of proving obviousness is more easily satisfied when, as in this case, the prior art on which the claim of obviousness is based was not considered by the Examiner. (169)
Preponderance of the Evidence:
A preponderance of the evidence is evidence that shows it is more probable than not that the patent was obvious. This is a lower standard of proof than either clear or convincing evidence or the beyond a reasonable doubt standard used in criminal cases.
After reviewing the jury instructions, the subjects were presented with several questions. (170) First, we asked subjects for their ultimate decision on invalidity. (171) For subjects who received the first two conditions, we asked whether the patent challenger had proven by clear and convincing evidence that the patent was obvious. For subjects who received the preponderance condition, we asked the same question but substituted "preponderance of the evidence" for "clear and convincing." Second, subjects were asked about their level of confidence in their answer regarding invalidity on a 1-7 scale (1 = least confident, 7 = most confident). A third question asked the subjects to indicate "how likely do you think it is that Acme's patent is obvious" on a 0-100% scale (in 10% increments).
On the next page of the experiment, we included three validation questions designed to exclude subjects who had not read and/or adequately understood the fact pattern. (172) On the final page, the study concluded with several demographic questions about the subject's age, race, gender, education, U.S. citizenship, patent experience, and golf experience.
We ran a preliminary study to validate the language used. Approximately ten people read and completed the experiment. We discussed the experiment with these testers and made minor linguistic adjustments to improve comprehension. Subsequently, we conducted a pilot study on incoming first-year law students at Chicago-Kent College of Law during orientation in August 2011. (173) After the pilot study, we believed that the study was understandable to subjects.
We conducted the experiment upon members of the general public eligible for U.S. jury duty in November 2011 (N = 500). (174) Subjects were recruited through Amazon's Mechanical Turk, (175) a website that allows participants to complete Human Intelligence Tasks for compensation. In the initial iteration, we compensated subjects $2 for successfully completing the experiment. (176) Recent studies have used Mechanical Turk to successfully replicate the results of well-known social science experiments, including the prisoner's dilemma, providing evidence that subjects recruited through Mechanical Turk "behave similarly to subjects in physical laboratories." (177) Research using Mechanical Turk-based experiments has been published and presented in a number of academic fields, including peer-reviewed journals, which suggests that it meets scholarly expectations for quality web-based experiments. (178)
In an attempt to mirror jury-eligible persons, we limited the subjects to those within the United States who were U.S. citizens, (179) who understood English, and who were at least eighteen years old. (180) Subjects who wished to participate were redirected to the experiment, which was hosted by Qualtrics, a widely used online survey program. (181) On average, subjects spent seventeen minutes completing the experiment. The program prohibited subjects from submitting multiple responses.
Based on self-reported demographic information, the subjects of our online experiment were more female (61%), younger (61% between 18-34 years old, 38% between 35-64 years old), somewhat less racially and ethnically diverse (82% white, 7% African-American, 5% Hispanic, 4% Asian-American, 3% other), and better educated (48% had undergraduate degree or higher) than the general population. (182) When compared to a sample of jurors in long-term federal trials (20+ days) in a study by the Federal Judicial Center, (183) our experiment's subjects were of similar diversity with respect to race and sex, but were better educated and younger. (184) Notably, the subjects' demographics were more representative of the overall U.S. population than the law students used in our pilot study, as well as the undergraduate students used in previous experimental studies regarding jury behavior that have been published in peer-reviewed psychology journals. (185)
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||I. Introduction through Methodology and Study Design, p. 429-458|
|Author:||Schwartz, David L.; Seaman, Christopher B.|
|Publication:||Harvard Journal of Law & Technology|
|Date:||Mar 22, 2013|
|Previous Article:||Discovering Facebook: social network subpoenas and the Stored Communications Act.|
|Next Article:||Standards of proof in civil litigation: an experiment from patent law.|