PANEL COMPOSITION AT THE COURT OF APPEAL FOR ONTARIO.
I INTRODUCTION 12 II BACKGROUND OF THE COURT 13 I. The Court of Appeal for Ontario 13 III RELEVANT LITERATURE 15 I. Yahya and Stribopoulos: an Empirical Study of the Court 15 II. A Check for Randomness: Regressions 16 III. A Check for Randomness: Panel Composition and Simulations 18 IV ANALYSIS OF THE COURT 18 I. Scope of Analysis 18 II. Statistical Analysis: Panel Presence 20 III. Statistical Analysis: Panel Presence and Regressions 22 IV. Statistical Analysis: Panel Composition 24 V DISCUSSION AND CONCLUSION 28 I. Discussion of Findings 28 II. Study Limitations 29 III. Conclusion 30
Canada is a nation governed by the rule of law; this is a "fundamental postulate of our constitutional structure." (1) An independent and impartial judiciary is critical to maintaining the rule of law. Indeed, in criminal matters the right to "a fair and public hearing by an independent and impartial tribunal" is guaranteed in Section 11(d) of the Canadian Charter of Rights and Freedoms. (2) Canadians therefore have a right to receive a fair hearing. Further, as the Supreme Court of Canada has held, "the courts should be held to the highest standards of impartiality... Fairness and impartiality must be both subjectively present and objectively demonstrated to the informed and reasonable observer." (3)
The personal attributes of the judge--whether age, sex, political views or otherwise--should therefore have no bearing on the outcome of a given case. And yet, judges are human and carry biases and beliefs that may influence their decisions. Indeed, empirical studies have shown that judges' political affiliations may affect their judicial decisions. (4) When courts hear cases in panels, as opposed to en banc, the possibility exists that the composition of the panel will influence the outcome of the case. The process of appointing judges to panels therefore becomes critical. Are chief justices selectively assigning judges to panels to craft the law in their preferred direction? Or, are panels selected randomly, ensuring that human bias does not affect panel composition? Should panel selection necessarily be random, to avoid any potential for manipulation of panels?
From a normative perspective, random panel composition is not without downsides. Ensuring a random process can be virtuous, as it helps to avoid panels being gamed to sway case outcomes. Yet, it may also fail to take advantage of the beneficial aspects of a non-random process. For example, a judge with criminal law expertise may be uniquely qualified to hear a complex criminal matter, while a judge with corporate law expertise may bring a valued perspective to a corporate matter. Sitting junior judges alongside senior judges may enhance their training. Ensuring diversity of age, gender and culture among judges on a panel may also have advantages. A strictly random process would fail to take advantage of these possible benefits.
As the highest court for the province of Ontario, the Court of Appeal for Ontario (the "Court") is one of the most important appellate courts in the nation. Given the large number of judges on the Court and the small panel sizes used, there is the chance that case outcomes are affected by the composition of the panels. That is, the identity and personal attributes of the specific judges on a panel could be impacting the outcome of a given case. However, research to date has not examined panel composition at the Court in a detailed manner.
This paper will examine panel composition at the Court, seeking to determine whether panel composition is random or non-random. The sample analyzed in this study comprises all judgments of the Court in the 2013 calendar year. Overall, this study has led me to conclude that: (i) panel composition was consistent with randomness for full judgments of the Court, but (ii) panel composition was not consistent with randomness for endorsements of the Court. However, caution must be taken so as not to over-emphasize these findings. As will be discussed, they are subject to significant qualifications and limitations.
In Part II I will provide a brief overview of the Court and its panel selection process. In Part III I will discuss literature pertinent to the Court specifically and panel selection more generally. In Part IV I will review my statistical analysis of the Court and in Part V I will conclude with a discussion of my findings and potential implications.
II BACKGROUND OF THE COURT
I. THE COURT OF APPEAL FOR ONTARIO
The Court serves as the highest court for the province of Ontario, and hears appeals primarily from decisions of the Ontario Court of Justice and the Superior Court of Justice; the Court also hears a small number of appeals from the Ontario Review Board. The Court receives and disposes of a significant number of appeals. Over the five-year period from 2009 to 2013, the Court disposed of an average of 1,615 appeals per year. (5) Yet, fewer than 3% of the Court's decisions are appealed to the Supreme Court of Canada. (6) For the vast majority of cases, therefore, the Court is the final court of appeal.
As of March 1, 2015, there are 28 justices of the Court. The majority of cases are heard in panels of three, though occasionally panels of five are used or an appeal is disposed of by a single judge. Given the large number of judges and small panel sizes, the potential for panel effects looms large. That is, given the possibility that one panel of three judges may not always reach the same decision as another panel of three judges, the composition of the panels can impact case outcomes. Indeed, as Ian Greene and others put it, there is a "luck of the draw dimension" to the process of appellate decision making. (7)
The Ontario Courts of Justice Act (the "Act") is largely silent on panel selection and appears to leave the process entirely within the discretion of the Chief Justice. The Act states that "the Chief Justice of Ontario has general supervision and direction over the sittings of the Court of Appeal and the assignment of the judicial duties of the court." (8) The Act further states that panels must be composed of an uneven number of judges. (9) Another empirical study of the Court, discussed later in this paper, notes that the Registrar is responsible for determining panel composition. (10)
In practice, the Court strives to assign judges to cases using a random process, while making efforts to have at least one senior judge and one subject-area specialist per panel. (11) For example, a criminal panel would have at least one senior judge and at least one judge with expertise in criminal law. While, as a generalist court, the judges hear appeals across different areas of law, in practice, judges with expertise in a particular area may hear more appeals in that area.
Every week, the Court holds at least one criminal panel and one civil panel. Before judges are assigned to panels, cases are scheduled by a coordinator. The coordinator is unaware of the legal substance of the cases. Once the cases have been scheduled, the Senior Legal Officer, taking into account scheduling conflicts, manually assigns judges to panels. The number of panels depends on the number of cases scheduled to be heard. Often there are three or four panels in any given week. Judges will typically hear cases for two weeks, followed by two weeks of crafting judgments. While the Chief Justice and Associate Chief Justice may review the draft schedules, they very rarely intervene.
Overall, this process attempts to strike a balance between taking advantage of judicial resources--specifically, practice area expertise and seniority--while attempting to maintain panel composition consistent with a random process. Objectively, however, a process designed to take advantage of seniority and expertise is not random.
III RELEVANT LITERATURE
I. YAHYA AND STRIBOPOULOS: AN EMPIRICAL STUDY OF THE COURT
Yahya and Stribopoulos' 2007 article analyzed all decisions of the Court and all votes from 1990 to 2003, tracking the type of litigant, political party that appointed the judge (hereinafter referred to as a judge's political party) (12) and gender of the judge. While the focus of this study was on case outcomes, as a rare empirical study of the Court it nonetheless holds valuable lessons for students of the Court.
Yahya and Stribopoulos note that across their sample of over 4,000 cases, 95% of the Court's decisions were unanimous. (13) Further, there was little variation in this rate across case categories. Yahya and Stribopoulos did not, however, include endorsements; these shorter decisions represent a significant amount of the Court's work but due to their brevity and, in the pre-internet age, general lack of publication, they are of little precedential value.
While the greater part of Yahya and Stribopoulos' study was on case outcomes, they did conduct a brief analysis of panel composition. (14) For example, they found that 59% of panels were of mixed gender, 40.5% were all male and just 0.5% were all female. (15) Further, they noted that 73% of panels were mixed by political party, while 18% were composed of all Conservative judges and 9% were composed of all Liberal judges. (16) They also described the percentages of votes cast by female judges, male judges, Conservative judges and Liberal judges. However, none of these figures were analyzed for statistical significance, and the discussion of panel composition was brief.
The bulk of the paper was devoted to an analysis and discussion of case outcomes as a function of the area of law, the disposition of the trial court, the nature of the complainant, and the gender and political party of the judges. while noting a "remarkable amount of cohesiveness" (17) on the Court, the authors found some influence of political party of appointment on judicial decisions. For example, in certain contexts, Conservative judges preferred more conservative outcomes in Charter litigation. (18) However, when panels were mixed--that is, they included both Conservative and Liberal judges--the influence of politics was dampened. (19) Gender also had a measure of impact in case outcomes; female judges favoured complainants in sexual abuse and domestic violence cases, and favoured mothers in family law cases. (20) The converse was true for male judges. (21) But again, these outcomes dampened with mixed panels. (22)
While Yahya and Stribopoulos made a host of interesting findings relating to case outcomes, they spent little time discussing panel composition. Nonetheless, their findings led them to recommend a panel selection process that "deliberately ensures political and gender diversity while also respecting the wisdom of randomness in assembling the panels." (23) Indeed, given their findings on case outcomes, the authors proposed that a "process that ensures political and gender diversity on appeal court panels would be an easy task and would go a considerable distance toward eliminating reasonable perceptions of bias that this study would now seem to have empirically validated." (24) In light of their finding that decision making at the Court may be influenced by judicial characteristics, the authors proposed a panel composition process that embraces both diversity and randomness.
II. A CHECK FOR RANDOMNESS: REGRESSIONS
Two recent articles on the U.S. Courts of Appeals employed research methods that are highly applicable to this study. Accordingly, before proceeding to discuss my analysis, I will provide a brief discussion of these articles.
In 2006, Cass Sunstein and others produced an empirical study of the U.S. Courts of Appeals. (25) Sunstein found that judges appointed by Democrats (Democratic judges) vote differently than judges appointed by Republicans (Republican judges). However, while personal judicial characteristics affect judicial decision making, these differences are not completely determinative of case outcomes. (26) Further, Sunstein noted certain panel effects, or in other words, that the composition of the panels may impact voting patterns and case outcomes. Sunstein observed that when panels are mixed, ideological voting is, in certain contexts, dampened. Conversely, when panels are composed entirely of judges from one party, ideological voting is, in certain contexts, amplified. (27)
In 2010, Matthew Hall built on and critiqued Sunstein's work. (28) Hall proceeded by attempting to adjust for certain omitted variables and alleged methodological errors in Sunstein's study. For example, Sunstein had explicitly assumed that the assignment of judges to panels was done randomly. (29) Following this logic, Hall posited that one should "not be able to predict a judge's party by knowing the types of cases assigned to that judge." (30) In studying panel composition, however, Hall regressed a judge's political party on the types of cases they heard. Hall found that Sunstein's claim of random panel assignments was not borne out, and that, in certain instances, a judge's political party could be determined based on the types of cases they heard. (31) Interestingly, Hall reviewed the courts of appeals' operating procedures and conducted interviews with court officials to find out the processes actually used. Some circuits randomly assigned judges via a computer program or by drawing names from a hat. However, in other circuits, clerks of chief justices manually assigned judges to cases, introducing the potential for bias. (32) Hall then removed the non-random circuits from the study and found that, perhaps unsurprisingly, at the remaining circuits, random panel selection was observed. (33)
Hall then proceeded to analyze case outcomes, after correcting for what he viewed as methodological errors in Sunstein's work, including the mistaken assumption of random panel assignments. He found that Sunstein underestimated the effect of partisanship on case outcomes in certain areas and reached mistaken conclusions regarding the existence of a partisan effect on outcomes in other areas. (34) Further, according to Hall, Sunstein overestimated the similarity of partisan effects between circuits. (35)
While the majority of Hall's paper focused on analyzing case outcomes, for the purposes of this paper, the discussion of randomness and panel composition is particularly pertinent. Hall, in his conclusion, noted that claims of randomness in the panel selection process must be evaluated and tested rigorously. However, regressing types of cases heard and political party of appointment is, on its own, an incomplete test of randomness. It fails to account for panel composition, while looking only at the cases heard by individual judges. For example, consider two scenarios of three panels of three judges. In one scenario, all three panels are composed of two Republicans and one Democrat. In the second scenario, two panels are all Republican, while one panel is all Democratic. Using only regressions to check for randomness could fail to detect any difference between the two; after all, they both end up with six Republican votes and three Democratic votes. Accordingly, a more detailed analysis that captures the composition of the panels is called for.
III. A CHECK FOR RANDOMNESS: PANEL COMPOSITION AND SIMULATIONS
A draft paper by Chilton and Levy addressed this exact shortcoming. (36) After noting, as Hall did, that random federal appellate court panel assignments are often assumed, the authors undertook a large scale simulation of panel composition. They then compared these results to the actual panels and analyzed randomness by political party. The authors noted that, for certain circuits, panel composition displays evidence of non-randomness and the authors therefore challenged the continued assumption of randomness in further empirical research. (37)
Chilton and Levy's rigorous statistical approach serves to complement the regression work conducted by Hall. My statistical analysis is informed by both of these approaches. Like Hall, I use regressions of cases heard and judicial characteristics. Like Chilton and Levy, I compare expected panel composition to actual panel composition.
IV ANALYSIS OF THE COURT
I. SCOPE OF ANALYSIS
To test the randomness of panel composition at the Court, I prepared a database of all panels in the 2013 calendar year. Each decision, as reported on the Court's website (38), was coded across the following dimensions: area of law; type of judgment (i.e. a full judgment or an endorsement); judges hearing the appeal; and unanimity of the decision. Unlike Yahya and Stribopoulos, I include endorsements in addition to full judgments; while endorsements may lack precedential value, they nonetheless represent panels of the Court and accordingly are relevant for my analyses. (39) The data are presented in Table 1, below.
Over the course of 2013, the Court released 729 decisions, of which 269 were full judgments. Approximately 40% of the Court's work related to criminal appeals. As noted by Yahya and Stribopoulos, the court is a highly cohesive one (40); there were just 11 dissenting opinions in my sample. Throughout my analysis, I examined "all appeals" (i.e. both endorsements and full judgments) and "full judgments only" separately, to allow for the possibility that panel composition may be different in cases that are of greater precedential value, as opposed to endorsements.
For each judge of the Court, I recorded his or her gender, political party, years of experience and area of expertise. Political party refers to the party that appointed the judge to the Court. Area of expertise was based on the biographies available on the Court's website or, alternatively, as described in media reports. I used three categories of expertise: private, criminal, and constitutional. After compiling the database, I proceeded to analyze the data for statistical significance, as described below.
II. STATISTICAL ANALYSIS: PANEL PRESENCE
In the first stage of my statistical analysis, I counted and regressed the types of cases that judges hear. I also used statistical methods to determine if case assignment was random by gender, political party, years of experience and area of expertise. (41) I refer to the number of cases heard as "panel presence" and the number of full judgments heard as "judgment presence." Both panel presence and judgment presence were compared to the corresponding proportion of judges on the Court. I utilized a two-tailed independent samples t-test, assuming equal variance, to determine whether caseload assignment was not random by gender, party and experience (grouping by greater or less than the mean number of years on the Court). (42) A p-value under 0.05 would indicate that, at a 95% confidence level, I could reject the null hypothesis of random caseload assignment.
First, I analyzed panel presence and judgment presence based on gender. Of the 26 judges in the sample, 17 were male and nine were female. The results of this analysis are presented in Table 2, below.
The panel presence and judgment presence of male and female judges are in line with the ratio of male and female judges on the Court. Accordingly, no statistical significance was found and random caseload assignment cannot be rejected.
Next, I analyzed panel presence and judgment presence based on political party. Of the 26 judges in the sample, 15 were Liberal and 11 were Conservative. The results of this analysis are presented in Table 3, below.
The panel presence and judgment presence of Liberal and Conservative judges was in line with the ratio of Liberal and Conservative judges on the Court. Consequently, no statistical significance was found and random caseload assignment cannot be rejected.
Next, I analyzed panel presence and judgment presence based on years of experience. Of the 26 judges in the sample, 13 had more years of experience than the mean on the court and 13 had less. The results of this analysis are presented in Table 4, below.
The panel presence and judgment presence of senior and junior judges was in line with the ratio of senior and junior judges on the Court. Accordingly, no statistical significance was found and random caseload assignment cannot be rejected.
Measuring statistical significance by area of expertise is slightly more complicated. There are more than two possible areas of expertise; judges were coded as experts in one of three possible areas, as discussed earlier. (43) Therefore, a one-way ANOVA was used to determine whether assignment of caseload was not random by area of expertise. A p-value under 0.05 would indicate that, at a 95% confidence level, I could reject the null hypothesis of random caseload assignment. However, per Table 5, below, the p-values are significantly greater than 0.05 and therefore I cannot reject the null hypothesis of random caseload assignment.
At this stage, no evidence of non-random panel selection has been found. As the tables above indicate, the assignment of judges to cases across gender, political party, years of experience and area of expertise are all consistent with random outcomes.
III. STATISTICAL ANALYSIS: PANEL PRESENCE AND REGRESSIONS
As Hall noted, if case assignments are random, then the types of cases that a judge hears should not be able to predict the judge's political party. (44) The preceding analysis confirms that there is no evidence for non-randomness across the analyzed characteristics. However, multiple variable regressions can be used to analyze multiple variables simultaneously. I therefore ran multiple variable linear regressions, with "panel presence" and "judgment presence", both expressed as rates of total panels and total judgment panels, respectively, as dependent variables. The independent variables were gender, political party, years of experience and area of expertise. (45) As the tables below indicate, none of the independent variables were statistically significant at the 95% confidence level, as would be evidenced by a p-value less than 0.05. The lack of statistical significance was consistent in both panel presence (Table 6) and judgment presence (Table 7). Further regressions, including isolating criminal cases and isolating endorsements also showed no statistical significance.
To assist the reader in understanding the results of Table 6, below, let us take the independent variable "Male" as an example. The coefficient of -0.014 indicates that a male judge will have a panel presence, on average, of 1.4% less than that of a female judge, holding all else constant. However, the p-value is significantly greater than 0.05, indicating that we can have little confidence in the true value of this figure. Indeed, we can only be 95% confident that the value of this figure is between -0.059 and +0.030. Given that zero is well-within the range of confidence, we cannot even be certain whether the coefficient is positive or negative.
To assist the reader in understanding the results of Table 7, below, let us take the independent variable "Conservative" as an example. The coefficient of -0.003 indicates that a Conservative judge will have a judgment presence, on average, of 0.3% less than that of a Liberal judge, holding all else constant. However, the p-value is significantly greater than 0.05, indicating that we can have little confidence in the true value of this figure. Indeed, we can only be 95% confident that the value of this figure is between -0.061 and +0.055. Given that zero is well-within the range of confidence, we cannot even be certain whether the coefficient is positive or negative.
Both regressions produce low [r.sup.2] values. [R.sup.2] is a measure of how much of the variance of the dependent variable is accounted for by the independent variables. In the case of Table 6, only 11.2% of the variance in panel presence is accounted for by the independent variables. In the case of Table 7, only 5.4% of the variance in judgment presence is accounted for by the independent variables.
IV. STATISTICAL ANALYSIS: PANEL COMPOSITION
All of the analysis conducted thus far has failed to show evidence of non-randomness. However, it has focused only on case assignments, and not on panel composition. Chilton and Levy's study offers one solution to this problem: simulating panel composition and comparing the results of the actual panels to a randomly generated outcome. Alternatively, computing expected values, using compound proportions, can approximate simulations.
Compound proportions utilize the probabilities of certain events occurring to calculate expected values. To assist the reader in understanding how this process works in the context of this study, let us consider the expected value of 23 panels with three female judges, as noted in Table 8, below. As noted in Table 2, there were 17 male and nine female judges on the Court during the time frame under analysis in this study. In composing a panel, and assuming a purely random selection process, the probability of the first judge selected being female would therefore be (9/26) = 0.346. Of the remaining judges, eight of the 25 are female; the probability of the second judge being female would therefore be (8/25) = 0.320. Of the remaining judges, seven of the 24 are female; the probability of the third judge being female would therefore be (7/24) = 0.292. Therefore, the cumulative probability of an all-female panel is 0.346 * 0.320 * 0.292 = 0.032. We then multiply the probability of an all-female panel by the total number of panels to arrive at the expected number of all-female panels. In this case, there are 718 panels; 0.032 * 718 = 23. Therefore, we would expect 23 panels composed of all-female judges. This process was followed for both total panels and panels with full judgments only. (46) To evaluate randomness in panel composition, I compared observed frequency to expected frequency, and sought to determine if these values differed in a statistically significant manner. A chi-square test was used to test for statistical significance; if the chi-square value was greater than the critical value, I could reject the null hypothesis that there was a random distribution.
First I analyzed the actual and expected panel composition, for all panels, by gender. For example, there were 154 panels with zero women, whereas based on mathematical probabilities, I would have expected 188 such panels. The results of this analysis are in Table 8, below.
Overall, the distribution of panels by gender was inconsistent with a random outcome. This result is largely driven by a greater number of panels with one female judge than would be expected; conversely, there are fewer panels with zero female judges than would be expected. Taken together, these factors drive the non-random outcome.
Next, I analyzed the actual and expected panel composition, for full judgments only, by gender. For example, there were 64 panels with zero women, whereas based on mathematical probabilities I would have expected 68 such panels. The results of this analysis are in Table 9, below.
Unlike the finding in Table 8, when looking at only full judgments, the distribution of panels by gender was consistent with a random outcome.
Next, I analyzed the actual and expected panel composition, for all panels, by political party. For example, there were 113 panels with zero Conservatives, whereas based on mathematical probabilities, I would have expected 126 such panels. The results of this analysis are in Table 10, below.
Overall, the distribution of panels by political party was inconsistent with a random outcome. The non-random outcome appears to be driven by fewer all-Liberal and all-Conservative panels than would be expected. Indeed, the number of mixed panels (i.e. panels with both Conservative and Liberal judges) is greater than would be expected.
Next, I analyzed the actual and expected panel composition, for full judgments only, by political party. For example, there were 39 panels with zero Conservatives, whereas based on mathematical probabilities, I would have expected 46 such panels. The results of this analysis are in Table 11, below.
While the panel composition by political party was not consistent with randomness when looking at all panels, in the context of full judgment panels, it is not possible to reject the null hypothesis of random panels.
Next, I analyzed the actual and expected panel composition, for all criminal panels, by area of expertise. For example, there were 118 panels with zero criminal law experts, whereas based on mathematical probabilities, I would have expected 156 such panels. The results of this analysis are in Table 12, below.
Like my findings above, when looking at all panels, the results are not consistent with a random outcome. The non-random outcome is driven by fewer panels with zero criminal experts than would be expected. Indeed, it appears as though criminal experts are more frequently assigned to criminal panels than would be expected.
Next, I analyzed the actual and expected panel composition, for full criminal judgments, by area of expertise. For example, there were 48 panels with zero criminal law experts, whereas based on mathematical probabilities, I would have expected 53 such panels. The results of this analysis are in Table 13, below.
As the results of the above analysis indicate, once again, when looking at full judgments only, the results are consistent with a random outcome. (47)
V DISCUSSION AND CONCLUSION
I. DISCUSSION OF FINDINGS
To summarize, panel composition was consistent with randomness in full judgments by gender, political party and, in the context of criminal cases, by area of expertise. However, when looking at all panels, including full judgments and endorsements, panel composition was not consistent with a random outcome.
How, then, to explain this outcome? Across all other measures, including counting of panel presence (Tables 2-5) and regressions (Tables 6-7) this study fails to provide support for non-randomness in panel assignments. Yet, the distribution of panel composition displays evidence of non-randomness. Further, this non-randomness has manifested itself in three interesting ways: (i) fewer panels with zero women than would be expected; (ii) more panels with mixed Liberal-Conservative composition than would be expected; and (iii) more criminal panels with criminal law experts than would be expected.
The second of these findings is explainable, from a statistical perspective, by the difference between regressions and panel composition analysis. That is, fewer all-Liberal and all-Conservative panels, balanced out by more mixed panels, would not be captured in a regression. Earlier, I presented two hypothetical scenarios of three panels of three judges to illustrate this point. In one scenario, all three panels are composed of two Republicans and one Democrat. In a second possible scenario, two panels are all Republican, while one panel is all Democratic. Using only regressions to check for randomness could fail to detect any difference between the two; after all, they both end up with six Republican votes and three Democratic votes.
However, the first and third of these findings are not easily explainable by the difference between regressions and panel composition analysis. In the first instance, one might expect a regression to indicate that women hear more cases. In the third instance, one might expect a regression to indicate that criminal experts hear more criminal cases. However, these panel composition findings are not significantly large such that the panel assignment data in Tables 2-7 yield any outcomes inconsistent with random panel selection. Perhaps the 0.05 p-value cut-off utilized in the regression, demanding a 95% confidence level, was too restrictive. Possibly, the Court could be ensuring randomness for full judgments, while in the case of endorsements, such care was not taken. Indeed, endorsements are of less precedential value; perhaps the importance attached to random panel assignments is reduced accordingly.
The Court must dispose of a large number of appeals. Further, endorsements generally relate to less complex issues as compared to full judgments. Disposing of endorsements may therefore be more straightforward than disposing of full judgment cases. Indeed, many of the reported reasons for endorsements are exceedingly brief. It could therefore be that ensuring randomness on endorsement panels is not a priority. This notion of straightforward cases may also explain, in part, the Court's consistently high rate of unanimous decisions.
Alternatively, given the small size of the sample--just a one-year period--it is possible this result is anomalous. Chilton and Levy cite the example of flipping a coin and note that "even if a coin is fair, it is possible to flip heads ten times in a row--it is just highly unlikely." (48) It is certainly possible that the outcome we observed, while statistically inconsistent with a random outcome, is nonetheless the product of a random process. Conversely, the opposite could also be true. That is, even if a coin is weighted or crooked, it is possible to flip heads and tails five times out of ten, each. In a one-year study, therefore, it is possible we have failed to detect panel composition patterns that may emerge when studying a greater time-frame.
If one believes that these non-random outcomes are not the product of mere chance, it is certainly arguable that they are normatively beneficial. Having more women on the bench, ensuring a diversity of political views on each panel and drawing upon the knowledge of criminal experts are all reasonable ideas. Moreover, to the extent these outcomes also maintain an element of randomness, as indicated by the regression analysis, they are in accordance with Yahya and Stribopoulos' view that both diversity and randomness should be respected. (49) Indeed, as noted earlier, the Court does make efforts to have at least one senior judge and one subject-area specialist per panel. It is not, however, the purpose of this paper to argue for or against any such policy initiatives. Further, these non-random panel composition effects were not present when observing only full judgment panels.
I did consider the possibility that the exclusion of the 11 cases with five-judge panels influenced my panel composition findings. Further, I also considered the possibility that excluding judges who sat on less than ten panels could influence the findings on panel composition. However, re-running the above tables, including both five-judge panels and judges with less than ten panel-sittings yields the same findings in terms of statistical significance. That is, panel composition remains consistent with randomness when observing only full judgments, but is not consistent with randomness when considering all judgments.
II. STUDY LIMITATIONS
While the results of this study support the notion that panel composition for full judgments at the Court is largely random, caution must be taken not to over-emphasize this finding. Importantly, this study was only for one 12-month period. The external validity of the study may therefore be limited; perhaps this 12-month period was anomalous, or in some way does not represent the regular proceedings of the Court. Indeed, given the large role that the availability of particular judges plays in scheduling the panels, there is a risk that conflicting commitments of judges could unduly influence the data. That being said, despite the sample size, the data had sufficient variability to yield statistically significant differences for the time period under consideration.
Further, while I repeatedly rely on variables as representing averages across the court, it must be noted that there are individuals behind these numbers. Indeed, the Court of Appeal for Ontario is a generalist court, and its judges hear cases across many areas of law. Accordingly, reducing the leading jurists of our province to "ones and zeros" may be oversimplifying matters. Any conclusions that could be drawn about judicial attributes are, due to the small size of the study, inherently linked to the individual judges. For example, Justice Epstein sat on 121 panels in my sample; this accounts for 13% of all Conservative panel-sittings, while also accounting for 15% of all female panel-sittings. Therefore, any observations I might otherwise be drawing about females or Conservatives are significantly influenced by Justice Epstein as an individual.
This study used methods similar to those of Hall and Chilton and Levy. Like Hall, I used regressions to check for randomness in cases heard (Tables 6-7). Like Chilton and Levy, I compared observed panel composition to expected values (Tables 8-13). Additionally, I compared the characteristics of individual judges to the number of cases they heard (Tables 2-5).
Tables 2-7 indicate that panel selection is consistent with a random process. That is, in counting and regressing the types of cases, when observing both all panels and full judgments only, all of this study's findings were consistent with random outcomes. Table 2 illustrates that the number of cases heard by male and female judges is consistent with their representation on the Court. Table 3 illustrates that the number of cases heard by Liberal and Conservative judges is consistent with their representation on the Court. Table 4 illustrates that the number of cases heard by senior and junior judges is consistent with their representation on the Court. Not surprisingly, then, the regressions in Tables 6-7 indicate that the characteristics of a judge yield no statistically significant information regarding their likelihood of hearing cases.
However, while Tables 2-7 deal with overall panel selection, Tables 8-13 consider the actual composition of the panels. Interestingly, when considering all panels, including endorsements, panel composition is not consistent with a random outcome. This holds when looking at panel composition by gender, political party and area of expertise. However, when observing only full judgments, the results are consistent with random panel composition.
In attempting to explain this anomaly, I theorized that, given the large number of appeals before the Court, many do not relate to overly complex issues and can be disposed of in a relatively straightforward manner. Indeed, many of the reported reasons for endorsements are exceedingly brief. Accordingly, ensuring randomness for such panels may not be a priority. This notion of straightforward cases may also explain, in part, the Court's consistently high rate of unanimous decisions.
It could also be that the panel composition results, while in part inconsistent with a random outcome, are nonetheless the product of a random process. The small sample size and possibility that 2013 was somehow an anomalous year reinforce the need for caution and the importance of not overstating these findings. Further study could verify whether these findings are consistent with other time periods at the Court. The data in this study yield some interesting results. With respect to full judgment panels, however, this study does serve to reinforce the idea that our courts, with regards to panel composition, are impartial bodies.
(*) B.A. in Economics with Distinction in all Subjects (Cornell University), JD/MBA (University of Toronto). The author is grateful to Professors Alarie and Green for inspiring my interest in the subject. The author acknowledges the valuable assistance of the editors of the University of Toronto Faculty of Law Review, especially that of Mark Coombes.
(1) Roncarelli v Duplessis,  SCR 121 at 142.
(2) Canadian Charter of Rights and Freedoms, Part I of the Constitution Act, 1982, being Schedule B to the Canada Act 1982 (UK), 1982, c 11, s 11(d).
(3) R v S (RD),  3 SCR 484 at paras 93-94.
(4) Examples of such studies will be discussed further in this paper, and include: Moin Yahya & James Stribopoulos, "Does a Judge's Party of Appointment or Gender Matter to Case Outcomes? An Empirical Study of the Court of Appeal for Ontario (Canada)" (2007) 45:2 Osgoode Hall LJ 315; Cass Sunstein et al, Are Judges Political? An Empirical Analysis of the Federal Judiciary (Washington, DC: Brookings Institution Press, 2006). For a comparative study of the Supreme Court of Canada and the U.S. Supreme Court, see Matthew Wetstein et al, "Ideological Consistency and Attitudinal Conflict: A Comparative Analysis of the U.S. and Canadian Supreme Courts" (2009) 42(6) Comparative Political Studies 763.
(5) "2013 Annual Report", Court of Appeal for Ontario, online: <http://www.ontariocourts.ca/coa/en/ps/>.
(6) "About the Court", Court of Appeal for Ontario, online: <http://www.ontariocourts.ca/coa/en/>.
(7) Ian Greene et al, Final Appeal: Decision Making in Canadian Courts of Appeal (Toronto: James Lorimer & Co, 1998) at 205.
(8) Courts of Justice Act, RSO 1990, c C-43, s 5(1).
(9) Ibid, s 7(1).
(10) Moin Yahya & James Stribopoulos, "Does a Judge's Party of Appointment or Gender Matter to Case Outcomes? An Empirical Study of the Court of Appeal for Ontario (Canada)" (2007) 45:2 Osgoode Hall LJ 315 at 363.
(11) Interview of John Kromkamp, Senior Legal Officer, Court of Appeal for Ontario, by Jeremy Drucker (4 December 2014). The information that follows in this section is also taken from the interview with Mr. Kromkamp.
(12) Note I will refer to judges as "Liberal judges" or "Conservative judges". When discussing American cases, I will use the terms "Democratic judges" and "Republican judges". In all instances I am referring to the political party that appointed the judge to the court.
(13) Yahya & Stribopoulos, supra note 10 at 318.
(14) Ibid at 340.
(17) Ibid at 361.
(18) Ibid at 318.
(19) Ibid at 318-319.
(20) Ibid at 319.
(23) Ibid at 363.
(25) Cass Sunstein et al, Are Judges Political? An Empirical Analysis of the Federal Judiciary (Washington, DC: Brookings Institution Press, 2006).
(26) Ibid at 11-12.
(27) Ibid at 10.
(28) Matthew Hall, "Randomness Reconsidered: Modeling Random Judicial Assignment in the U.S. Courts of Appeals" (2010) 7:3 Journal of Empirical Legal Studies 574.
(29) Sunstein et al, supra note 25 at 7.
(30) Hall, supra note 28 at 577-578.
(32) Ibid at 578-579.
(33) Ibid at 579.
(34) Ibid at 581.
(36) Adam Chilton & Marin Levy, "Challenging the Randomness of Panel Assignment in the Federal Courts of Appeals" (2015) Duke Law School Public Law & Legal Theory Series, draft available at SSRN <http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2520980>.
(37) Ibid at 42-43.
(38) "Decisions of the Court of Appeal", Court of Appeal for Ontario, online: <http://www.ontariocourts.ca/ decisions_index/en/>.
(39) Note, however, that I excluded cases that were disposed of by a single judge; there is no "panel composition" in these cases. There were 31 such cases.
(40) Yahya & Stribopoulos, supra note 10 at 337.
(41) Note that only judges who served on ten or more panels were included. Certain judges of the Superior Court, for example, are occasionally invited to hear cases with the Court. They were not included.
(42) Note that an independent samples t-test would assume, among other things, that: (i) the samples are independent; and (ii) panel presence is normally distributed across each independent variable. These two assumptions are, in this context, questionable and, accordingly, caution should be exercised in placing too much reliance on the p-values. However, a Mann-Whitney U-Test, which does not assume normal distribution, was also utilized, and the same result--no statistical significance--was observed.
(44) Hall, supra note 28 at 577-578.
(45) Given that gender, political party and area of expertise are categorical variables, dummy variables were utilized.
(46) Note there were 11 decisions with five judge-panels. They were not included for this portion of the analysis.
(47) Note that where more than 20% of "bins" in the chi-square test have cell counts of less than five, the chi-square test may not be appropriate. Accordingly, I also combined the bins of 2 and 3 criminal experts; this yielded a chi-square value of 3.051, which is less than the critical value for two degrees of freedom of 5.991. Therefore, the conclusion remains the same; the results are consistent with a random outcome.
(48) Chilton & Levy, supra note 36 at 26.
(49) Yahya & Stribopoulos, supra note 10 at 363.
Table 1: Case Count and Type Case Type All Appeals % of Total Full Judgments % of Total Disposed Only Administrative 9 1.2% 4 1.5% Constitutional 33 4.5% 23 8.6% Criminal 308 42.2% 107 39.8% Family 59 8.1% 13 4.8% Private 240 32.9% 85 31.6% Public 71 9.7% 37 13.8% n/a (*) 9 1.2% 0.0% Grand Total 729 100.0% 269 100.0% (*) In these cases, the judgments were exceedingly brief and the trial judgment was not reported, such that no evaluation of the body of law could be made.
Table 2: Panel Presence and Judgment Presence by Gender Number % of Panel % of Statistical of Total Presence Total Significance Judges Female 9 34.6% 794 36.5% Male 17 65.4% 1383 63.5% t(24) = -0.468 Total 26 100.0% 2177 100.0% p = 0.644 Judgment % of Statistical Presence Total Significance Female 282 34.7% Male 531 65.3% t(24) = -0.016 Total 813 100.0% p = 0.987
Table 3: Panel Presence and Judgment Presence by Political Party Number % of Panel % of Statistical of Total Presence Total Significance Judges Liberal 15 57.7% 1257 57.7% Conservative 11 42.3% 920 42.3% t(24) = 0.012 Total 26 100.0% 2177 100.0% p = 0.991 Judgment % of Statistical Presence Total Significance Liberal 474 58.3% Conservative 339 41.7% t(24) = 0.136 Total 813 100.0% p = 0.893
Table 4: Panel Presence and Judgment Presence by Years of Experience Number % of Panel % of Statistical of Total Presence Total Significance Judges > Average 13 50.0% 1145 52.6% < Average 13 50.0% 1032 47.4% t(24) = 0.625 Total 26 100.0% 2177 100.0% p = 0.538 Judgment % of Statistical Presence Total Significance > Average 442 54.4% < Average 371 45.6% t(24) = 0.978 Total 813 100.0% p = 0.338
Table 5: Panel Presence and Judgment Presence by Area of Expertise Number % of Panel % of Statistical of Total Presence Total Significance Judges Constitutional 4 15.4% 302 13.9% Criminal 5 19.2% 521 23.9% F(3,22) Private 14 53.8% 1111 51.0% = 0.695 n/a 3 11.5% 243 11.2% p = 0.565 Total 26 100.0% 2177 100.0% Judgment % of Statistical Presence Total Significance Constitutional 126 15.5% Criminal 186 22.9% F(3,22) Private 420 51.7% = 0.387 n/a 81 10.0% p = 0.764 Total 813 100.0%
Table 6: Regression of Panel Presence and Judicial Characteristics Dependent Panel Presence [r.sup.2] = 0.112 Variable: 95% Confidence Ranges Variable Coefficient P-Value Lower Upper Constant 0.111 0.005 0.038 0.185 Male (0.014) 0.506 (0.059) 0.030 Conservative (0.003) 0.888 (0.054) 0.047 Years on Court 0.000 0.821 (0.003) 0.004 Crim. Expert 0.040 0.212 (0.025) 0.106 Private Expert 0.005 0.832 (0.045) 0.056
Table 7: Regression of Judgment Presence and Judicial Characteristics Dependent Judgement Presence [r.sup.2] = 0.054 Variable: 95% Confidence Ranges Variable Coefficient P-Value Lower Upper Constant 0.105 0.016 0.022 0.189 Male (0.004) 0.869 (0.054) 0.046 Conservative (0.003) 0.921 (0.061) 0.055 Years on Court 0.001 0.755 (0.004) 0.005 Crim. Expert 0.030 0.410 (0.044) 0.104 Private Expert 0.005 0.863 (0.053) 0.062
Table 8: Panel Composition: Observed vs. Expected, by Gender (All Panels) # Women Observed Expected [x.sup.2] = [SIGMA] [(O - E).sup.2] / E 0 154 188 6.078 1 371 338 3.219 2 166 169 0.053 3 27 23 0.624 Total 718 718 9.975 [x.sup.2] = 9.975 [x.sup.2] crit (3) = 7.815 9.975 > 7.815, can reject null @ p < 0.05
Table 9: Panel Composition: Observed vs. Expected, by Gender (Full Judgments Only) # Women Observed Expected [x.sup.2] = [SIGMA] [(O - E).sup.2] / E 0 64 68 0.266 1 130 123 0.414 2 58 61 0.192 3 9 8 0.038 Total 261 261 0.910 [x.sup.2] = 9.975 [x.sup.2] crit (3) = 7.815 0.910 < 7.815, cannot reject null @ p < 0.05
Table 10: Panel Composition: Observed vs. Expected, by Political Party (All Panels) # Conservatives Observed Expected 0 113 126 1 334 319 2 247 228 3 24 46 Total 718 718 # Conservatives [x.sup.2] = [SIGMA] [(O - E).sup.2] / E 0 1.274 1 0.709 2 1.614 3 10.207 Total 13.803 [x.sup.2] = 13.803 [x.sup.2] crit (3) = 7.815 13.803 > 7.815, can reject null @ p < 0.05
Table 11: Panel Composition: Observed vs. Expected, by Political Party (Full Judgments Only) # Conservatives Observed Expected 0 39 46 1 129 116 2 85 83 3 8 17 Total 261 261 # Conservatives [x.sup.2] = [SIGMA] [(O - E).sup.2] / E 0 0.975 1 1.470 2 0.058 3 4.427 Total 6.931 [x.sup.2] = 6.931 [x.sup.2] crit (3) = 7.815 6.931 < 7.815, cannot reject null @ p < 0.05
Table 12: Panel Composition: Observed vs. Expected, by Criminal Expertise (All Criminal Panels) # Crim experts Observed Expected 0 118 156 1 137 123 2 39 25 3 10 1 Total 304 304 # Crim experts [x.sup.2] = [SIGMA] [(O - E).sup.2] / E 0 9.047 1 1.650 2 8.499 3 66.696 Total 85.891 [x.sup.2] = 85.891 [x.sup.2] crit (3) = 7.815 85.891 > 7.815, can reject null @ p < 0.05
Table 13: Panel Composition: Observed vs. Expected, by Criminal Expertise (Criminal Judgments Only) # Crim experts Observed Expected 0 48 53 1 43 42 2 12 8 3 1 0 104 104 # Crim experts [x.sup.2] = [SIGMA] [(O - E).sup.2] / E 0 0.508 1 0.024 2 1.543 3 0.900 2.975 [x.sup.2] = 2.975 [x.sup.2] crit (3) = 7.815 2.975 < 7.815, cannot reject null @ p < 0.05
|Printer friendly Cite/link Email Feedback|
|Publication:||University of Toronto Faculty of Law Review|
|Date:||Mar 22, 2015|
|Previous Article:||ACCESS TO STATE SECRETS IN TERRORISM PROSECUTIONS: A CASE COMMENT ON R. V. ALIZADEH AND THE STRICTURES OF THIRD PARTY DISCLOSURE.|
|Next Article:||GRASSY NARROWS X: AN ALTERNATIVE ARGUMENT.|