The opinion of the court was delivered by: Shira A. Scheindlin, U.S.D.J.
Police officer are permitted to briefly stop any individual, but only upon reasonable suspicion that he is committing a crime.*fn1 The source of that limitation is the Fourth Amendment to the United States Constitution, which guarantees that "the right of the people to be secure in their persons, houses, papers, and effects, against unreasonable searches and seizures, shall not be violated." The Supreme Court has explained that this "inestimable right of personal security belongs as much to the citizen on the streets of our cities as to the homeowner closeted in his study to dispose of his secret affairs."*fn2 The right to physical liberty has long been at the core of our nation's commitment to respecting the autonomy and dignity of each person: "No right is held more sacred, or is more carefully guarded, by the common law, than the right of every individual to the possession and control of his own person, free from all restraint or interference of others, unless by clear and unquestionable authority of law."*fn3 Safeguarding this right is quintessentially the role of the judicial branch.
No less central to the courts' role is ensuring that the administration of law comports with the Fourteenth Amendment, which "undoubtedly intended not only that there should be no arbitrary deprivation of life or liberty, or arbitrary spoliation of property, but that equal protection and security should be given to all under like circumstances in the enjoyment of their personal and civil rights."*fn4
On over 2.8 million occasions between 2004 and 2009, New York City police officers stopped residents and visitors, restraining their freedom, even if only briefly.*fn5 Over fifty percent of those stops were of Black people and thirty percent were of Hispanics, while only ten percent were of Whites. The question presented by this lawsuit is whether the New York City Police Department ("NYPD") has complied with the laws and Constitutions of the United States and the State of New York. Specifically, the four named plaintiffs allege, on behalf of themselves and a putative class, that defendants have engaged in a policy and/or practice of unlawfully stopping and frisking people in violation of their Fourth Amendment right to be free from unlawful searches and seizures and their Fourteenth Amendment right to freedom from discrimination on the basis of race.
To support their claims, plaintiffs have enlisted the support of Jeffrey Fagan, a professor of criminology at Columbia Law School, who has submitted an extensive report analyzing the NYPD's practices.*fn6 The City of New York ("City") and the other defendants object to the introduction of Fagan's opinions, arguing that he lacks the qualifications to make the assessments that he makes, that his methodologies are fatally flawed, and that many of his opinions constitute inadmissible conclusions of law.*fn7
NYPD officers are required to fill out a detailed worksheet describing the events before and during every stop that they perform. All of these records are compiled in a database -- a database that now contains a wealth of information about millions of interactions between police officers and civilians. The information is both incredibly rich and inevitably incomplete: rich because the dozens of boxes on the worksheet are designed to solicit the very information -- who, when, where, why and how -- that courts (and the NYPD itself) use to evaluate whether a stop was lawful; incomplete because a fill-in-the-blank document can never fully capture the nuances of a human interaction, because these worksheets capture only the quick responses of police officers rather than of the civilians who have been stopped, and because police officers do not always fill them out perfectly.
How should a jury evaluate the NYPD's stop-and-frisk policy? What should attorneys and witnesses be permitted to tell the jury about the 2.8 million interactions between officers and the people they have stopped? And what should the Court tell those jurors? Both parties agree that the database contains valuable and relevant information. But they disagree vehemently over how to accurately summarize the information and how to fairly describe it to the jury. Defendants' motion to exclude the opinions of Professor Fagan therefore presents this Court with important questions regarding expert testimony and trial management.
With one important exception, Fagan's report is methodologically sound and, under the Federal Rules of Evidence, admissible. I will permit Fagan's generalizations where they are reasonable interpretations of the data and I will prohibit them where I find that they are inaccurate or have little probative value. For the reasons below, defendants' motion is granted in part and denied in part.
A. Professor Fagan's Qualifications
Fagan is the Isidor and Seville Sulzbacher Professor of Law at
Columbia Law School; director of the school's Center for Crime,
Community, and Law; a Senior Research Scholar at Yale Law School; and
a Fellow of the American Society of Criminology.*fn8
He has published dozens of refereed journal articles and chapters on
an array of topics in criminology including issues related to
juveniles, deterrence, capital punishment, race, and New York City.*fn9
He has been studying and writing about the policies at issue
in this case for over a decade.*fn10 Perhaps
most prominently, in 1999 Fagan conducted a study for the Civil Rights
Bureau of the New York State Office of the Attorney General,
statistically analyzing the NYPD's data on approximately 175,000 stops
and frisks and "focusing specifically on racial disparities in stop
rates and the extent to which stops complied with the Fourth
Amendment."*fn11 The results of his analysis were
published that year in The New York Police Department's "Stop and Frisk" Practices: A Report to
the People of the State of New York from the Office of the Attorney
As defendants point out, however, Fagan is not a lawyer and has never taken courses at a law school.*fn13 His graduate degrees are in industrial and civil engineering, with a focus on policy science and criminal justice.*fn14 Furthermore, Fagan "has never worked in a law enforcement field, has never completed a [stop and frisk] form, never conducted a Stop, Question & Frisk ("SQF") and never observed more than a few SQFs or gone for a ride along with a NYPD officer to even observe a SQF."*fn15
After conducting a stop, NYPD officers are required to fill out a "Stop, Question and Frisk Report Worksheet," which is a two-sided form commonly known as a UF-250.*fn16 Approximately 2.8 million of these worksheets were filled out between 2004 and 2009 and the NYPD entered the information from each of the worksheets into a database and produced it to plaintiffs and Fagan as electronic files.*fn17 Each UF-250 includes information about the suspect's demographic characteristics (age, gender, race/ethnicity); the date, time, duration, location, and outcome of the stop (e.g., frisk, search, type of weapon seized if any, type of other contraband found if any, summons issued, arrest); the suspected crime for which the person was stopped; and whether and what kind of physical force was used. Because the suspected crimes were recorded "using individualized and often idiosyncratic notation," Fagan coded the notations into a set of 131 specific criminal charges and then distributed each "suspected crime" into one of twenty aggregate crime categories (e.g., violent crime, minor violent crime, fraud, drugs).*fn18
On each UF-250, there are twenty boxes that can be checked by police officers regarding the factors -- or as Fagan calls them, the "indicia of suspicion" -- that motivated the stop. There are ten indicia on Side 1 of the worksheet ("circumstances of stop" or "stop circumstances") and ten more on Side 2 ("additional factors"). The worksheet also contains nine checkboxes regarding the indicia of suspicion that motivated any frisk that took place and four checkboxes regarding the indicia of suspicion that motivated any search.
Fagan's report relied on detailed demographic information, organized by police precinct and census tract, which he compiled from a variety of resources including the United States Census, the federal government's American Community Survey, and a commercial database called ESRI. Fagan used police precincts as his principal unit of analysis because "precincts are the units where police patrol resources are aggregated, allocated, supervised and monitored" and because "precinct crime rates are the metrics for managing and evaluating police performance."*fn19 The demographic data he collected includes information on race, ethnicity, age, income, unemployment, housing vacancy, residential mobility, and physical disorder.*fn20 The City provided him with data on crime complaints from 2004-2009. This data specifies the location of a complaint and type of alleged crime; Fagan categorized the alleged crimes using the same categories that he used to analyze the UF-250s, which "provided a foundation for benchmarking the types and rates of suspected crimes in the stops with the observed rates of reported specific crimes in each police precinct."*fn21 The City also provided Fagan with "patrol strength data" regarding the allocation of police resources to particular neighborhoods. Finally, Fagan included in his analysis information about the location of public housing (where there is often a large police presence) and population density (which impacts the likelihood of police-civilian interactions).*fn22
C. Fagan's Analysis Regarding Plaintiffs' 14th Amendment Equal Protection Claims and Defendants' Criticism of That Analysis In order to test plaintiffs' 14th Amendment claim that defendants' stop-and-frisk practices treat Blacks and Hispanics differently than they treat Whites, Fagan designed and ran regressions that sought to determine the impact of a person's race on outcomes such as being stopped, being frisked, being subjected to force during an arrest, etc.*fn23 Fagan's regressions compared the influence of race on these outcomes with the influence of non-race factors such as residency in a poor or high crime neighborhood. These analyses control for the fact that in New York City, as a general matter, Blacks and Hispanics live in higher crime neighborhoods than do Whites.*fn24
Fagan created a benchmark against which "to determine if police are selectively, on the basis of race or another prohibited factor, singling out persons for stops, questioning, frisk or search."*fn25 Police officers may lawfully stop an individual only when they have reasonable suspicion to believe that the person has committed, is committing, or is about to commit a crime. The rates at which different groups of people engage in behavior that raises such reasonable suspicion is therefore relevant to the determination of whether the police are treating people equally. According to Fagan, "a valid benchmark requires estimates of the supply of individuals of each racial or ethnic group who are engaged in the targeted behaviors and who are available to the police as potential targets for the exercise of their stop authority."*fn26 Fagan used two variables in constructing a benchmark that would fulfill these requirements: the local rate of crime and the racial distribution of the local population.*fn27 This benchmark was designed, in part, "to test the extent to which the racial composition of a precinct, neighborhood, or census tract -- separate and apart from its crime rate -- predicts the stop-and-frisk rate in that precinct, neighborhood, or census tract."*fn28
Based on his statistical analyses, Fagan reached the following conclusions regarding disparate treatment:
The racial composition of a precinct, neighborhood, and census tract is a statistically significant, strong and robust predictor of NYPD stop-and-frisk patterns even after controlling for the simultaneous influences of crime, social conditions, and allocation of police resources.
NYPD stops-and-frisks are significantly more frequent for Black and Hispanic residents than they are for White residents, even after adjusting for local crime rates, racial composition of the local population, police patrol strength, and other social and economic factors predictive of police enforcement activity.
Blacks and Latinos are significantly more likely to be stopped by NYPD officers than are Whites even in areas where there are low crime rates and where residential populations are racially heterogenous or predominately White.
Black and Hispanic individuals are treated more harshly during stop-and-frisk encounters with NYPD officers than Whites who are stopped on suspicion of the same or similar crimes.*fn29
Notably, Fagan did not include in his benchmark the rates of criminal activity by race. This decision constitutes the parties' central disagreement regarding Fagan's analysis of disparate treatment. Defendants believe that crime rates by race, as reflected in the complaints of crime victims and in the NYPD's arrest data, is the best benchmark: "In an analysis concerned with whom the police are stopping, a reliable benchmark must take into account who is committing the crime."*fn30 Defendants argue that "Blacks and Hispanics comprise a majority of violent crime suspects in all precincts except one in the City, and in most precincts are the overwhelming majority of suspects."*fn31 Defendants point out that Fagan has used arrest data in at least two previous studies, even though arrest data was less complete at the time of those studies than it is today.*fn32
Fagan explains that he chose not to use data from arrests and suspect identifications here because that data is incomplete; imputing the characteristics of the known data to the missing data, Fagan believes, would raise serious risks of selection bias.*fn33 Because suspect race is only known in fifty to sixty percent of cases, extrapolation of that known racial distribution to the remaining forty or fifty percent of cases may not be appropriate, Fagan argues, particularly if the suspect crimes that animate a large share of stops (such as drug possession) do not correlate well to crime reports that identify the race of a suspect (such as assault). In the years since his earlier reports were written, Fagan explains, "the weight of opinion among researchers who were doing this kind of work" is that his current benchmark is an improvement on his earlier benchmarks.*fn34
D. Fagan's Analysis Regarding Plaintiffs' Fourth Amendment Reasonable Suspicion Claim and Defendants' Criticism of That Analysis In order to assess plaintiffs' claim that defendants have engaged in a practice of stopping and frisking New Yorkers without reasonable suspicion and in violation of the Fourth Amendment, Fagan analyzed the combinations of boxes that officers checked on the UF-250s. He did this in two ways. First, he assumed that the forms had been filled out accurately and completely and sought to determine whether reasonable suspicion existed in any given stop based on the boxes that were checked off on the worksheet. Second, by searching for patterns in the worksheet data from across the City and over the 2004-2009 period, Fagan sought to determine whether the data on the forms is accurate and whether the NYPD's use of the forms is an effective way to ensure that officers are complying with the law.*fn35
1. Analysis and Findings Regarding UF-250s, Assuming Their Veracity and Completeness
Because there are ten "stop circumstances" on Side 1 of the form and ten "additional factors" on Side 2, and because officers are not limited in the number of boxes they can check (although they are required to check at least one Side 1 stop circumstance), there are an enormous number of potential combinations of boxes that can be checked. Fagan created the following system for determining whether or not a stop was lawful: First, he categorized the stop factors on Side 1 as either "justified" or "conditionally justified." Second, he defined a stop itself as "justified," "unjustified," or "indeterminate" based on which boxes had been checked. He did this by analyzing case law, as described in Appendix D of his report. The following is a summary of Fagan's algorithm and categorization scheme:
Category 1: Stops are justified if one or more of the following three "justified" stop circumstances on Side 1 are checked off: (1) "Actions Indicative Of 'Casing' Victims Or Location"; (2) "Actions Indicative Of Engaging In Drug Transaction"; (3) "Actions Indicative Of Engaging In Violent Crimes."
Category 2: Stops are justified if at least one of the following six "conditionally justified" stop circumstances on Side 1 are checked off and at least one of the additional circumstances on Side 2 are checked off. The conditionally justified stop circumstances are (1) "Carrying Objects In Plain View Used In Commission Of Crime e.g., Slim Jim/Pry Bar, etc."; (2) "Suspicions Bulge/Object (Describe)"; (3) "Actions Indicative Of Acting As A Lookout"; (4) "Fits Description"; (5) "Furtive Movements"; (6) "Wearing Clothes/Disguises Commonly Used In Commission Of Crime."
Category 3: Stops are unjustified if no stop circumstances on Side 1 are checked off, even if one or more additional circumstances on Side 2 are checked off.
Category 4: Stops are unjustified if only one conditionally justified stop circumstance on Side 1 is checked off and no additional circumstances on Side 2 are checked off.
Category 5: Stops are justified if two or more conditionally justified stop circumstances on Side 1 are checked off.
Category 6: Stops are indeterminate if "Other Reasonable Suspicion Of Criminal Activity (Specify)" is the only stop circumstance checked off on Side 1, regardless of whether one or more additional circumstances on Side 2 are checked off and regardless of what is written in the blank space under the "Other" box.
Based on this classification system, Fagan concluded the following about the stops conducted by the NYPD:
More than 170,000 stops, or 6.41% of all stops (6.71% of non-radio run stops, and 5.26% of radio runs), recorded by NYPD officers between 2004 and 2009 were Unjustified.
For more than 400,000 stops, or approximately 15%, the corresponding UF250 forms do not provide sufficient detail to determine the stops' legality.*fn36
Defendants level many criticisms at Fagan's classification system,*fn37 including the following: First, the legality of a given stop cannot be determined based solely on the information on the UF-250, since the worksheet is simply a summary of the events and cannot substitute for a proper evaluation of the totality of the circumstances. Second, Fagan's descriptions of stops as justified, unjustified, or indeterminate constitute inadmissible legal conclusions. Third, Fagan did not incorporate into his analysis the handwritten notes on the worksheets that are made when the box marked "Other" is checked (Category 6), even when those notes provided an explanation of why reasonable suspicion existed. Fourth, Fagan classified Category 3 stops as unjustified even when multiple Side 2 circumstances were checked and Category 6 stops as indeterminate even when the "Other" box was coupled with multiple Side 2 circumstances; these decisions are not supported by the caselaw, which permits some stops that fall into those categories. Fifth, Fagan classified Category 4 stops as unjustified even though courts have permitted stops on the basis of only one "Conditionally Justified" factor. Sixth, Fagan failed to incorporate the location of a stop in determining whether it took place in a high crime area, relying instead on whether the Side 2 high crime area box had been checked, and he failed to incorporate descriptive information about the person stopped (such as height, weight, etc.) that might explain why an individual fit the description of a perpetrator of a crime.
2. Analysis of the Accuracy and Effectiveness of the UF-250s and the Stop-and-Frisk Policy
Fagan also sought to determine the extent to which the information on the UF-250s was accurate and complete. This analysis was largely independent of the justified/unjustified classification model described above. The most important elements of Fagan's analysis involved the trends in the usage of various stop factors and the rates at which stops yielded arrests, summonses, and seizures of weapons and contraband (what he calls the "hit rate").
For example, Fagan found that police officers check the Side 2 box "Area Has High Incidence of Reported Offense Of Type Under Investigation" in approximately fifty-five percent of all stops, regardless of whether the stop takes place in a precinct or census tract with average, high, or low crime.*fn38 Relatedly, the Side 1 box "Furtive Movements" is checked in over forty-two percent of stops; in 2009 it was checked off in nearly sixty percent of stops.*fn39 However, the arrest rates in stops where the high crime area or furtive movement boxes are checked off is actually below average.*fn40
Fagan has found that over the study period, "the percentage of stops whose suspected crime is uninterpretable has grown dramatically from 1.12% in 2004 to 35.9% in 2009."*fn41 Fagan calculates that "5.37 percent of all stops result in an arrest," that [s]ummonses are issued at a slightly higher rate: 6.26 percent overall," and that "[s]eizures of weapons or contraband are extremely rare. Overall, guns are seized in less than one percent of all stops: 0.15 percent . . . . ...