The opinion of the court was delivered by: Nicholas G. Garaufis, United States District Judge
From 1999 to 2007, the New York City Fire Department used written examinations that had discriminatory effects on minority applicants and failed to test for relevant job skills. In July 2009 and January 2010, this court held that New York City's use of these examinations discriminated against black and Hispanic applicants in violation of Title VII of the Civil Rights Act of 1974, and against black applicants in violation of the United States Constitution. Today, the court concludes that the Fire Department's current written examination, Exam 6019, does not comply with Title VII. As a result, the court temporarily enjoins the City from using Exam 6019 to appoint entry-level firefighters.
The court's analysis is complex, but its conclusion is simple. The City has not shown that the current examination identifies candidates who will be successful firefighters. Because the test questions do not measure the abilities required for the job of entry-level firefighter, the examination cannot distinguish between qualified and unqualified candidates, or even between more and less qualified candidates. In the words of the Second Circuit Court of Appeals, the examination "satisfies a felt need for objectivity, but it does not necessarily select better job performers."*fn1 What the examination does do is screen and rank applicants in a manner that disproportionately excludes black and Hispanic applicants. As a result, hundreds of minority applicants are being denied the opportunity to serve as New York firefighters, for no legitimate or justifiable reason.
The court is not alone in its opinion that Exam 6019 fails to test for useful skills and abilities. The firefighters and fire lieutenants who reviewed the examination before it was administered overwhelmingly agreed that large portions of the exam should not be used. They also offered the following comments:
I feel all these questions are unfair. They have nothing to do with an entry-level exam.
No good. These questions should be used to help in a psychological profile of the applicant. They should not be used for an entrance exam.
This should not be part of the test. It is subjective.
Prior firehouse knowledge needed. Members/candidates with prior firehouse or fire ground knowledge will have a great unfair advantage compared to the general public.*fn2
The City ignored the opinions of its own firefighters when creating Exam 6019. As a result, the City administered an invalid, discriminatory exam to nearly 22,000 job applicants.
This court previously ordered the parties to begin constructing a new, valid firefighter selection procedure under the guidance of Special Master Mary Jo White. The question now is whether and how the City may use Exam 6019 to appoint new firefighters in the interim. At this time, the court does not have enough information to decide on a permanent course of action. The City asserts, without offering any documentary or testimonial support, that it needs to hire a new firefighter class immediately. But the City cancelled its last class of appointees in 2009, and earlier this year Mayor Michael Bloomberg advocated closing 20 fire companies and reducing staffing in 60 additional engine companies.*fn3 Before this court can permit the City to use Exam 6019 in any manner, the City must explain what has changed and why the need to appoint a few hundred rookie firefighters using and invalid test outweighs the need to avoid racial discrimination in municipal hiring. Accordingly, this court will hold a hearing as soon as possible to consider the remedial measures it should take in light of today's decision and the City's needs.
Between 1999 and 2008, the City used two competitive examination processes, Exam 7029 and Exam 2043, to screen and select applicants for entry-level firefighter positions. In 2002 and 2005, the Intervenors filed charges with the Equal Employment Opportunity Commission ("EEOC"), alleging that the exams violated Title VII. (See Int. D.I. 56.1 ¶¶ 4-5.) In 2004, the EEOC determined that Exam 7029 adversely affected black applicants and was invalid. In 2005, the EEOC made the same determination regarding Exam 2043. (Id.) The City refused to conciliate, and the EEOC referred the charges to the United States Department of Justice ("DOJ"). (Id. ¶ 6 & Ex. K.) The DOJ filed the instant lawsuit in May 2007. (See Compl.(Docket Entry # 1).)
The City began developing its current test, Exam 6019, in August 2006, after the EEOC determined that Exams 7029 and 2043 were invalid. (See 6019 Test Development Report (Def. Ex. A-3) ("Test Dev. Rep.") 2.) The City administered Exam 6019 on January 20, 2007. (Pl. Ex. 1.) Approximately 21,983 candidates completed the exam, and 21,235 candidates passed. (Exam 6019 Analyses and Scoring Report (Def. Ex. A-5)("Exam Analysis") 3; Pl. Ex. 4a.) The City established the Exam 6019 "eligibility list" -- i.e., the rank-order list of those who passed -- in June 2008, and hired its first (and to date, only) academy class off the list in July 2008. (Seeley Decl. (Docket Entry # 316-1), Ex. C; HT 228-29.)
In July 2009, this court held that the City's use of Exams 7029 and 2043 as pass/fail and rank-ordering devices constituted disparate-impact discrimination in violation of Title VII. See United States v. City of New York, 637 F. Supp. 2d 77 (E.D.N.Y. 2009) ("Disparate Impact Opinion" or "D.I. Op."). In January 2010, this court also held that the City's actions constituted intentional discrimination in violation of Title VII and the Fourteenth Amendment. United States v. City of New York, 683 F. Supp. 2d 225 (E.D.N.Y. 2010). Following these decisions, the court issued a preliminary relief order directing the parties to take certain actions to begin remedying the City's violations. See United States v. City of New York, 681 F. Supp. 2d 274 (E.D.N.Y. 2010) ("Initial Remedy Order"). Among other things, the court directed the parties to prepare for a hearing (the "6019 Hearing") regarding the validity of Exam 6019, which in turn would determine whether and how the City could hire from the Exam 6019 eligibility list on an interim basis while a new, valid selection procedure was being developed. Id. at 278.
Under the supervision of Magistrate Judge Roanne Mann and Special Master White,*fn4 the parties engaged in a lengthy and occasionally contentious discovery process in preparation for the 6019 Hearing. On June 29, 2010, the City informed the court that it intended to hire approximately 300 firefighters from the Exam 6019 eligibility list, to initiate the new academy class in either the last week of August 2010 or the first week of September 2010, and to begin notifying successful candidates approximately 30 days in advance of the start of the class. (Docket Entry # 456.) Accordingly, the court and the Special Master accelerated the schedule for the 6019 Hearing. The parties submitted voluminous pre-hearing briefing and numerous exhibits. Over the course of a two-day hearing on July 20 and 21, 2010, the court heard testimony from the City's test-construction expert, Dr. Catherine Cline; the Plaintiffs' test-construction expert, Dr. David P. Jones; and Donay Queenan, the FDNY's Assistant Commissioner for Human Resources. This opinion now follows.
B. The Exam 6019 Hiring Process
Exam 6019 is an objectively scored paper-and-pencil test consisting of 195 multiple-choice questions. (Pl. PFF ¶ 3.) The exam comprises three components: a timed component, a "situational judgment exercise" ("SJE") component, and a cognitive component. (Id.) The City weighted the scores on these components differently, with the cognitive component weighted the most and the timed component weighted the least. (Pl. Ex. 2.) The City scaled each candidate's combined score so that candidates who correctly answered all questions received a score of 100, and candidates who correctly answered 70% of all questions received a score of 70. (Pl. PFF ¶ 9; Exam Analysis 5.)
The City used a cutoff score of 70 to determine which candidates passed Exam 6019. (Pl. Ex. 1.) Candidates who failed the exam were excluded from further consideration for the job. The City calculated each passing candidate's "Adjusted Final Average" by adding any applicable residency, veteran, and legacy bonus points to the candidate's exam score.*fn5 (PL. PFF ¶ 15.) The City then assigned each candidate a list number (or rank) based on the candidate's Adjusted Final Average, with the lowest list numbers (i.e., the highest ranks) assigned to the candidates with the highest Adjusted Final Averages. Candidates with the same Adjusted Final Average were ranked based upon their Social Security numbers. (Id. ¶ 17.)
Candidates' exam scores and resulting list ranking determine the order in which they are processed for hiring. Candidates are invited to take the Candidate Physical Ability Test ("CPAT") based on their rank on the Exam 6019 eligibility list. (Id. ¶ 21.) To be appointed, candidates passing the CPAT also have to appear on a certification list, meet all requirements for appointment set forth in the Exam 6019 notice of examination, and pass a medical and psychological examination. (Id.) Because the City hires firefighters in classes -- typically between 150 and 300 hires at a time -- it processes candidates off of the eligibility list in large groups, as many as 1,000 at a time. (6019 Hearing Transcript ("HT") 227-30.) The candidates who are found to be qualified are hired in rank-order off of the eligibility list, meaning that a candidate who has completed all steps in the selection process may still not be hired if the City fills its academy class before the candidate's list number is reached. (Pl. PFF ¶ 26.)
Since establishing the Exam 6019 eligibility list in June 2008, the City has hired only one academy class from it. The lowest-ranked applicant actually appointed by the City thus far was ranked 834th on the eligibility list. (Siskin Decl. (Docket Entry # 316-3) ¶ 11 & n.5.)
II. THE COURT'S REMEDIAL AUTHORITY
Congress enacted Title VII "to assure equality of employment opportunities and to eliminate those discriminatory practices and devices which have fostered racially stratified job environments to the disadvantage of minority citizens." McDonnell Douglas Corp. v. Green, 411 U.S. 792, 800 (1973). In order to meet this sweeping mandate, Congress "took care to arm the courts with full equitable powers." Albemarle Paper Co. v. Moody, 422 U.S. 405, 418 (1975). Title VII authorizes district courts to choose from a wide spectrum of remedies for illegal discrimination, ranging from compensatory relief such as back pay to "affirmative relief" such as the imposition of hiring quotas. See 42 U.S.C. 2000e-5(g); Berkman v. City of New York, 705 F.2d 584, 595-96 (2d Cir. 1983); see also Rios v. Enterprise Asso. Steamfitters Local 638 etc., 501 F.2d 622, 629 (2d Cir. 1974) ("Once a violation of Title VII is established, the district court possesses broad power as a court of equity to remedy the vestiges of past discriminatory practices.").
While certain types of relief authorized by Title VII are discretionary and context-specific, other types are essentially requisite. Compare United States v. Starrett City Associates, 840 F.2d 1096, 1102 (2d Cir. 1988) (racial quotas appropriate only in limited circumstances) with Albemarle Paper Co., 422 U.S.at 422 (back pay should be awarded in almost all circumstances). In particular, once liability for racial discrimination has been established, the district court "has not merely the power but the duty" to "bar like discrimination in the future." Id. at 418 (quoting Louisiana v. United States, 380 U.S. 145, 154 (1965)). This so-called "compliance relief" is designed to assure future compliance with Title VII. Berkman, 705 F.2d at 595. In the context of discriminatory testing regimes, such relief involves "restricting the use of an invalid exam, specifying procedures and standards for a new valid selection procedure, and authorizing interim hiring that does not have a disparate racial impact." Guardians Assoc. of New York City Police Dept., Inc. v. Civil Service Comm'n, 630 F.2d 79, 108 (2d Cir. 1980) ("Guardians"); see also id. at 109 ("Once an exam has been adjudicated to be in violation of Title VII, it is a reasonable remedy to require that any subsequent exam or other selection device receive court approval prior to use."). In this case, the City has stopped using Exams 2043 and 7029, and the court has already ordered the parties to begin constructing a new, valid selection procedure for entry-level firefighters, see February 24, 2010 Docket Entry. The court's remaining duty is to determine whether the City's interim hiring based on Exam 6019 is compatible with Title VII.
Before undertaking this analysis, the court offers a word about its methodology. The existence of a Title VII violation affords the court broad equitable remedial authority, and Guardians suggests that courts are free to dismantle interim hiring procedures based solely on their disparate impact. Nonetheless, this court believes that the appropriate course is to subject Exam 6019 to a standard Title VII analysis -- that is, to assess both the exam's racial impact and whether it is job-related or consistent with business necessity -- before ruling on its use. In part, this belief stems from the simple intuition that the best way to police the City's compliance with Title VII is to actually measure the City's actions against the statute's requirements. Another reason, however, is that the law in this area is in a state of flux. In Ricci v. DeStefano, 129 S.Ct. 2658 (2009), the Supreme Court held that an employer may not, consistent with the disparate-treatment provisions of Title VII, set aside the results of an exam that disparately impacts employee candidates without a strong basis in evidence that the exam actually violates Title VII. Id. at 2673-77. Ricci's specific holding does not control here, since a federal court attempting to remedy identified discrimination enjoys far more authority than an employer attempting to remedy potential discrimination. See, e.g., Albemarle Paper Co., 422 U.S. at 418. Nonetheless, Ricci announces general principles that could as easily apply to courts as employers, and this court is hesitant to ignore them absent further guidance from the Supreme Court or the Second Circuit. One of these principles is that government actors -- who are bound by the Equal Protection Clause, which is analogous to Title VII's disparate-treatment provision -- must identify, rather than extrapolate, the existence of a Title VII violation before taking race-conscious remedial action such as setting aside or restricting the use of exam results. As stated before, a federal court operating in a remedial setting may well be exempt from this principle, but the court sees no reason to use this as a test case. Therefore, the prudent step is to evaluate Exam 6019 using a Title VII analysis.
At the same time, the court does not wish to overstate its actions or overstep its authority. Exam 6019 is not the subject of this lawsuit, and no party has filed an independent action alleging that it is unlawful. This court is only reaching Exam 6019 as an exercise of its remedial jurisdiction, and while that jurisdiction is broad, it is not unlimited. Thus, as this court observed previously, [It is not] necessary (or even appropriate) for this court to conclusively determine the legality of Exam 6019 at this juncture. The court simply wants to know, before it decides on an interim hiring remedy, whether the pass/fail and rank-ordering uses of Exam 6019 have a disparate impact on black and Hispanic test-takers, and if so, whether those uses are job-related. The exam's precise legal status, its superiority or inferiority to alternate procedures, and the rights of the candidates who took it are all important questions that may be taken up elsewhere.
But they are ancillary to the question facing the court, which is what use, if any, should be made of the Exam 6019 eligibility list in the event that the FDNY begins hiring firefighters before a new examination is developed.
United States v. City of New York, 2010 U.S. Dist. LEXIS 31397, at *5-6 (E.D.N.Y. Mar. 31, 2010). Therefore, although this court is analyzing Exam 6019 under a Title VII liability framework, the freestanding "legality" of Exam 6019 is not at issue, and the court's conclusions are meant only to guide its future remedial actions.
III. THE PLAINTIFFS' PRIMA FACIE CASE
Courts assess disparate-impact discrimination under Title VII using a three-step burden-shifting framework. A plaintiff must first "establish by a preponderance of the evidence that the employer 'uses a particular employment practice that causes a disparate impact on the basis of race, color, religion, sex, or national origin.'" Robinson v. Metro-North Commuter R.R. Co., 267 F.3d 147, 160 (2d Cir. 2001) (quoting 42 U.S.C. § 2000e-2(k)(1)(A)(i)). "To make this showing, a plaintiff must (1) identify a policy or practice, (2) demonstrate that a disparity exists, and (3) establish a causal relationship between the two." Id. at 160. If the plaintiff succeeds, the burden shifts to the defendant to demonstrate that the challenged practice or policy is "job related for the position in question and consistent with business necessity." 42 U.S.C. § 2000e-2(k)(1)(A)(i). The burden then shifts back to the plaintiff "to establish the availability of an alternative policy or practice that would also satisfy the asserted business necessity, but would do so without producing the disparate effect."*fn6 Robinson, 267 F.3d at 161 (citing 42 U.S.C. § 2000e-2(k)(1)(A)(ii), (C)).
Statistics alone can make out a prima facie case, provided they "reveal a disparity substantial enough to raise an inference of causation." EEOC v. Joint Apprenticeship Comm. of Joint Indus. Bd. of Elec. Indus., 186 F.3d 110, 117 (2d Cir. 1999). The plaintiffs' statistical evidence "must reflect a disparity so great that it cannot be accounted for by chance." Id.; see also Robinson, 267 F.3d at 160. In this case, Plaintiffs have demonstrated -- and the City concedes -- that the City's use of Exam 6019 as a pass/fail and rank-ordering device results in a statistically significant racial disparity sufficient to make out a prima facie case of disparate impact discrimination. See Initial Remedy Order, 681 F. Supp. 2d at 295 & n.2; see also HT 6-7.
Plaintiffs' expert, Dr. Bernard Siskin, has demonstrated that the City's pass/fail use of Exam 6019 with a cutoff score of 70 has a statistically significant adverse impact upon black and Hispanic applicants. The disparity between the pass rates of black applicants and the pass rate of white applicants is equivalent to 22.70 units of standard deviation.*fn7 (Siskin Decl. ¶ 7.) The practical effect of this disparity is that 298 black candidates who would not have failed the examination but for the disparity were eliminated from consideration for the position of entry-level firefighter. (Id. ¶ 8.) The disparity between the pass rates of Hispanic applicants and the pass rate of white applicants is equivalent to 11.35 units of standard deviation. (Id. ¶ 9.) As a practical matter, this disparity eliminated 132 Hispanic candidates from consideration. (Id. ¶ 8.)
Dr. Siskin's analysis also demonstrates that the City's use of Exam 6019 as a rank-ordering device has a statistically significant adverse impact on black and Hispanic applicants, and that this adverse impact will only worsen as the City continues to use the eligibility list to hire firefighters. As Dr. Siskin points out, many applicants who nominally passed Exam 6019 have effectively failed the exam because they did not score high enough to actually be hired. Currently, the lowest-ranked applicant who has been appointed as a firefighter was ranked 834th on the Exam 6019 eligibility list.*fn8 (Id. ¶ 11 & n.5.) Based on the number of firefighters that the City has hired off the 6019 list to date, Dr. Siskin calculates that 4.10% of white test-takers and 2.36% of black test-takers scored high enough to actually be appointed. The disparity in the effective pass rates of white and black applicants is equivalent to 5.01 units of standard deviation. Dr. Siskin estimates that, absent the disparity, 21 additional black applicants would have been appointed from the Exam 6019 eligible list. (Id. ¶ 11.) If the City continues to use the Exam 6019 eligible list in rank-order and reaches twice as far down the list as it currently has, the disparity between the effective pass rates of white and black applicants will be equivalent to 5.94 units of standard deviation, representing an estimated 35 additional black applicants who would be hired absent the disparity. (Id. ¶ 12.) If the City reaches three times as far down the list, the disparity between the effective pass rates of white and black applicants will be equivalent to 8.11 units of standard deviation, representing an estimated 58 additional black applicants who would be hired absent the disparity. (Id. ¶ 13.) And if the City reaches four times as far down the list, the disparity between the effective pass rates of white and black applicants will be equivalent to 9.59 units of standard deviation, representing an estimated 74 additional black applicants who would be hired absent the disparity.*fn9 (Id. ¶ 13.)
These statistics are sufficient to establish a prima facie case of disparate-impact discrimination. As noted previously in this case, "[t]he Second Circuit has repeatedly recognized that standard deviations of more than 2 or 3 units can give rise to a prima facie case of disparate impact because of the low likelihood that such disparities have resulted from chance." D.I. Op., 637 F. Supp. 2d at 93 (citing Malave v. Potter, 320 F.3d 321, 327 (2d Cir. 2003); Waisome v. Port Auth. of N.Y. & N.J., 948 F.2d 1370, 1376 (2d Cir. 1991); Ottaviani v. State University of New York, 875 F.2d 365, 372 (2d Cir. 1989); Guardians, 630 F.2d at 86); see also Hazelwood School Dist. v. United States, 433 U.S. 299, 309 n.14 (1977) ("general rule" in employment discrimination cases with large samples is that "if the difference between the expected value and the observed number is greater than two or three standard deviations, then the hypothesis that [employees] were hired without regard to race would be suspect"). The calculated standard deviations in this case range from 5.01 to 22.7 units, well in excess of the Second Circuit's benchmark. (Siskin Decl. ¶¶ 7, 11.)
The City does not contest Dr. Siskin's results or methodology, and conceded at the 6019 Hearing that the Plaintiffs' evidence creates a prima facie case of disparate-impact discrimination. (See HT 6-7.) Accordingly, the dispositive question is whether the City has carried its burden of demonstrating that Exam 6019 is job-related and justified by business necessity.
IV. THE CITY'S BUSINESS NECESSITY DEFENSE
Because Plaintiffs have shown that the City's uses of Exam 6019 disparately impacts black and Hispanic candidates, the City bears the burden of showing that those uses are justified by legitimate business and job-related considerations.*fn10 See Gulino v. New York State Educ. Dep't, 460 F.3d 361, 385 (2d Cir. 2006). In Gulino, the Second Circuit explained the business necessity defense as follows:
[T]he basic rule has always been that "discriminatory tests are impermissible unless shown, by professionally acceptable methods, to be predictive of or significantly correlated with important elements of work behavior which comprise or are relevant to the job or jobs for which candidates are being evaluated." This rule operates as both a limitation and a license for employers: employers have been given explicit permission to use job related tests that have a disparate impact, but those tests must be "demonstrably a reasonable measure of job performance."
460 F.3d at 383 (quoting Albemarle Paper Co. v. Moody, 422 U.S. 405, 431, 426 (1975)). In EEOC parlance, an exam that accurately predicts on-the-job performance is said to be "valid," while an exam that does not is invalid. See, e.g., 29 CFR § 1607.5.
The validity of an exam turns in large part on the process by which it was constructed. See Guardians, 630 F.2d at 95. Accordingly, the court begins by reviewing Exam 6019's construction before turning to the substantive legal standards ...