The opinion of the court was delivered by: CARTER
On June 30, 1979, Examination No. 8155 was given to provide an eligibility list of applicants for appointment as police officers on the New York City police force. Approximately 36,797 applicants took the examination. The police department had indicated that it could absorb an eligibility list of some 12,000 candidates over the next four years. Because the 12,000th candidate scored 94, that became the cut off point for placement on the eligibility list. Roughly 13,000 persons qualified with scores of 94 or above. The city proposes to select applicants for the police force by their ranked order of combined examination scores and veterans' preference points until the list is exhausted. In November, 1979, defendants hired 415 police recruits from the eligible list: 318 white males, 49 white females, 11 black males, 3 black females, 29 hispanic males, 2 hispanic females and 3 oriental males.
Plaintiffs, the Guardians Association of the New York City Police Department, Inc., the Hispanic Society of the New York City Police Department, Inc. and Nydia I. Diaz, James Michael Hidalgo, Wilfred Cebollero, Andre Lopez, Reinaldo Salgado, Denise Santos, Deborah Holmes and Pamela Obey, individual black and hispanic applicants who either received a passing grade too low on the eligibility list to afford an expectation of appointment in the foreseeable future, or who scored below the qualifying 94, instituted this litigation. They challenge the legality of the examination and seek a preliminary and permanent injunction barring the planned use of the test's results. Defendants, the Civil Service Commission, the Department of Personnel and the Police Department of the City of New York, are responsible for the preparation, administration and use of the examination.
As indicated, the eligibility list has already been utilized, and defendants propose to hire another class of recruits from the list on Monday, January 14, 1980. With the parties' consent the hearing on the preliminary injunction was consolidated with a trial on the merits which took place on November 13, 14 and 15.
At the hearing defendants presented testimony indicating how the examination had been devised and structured. Plaintiffs presented similar testimony by individual participants in the preparation of the test, as well as evidence concerning the adverse effect on minorities of the examination and of use of an eligibility list with ranking based on the candidates' scores. Both sides presented expert testimony concerning the test's validity. The parties filed their briefs in support of their contentions on or about December 3, 1979, but neither side filed proposed findings of fact and conclusions of law until on or about December 13, 1979. On December 10, 1979, the government filed a brief amicus curiae asserting that Examination No. 8155 was unlawful, and thereafter participated as amicus curiae in the proceedings.
On December 17, 1979, the court advised the parties orally that it was of the view that the examination and the rank order selection of designated eligible candidates violated Title VII of the Civil Rights Act, 42 U.S.C. § 2000e et seq., and that no appointments from the list in ranked order should be made. The court anticipated that this oral indication of its determination would enable the city to make temporary adjustments in the selection process to bring the contemplated January appointments into compliance with the law by taking affirmative action to eliminate discrimination against blacks and hispanics. At a subsequent hearing on defendants' order to show cause and motion for reconsideration, counsel for defendants stated that defendants would not modify the selection process by the appointment of black and hispanic applicants out of rank order, and that they were determined to utilize the discriminatory listing. The matter was then heard by the Court of Appeals on defendants' petition for a writ of mandamus. The Court of Appeals ordered this court not to bar the city from making the contemplated appointments without findings of fact and conclusions of law being filed in compliance with Rules 52(a) and 65(d), F.R.Civ.P. The court was further ordered to refrain from enjoining defendants from selecting police recruits from the existing list until 48 hours after such findings and conclusions had been filed.
On January 11, 1980, a hearing on relief was held. The Policewomen's Endowment Association was allowed to intervene as amicus curiae. Plaintiffs offered testimony on the black-hispanic make-up of the relevant labor pool from which applicants for employment with the New York City police force were expected to come. Defendants have qualified for immediate appointment on January 14, 1980, some 380 candidates, of whom 280 are white males and 62 are white females, 9 are black males and 5 are black females, 20 are hispanic males and 3 are hispanic females, and 1 is an oriental male.
Preparation of Examination No. 8155
The development of the test and the job analysis employed was described at the November hearings by Esther Juni, administrative staff analyst of the department of personnel and chief of the police unit of the criminal justice task force. Juni disclaimed expertise in both testing and police work. This was only the second entry level examination she had ever worked on, the first being an examination for police administrative aides manning the 911 system where the skills tested were basically clerical and communicative. Moreover, no outside experts in testing had been brought in to aid defendants in devising a test in compliance with the Uniform Guidelines on Employee Selection Procedures ("Uniform Guidelines") promulgated by the Equal Employment Opportunity Commission, Department of Justice, Department of Labor and the United States Civil Service Commission, 5 C.F.R. § 300.103(e), 28 C.F.R. § 50.14, and 29 C.F.R. § 1607, effective September 25, 1978.
Juni's testimony was hardly a model of clarity. It was often confusing, contradictory and obscure. However, from her testimony and the report outlining the procedures followed in job analysis and in preparation of Examination No. 8155, which I believe she testified she prepared or whose preparation she supervised, the following facts were gleaned.
Preparation of the test began in the fall of 1978 when, as a first step, the department of personnel asked the police department to supply 10 staff analysts to interview 49 police officers and 49 of their supervisors. These officers, in Juni's words, were chosen on the basis of "good job performance record, their ethnicity, the sexual composition (sic), types of duty they performed, the tours they worked on, the boroughs they worked in and it was made to be a comprehensive analysis of the positions of the police officers" (Tr. 9-10). The officers were asked to identify the tasks performed by police officers, their frequency and importance, the amount of time spent thereon, and the knowledge, skills and abilities (labeled KSA's)
necessary to perform each identified function. By this process some 71 tasks were identified.
Presumably this group of officers was chosen as "job knowledge experts,"
but except that they were apparently selected to comprise a cross section of good performers, we do not know their level of expertise or indeed whether they qualify as job knowledge experts
as that term is used in the department of personnel manual on job analysis ("Procedures "). There is no evidence that Juni or anyone else examined the officer-consultants on this question before asking them to define the tasks police perform.
Next, a panel of police officers was chosen to refine the identified tasks further. Again, except that the panel was chosen by officers on the police force, using the same criteria Juni described above, we were not advised whether the panel members possessed any special job knowledge expertise warranting their selection over others or even insuring their professional competence to do the task assigned. At any rate, the panel reduced the 71 identified tasks to 42. Thereafter a questionnaire was sent to 5,600 officers for their rating of the 42 tasks in terms of frequency, time spent and level of importance. Some 2,500 responses were received. At the same time, observers from John Jay College, watching police officers at work, recorded their observations of 28 of the identified 42 tasks. These observations were then correlated with the officers' responses to the questionnaires as to the frequency with which tasks were performed. The questionnaires were fed into a computer programmed to take data on time, frequency and importance, average them and produce a list of the 42 tasks in hierarchical order based on these three factors.
The tasks were next divided into 5 clusters of tasks "that seem to us to go together on a rational basis," "that looked somewhat similar," "had some kind of common factor" (Tr. 19). A panel of officers knowledgeable about the tasks involved was chosen for each cluster. If, for example, one of the clusters was the arrest process, police officers very familiar with the arrest process, "meaning they were in a precinct that maybe made a large number of arrests, etc." were chosen (Tr. 20). These panels were to determine the KSA's needed for each task within the cluster. Again the job knowledge expertise of these people is not identified or defined, nor was that factor determined by Juni or anyone else from the department of personnel. Juni's report refers to this group as "job knowledge experts" without further elaboration. The KSA's to be assigned to each task by the panels were characterized as filling out forms; human relations, including communication techniques; understanding written instructions and applying appropriate procedures; comprehending and applying sections of the law; and recalling details and facts. The KSA's were never more specifically defined, although the human relations KSA's were described as interacting with people.
The 5 panels never exchanged ideas or concepts, and we do not know whether the description and identification of the KSA's were the same or different from panel to panel. The KSA's were then rated by the 5 panels as to the relative importance each had in the successful completion of each task within the assigned cluster. John Galea, one of the officers detailed to one of the panels assigning the relevant KSA's to each task, testified, however, that his group made no effort to differentiate the human relations KSA components assigned to different tasks. The panels' conclusions were fed into a computer which multiplied the weight of each task by the weight of the KSA's necessary to accomplish it. Based on the weighted result, the test plan was devised with an appropriate percentage of the examination being allocated to each of the 5 KSA's.
Then police officers were asked to propose questions covering the 5 KSA's. According to Juni the police department was asked to choose for this assignment "police officers who were familiar with the job of police officer" (Tr. 24). Some 10-11 officers were given this responsibility. They were not shown the KSA lists or the tasks with KSA weight accorded to them. Gerald Burkhalter, one of this group, testified that he was asked by an officer in the personnel bureau of the police department to participate. He was told that they were looking for people of various ethnic groups and that he had patrol experience and a college background. Presumably these factors and his being black qualified him for the assignment. Between December, 1978 and June, 1979, he spent each Friday writing questions. The first two months he wrote questions based on materials from the Police Academy. He was not shown the tasks until he was over two months into the assignment and never saw the KSA's. He stated that he had raised the issue of the adverse impact of written tests on minorities several times with Juni and in the equal opportunity office of the police department where he worked (Tr. 408). Although police officers proposed some of the questions, a good proportion of the questions were written by employees of the department of personnel.
A panel of police officers reviewed the questions to determine whether the police procedures described in the examination were consonant with current practices. Nicholas Estavillo, asked to work with this panel, joined it at its second meeting. The group was told what the questions dealt with, and discussed whether there was any conflict between the questions and police department procedures. Estavillo concluded that what was being tested was reading comprehension, and expressed this view to Juni and others on several occasions (Tr. 440). Department of personnel employees reviewed all of the proposed questions to make certain they were stated clearly, simply, and grammatically, but these employees had no clear idea of what KSA's the questions they clarified and simplified purported to test. There was no final review by police or department of personnel officials to determine whether the questions as finalized actually tested for the KSA's they purported to measure.
The examination was divided into 100 multiple choice questions. Questions 1-15 dealt with recalling details and facts; questions 16-24 with filling out forms; questions 25-38 with comprehending and applying sections of the law; questions 39-62, 64-68, and 71-74 with understanding written instructions and applying appropriate procedures; and all the rest with human relations, including communications techniques. It was decided not to pre-select a passing score, but to qualify the 12,000 top scores.
Juni testified that she did not measure police work behavior but, rather, the KSA's necessary to perform a police officer's functions; that the test did not mirror the tasks of officers on the job, but the KSA's needed for the job (Tr. 48). She conceded that the work product that is the police officer's basic function was not completely described, and testified that while some of the tasks concern work product, work product must be inferred from others (Tr. 50). When asked for an operational definition of the KSA's, she replied that the KSA's are themselves simply stated. The ...