Documentation of Impact and Validity Evidence

This is an archived document, to be used for reference.

Documentation of Impact and Validity Evidence

A. Required information. Users of selection procedures other than those users complying with section 15A(1) below should maintain and have available for each job information on adverse impact of the selection process for that job and, where it is determined a selection process has an adverse impact, evidence of validity as set forth below.

(1) Simplified recordkeeping for users with less than 100 employees. In order to minimize recordkeeping burdens on employers who employ one hundred (100) or fewer employees, and other users not required to file EEO-1, et seq., reports, such users may satisfy the requirements of this section 15 if they maintain and have available records showing, for each year:

(a) The number of persons hired, promoted, and terminated for each job, by sex, and where appropriate by race and national origin;

(b) The number of applicants for hire and promotion by sex and where appropriate by race and national origin; and

These records should be maintained for each race or national origin group (see section 4 above) constituting more than two percent (2%) of the labor force in the relevant labor area. However, it is not necessary to maintain records by race and/or national origin (see §4 above) if one race or national origin group in the relevant labor area constitutes more than ninety-eight percent (98%) of the labor force in the area. If the user has reason to believe that a selection procedure has an adverse impact, the user should maintain any available evidence of validity for that procedure (see sections 7A and 8).

(2) Information on impact—(a) Collection of information on impact. Users of selection procedures other than those complying with section 15A(1) above should maintain and have available for each job records or other information showing whether the total selection process for that job has an adverse impact on any of the groups for which records are called for by sections 4B above. Adverse impact determinations should be made at least annually for each such group which constitutes at least 2 percent of the labor force in the relevant labor area or 2 percent of the applicable workforce. Where a total selection process for a job has an adverse impact, the user should maintain and have available records or other information showing which components have an adverse impact. Where the total selection process for a job does not have an adverse impact, information need not be maintained for individual components except in circumstances set forth in subsection 15A(2)(b) below. If the determination of adverse impact is made using a procedure other than the “four-fifths rule,” as defined in the first sentence of section 4D above, a justification, consistent with section 4D above, for the procedure used to determine adverse impact should be available.

(b) When adverse impact has been eliminated in the total selection process. Whenever the total selection process for a particular job has had an adverse impact, as defined in section 4 above, in any year, but no longer has an adverse impact, the user should maintain and have available the information on individual components of the selection process required in the preceding paragraph for the period in which there was adverse impact. In addition, the user should continue to collect such information for at least two (2) years after the adverse impact has been eliminated.

(c) When data insufficient to determine impact. Where there has been an insufficient number of selections to determine whether there is an adverse impact of the total selection process for a particular job, the user should continue to collect, maintain and have available the information on individual components of the selection process required in section 15(A)(2)(a) above until the information is sufficient to determine that the overall selection process does not have an adverse impact as defined in section 4 above, or until the job has changed substantially.

(3) Documentation of validity evidence—(a) Types of evidence. Where a total selection process has an adverse impact (see section 4 above) the user should maintain and have available for each component of that process which has an adverse impact, one or more of the following types of documentation evidence:

(i) Documentation evidence showing criterion-related validity of the selection procedure (see section 15B, below).

(ii) Documentation evidence showing content validity of the selection procedure (see section 15C, below).

(iii) Documentation evidence showing construct validity of the selection procedure (see section 15D, below).

(iv) Documentation evidence from other studies showing validity of the selection procedure in the user's facility (see section 15E, below).

(v) Documentation evidence showing why a validity study cannot or need not be performed and why continued use of the procedure is consistent with Federal law.

(b) Form of report. This evidence should be compiled in a reasonably complete and organized manner to permit direct evaluation of the validity of the selection procedure. Previously written employer or consultant reports of validity, or reports describing validity studies completed before the issuance of these guidelines are acceptable if they are complete in regard to the documentation requirements contained in this section, or if they satisfied requirements of guidelines which were in effect when the validity study was completed. If they are not complete, the required additional documentation should be appended. If necessary information is not available the report of the validity study may still be used as documentation, but its adequacy will be evaluated in terms of compliance with the requirements of these guidelines.

(c) Completeness. In the event that evidence of validity is reviewed by an enforcement agency, the validation reports completed after the effective date of these guidelines are expected to contain the information set forth below. Evidence denoted by use of the word “(Essential)” is considered critical. If information denoted essential is not included, the report will be considered incomplete unless the user affirmatively demonstrates either its unavailability due to circumstances beyond the user's control or special circumstances of the user's study which make the information irrelevant. Evidence not so denoted is desirable but its absence will not be a basis for considering a report incomplete. The user should maintain and have available the information called for under the heading “Source Data” in sections 15B(11) and 15D(11). While it is a necessary part of the study, it need not be submitted with the report. All statistical results should be organized and presented in tabular or graphic form to the extent feasible.

B. Criterion-related validity studies. Reports of criterion-related validity for a selection procedure should include the following information:

(1) User(s), location(s), and date(s) of study. Dates and location(s) of the job analysis or review of job information, the date(s) and location(s) of the administration of the selection procedures and collection of criterion data, and the time between collection of data on selection procedures and criterion measures should be provided (Essential). If the study was conducted at several locations, the address of each location, including city and State, should be shown.

(2) Problem and setting. An explicit definition of the purpose(s) of the study and the circumstances in which the study was conducted should be provided. A description of existing selection procedures and cutoff scores, if any, should be provided.

(3) Job anlysis or review of job information. A description of the procedure used to analyze the job or group of jobs, or to review the job information should be provided (Essential). Where a review of job information results in criteria which may be used without a full job analysis (see section 14B(3)), the basis for the selection of these criteria should be reported (Essential). Where a job analysis is required a complete description of the work behavior(s) or work outcome(s), and measures of their criticality or importance should be provided (Essential). The report should describe the basis on which the behavior(s) or outcome(s) were determined to be critical or important, such as the proportion of time spent on the respective behaviors, their level of difficulty, their frequency of performance, the consequences of error, or other appropriate factors (Essential). Where two or more jobs are grouped for a validity study, the information called for in this subsection should be provided for each of the jobs, and the justification for the grouping (see section 14B(1)) should be provided (Essential).

(4) Job titles and codes. It is desirable to provide the user's job title(s) for the job(s) in question and the corresponding job title(s) and code(s) from U.S. Employment Service's Dictionary of Occupational Titles.

(5) Criterion measures. The bases for the selection of the criterion measures should be provided, together with references to the evidence considered in making the selection of criterion measures (essential). A full description of all criteria on which data were collected and means by which they were observed, recorded, evaluated, and quantified, should be provided (essential). If rating techniques are used as criterion measures, the appraisal form(s) and instructions to the rater(s) should be included as part of the validation evidence, or should be explicitly described and available (essential). All steps taken to insure that criterion measures are free from factors which would unfairly alter the scores of members of any group should be described (essential).

(6) Sample description. A description of how the research sample was identified and selected should be included (essential). The race, sex, and ethnic composition of the sample, including those groups set forth in section 4A above, should be described (essential). This description should include the size of each subgroup (essential). A description of how the research sample compares with the relevant labor market or work force, the method by which the relevant labor market or work force was defined, and a discussion of the likely effects on validity of differences between the sample and the relevant labor market or work force, are also desirable. Descriptions of educational levels, length of service, and age are also desirable.

(7) Description of selection procedures. Any measure, combination of measures, or procedure studied should be completely and explicitly described or attached (essential). If commercially available selection procedures are studied, they should be described by title, form, and publisher (essential). Reports of reliability estimates and how they were established are desirable.

(8) Techniques and results. Methods used in analyzing data should be described (essential). Measures of central tendency (e.g., means) and measures of dispersion (e.g., standard deviations and ranges) for all selection procedures and all criteria should be reported for each race, sex, and ethnic group which constitutes a significant factor in the relevant labor market (essential). The magnitude and direction of all relationships between selection procedures and criterion measures investigated should be reported for each relevant race, sex, and ethnic group and for the total group (essential). Where groups are too small to obtain reliable evidence of the magnitude of the relationship, need not be reported separately. Statements regarding the statistical significance of results should be made (essential). Any statistical adjustments, such as for less then perfect reliability or for restriction of score range in the selection procedure or criterion should be described and explained; and uncorrected correlation coefficients should also be shown (essential). Where the statistical technique categorizes continuous data, such as biserial correlation and the phi coefficient, the categories and the bases on which they were determined should be described and explained (essential). Studies of test fairness should be included where called for by the requirements of section 14B(8) (essential). These studies should include the rationale by which a selection procedure was determined to be fair to the group(s) in question. Where test fairness or unfairness has been demonstrated on the basis of other studies, a bibliography of the relevant studies should be included (essential). If the bibliography includes unpublished studies, copies of these studies, or adequate abstracts or summaries, should be attached (essential). Where revisions have been made in a selection procedure to assure compatability between successful job performance and the probability of being selected, the studies underlying such revisions should be included (essential). All statistical results should be organized and presented by relevant race, sex, and ethnic group (essential).

(9) Alternative procedures investigated. The selection procedures investigated and available evidence of their impact should be identified (essential). The scope, method, and findings of the investigation, and the conclusions reached in light of the findings, should be fully described (essential).

(10) Uses and applications. The methods considered for use of the selection procedure (e.g., as a screening device with a cutoff score, for grouping or ranking, or combined with other procedures in a battery) and available evidence of their impact should be described (essential). This description should include the rationale for choosing the method for operational use, and the evidence of the validity and utility of the procedure as it is to be used (essential). The purpose for which the procedure is to be used (e.g., hiring, transfer, promotion) should be described (essential). If weights are assigned to different parts of the selection procedure, these weights and the validity of the weighted composite should be reported (essential). If the selection procedure is used with a cutoff score, the user should describe the way in which normal expectations of proficiency within the work force were determined and the way in which the cutoff score was determined (essential).

(11) Source data. Each user should maintain records showing all pertinent information about individual sample members and raters where they are used, in studies involving the validation of selection procedures. These records should be made available upon request of a compliance agency. In the case of individual sample members these data should include scores on the selection procedure(s), scores on criterion measures, age, sex, race, or ethnic group status, and experience on the specific job on which the validation study was conducted, and may also include such things as education, training, and prior job experience, but should not include names and social security numbers. Records should be maintained which show the ratings given to each sample member by each rater.

(12) Contact person. The name, mailing address, and telephone number of the person who may be contacted for further information about the validity study should be provided (essential).

(13) Accuracy and completeness. The report should describe the steps taken to assure the accuracy and completeness of the collection, analysis, and report of data and results.

C. Content validity studies. Reports of content validity for a selection procedure should include the following information:

(1) User(s), location(s) and date(s) of study. Dates and location(s) of the job analysis should be shown (essential).

(3) Job analysis—Content of the job. A description of the method used to analyze the job should be provided (essential). The work behavior(s), the associated tasks, and, if the behavior results in a work product, the work products should be completely described (essential). Measures of criticality and/or importance of the work behavior(s) and the method of determining these measures should be provided (essential). Where the job analysis also identified the knowledges, skills, and abilities used in work behavior(s), an operational definition for each knowledge in terms of a body of learned information and for each skill and ability in terms of observable behaviors and outcomes, and the relationship between each knowledge, skill, or ability and each work behavior, as well as the method used to determine this relationship, should be provided (essential). The work situation should be described, including the setting in which work behavior(s) are performed, and where appropriate, the manner in which knowledges, skills, or abilities are used, and the complexity and difficulty of the knowledge, skill, or ability as used in the work behavior(s).

(4) Selection procedure and its content. Selection procedures, including those constructed by or for the user, specific training requirements, composites of selection procedures, and any other procedure supported by content validity, should be completely and explicitly described or attached (essential). If commercially available selection procedures are used, they should be described by title, form, and publisher (essential). The behaviors measured or sampled by the selection procedure should be explicitly described (essential). Where the selection procedure purports to measure a knowledge, skill, or ability, evidence that the selection procedure measures and is a representative sample of the knowledge, skill, or ability should be provided (essential).

(5) Relationship between the selection procedure and the job. The evidence demonstrating that the selection procedure is a representative work sample, a representative sample of the work behavior(s), or a representative sample of a knowledge, skill, or ability as used as a part of a work behavior and necessary for that behavior should be provided (essential). The user should identify the work behavior(s) which each item or part of the selection procedure is intended to sample or measure (essential). Where the selection procedure purports to sample a work behavior or to provide a sample of a work product, a comparison should be provided of the manner, setting, and the level of complexity of the selection procedure with those of the work situation (essential). If any steps were taken to reduce adverse impact on a race, sex, or ethnic group in the content of the procedure or in its administration, these steps should be described. Establishment of time limits, if any, and how these limits are related to the speed with which duties must be performed on the job, should be explained. Measures of central tend- ency (e.g., means) and measures of dispersion (e.g., standard deviations) and estimates of realibility should be reported for all selection procedures if available. Such reports should be made for relevant race, sex, and ethnic subgroups, at least on a statistically reliable sample basis.

(6) Alternative procedures investigated. The alternative selection procedures investigated and available evidence of their impact should be identified (essential). The scope, method, and findings of the investigation, and the conclusions reached in light of the findings, should be fully described (essential).

(7) Uses and applications. The methods considered for use of the selection procedure (e.g., as a screening device with a cutoff score, for grouping or ranking, or combined with other procedures in a battery) and available evidence of their impact should be described (essential). This description should include the rationale for choosing the method for operational use, and the evidence of the validity and utility of the procedure as it is to be used (essential). The purpose for which the procedure is to be used (e.g., hiring, transfer, promotion) should be described (essential). If the selection procedure is used with a cutoff score, the user should describe the way in which normal expectations of proficiency within the work force were determined and the way in which the cutoff score was determined (essential). In addition, if the selection procedure is to be used for ranking, the user should specify the evidence showing that a higher score on the selection procedure is likely to result in better job performance.

(8) Contact person. The name, mailing address, and telephone number of the person who may be contacted for further information about the validity study should be provided (essential).

(9) Accuracy and completeness. The report should describe the steps taken to assure the accuracy and completeness of the collection, analysis, and report of data and results.

D. Construct validity studies. Reports of construct validity for a selection procedure should include the following information:

(1) User(s), location(s), and date(s) of study. Date(s) and location(s) of the job analysis and the gathering of other evidence called for by these guidelines should be provided (essential).

(3) Construct definition. A clear definition of the construct(s) which are believed to underlie successful performance of the critical or important work behavior(s) should be provided (essential). This definition should include the levels of construct performance relevant to the job(s) for which the selection procedure is to be used (essential). There should be a summary of the position of the construct in the psychological literature, or in the absence of such a position, a description of the way in which the definition and measurement of the construct was developed and the psychological theory underlying it (essential). Any quantitative data which identify or define the job constructs, such as factor analyses, should be provided (essential).

(4) Job analysis. A description of the method used to analyze the job should be provided (essential). A complete description of the work behavior(s) and, to the extent appropriate, work outcomes and measures of their criticality and/or importance should be provided (essential). The report should also describe the basis on which the behavior(s) or outcomes were determined to be important, such as their level of difficulty, their frequency of performance, the consequences of error or other appropriate factors (essential). Where jobs are grouped or compared for the purposes of generalizing validity evidence, the work behavior(s) and work product(s) for each of the jobs should be described, and conclusions concerning the similarity of the jobs in terms of observable work behaviors or work products should be made (essential).

(5) Job titles and codes. It is desirable to provide the selection procedure user's job title(s) for the job(s) in question and the corresponding job title(s) and code(s) from the United States Employment Service's dictionary of occupational titles.

(6) Selection procedure. The selection procedure used as a measure of the construct should be completely and explicitly described or attached (essential). If commercially available selection procedures are used, they should be identified by title, form and publisher (essential). The research evidence of the relationship between the selection procedure and the construct, such as factor structure, should be included (essential). Measures of central tendency, variability and reliability of the selection procedure should be provided (essential). Whenever feasible, these measures should be provided separately for each relevant race, sex and ethnic group.

(7) Relationship to job performance. The criterion-related study(ies) and other empirical evidence of the relationship between the construct measured by the selection procedure and the related work behavior(s) for the job or jobs in question should be provided (essential). Documentation of the criterion-related study(ies) should satisfy the provisions of section 15B above or section 15E(1) below, except for studies conducted prior to the effective date of these guidelines (essential). Where a study pertains to a group of jobs, and, on the basis of the study, validity is asserted for a job in the group, the observed work behaviors and the observed work products for each of the jobs should be described (essential). Any other evidence used in determining whether the work behavior(s) in each of the jobs is the same should be fully described (essential).

(8) Alternative procedures investigated. The alternative selection procedures investigated and available evidence of their impact should be identified (essential). The scope, method, and findings of the investigation, and the conclusions reached in light of the findings should be fully described (essential).

(9) Uses and applications. The methods considered for use of the selection procedure (e.g., as a screening device with a cutoff score, for grouping or ranking, or combined with other procedures in a battery) and available evidence of their impact should be described (essential). This description should include the rationale for choosing the method for operational use, and the evidence of the validity and utility of the procedure as it is to be used (essential). The purpose for which the procedure is to be used (e.g., hiring, transfer, promotion) should be described (essential). If weights are assigned to different parts of the selection procedure, these weights and the validity of the weighted composite should be reported (essential). If the selection procedure is used with a cutoff score, the user should describe the way in which normal expectations of proficiency within the work force were determined and the way in which the cutoff score was determined (essential).

(10) Accuracy and completeness. The report should describe the steps taken to assure the accuracy and completeness of the collection, analysis, and report of data and results.

(11) Source data. Each user should maintain records showing all pertinent information relating to its study of construct validity.

(12) Contact person. The name, mailing address, and telephone number of the individual who may be contacted for further information about the validity study should be provided (essential).

E. Evidence of validity from other studies. When validity of a selection procedure is supported by studies not done by the user, the evidence from the original study or studies should be compiled in a manner similar to that required in the appropriate section of this section 15 above. In addition, the following evidence should be supplied:

(1) Evidence from criterion-related validity studies—a. Job information. A description of the important job behavior(s) of the user's job and the basis on which the behaviors were determined to be important should be provided (essential). A full description of the basis for determining that these important work behaviors are the same as those of the job in the original study (or studies) should be provided (essential).

b. Relevance of criteria. A full description of the basis on which the criteria used in the original studies are determined to be relevant for the user should be provided (essential).

c. Other variables. The similarity of important applicant pool or sample characteristics reported in the original studies to those of the user should be described (essential). A description of the comparison between the race, sex and ethnic composition of the user's relevant labor market and the sample in the original validity studies should be provided (essential).

d. Use of the selection procedure. A full description should be provided showing that the use to be made of the selection procedure is consistent with the findings of the original validity studies (essential).

e. Bibliography. A bibliography of reports of validity of the selection procedure for the job or jobs in question should be provided (essential). Where any of the studies included an investigation of test fairness, the results of this investigation should be provided (essential). Copies of reports published in journals that are not commonly available should be described in detail or attached (essential). Where a user is relying upon unpublished studies, a reasonable effort should be made to obtain these studies. If these unpublished studies are the sole source of validity evidence they should be described in detail or attached (essential). If these studies are not available, the name and address of the source, an adequate abstract or summary of the validity study and data, and a contact person in the source organization should be provided (essential).

(2) Evidence from content validity studies. See section 14C(3) and section 15C above.

(3) Evidence from construct validity studies. See sections 14D(2) and 15D above.

F. Evidence of validity from cooperative studies. Where a selection procedure has been validated through a cooperative study, evidence that the study satisfies the requirements of sections 7, 8 and 15E should be provided (essential).

G. Selection for higher level job. If a selection procedure is used to evaluate candidates for jobs at a higher level than those for which they will initially be employed, the validity evidence should satisfy the documentation provisions of this section 15 for the higher level job or jobs, and in addition, the user should provide: (1) a description of the job progression structure, formal or informal; (2) the data showing how many employees progress to the higher level job and the length of time needed to make this progression; and (3) an identification of any anticipated changes in the higher level job. In addition, if the test measures a knowledge, skill or ability, the user should provide evidence that the knowledge, skill or ability is required for the higher level job and the basis for the conclusion that the knowledge, skill or ability is not expected to develop from the training or experience on the job.

H. Interim use of selection procedures. If a selection procedure is being used on an interim basis because the procedure is not fully supported by the required evidence of validity, the user should maintain and have available (1) substantial evidence of validity for the procedure, and (2) a report showing the date on which the study to gather the additional evidence commenced, the estimated completion date of the study, and a description of the data to be collected (essential).

(Approved by the Office of Management and Budget under control number 3046-0017)

(Pub. L. 96-511, 94 Stat. 2812 (44 U.S.C. 3501 et seq.))

[43 FR 38295, 38312, Aug. 25, 1978, as amended at 46 FR 63268, Dec. 31, 1981]

§1607.15 Documentation of impact and validity evidence.