Attempting to increase transparency surrounding the use—and potential disparate impact—of artificial intelligence and data analytics in employment determinations, the New York City Department of Consumer and Worker Protection has issued final rules governing the use of automated employment decision tools ("AEDTs") in hiring and promotions in the City. The rules clarify the extensive requirements imposed by New York City Local Law 144 (the "Law"), which is slated to take effect on July 5, 2023. Key takeaways include the obligation to conduct an exhaustive "bias audit" of covered computer-based techniques that issue predictions or rankings for candidates, and the requirement to conspicuously publish the results of such audit to the public.

Attempting to increase transparency surrounding the use—and potential disparate impact—of artificial intelligence and data analytics in employment determinations, the New York City Department of Consumer and Worker Protection has issued final rules governing the use of automated employment decision tools ("AEDTs") in hiring and promotions in the City. The rules clarify the extensive requirements imposed by New York City Local Law 144 (the "Law"), which is slated to take effect on July 5, 2023. Key takeaways include the obligation to conduct an exhaustive "bias audit" of covered computer-based techniques that issue predictions or rankings for candidates, and the requirement to conspicuously publish the results of such audit to the public.

What is an AEDT?

The final rules make clear that the Law is designed to regulate certain "mathematical, computer-based techniques"—including artificial intelligence and data analytics—that assess data provided by candidates for employment or promotion, and either (A) generate a prediction about that candidate, or (B) classify the candidate in a particular way. Per the rules, the predictions or classifications can take the form of scores, tags or categories, candidate rankings, or recommendations as to next steps in the hiring or promotion process.

These computer-based techniques are AEDTs covered by the Law when the employer^[1] using them (A) relies solely on the predictions or classifications these techniques generate to make a determination; (B) gives more weight to such predictions or classifications than to other criteria; or (C) uses such predictions or classifications to overrule determinations resulting from other factors.

Imagine that a City employer intends to hire a server for a restaurant in Brooklyn. In an effort to streamline the hiring process, that employer plans to use an AI or data analytics tool that identifies restaurant-related keywords in candidate résumés and ranks applicants based upon the keywords that appear. It then plans to automatically interview or hire the individual who ranks highest. That employer intends to use an AEDT, and must comply with the resulting obligations under the Law.

What must an employer do to use an AEDT?

The Law forbids a City employer from using a given AEDT unless it has conducted a bias audit of that tool within the past 12 months and publicized the results of the audit in a particular manner. An employer who has never used a given AEDT must conduct an audit before doing so, and employers actively using AEDTs must routinely audit them each year.

The audit must assess data to evaluate any bias that may result from using the tool. As such, an employer may need to use historical data from other employers to complete the audit, or may need to use test data for that purpose, subject to restrictions reflected in the final rules.

What does a bias audit entail?

The Law is fundamentally concerned with determining whether a given AEDT impacts individuals differently due to their race, ethnicity, and/or gender. For purposes of the audit, the final rules break these demographics into what they call "categories." The relevant categories for assessment are male, female, Hispanic or Latino, White, Black or African American, Native Hawaiian or Pacific Islander, Asian, Native American or Alaska Native, and Two or More Races (Not Hispanic or Latino). The final rules also require employers to assess "intersectional categories," which further break down race- or ethnicity-based data by gender (comparing the AEDT's treatment of Hispanic males to that of Hispanic females, for example).

The exhaustive requirements for a bias audit differ based on the type of tool used. Relevant calculations can include:

• the number of individuals selected for advancement or classification in each demographic category, divided by the total number of candidates who applied from that category (called the "Selection Rate");
• the Selection Rate for a particular category of individuals, divided by the Selection Rate for the most selected category of individuals; and/or
• the rate at which individuals in a given category receive more than the median score in a scored exercise (called the "Scoring Rate"), divided by the Scoring Rate for the individuals in the highest-scoring category.

The final rules make clear that the audit calculations must be conducted by an "objective and impartial" independent auditor. Among other restrictions, that individual cannot be an employee of the employer seeking to use the AEDT, must not have a financial interest in that employer, and must not have been involved in using or developing the AEDT at issue.

Once the bias audit is complete, what must an employer do with the data produced?

After completing the bias audit, the employer must publish the results and must notify City resident candidates in advance that an AEDT will be used in hiring or promotions.

For wider publication, the final rules require that an employer clearly and conspicuously disclose on its website (A) the date of the most recent bias audit; (B) a particular summary of results, including certain calculations described above; and (C) the date the employer first used the AEDT in making employment determinations.

What sort of notice must an employer provide to candidates regarding the use of AEDTs?

City employers who use AEDTs must inform City resident candidates for hiring and promotion that an AEDT will be used as an evaluative tool and must specify the job qualifications and characteristics that the AEDT will use in completing its evaluation. The employer must provide this notice at least 10 business days prior to using the AEDT. It must also notify such candidates that they may request an alternative selection process or accommodation and must include particular instructions regarding that request.

With regard to format, an employer can provide notice to new applicants or promotion candidates in the employment section of its website, in a job posting, or via U.S. mail or email. Employees seeking promotion can also receive notice via an employer's written policy or procedure.

Separate requirements also apply for disclosing the type of data collected, the source of data, and the employer's data retention policy.

What are the penalties for an employer's failure to comply with the Law?

The Law carries potentially significant financial consequences for noncompliance. It imposes a $500 penalty for a first violation, and a penalty of up to $1500 for each subsequent violation, which occurs each day that the AEDT is used in violation of the Law. In addition, if an employer fails to provide the required notice to a given candidate, that also constitutes a separate violation, with a penalty of up to $1500 for each individual. Any candidate for hiring or promotion can bring a civil action against the employer for its failure to comply, and the City's Corporation Counsel may also initiate a litigation to mandate compliance.

The rule's requirements are expansive, and only some of those requirements have been addressed here. 

[1] While this article refers to "employers" throughout, the requirements outlined herein apply equally to employers and covered employment agencies.