[co-author: Henry Thompson-Smith]
New York City employers who use artificial intelligence technologies to recruit and hire candidates and promote employees will soon face significant compliance obligations aimed at preventing bias when using such technology. In response to the increased use of AI in the recruitment process, the New York City council recently passed Local Law Int. No. 1894-A, amending the City’s administrative code to protect employees from unlawful discriminatory bias when automated employment decision tools are used in the hiring and promotion process. Employers have until January 1, 2023, to arrange for an independent bias audit of any automated employment decision tools in use and comply with notice requirements. Here’s a breakdown of what employers need to know.
What Is an Automated Employment Decision Tool?
An automated employment decision tool is “any computational process, derived from machine learning, statistical modeling, data analytics, or artificial intelligence, that issues simplified output, including a score, classification, or recommendation” that substantially assists or replaces discretionary decision making with respect to screening candidates for employment or promotion. The term does not include routine data processing mechanisms like junk email filters, firewalls, spreadsheets, databases, or other compilations of data.
Who Conducts a Bias Audit?
A bias audit consists of an impartial evaluation of the automated employment decision tool by an independent auditor. The bias audit must assess any disparate impact of the automated employment tools on individuals because of their race, ethnicity, or sex.
What Must Employers Do?
Employers and employment agencies using an automated employment tool to screen a candidate for employment or employee for promotion must:
- Arrange for an independent bias audit of any automated employment decision tool to assess the tool’s disparate impact on individuals based on their race, ethnicity, or sex no more than one year prior to the use of the tool;
- Make publicly available on its website a summary of the results of the most recent bias audit and the distribution date of the tool before the tool is used; and
- Provide candidates for employment and employees who reside in New York City with notice, no less than 10 business days before the use of the tool, of the following:
- The use of an automated employment decision tool for assessment or evaluation;
- The ability for the candidate to request an alternative selection process or accommodation; and
- The job qualifications and characteristics that the automated employment decision tool will use in the assessment of the candidate or employee.
Additionally, within 30 days of a written request by the candidate or employee, the employer must provide information regarding the type of data collected for use by the automated employment decision tool, the source of such data, and information about the employer or employment agency’s data retention policies. Alternatively, this information may be disclosed on an employer website. Employers do not need to disclose this information if disclosure would violate a local, state, or federal law or interfere with a law enforcement investigation.
What Are the Penalties for Non-Compliance?
Employers who violate this new law will be assessed a civil penalty of $500 for the first violation. Subsequent violations will vary between $500 and $1,500. Each day the automated employment decision tool is used in violation of the law counts as a separate violation.
Additionally, the notice requirements consist of three distinct obligations that will constitute separate violations. There is no cap on civil penalties, which means employers can rack up significant fines if they fail to abide by the provisions of this law. NYC’s Office of Administrative Trials and Hearings or another designated agency will oversee the recovery of civil penalties, while the City’s Corporation Counsel (or a designee) will have enforcement power.
Next Steps for Employers
If you use AI technologies in the recruitment or promotion process, you have about a year to have any such tools independently audited for discriminatory bias. The results of the bias audit will need to be posted on your website. Along with this assessment, you will need to take steps to implement the detailed notice requirements of the law.
Accordingly, you will need to take a hard look at your hiring procedures to determine if the inherent waiting period built into the notice process will cause any undue delay in the recruitment process and plan accordingly. For example, an employer using AI technology to review resumes or conduct candidate assessment tests need to immediately provide the requisite notice to the candidate upon submission of their resume or application materials and then wait at least 10 business days before utilizing said technology (if the candidate does not opt-out). If your business has pressing hiring demands, this 10-day wait period could be critical. Further, you should be prepared with an “alternative selection process or accommodation” should a candidate opt-out of the automated process. The City has not yet provided any guidance on this alternative process.
With the expected confusion (and frustration) over the notice process, adequacy of alternative selection processes or accommodations, sufficiency of independent audits, and, to cap it all off, the new collection and enforcement powers granted to OATH and the Corporation Counsel, additional guidance is likely to be forthcoming before the effective date.
