New York City will become the first US jurisdiction to implement specific regulations governing the use of AI in matters related to employment. Starting from July 5, 2023, the Department of Consumer and Worker Protection (DCWP) will initiate the enforcement of Local Law 144 of 2021 (NYC 144). The NYC 144 imposes restrictions on the use of automated employment decision tools (AEDT), mandates yearly bias audits, and requires employers to notify individuals who are subjected to decisions made by the AEDT.

Following a lengthy notice and comment period regarding the initial regulations that were put forth in April 2022, the DCWP proceeded to release final regulations on April 5, 2023, in alignment with the new legislation (the DCWP Regulations). As the implementation of NYC 144 commenced on July 5, employers were afforded slightly more than two months to assess their hiring procedures, determine whether they use AEDTs in covered employment decisions, and take the stipulated measures to ensure compliance.

Regulatory Requirements

Scope

Under the regulations, employers in New York City must determine whether they utilize AEDTs that would bring them under the scope of NYC 144. The term 'AEDTs' is defined as:

• any computational process, derived from machine learning, statistical modeling, data analytics, or AI;
• that issues simplified output, including a score, classification, or recommendation; and
• that is used to substantially assist or replace discretionary decision-making for making employment decisions that impact natural persons.

Within this definition, AEDTs encompass mathematical, computer-driven techniques that produce forecasts, such as scores, rankings, or classifications, wherein computer participation is involved in identifying inputs and their corresponding significance.

The term 'employment decision' is defined under the law to exclusively pertain to choices associated with hiring or promoting an employee within the city, excluding decisions regarding compensation, termination, and similar matters. Furthermore, this definition is constrained decisions involving candidates who have applied for a position, thereby excluding its applicability to broad instruments designed for identifying potential candidates.

In order to qualify as having 'substantially assisted' in discretionary decision-making, the tool must satisfy one of the following conditions:

• it was the only or highest-weighted factor influencing the employment decision; or
• it was used to overrule conclusions derived from other factors, including decisions made by human judgment.

For example, a tool is considered to have 'substantially assisted' in making an employment decision in the following scenarios:

• a software designed for screening resumes determines which candidates are selected for an interview;
• a software designed for screening resumes with a human reviewer to assign scores to applicants, determining which candidates are selected for an interview. In this case, the software's score holds greater significance than the human evaluator's score; or
• a resume screening software reviews a list of candidates chosen by humans for interviews and eliminates candidates it deems are not a "good fit".

Bias Audits

In accordance with NYC 144, employers are obligated to conduct bias audits to evaluate their AEDTs within a year prior to the use of such tools. The precise procedures for conducting these audits are outlined in the DCWP Regulations. Thus, an employer may not use or continue to use an AEDT if more than one year has passed since the last bias audit on any particular AEDT. In practical terms, any employer that currently uses AEDTs for employment decisions may be required to suspend the use of the AEDT until the obligations of performing bias audits are fulfilled.

The bias audit must calculate and assess the tool's selection rate and impact ratio for specific demographic classifications, such as race/ethnicity and sex. This audit must be conducted by an impartial and independent auditor. To be independent, the auditor must refrain from:

• having used, developed, or distributed the AEDT itself;
• possessing any financial interest in the employer seeking AEDT implementation or in any AEDT vendors; or
• had any employment relationship with the employer seeking to use AEDTs or any AEDT vendors.

Once the bias audit has been conducted by the independent auditor, employers must publish the AEDT distribution date, the audit date, and a summary of the outcomes on the employment segments of their publicly accessible websites. The disclosure should be presented in a clear and conspicuous manner. The summary must provide details of the bias audit, including an explanation of the data used to conduct the audit, the number of applicants, and the calculated impact ratios for all assessed categories. This information must remain accessible on the employer's website for a minimum of six months following the latest use of the AEDT for an employment decision.

Notice to NYC Residents

Furthermore, NYC 144 requires employers to provide notice to all job candidates and employees who reside in New York City and are subject to decisions carried out by AEDTs within 10 business days prior to the use of AEDT for their assessment. Specifically, the notification must state the intention to employ an AEDT in connection with the evaluation of that particular individual and list the qualifications and characteristics of the job that the AEDT will consider in its assessment. Additionally, the notice must include instructions for individuals to request an alternative selection process, although the law specifies that it does not require employers to offer an alternative process.

In addition, unless already disclosed on the employer's website, NYC 144 requires employers to share their data collection practices and data retention policy within 30 days upon a written request from a candidate or employee.

The notice requirements outlined by NYC 144 can be fulfilled through a single, clear, and conspicuous notice posted on the employer's website, on a job listing, or via a mailed letter or email to the candidate or employee.

Enforcement

Violation of NYC 144 results in the imposition of a civil penalty, amounting to a maximum of $500 for the initial offense and every subsequent violation that occurs on the same day. For each subsequent violation following the initial occurrence, the penalty escalated to a maximum of $1,500. Furthermore, for each day during which an AEDT is used in violation of the law, an individual violation is deemed to occur, giving rise to further penalties.

Moreover, failure to comply with the law's notice requirements constitutes a distinct violation. It's important to note that NYC 144 does not provide a private right of action.

Implications

The enactment of NYC 144 is the most recent advancement in New York's legislative drive to foster transparency and safeguard privacy rights in data-related practices. For example, in 2022, the State of New York introduced legislation requiring employers to post conspicuous notices about employee electronic monitoring practices. Additionally, in 2021, New York City passed laws governing the use of biometric identifiers in commercial establishments, alongside obligations to inform tenants about specific data collection practices.

Nevertheless, as the first law specifically governing the use of AI in employee hiring and promotions within the US, NYC 144 could potentially lay down a blueprint for other policymakers interested in addressing privacy and bias concerns associated with AI deployment in this realm. At the same time, the U.S. Equal Employment Opportunity Commission (EEOC) and other regulatory bodies have been very clear that they can oversee and enforce regulations concerning the application of AI and algorithms in employment and other contexts, pursuant to existing laws.

Mark Francis Partner
[email protected]
Sophie Kletzien Associate
[email protected]
Holland & Knight LLP, New York