NYC Issues Proposed Regulations Regarding Automated Employment Decision Tools Law

On September 23, 2022, the New York City Department of Consumer and Worker Protection (DCWP) published proposed regulations regarding the city’s Automated Employment Decision Tools (AEDT) law, which goes into effect on January 1, 2023. As previously reported on Cooley’s cyber/data/privacy insights blog, the AEDT law will require employers and employment agencies in New York City to comply with an extensive series of requirements before using an AEDT in the hiring and promotion process. Among other things, employers will be required to:

• Complete an independent bias audit of the tool.
• Provide a publicly available summary regarding the audit and the distribution date of the tool.
• Give notice to candidates and employees who have applied for a position and reside in the city that the tool will be used – as well as the job qualifications and characteristics the tool will measure in assessing candidates – and permit candidates to request an alternative selection process or accommodation.
• Make available information about the source and type of data collected by the tool, and the employer’s data retention policy, unless disclosure of this information would violate law or interfere with a law enforcement investigation.

In enacting the AEDT law, New York City joined other jurisdictions, such as Illinois and Maryland, which regulate artificial intelligence-driven tools in the employment process, due to concerns of bias or disparate impact on certain groups.

Key definitions

The DCWP will be collecting comments on the proposed regulations in advance of a public hearing scheduled for October 24, 2022. Among other things, the proposed regulations seek to define certain key terms and clarify requirements for the bias audit, the publication of the results of the bias audit, and the notices that employers and agencies must provide to employees and candidates. We’ve outlined some of these elements here.

Substantially assist or replace discretionary decision making

The AEDT law defines an “automated employment decision tool” as “any computational process, derived from machine learning, statistical modeling, data analytics, or artificial intelligence, that issues simplified output, including a score, classification, or recommendation, that is used to substantially assist or replace discretionary decision making for making employment decisions that impact natural persons.” The proposed regulations clarify that the phrase “substantially assist or replace discretionary decision making” means to:

• Rely solely on a simplified output (e.g., score, tag, classification, ranking, etc.) with no other factors considered.
• Use a simplified output as one of a set of criteria where the output is weighted more than any other criteria.
• Use a “simplified output to overrule or modify conclusions derived from other factors including human decision-making.” (A “simplified output” is further defined as a prediction or classification that can take the form of a score, tag or categorization, recommendation, or ranking.)

Candidate for employment

The AEDT law applies to employment decisions affecting “candidates for employment or employees for promotion.” The proposed regulations define the former as “a person who has applied for a specific employment position by submitting the necessary information and/or items in the format required by the employer or employment agency.” As such, the proposed definition clarifies that candidates must have made affirmative steps by submitting information or other items to be covered under the law.

Independent auditor

The AEDT law requires that an independent auditor conduct a bias audit of any AEDT tool prior to using it. The proposed regulations define an “independent auditor” as “a person or group that is not involved in using or developing an AEDT that is responsible for conducting a bias audit of such AEDT.” Although this definition provides some guidance, it remains unclear what level of independence is required of the auditor, including whether the auditor may be otherwise employed by the AEDT’s vendor.

Bias audit requirements

As discussed above, an AEDT cannot be used unless it has been subject to an independent bias audit no more than one year prior to the use of the tool. The proposed regulations clarify the requirements of the bias audit, which now include calculations of a “selection rate” and “impact ratio” for each category required to be reported to the Equal Employment Opportunity Commission (EEOC) pursuant to the EEOC Employer Information Report EEO-1 (race, ethnicity and sex).

Under the proposed regulations, “selection rate” means the “rate at which individuals in a category are either selected to move forward in the hiring process or assigned a classification by an AEDT.” This rate is calculated by dividing the number of individuals in the category moving forward or assigned a classification by the total number of individuals in the category who applied for a position or were considered for promotion. On the other hand, “impact ratio” is defined as either “(1) the selection rate for a category divided by the selection rate of the most selected category, or (2) the average score of all individuals in a category divided by the average score of individuals in the highest scoring category.” Thus, an independent auditor conducting a bias audit must compare selection rates for each EEO-1 category and compare those rates to the most selected or favored category.

The kinds of calculations required in a bias audit depend on how the AEDT is used in the employment process. When an AEDT selects individuals to move forward in the hiring process or classifies them into groups, the bias audit must calculate for each category the selection rate, the impact ratio and, where an AEDT classifies individuals into groups, the calculations for the selection rate and impact ration must be performed for each such classification. However, when an AEDT scores applicants or candidates, the audit must calculate the average score for individuals in each category, and calculate the impact ratio for each category.

Publishing results of a bias audit

Prior to using an AEDT, employers must publish in a “clear and conspicuous manner” on their website’s careers or jobs section, the date of the most recent bias audit of the AEDT, a summary of the results of the audit, including selection rates and impact ratios for all categories, and the distribution date of the AEDT, or the date the employer began using a specific AEDT. The proposed regulations provide that employers can meet this requirement by including a hyperlink to an external website that contains the required information. Further, the required information must remain posted for at least six months after last using the AEDT for an employment decision.

Notice to candidates and employees

As discussed above, the AEDT law requires employers to provide notice to candidates and employees with specified information regarding the tool’s use and other requirements at least 10 business days prior to using the AEDT. The proposed regulations permit the required notices to be given to candidates who reside in the city by including the notice on the employer’s website’s careers or jobs section, in a job posting, or via email or mail. An employer can satisfy the notice requirements for current employees by including it in a written policy, procedure or job posting, or providing it in person, or via email or mail. Although the notice must include “instructions for how to request an alternative selection process or accommodation,” the proposed regulations ambiguously state that an employer is not required to provide an alternative selection process.

The proposed regulations also clarify the law’s requirement that employers provide information regarding the type of data collected and their data retention policy. Employers may satisfy this requirement by posting the information to their website’s careers or jobs section or by providing it in writing via mail or email. If the notice is not included on the website, an employer must post instructions on how to make a written request for such information on its website.

Next steps

While the proposed regulations are not final, employers can get ahead of the AEDT law by reviewing any AI tools used in the hiring or promotion process to determine whether those tools are covered by the law. Employers using such tools should reach out to the tool vendors regarding their compliance with the AEDT law. Employers also should train supervisors and human resources personnel regarding the law’s myriad requirements, including its publication and notice provisions.

We will continue to follow updates to the AEDT law after the October 24 hearing. If you have any questions about the AEDT law, please reach out to a member of the Cooley employment team.

This content is provided for general informational purposes only, and your access or use of the content does not create an attorney-client relationship between you or your organization and Cooley LLP, Cooley (UK) LLP, or any other affiliated practice or entity (collectively referred to as “Cooley”). By accessing this content, you agree that the information provided does not constitute legal or other professional advice. This content is not a substitute for obtaining legal advice from a qualified attorney licensed in your jurisdiction and you should not act or refrain from acting based on this content. This content may be changed without notice. It is not guaranteed to be complete, correct or up to date, and it may not reflect the most current legal developments. Prior results do not guarantee a similar outcome. Do not send any confidential information to Cooley, as we do not have any duty to keep any information you provide to us confidential. This content may be considered Attorney Advertising and is subject to our legal notices.