On January 17, 2024, the New York State Department of Financial Services (NY DFS) released for public comment a proposed circular letter (Proposed Letter) to regulate the use of artificial intelligence systems (AIS) and external consumer data and information sources (ECDIS) in insurance underwriting and pricing. Under the Proposed Letter, all insurers authorized to write insurance in New York will be expected to establish a governance and risk management framework to manage their use of ECDIS and AIS, and will be expected to carry out a robust testing program to ensure that ECDIS and AIS are not used in an unfairly discriminatory manner.
A. Overview of the Proposed Circular Letter
The Proposed Letter is far-reaching in scope and is intended to prevent insurers authorized to do business in New York from engaging in any form of unfair discrimination against any protected classes or similarly situated individuals when using AIS or ECDIS to price or underwrite insurance. The proposal will require similar governance and testing of ECDIS or AIS tools developed by third-party vendors. ECDIS is broadly defined to encompass all data or information used to supplement or act as a proxy for traditional insurance underwriting factors, with only limited exceptions.
Disproportionate and Adverse Effect. In order to prevent ECDIS and AIS from being used in an unfairly discriminatory manner to underwrite insurance, insurers will be required to assess whether their use of ECDIS or AIS has a disproportionate and adverse effect on similarly situated insureds or insureds of a protected class. Insurers may not use ECDIS or AIS unless they can demonstrate that there is legitimate, lawful, and fair explanation or rationale for any differential effects on similarly situated insureds and that they cannot underwrite insurance based on a less discriminatory alternative that would meet their legitimate business needs.
Testing. Insurers should test their AIS and ECDIS underwriting models for unfair or unlawful discrimination prior to putting them into production and then on a regular basis thereafter, including when material updates or changes are made to the AIS or ECDIS. Testing should include a quantitative assessment of the AIS or ECDIS model for unfair bias under which the insurer will analyze whether the AIS or ECDIS underwriting model has a statistically significant adverse effect on a protected class versus a control group based on multiple statistical metrics. Insurers are also expected to conduct a qualitative assessment of the AIS or ECDIS models in order to be able to articulate the logical relationship between the ECDIS or other AIS variable and the risks that are being insured.
Governance. The board of directors of the insurer is responsible for overseeing how the ECDIS and AIS programs are developed and managed, while senior management is responsible for implementation. Insurers are expected to develop written policies and procedures to govern the insurer’s use of ECDIS and AIS, including any EDCIS or AIS developed by third-party vendors. The written policies and procedures must be consistent with the guidelines set forth in the Proposed Letter, and should be annually reviewed and approved by the insurer’s board of directors (or a committee or senior management under the board’s delegated authority).
Risk Management. Insurers are also expected to establish a risk management framework for ECDIS and AIS that will be located either within their existing enterprise risk management function or as a standalone program. The framework will include standards for the development, implementation, use, and validation of AIS and EICDS. Insurers should use their existing internal audit function to assess the overall effectiveness of the AIS and ECDIS risk management program, including verifying that proper policies and procedures are in place and adhered to.
Transparency.The Proposed Letter clarifies that insurers using ECDIS or AIS are required under New York’s unfair trade practices law to disclose to an applicant their right to receive information explaining the insurer’s basis for any adverse underwriting decision, including decisions made using an AIS. Insurers who use ECDIS or AIS should provide a notice to the insured or potential insured disclosing: (i) whether the insurer uses AIS in the underwriting process, (ii) whether the insurer uses data from third-party vendors, (iii) that the applicant has the right to request information about the specific data that resulted in the underwriting or pricing decision, and (iv) the contact information the applicant should use to make such a request.
B. Comparison to Colorado and NAIC AI Regulations
This Proposed Letter comes in the heels of similar efforts by the National Association of Insurance Commissioners (NAIC) and the state of Colorado to govern the use of AIS in the insurance industry, although as currently drafted the Proposed Letter is broader in scope than either of these two other initiatives.
Colorado Regulations. On September 21, 2023, the Colorado Division of Insurance (CDI) adopted the first-of-its-kind regulation requiring life insurers to implement a governance and risk management framework governing the use of ECDIS and AIS. CDI has also proposed a regulation requiring life insurers to test their AIS and ECDIS underwriting models for racial bias (collectively, Colorado AI Regulations). The Proposed Letter is similar in structure to the Colorado AI Regulations as they both require life insurers to establish a governance and risk management framework and testing program to ensure the insurer is not engaging in unfair discrimination when using ECDIS and/or AIS to underwrite insurance.
However, the scope of the Proposed Letter is significantly broader than the Colorado Regulations as it applies to all forms of unfair discrimination, while the Colorado Regulations are currently limited to preventing unfair discrimination based on race. The Colorado Regulations are also currently limited to life insurers although the Colorado Division of Insurance is likely to expand their AIS and ECDIS governance framework to property & casualty as well as health insurer’s, albeit possibly with different testing requirements, and has conducted a stakeholder meeting on unfair discrimination in the passenger auto insurance market, with additional meetings expected in 2024. The Colorado Regulations also provide more detail on the methodology (i.e. Bayesian Improved First Name Surname Geocoding) to be used to estimate the race or ethnicity of each insured, whereas under the Proposed Letter the insurer has more discretion on which methodology it may use to ascertain whether their use of ECDIS or AIS constitutes unfair discrimination. Insurers may struggle to identify and develop methodologies to identify areas of unfair discrimination against non-racial or ethnic protected classes and the Proposed Letter does not provide clear guidance on what methodologies might comply with the Proposed Letter’s testing requirements.
NAIC Model Bulletin. The NAIC adopted on December 4, 2023 a model bulletin (NAIC Model Bulletin) to encourage insurers to adopt an AI Systems Program to govern their use of AIS in order to mitigate the risk of potential harm to consumers. The Proposed Letter is consistent with the NAIC Model Bulletin in setting the expectation that all insurers establish a governance and risk management framework to provide oversight of the insurer’s AIS program. Under both the NAIC Model Bulletin and Proposed Letter, the board of directors and senior management of the insurer have a key role to play in the development, implementation and monitoring of the governance framework and both assign the insurer’s internal audit function with the responsibility for assessing the effectiveness of the framework.
However, the Proposed Letter imposes more stringent guidelines for testing AIS and ECDIS for unfair discrimination, requiring that insurers provide extensive proof that their use of ECDIS and AIS does not unfairly discriminate against similarly situated insureds or a protected class. The NAIC Model Bulletin in contrast encourages insurers to test their AIS for potential unfair discrimination but does not require that the insurer provide affirmative proof that their AIS is not engaged in unfair discrimination. The Proposed Letter also concerns the use of both ECDIS and AIS whereas the NAIC Model Bulletin is limited primarily to AIS.
C. Feedback Request
Comments on the Proposed Letter are due to NY DFS by March 17, 2024 and may be submitted to innovation@dfs.ny.gov. Interested parties should use “Proposed Circular on the use of AIS and ECDIS in Insurance Underwriting and Pricing” in the subject line when submitting a comment.
[View source.]
