In December 2022, the National Association of Insurance Commissioners (NAIC) Big Data and Artificial Intelligence (H) Working Group published a Report on the use of artificial intelligence (AI) and machine learning (ML) technology by private passenger auto (PPA) insurers. The Report finds the overwhelming majority (88%) of surveyed insurers currently use, plan to use or plan to explore using AI/ML for their insurance operations, with the greatest use in claims, followed by marketing and fraud detection, and only a minority using AI/ML for underwriting, rating and loss prevention.

The AI/ML PPA analysis is the first of a series of AI/ML studies to be undertaken by state insurance regulators based on specific lines of business. A similar analysis is currently underway for homeowners’ and life insurance.

Background

Understanding how – and the extent to which – AI/ML technologies are employed for use in personal lines insurance operations (including through unaffiliated third parties) has been a priority of the NAIC and its Big Data/AI Working Group. The Working Group was created in large part to help regulators develop a better understanding of AI/ML and whether additional regulatory or supervisory action is required to protect consumers. To that end, the Working Group established a dedicated Workstream to develop AI/ML surveys and analyze their results.

For the initial AI/ML PPA survey, the Working Group established the following five objectives:

• Learn directly from the industry about what is happening in this space;
• Get a sense of the current level of risk and exposure and whether or how the industry is managing or mitigating that risk;
• Develop information for trending, such as how the risk is evolving over time, and the industry’s responsive actions;
• Inform a meaningful and useful regulatory approach, framework, and/or strategy for overseeing and monitoring this activity; and
• Learn from prior surveys to inform and improve future surveys.

Surveys were sent in 2021 to insurers that write PPA in one of the nine states participating in the survey¹ and had at least $75 million in national PPA insurance premium in 2020. The Working Group received responses from 193 insurers.

Defining AI/ML

A significant component of the Working Group’s effort has been to settle on a definition for AI/ML and define the scope of the systems that were to be included in (and excluded from) the AI/ML PPA Study. Ultimately, the Working Group defined AI/ML as:

An automated process in which a system begins recognizing patterns without being specifically programmed to achieve a predetermined result. This is different from a standard algorithm in that an algorithm is a process or set of rules executed to solve an equation or problem in a predetermined fashion. Evolving algorithms are considered a subset of AI/ML.

For purposes of the AI/ML PPA Study, regulators focused on “more advanced” models, which, among other items, included systems that:

• adapt and adjust to new data and experience without human intervention;
• arrive at results for which the outcomes and the stepwise approach toward the outcomes were not configured in advance by a human programmer;
• dynamically respond to conditions in the external environment without the specific nature of such responses being known in advance; and
• utilize neural networks and/or deep-learning algorithms, such as supervised, semi-supervised, and unsupervised learning algorithms.

Regulators expressly excluded from the scope of the AI/ML PPA Survey static scorecards that map risk characteristics to decisions and static ratemaking – including linear regression, generalized linear modeling and generalized additive modeling – and other approaches that insurers have employed for more than two decades.

Survey Results

As a preliminary matter, the AI/ML PPA Survey asked whether respondents use or intend to use AI/ML and, if so, asked them to respond to a number of questions based on one or more of the following six operational functions:

• Claims
• Fraud detection
• Marketing
• Rating
• Underwriting
• Loss prevention

In total, 88% of the 193 respondents indicated that they currently use, plan to use, or plan to explore using AI/ML (as defined above), while the remaining 12% indicated they had no plan to use or explore use of AI/ML for any of the six operational functions because (a) they saw no compelling business reason (b) due to a lack of resources or expertise or (c) they questioned whether AI/ML would necessarily yield better risk selection and product pricing. Companies responding in the affirmative reported varying levels of AI/ML use ranging from only 2% in the loss prevention area to 70% in claims operations. In particular, respondents reported using AI/ML for specific operational functions as follows:

Respondents also reported using AI/ML for a number of other operational functions, including: agency models; customer interactions; information technology models; knowledge management and language processing; social media sentiment analysis; premium audits; and video imaging to predict accidents.

In addition to covering uses of AI/ML, the survey also assessed levels of decision-making by AI/ML and the extent of use of models built in-house vs. purchased from third-party vendors. It also sought to elicit information on processes for customer data correction and model governance.

Respondents reported varying degrees of reliance on AI/ML for decision-making and execution. Claims relies more heavily on augmentation (defined as suggesting an answer and advising the human who is making the decision) and support (defined as a model that provides information but does not suggest a decision or action), as distinct from automation (defined as execution without human intervention). Claims assignment had the greatest usage of automation. Some respondents reported using automation for claims approval, only one for fraud detection and none reported automated claims denials. Marketing had the greatest uptake of automation, with the highest use for direct online advertising, direct online sales and offers to existing customers. Fewer companies reported use of AI/ML for Rating and Underwriting, which the Report attributes to greater reliance on traditional ratemaking techniques and older generation static predictive models. For Rating, the greatest use of automation is Retention Modeling,² Numerical Relativity Determination³ and “other” uses. For Underwriting, its use was limited to only for Automated Denials and “other” uses.

Of the 2,531 models identified by the survey, roughly 60% were developed by insurers in-house and 40% were developed by more than 70 third-party vendors. The survey lists the model developers named by the respondents in each functional area where used.

The survey of customer data correction opportunities proved to be of reduced value because the majority of companies declined to respond to the two survey questions: whether consumers are provided more information about data elements than required by law and whether the company has more consumer data correction processes than required by the Fair Credit Reporting Act (FCRA).

The purpose of the governance questions was to obtain a better understanding regarding insurers’ awareness of specific risk areas tied to the NAIC’s AI Principles and whether a principle has been documented within the governance program. As with consumer disputes, many companies did not respond, limiting the utility of the survey. Among the responses, the highest levels of documented governance processes were for Underwriting and Rating, except processes to ensure transparency including notice to consumers in Underwriting, where there were an equal number of yes and no responses.

Report Recommendations

The memorandum accompanying the Report included the following six recommendations as potential helpful next steps for discussions and decision-making at the NAIC:

1. Evaluate the survey analysis and determine whether to further explore the following subjects:
   • Insurer AI/ML model usage and the level of decision-making (i.e., the amount of human involvement in decision-making).
   • Insurer data elements.
   • Insurers’ governance frameworks and the documentation of such.
   • Consumer data recourse.
   • Third-party regulatory framework.
2. Create a risk hierarchy to prioritize the need for more model governance and insurer oversight. The general concept is that more oversight of a model will be needed as the consumer risk or impact increases from the modeling or models.
3. Evaluate consumer data recourse. Insurers report a wide variety of methods for consumers to evaluate and correct data used by insurers. Some methods are short and easy, such as using an app to correct data, and other methods are more time-consuming and require personal contact with the agent or company. In some cases, consumers may not know their data is being used, so consumer transparency is a priority. (Privacy Protections (D) Working Group)
4. Evaluate the regulatory framework around the use of third-party models and third-party data. Evaluate the ability of insurers and state insurance regulators to obtain needed information from third parties and for regulators to oversee this work either through the insurers or third parties in some way. (Workstream Two of the Big Data and Artificial Intelligence (H) Working Group)
5. Evaluate concerns about third-party concentration by insurer use. (Workstream Two of the Big Data and Artificial Intelligence (H) Working Group)
6. Determine whether additional white papers on best practices would be useful on subjects in the AI/ML space.

Future Reports on AI/ML Use in Home and Life Sectors

The PPA AI/ML analysis is the first of a series of AI/ML studies expected to be undertaken by state insurance regulators through the Working Group. A similar survey is under way for homeowners. The scope and nature of the life insurance survey remains ongoing, but the current expectation is that it will focus on large life insurers (greater than $250 million in premium on all individual policies in 2021), term life providers (that have issued policies on more than 10,000 lives), and certain specified “insurtech” companies that focus on life insurance. The Draft AI/ML Life Survey appears to require substantially more granular detail than that which was required by either the PPA or Homeowners’ Surveys.

_________________

1 Connecticut, Illinois, Iowa, Louisiana, Nevada, North Dakota, Pennsylvania, Rhode Island and Wisconsin.

[View source.]