Takeaway: Following the U.S. Supreme Court’s decision in Comcast Corp. v. Behrend, 133 S. Ct. 1426 (2013), the ability of a class action plaintiff to win class certification can often rise or fall with the plaintiff’s ability to develop a class-wide damages model. In recent years, an economist armed with a choice-based “conjoint survey” has emerged as the go-to method for developing a damages model, especially in a case asserting a benefit-of-the bargain theory of damages. The Ninth Circuit recently reversed the exclusion of just such an expert, even though the district court found the conjoint survey had “major methodological flaws” and the expert failed to comport with “accepted principles” in the scientific community. MacDougall v. Am. Honda Motor Co., No. 20-56060, 2021 WL 6101256 (9th Cir. Dec. 21, 2021). Even though unpublished, this decision will surely be relied upon by class action plaintiffs to defend against challenges to the admissibility of a conjoint survey analysis, no matter how flawed it is.

In MacDougall, the plaintiffs brought a putative class action against American Honda Motor Co., Inc. (“Honda”), alleging breach of warranty claims and violations of various consumer protection laws, including California’s Consumer Legal Remedies Act and Unfair Competition Law. According to the plaintiffs, certain Honda Odyssey minivan models suffer from a defect that causes the torque converter to jerk or judder while the vehicles are in motion. To calculate class-wide damages, the plaintiffs relied upon the testimony of Stefan Boedeker, an economist and statistician. Mr. Boedeker relied upon a conjoint survey, which asked consumers to choose between products with different sets of features or attributes. The survey responses were then used to determine a dollar value that consumers attach to each measured feature. In this case, Mr. Boedeker used the results of the conjoint study to measure the difference in value (and thus the damages owed) between the vehicles with and without the alleged defect.

After discovery, the plaintiffs moved for class certification, and in response, Honda opposed class certification and moved to strike Mr. Boedeker’s testimony under Federal Rule of Evidence 702 and Daubert. In a thorough opinion, the district court granted Honda’s motion to strike on the grounds that Mr. Boedeker’s underlying methodology rendered his report unreliable. MacDougall v. Am. Honda Motor Co., No. SACV 17-1079-JGB-DFM, 2020 WL 5583534 (C.D. Cal. Sept. 11, 2020), rev’d and remanded, No. 20-56060, 2021 WL 6101256 (9th Cir. Dec. 21, 2021).

The district court observed that, as a general rule, “technical inadequacies” in the administration of an expert’s survey or disputes about an expert’s credentials bear on the weight of an expert’s report, not its admissibility. 2020 WL 5583534, at *9 (citing Fortune Dynamic Inc. v. Victoria’s Secret Stores Brand Mgmt., Inc., 618 F.3d 1025, 1036 (9th Cir. 2010)). For this reason, the district court did not consider a number of contentions raised by Honda about the conjoint survey, including focalism bias in the survey questions, demographical disparities among survey respondents, and the lack of Mr. Boedeker’s expertise. But the district court found that “two major flaws” in Mr. Boedeker’s report warranted “striking the report in its entirety.” Id. at *5.

First, the district court found that the report failed to adequately account for supply-side considerations. Since market price depends upon both supply and demand, it is necessary to include accurate and realistic supply-side considerations in a conjoint survey, such as the actual price of a product. Without such supply-side considerations, a conjoint survey may grossly overestimate damages by measuring what a consumer is willing to pay for a given feature of a product, not the market price that consumers actually pay. While Mr. Boedeker recognized the importance of supply-side considerations, the district court found that he failed to use actual market prices in his survey and “[b]y untethering his survey price choices from real-world prices, Boedeker’s methodology sows doubt as to the reliability of his [damages] calculation.” Id. In ruling that “Boedeker’s speculative valuation of prices in his survey choices is sufficient to strike his report,” the district court collected cases from “across the country” where courts “have excluded choice-based conjoint analyses that fail to accurately account for supply-side considerations.” Id. at *6.

Second, the district court found that Mr. Boedeker failed to utilize a pretest of the final conjoint survey—an act typically done to ensure that the questions are not confusing, misleading, and accurately measure respondent preferences. Id. at *7. The district court ruled that the failure to pretest casts doubt on the reliability of the survey responses and also pointed to a number of examples in the data demonstrating unreliability: four percent of the survey sample expressed confusion over the survey questions, some chose “absurd or illogical responses that reflected a misunderstanding of the survey choices and instructions,” and “a remarkable 55.4% of individual responses reflected economically irrational choices as to either prices levels, the presence of defects, or both.” Id. at *8. As two more examples, the district court noted that one respondent indicated a willingness to pay three million dollars more for a vehicle with a defect than without, and another indicated a willingness to pay a whopping ten million dollars more for a vehicle without the defect.

“Altogether,” the district court ruled that “the irrationality exhibited in individual survey responses evidences a deeply flawed conjoint study that produced unreliable results unreflective of actual [willingness to pay].” Id. The district court concluded that Mr. Boedeker’s underlying methodology rendered his report unreliable under Rule 702 and Daubert and granted Honda’s motion to strike.

The Ninth Circuit summarily reversed. Without discussing the district court’s thorough analysis, the Ninth Circuit observed that the “district court relied on numerous cases that do not analyze the admissibility of conjoint analysis under Rule 702 or Daubert … .” MacDougall, 2021 WL 6101256 at *1. The Ninth Circuit then concluded that “Honda’s challenges—inter alia, the absence of market considerations, specific attribute selection, and the use of averages to evaluate the survey data—‘go to the weight given the survey, not its admissibility.’” Id. (quoting Wendt v. Host Int’l, Inc., 125 F.3d 806, 814 (9th Cir. 1997)).

As of this post, Honda has filed a motion for extension of time to file a petition for panel rehearing or rehearing en banc. It remains to be seen whether the Ninth Circuit’s decision will stand. If so, this decision, even though unpublished, will almost certainly be cited for the proposition that attacks on conjoint surveys go to the weight, not its admissibility, no matter how absurd some of the results may be.