Declining to rely on crowdsourced data, the National Advertising Division recommended that T-Mobile USA discontinue advertising claims about the speed of its network.

Competitor Verizon Communications challenged T-Mobile’s claim that it was the “fastest” 4G LTE network and its implied claims that Verizon’s LTE network was older and slower because it was built before T-Mobile constructed its network.

To support its claims, T-Mobile pointed to crowdsourced speed tests from the Ookla and Open Signal apps for the period of March 1, 2017, through April 17, 2017. The data—collected from users who downloaded one of the apps to test the speed of their devices—demonstrated that T-Mobile’s average download speed was 25.7 megabits per second, compared with Verizon’s 23.1 Mbps, with upload speeds of 12.1 Mbps for T-Mobile and 8.7 Mbps for Verizon.

Despite Verizon’s argument that it was continually making network improvements, the data showed it was still delivering slower speeds on its LTE network than T-Mobile was on its own network, the advertiser told the NAD.

Verizon countered that the speed tests were inaccurate. Several carriers, including Verizon, deprioritize customers who use more than 22 GB of data, but T-Mobile has a higher limit of 30 GB before deprioritization. Data users who experience deprioritization may be prompted to run additional speed tests that result in a “small but perceptible” impact on the average speed recorded on Verizon speed tests.

In addition, Verizon had only recently started data deprioritization, so the slowed speeds during the relevant period were a new experience for its customers, perhaps increasing the likelihood of speed testing after deprioritization. Other problems with the crowdsourced data included the failure to consider how different devices affect speed experience and that the speed tests did not factor in the effect of apps running in the background.

After finding that T-Mobile’s claim that it offers the fastest 4G LTE network is relevant to consumers, the NAD determined that the advertiser’s evidence did not provide a reasonable basis for the claim.

The Ookla speed tests can provide consumer-relevant information about the actual speeds consumers experience, particularly because the tests record data speed in a variety of real-world scenarios, the NAD acknowledged. But the Federal Communications Commission has cautioned that such measurements “are often done when … the service seems to be slow” and that randomized background scheduling is a more statistically valid approach to data collection.

“Because the speed tests may have oversampled deprioritized Verizon customers, T-Mobile’s data may have misrepresented the comparative 4G LTE speeds most Verizon customers actually experience,” the NAD wrote. “There was no evidence in the record demonstrating that the crowd sourced data on which T-Mobile relied was not unfairly biased in T-Mobile’s favor by data deprioritization policies, and, as a result, NAD concluded that the Ookla and Open Signal speed test results did not support a comparative claim that T-Mobile has the fastest 4G LTE network.”

The NAD recommended the advertiser discontinue the “fastest” claim and stop implying that its network is newer and Verizon’s is older.

To read the NAD’s press release about the decision, click here.

Why it matters: In a footnote, the NAD cautioned advertisers in the wireless service industry “to regularly monitor and reexamine their advertising claims to make certain that the underlying data upon which they are based is current so that their advertising claims are truthful, and recognize[] that changes to a network or network service conditions will impact whether and when a service provider can support comparative claims.” While the NAD acknowledged that crowdsourced data has some benefits, it cited the FCC for concerns about the proper collection of such information in order to provide a more accurate view of typical network performance.