Ad Watchers: How Should You Present Scientific Evidence to Support Your Ad Claims?

Ad Watchers episode four of season two provides advertisers insights into presenting evidence to support a claim at NAD. Co-hosts Eric Unis and Annie Ugurlayan walk listeners through what to expect when presenting a claim substantiation case.  

Annie begins by explaining that no matter the claim type, the stronger it is, the stronger its supporting evidence must also be. She states that different types of evidence bear different weights. For instance, human trials are more persuasive than animal trials when supporting a claim regarding pharmaceuticals marketed toward people. Annie also points out that it isn’t just express claims that need substantiation. Implied claims hold the same burden of proof. She uses the example of a “nontoxic” claim which would lead consumers to expect no health risks, even transient risks, to occur from the use or misuse of the product. Companies that make such implied claims must be able to provide evidence to support them.  

Eric enters the conversation by sharing that industry standards may guide NAD’s expectations for substantiation. However, to do so, the standard must be current and relevant. Practitioners may deviate from the industry standard if their product testing is a better mimic of real-world conditions. He cautions that, regardless of the standard, all studies must be conducted under consumer-relevant conditions. Companies may turn to proprietary testing when an industry standard doesn’t exist, but such testing must be reliable, statistically significant, and consumer-meaningful. Eric then explains that NAD puts more evidential weight on full studies than on abstracts or summaries. The organization prefers to review the methodology to evaluate reliability and relevance.  

Eric poses an interesting question to Annie, asking about NAD’s approach to complex scientific issues. Annie explains that, in such cases, the organization allows both sides to bring in expert witnesses to analyze evidence and present an opinion. Expert reports are most useful when no clear scientific standard has been set. However, these reports are not enough to substantiate a claim on their own. Advertisers must also submit true scientific studies. Eric adds that NAD may give an expert opinion little to no weight if it is contrary to scientific consensus. In other words, if an opinion is an outlier among specialists, NAD is less likely to find it reliable.  

At this point, the duo introduces distinguished guest Tom Rosholt, the owner and principal of Tom Rosholt Consulting, Inc. His company offers consulting on ad claim substantiation, analytics, and research. After introducing himself, he defines claim substantiation as the task of forging a connection between a message and a metric. His responsibility as a consultant is to ensure that the message and the metric connect before the claim goes to market. If a claim is already in the market, he analyzes data to reveal the strengths and weaknesses of the claim’s supporting evidence.  

Earlier in the episode, Eric mentioned the importance of statistical significance, and here, he asks Tom to define it. Tom answers, explaining that it is the scientific equivalent of being able to prove something beyond a reasonable doubt. He then answers Annie’s question, “What is one thing you wish lay people understood about scientific evidence?” by opining that lawyers too often look at statistics as the truth rather than an approximation of the truth. He uses the example of flipping a coin 10 times. The chance of getting heads each time is 50 percent. Though this is the case, that does not mean that the coin flip will result in heads five out of 10 times. In his experience, many fail to consider that the coin could land on heads as many as 10 times or as few as zero.   

Tom makes a few recommendations for representatives in claim substantiation cases. Be skeptical and view a claim as unsubstantiated until you see support for it and be aware of bias and error in testing. He ends by explaining that as a statistician, his approach to analyzing data is static and objective. It does not change depending on the product or industry he is researching. His goal is to present the numbers clearly and honestly.  

Eric and Annie wrap up the show with the following takeaways: