Ad Watchers: How Should You Present Scientific Evidence to Support Your Ad Claims?
July 27, 2022
Understanding how to present scientific evidence in a substantiation case can be difficult, especially when the data is complex or no industry standard exists.
In this episode of Ad Watchers, the hosts discuss a complex topic: how should marketers put scientific data to use? They answer this question by diving into different types of scientific evidence and which types of claims they support most effectively. For instance, they provide insight into when to bring forward an expert witness and the supplementary evidence needed to support their opinion. Statistician Tom Rosholt also joins them to share an analyst's approach to presenting unbiased and reliable data. Listen now to learn what to expect when presenting a case at NAD.
Ad Watchers episode four of season two provides advertisers insights into presenting evidence to support a claim at NAD. Co-hosts Eric Unis and Annie Ugurlayan walk listeners through what to expect when presenting a claim substantiation case.
Annie begins by explaining that no matter the claim type, the stronger it is, the stronger its supporting evidence must also be. She states that different types of evidence bear different weights. For instance, human trials are more persuasive than animal trials when supporting a claim regarding pharmaceuticals marketed toward people. Annie also points out that it isn’t just express claims that need substantiation. Implied claims hold the same burden of proof. She uses the example of a “nontoxic” claim which would lead consumers to expect no health risks, even transient risks, to occur from the use or misuse of the product. Companies that make such implied claims must be able to provide evidence to support them.
Eric enters the conversation by sharing that industry standards may guide NAD’s expectations for substantiation. However, to do so, the standard must be current and relevant. Practitioners may deviate from the industry standard if their product testing is a better mimic of real-world conditions. He cautions that, regardless of the standard, all studies must be conducted under consumer-relevant conditions. Companies may turn to proprietary testing when an industry standard doesn’t exist, but such testing must be reliable, statistically significant, and consumer-meaningful. Eric then explains that NAD puts more evidential weight on full studies than on abstracts or summaries. The organization prefers to review the methodology to evaluate reliability and relevance.
Eric poses an interesting question to Annie, asking about NAD’s approach to complex scientific issues. Annie explains that, in such cases, the organization allows both sides to bring in expert witnesses to analyze evidence and present an opinion. Expert reports are most useful when no clear scientific standard has been set. However, these reports are not enough to substantiate a claim on their own. Advertisers must also submit true scientific studies. Eric adds that NAD may give an expert opinion little to no weight if it is contrary to scientific consensus. In other words, if an opinion is an outlier among specialists, NAD is less likely to find it reliable.
At this point, the duo introduces distinguished guest Tom Rosholt, the owner and principal of Tom Rosholt Consulting, Inc. His company offers consulting on ad claim substantiation, analytics, and research. After introducing himself, he defines claim substantiation as the task of forging a connection between a message and a metric. His responsibility as a consultant is to ensure that the message and the metric connect before the claim goes to market. If a claim is already in the market, he analyzes data to reveal the strengths and weaknesses of the claim’s supporting evidence.
Earlier in the episode, Eric mentioned the importance of statistical significance, and here, he asks Tom to define it. Tom answers, explaining that it is the scientific equivalent of being able to prove something beyond a reasonable doubt. He then answers Annie’s question, “What is one thing you wish lay people understood about scientific evidence?” by opining that lawyers too often look at statistics as the truth rather than an approximation of the truth. He uses the example of flipping a coin 10 times. The chance of getting heads each time is 50 percent. Though this is the case, that does not mean that the coin flip will result in heads five out of 10 times. In his experience, many fail to consider that the coin could land on heads as many as 10 times or as few as zero.
Tom makes a few recommendations for representatives in claim substantiation cases. Be skeptical and view a claim as unsubstantiated until you see support for it and be aware of bias and error in testing. He ends by explaining that as a statistician, his approach to analyzing data is static and objective. It does not change depending on the product or industry he is researching. His goal is to present the numbers clearly and honestly.
Eric and Annie wrap up the show with the following takeaways:
Make sure your testing matches the claims and is reliable and consumer relevant. All testing should elicit statistically significant results pertinent to real-world use of the product.
Expert reports alone do not suffice to provide a reasonable basis. These reports need to provide supporting evidence for any conclusions.
Bring your statistician into the fold before making a claim. This ensures you have the information necessary to make the right claims and support claims as needed.
Be skeptical in your approach to claim substantiation. Assume that a claim is not supported until there is evidence to the contrary. Remember that evidence needs to be reliable, and results should be as specific as possible.
The Government Purchase of Private Data
In this episode of Privacy Abbreviated, professor Matthew Tokson joins our hosts to discuss how the collection and sale of private data may help government agencies circumvent legal requirements.
Ad Watchers: What is the appeal of an appeal? Getting to Know NARB
In this episode of Ad Watchers, your hosts discuss a critical link in the chain of advertising industry self-regulation: the National Advertising Review Board, or NARB, the appellate body for National Advertising Division cases.
Filling Privacy Gaps with Soft Law Solutions
In this episode of Privacy Abbreviated, our hosts are joined by the Future of Privacy Forum’s Jameson Spivack to discuss how industry-developed standards and best practices can guide policymaking allowing hard law to adopt the lessons learned from soft law.
Ad Watchers: The NAD Top 10 — Did you know?
In this episode of Ad Watchers, hosts Dan and Annie take you behind the scenes of the U.S. system of self-regulation in the advertising industry and discuss some of the myths, misconceptions, and misunderstandings advertisers have.