Ad Watchers: How Should You Present Scientific Evidence to Support Your Ad Claims?
July 27, 2022
Understanding how to present scientific evidence in a substantiation case can be difficult, especially when the data is complex or no industry standard exists.
In this episode of Ad Watchers, the hosts discuss a complex topic: how should marketers put scientific data to use? They answer this question by diving into different types of scientific evidence and which types of claims they support most effectively. For instance, they provide insight into when to bring forward an expert witness and the supplementary evidence needed to support their opinion. Statistician Tom Rosholt also joins them to share an analyst's approach to presenting unbiased and reliable data. Listen now to learn what to expect when presenting a case at NAD.
Ad Watchers episode four of season two provides advertisers insights into presenting evidence to support a claim at NAD. Co-hosts Eric Unis and Annie Ugurlayan walk listeners through what to expect when presenting a claim substantiation case.
Annie begins by explaining that no matter the claim type, the stronger it is, the stronger its supporting evidence must also be. She states that different types of evidence bear different weights. For instance, human trials are more persuasive than animal trials when supporting a claim regarding pharmaceuticals marketed toward people. Annie also points out that it isn’t just express claims that need substantiation. Implied claims hold the same burden of proof. She uses the example of a “nontoxic” claim which would lead consumers to expect no health risks, even transient risks, to occur from the use or misuse of the product. Companies that make such implied claims must be able to provide evidence to support them.
Eric enters the conversation by sharing that industry standards may guide NAD’s expectations for substantiation. However, to do so, the standard must be current and relevant. Practitioners may deviate from the industry standard if their product testing is a better mimic of real-world conditions. He cautions that, regardless of the standard, all studies must be conducted under consumer-relevant conditions. Companies may turn to proprietary testing when an industry standard doesn’t exist, but such testing must be reliable, statistically significant, and consumer-meaningful. Eric then explains that NAD puts more evidential weight on full studies than on abstracts or summaries. The organization prefers to review the methodology to evaluate reliability and relevance.
Eric poses an interesting question to Annie, asking about NAD’s approach to complex scientific issues. Annie explains that, in such cases, the organization allows both sides to bring in expert witnesses to analyze evidence and present an opinion. Expert reports are most useful when no clear scientific standard has been set. However, these reports are not enough to substantiate a claim on their own. Advertisers must also submit true scientific studies. Eric adds that NAD may give an expert opinion little to no weight if it is contrary to scientific consensus. In other words, if an opinion is an outlier among specialists, NAD is less likely to find it reliable.
At this point, the duo introduces distinguished guest Tom Rosholt, the owner and principal of Tom Rosholt Consulting, Inc. His company offers consulting on ad claim substantiation, analytics, and research. After introducing himself, he defines claim substantiation as the task of forging a connection between a message and a metric. His responsibility as a consultant is to ensure that the message and the metric connect before the claim goes to market. If a claim is already in the market, he analyzes data to reveal the strengths and weaknesses of the claim’s supporting evidence.
Earlier in the episode, Eric mentioned the importance of statistical significance, and here, he asks Tom to define it. Tom answers, explaining that it is the scientific equivalent of being able to prove something beyond a reasonable doubt. He then answers Annie’s question, “What is one thing you wish lay people understood about scientific evidence?” by opining that lawyers too often look at statistics as the truth rather than an approximation of the truth. He uses the example of flipping a coin 10 times. The chance of getting heads each time is 50 percent. Though this is the case, that does not mean that the coin flip will result in heads five out of 10 times. In his experience, many fail to consider that the coin could land on heads as many as 10 times or as few as zero.
Tom makes a few recommendations for representatives in claim substantiation cases. Be skeptical and view a claim as unsubstantiated until you see support for it and be aware of bias and error in testing. He ends by explaining that as a statistician, his approach to analyzing data is static and objective. It does not change depending on the product or industry he is researching. His goal is to present the numbers clearly and honestly.
Eric and Annie wrap up the show with the following takeaways:
Make sure your testing matches the claims and is reliable and consumer relevant. All testing should elicit statistically significant results pertinent to real-world use of the product.
Expert reports alone do not suffice to provide a reasonable basis. These reports need to provide supporting evidence for any conclusions.
Bring your statistician into the fold before making a claim. This ensures you have the information necessary to make the right claims and support claims as needed.
Be skeptical in your approach to claim substantiation. Assume that a claim is not supported until there is evidence to the contrary. Remember that evidence needs to be reliable, and results should be as specific as possible.
The Future of EU-U.S. Data Transfers
On October 7, the negotiations between the U.S. and European Commission regarding the future of the data privacy frameworks behind the Privacy Shield program were completed with the release of a Presidential Executive...
Ad Watchers: Where is the Line Between Ethical Design and Dark Patterns?
Think about the times you felt tricked or frustrated by a membership or subscription that had a seamless signup process but was later difficult to cancel. Something that should be simple and transparent can be complicated, intentionally or unintentionally, in ways that impair consumer choice. These are examples of dark patterns. Unfortunately,...
The Metaverse Will Blur the Lines Between Physical and Online Privacy
On this episode of Privacy Abbreviated, hosts Dona and Catherine are joined by Tracy Shapiro, a privacy expert, and partner at Wilson, Sonsini, Goodrich, & Rosati. Together, they discuss the many questions related to how virtual reality will force privacy standards to evolve in the coming years. Though no one has concrete answers yet, Tracy offers her predictions on...
Ad Watchers: How is Direct Selling Advertising Different?
Ad Watchers hosts are joined by Direct Selling Self-Regulatory Council Vice President, Peter Marinello, who shares with listeners how DSSRC has worked to support the direct selling industry. Listen now to understand self-regulation's impact on the market and what to expect during a direct selling self-regulatory case.