Summary
A message highlighting the tobacco companies that are funding the campaign to oppose California's Proposition 31 was highly effective at increasing support for the measure.
Research Outcomes and Key Findings
Grow Progress used its Rapid Message Testing tool to evaluate the effects of Facebook posts supporting California's Prop 31, which would ban the sale of flavored tobacco products in the state.
All four messages were highly effective at increasing support for the measure. The Rapid Message Testing Tool's "Change Results Display" feature helped identify that "Funders" was the clear winner among the four.
Overnight delivery: This test was launched at 4:15 pm on Wednesday, October 5, and returned results at 9:49 am the next morning.
Affordability: For this test, the sample size was doubled for more precision and the ability to confidently observe subgroup effects. Four distinct image messages were evaluated. The test cost 20 credits to run,approximately $4,000 for most customers.
Question
What works to change minds on California's flavored tobacco sales ban?
The Yes on 31: Protect California Kids campaign shared a series of Facebook posts with a wide range of arguments in support of the ballot proposition. The test evaluated messages that highlighted a personal story of loss, the LA Times endorsement of the measure, flavored tobacco's effect on youth, and the funders of the No side of the measure. The test was designed simply to observe the effects of the messages in changing vote choice on the measure.
What Worked?
All messages significantly increased support for the ballot measure, with effects ranging from 14 to 18 percentage points. "Funders" appeared to be the most effective, increasing the "Yes" vote by 18 percentage points.
So should the campaign choose "Funders"? The other messages were nearly as persuasive, at first glance. Further analysis shows that "Funders" is the strongest message.
The Rapid Message Testing tool allows users to instantly re-analyze the data to only show results for a particular subgroup, or to display movement toward or away from answer options other than "Yes." In this case, the analysis also examined which message was most effective at reducing the "No" vote.
The results show that "Funders" and "La Times" both reduced opposition to the measure by 10 percentage points. Therefore, the "net effect" of "Funders" was clearly the largest, increasing net support by 28 points.
Next Steps
When a message increases net support by 28 points in testing, the campaign should deploy it to as many people as possible. Further analysis showed that "Funders" worked among every subgroup and should be highly effective at changing minds.
If the campaign were to widely deploy this message, it could make sense to test again once the message has had the chance to "burn in",after most people have been exposed to the message, it's possible that another message with information that's new to the electorate would become more effective.
Appendix: Messages
Four distinct messages were tested:
- Funders - Highlighted tobacco company funding of opposition
- La Times - Featured the LA Times endorsement
- Personal - Shared a personal story of loss
- Youth - Focused on flavored tobacco's effect on youth
