Findings derived from two billion personalized experiences confirm value of AI over manual A/B testing
NEW YORK — Monetate, the worldwide leader in personalization, released new research revealing the proven efficacy of AI-powered methods over traditional approaches to testing, segmentation and 1-to-1 personalization. Leveraging data from more than two billion personalized experiences, Monetate uncovered that less than 10 percent of traditional A/B tests – which measure the impact of changes to user experiences – reach statistical significance. Not only does this indicate that manual A/B tests do not deliver the conclusive results marketers require to generate better user experiences and increase engagements, but it also suggests the opportunity for marketers to explore AI as a better alternative.
“Our data findings underscore why marketers should be considering how AI can improve their testing practices and ultimately increase engagement”
Marketers have long relied on manual A/B testing to improve customer experiences and deliver strong business results. While this method for website optimization has created a strong foundation for personalization, the emergence of AI for testing and segmentation is prompting marketers to reevaluate their engagement toolkit.
To help marketers understand whether AI-enabled tools are worth investing in, Monetate examined data from the Monetate Intelligent Personalization Engine™ to uncover how manual methods compare to AI solutions. The AI counterparts examined during the research were Majority Fit Experiences (MFEs) – which monitor the results of an A/B test and gradually direct new traffic to the more successful content – and Individual Fit Experiences (IFEs) – which use first and third party data to inform 1-to-1 individualized content decisions that provide each site visitor with a unique combination of content that is most likely to be relevant to them. The results confirmed that traditional testing often falls short, with the statistical significance of IFEs representing a more than 290 percent increase compared to traditional A/B tests.
Additional key findings include:
- 84 percent of MFEs outperformed their corresponding A/B test relative to a specific goal metric, indicating that AI personalization solutions offer increased potential to optimize for business KPIs
- The difference in KPIs was particularly noticeable in Revenue Per Session, in which the lift of the MFE over the A/B test exceeded nine percent, and New Customer Acquisition, which saw a lift of more than 15 percent
- In experiences that were designed to encourage user action—such as email signups or link clinks—the IFE outperformed a non-personalized control group 65 percent of the time, showing an average lift in click rates of more than 41 percent
“Our data findings underscore why marketers should be considering how AI can improve their testing practices and ultimately increase engagement,” said Brian O’Neill, CTO of Monetate. “While traditional A/B testing still deserves a place in web optimization programs, supplementing those practices with AI-powered personalization will enable marketers to more effectively meet their goals without draining resources, creating a better framework to focus on customer lifetime value.”
To learn more about Monetate’s findings, read this blog post.
About Monetate
Monetate, the global leader in personalization software for consumer-facing brands, enables marketers to create experiences with improved relevance for every customer, increasing their engagement and boosting business outcomes. Powered by artificial intelligence, the Monetate Intelligent Personalization Engine makes it easy to test and optimize, segment and target, recommend products, and even create true 1-to-1 omnichannel experiences. The company was recognized as an industry leader by two leading analyst firms in June 2018 and August 2018. Founded in 2008, with offices in the U.S. and Europe, Monetate is used by leading brands around the world and influences billions of dollars in revenue every year for QVC, Newegg, Timberland, Carnival, The North Face, and hundreds of other market leaders. Learn more at www.monetate.com.
