Lessons Learned: How Cognitive Dissonance Can Impact Email Performance

At Zembula, some of our most valuable insights come from the unexpected. Often, the smallest changes to our Dimensions™ solution can lead to massive performance swings — both positive and negative.

The biggest lessons usually emerge from those unexpected negative results. We recently experienced this with a campaign that revealed critical insight into messaging consistency in email personalization.

Here’s what happened.

The discovery: A minor change with a major impact

We were working with a brand that launched a new campaign using a dynamic Smart Banner™ at the top of their daily email. In this campaign, the brand requested that we exclude prices in the cart or browse abandonment banners from being displayed in the emails.

It seemed like a minor tweak. So, we thought little of it at the time.

We built out the Dimensions campaign and launched it as part of a longitudinal holdout test to track performance over four weeks. The goal was to see how much revenue the new program would drive.

But when the results came in, we were surprised — and not in a good way. The new program was driving negative revenue performance. 

We couldn’t believe it, so what had gone wrong?

Digging into the results: A hidden problem

We looked closer, lining up the daily revenue performance with the subject lines and email creative from the test, which included over 40 emails.

That’s when the answer became clear.

The negative results happened on days the brand ran a category-specific sale. However, on days with sitewide sales or no sales, it performed positively. What was the difference?

The visibility of the price.

The role of cognitive dissonance

Here’s what we realized:

When the subject line and hero banner announced a category sale citing a specific percentage off, but the subscriber saw their abandoned cart item wasn’t part of it, it created a moment of cognitive dissonance – meaning, the customer would click to buy their carted item, only to find out it wasn’t part of the sale and leave the website.

What may seem like a slight disconnect between the message and the actual shopping experience was enough to take the campaign from significantly positive to negative, especially because the brand had frequent category sales during the test period.

The solution: Transparency and pricing consistency

The fix was simple but effective: display the pricing in the cart and browse abandonment Smart Banners. 

By showing the price upfront, the subscriber could decide whether to purchase the item in the banner at full price or explore the category sale — without clicking through and getting discouraged by the mismatch. In other words, they are given accurate and transparent price information upfront to guide their buying decision.

From this experience, we learned two important lessons:

1. Small inconsistencies can impact performance: Even a minor inconsistency between what a customer expects and what they see can lead to significant performance swings.

2. Displaying accurate pricing is critical in email: If a subscriber clicks through expecting a sale and the item isn’t discounted, it’s likely to cost you the sale.

We had never encountered this issue previously because it takes a high volume of data and the right conditions to see these effects clearly in testing. For example, getting clicks doesn’t guarantee conversions, and if we had only tested based on clicks, we wouldn’t have learned this lesson.

This is why using a real-time solution for products is so important for enterprise retailers. Revenue performance is often driven by nuance. Enterprise retailers must ensure email content is as up-to-date as possible when opened because it can directly lead to changes in revenue performance.

It’s another reason long-term longitudinal testing is superior for finding potential issues with your messaging. A single-day A/B test may have never caught this revenue dip. (Need a refresher on why longitudinal testing is the gold standard? Read on about it here!).

Once we identified the problem, we applied this learning across future campaigns to make incremental improvements with every email sent.

Like what you learned? Subscribe to our newsletter for more insights and discoveries from us!

Prefer getting your newsletters through LinkedIn? Sign up for Zembula Insights for LinkedIn here.