Popups. Sliders. Takeovers. We all hate them. Right?
Everyone says popups suck yet sites that use popups consistently outperform sites without them.
If this is true, then why do so many sites choose not to deploy a well designed and targeted popup strategy?
Why? Because the folks with the data, the digital analytics and optimization teams, tend to be a bit too submissive to their product designer, product owner, business executive colleagues who often throw their weight around with strong opinions claiming that popups will destroy the user experience and my all-time favorite executive retort, “when I was at my last company, we deployed a popup and it destroyed our business so we aren’t doing that here!!!”
An A/B Test Compromise
Can i suggest that we compromise by running a targeted A/B optimization campaign to test our assumptions that popups work vs. your assumptions that they will destroy the business?
In it’s simplest form, an A/B test is simply a comparison between two different versions of our website — in our example the existing website without a popup versus a new version that features a targeted popup.
If you are new to the concept of A/B testing, there is a lot of great information out there, i would suggest starting with this in-depth introductory article from CXL: A/B Testing Mastery: From Beginner to Pro
Oh, and by the way, the experts at CXL have a popup on this page ☺️
A Real World Example
Just as the decision not to run a popup campaign shouldn’t be based only on past experience or gut instinct, you shouldn’t formulate a popup campaign without informing it using data.
One of the biggest mistakes we see companies making in their optimization programs is running all kinds of A/B tests based on seemingly random ideas, “I think we would convert more visitors if we changed our button color to cornflower blue.” The most successful optimization programs are backed by data, data is used to ideate potential high value A/B tests, data is used to target those tests, and data is used to provide rich learnings from the results of those tests.
At 33 Sticks, we use data to inform insanely high value optimization programs for massive global brands, to the world’s most respected publishing companies, to niche companies like Renew Financial that specialize in affordable financing for renewable energy and energy efficiency projects.
During a targeted analysis of Renew Financial’s most highly trafficked landing pages, the 33 Sticks team identified a repeatable pattern of high volume entry pages associated with a much higher than average Bounce Rates.
These high volume, high bounce, entry pages were all contained within the site’s “resource” section, a section of the site that contains informative articles from everything on how to properly heat and cool your home to tips on when to replace your roof.
As many of us with a blog know from our own data, blog articles are great at attracting new site visitors but most of them are “one and done,” they get what they were looking for and leave. The assumption here was we were observing this same “one and done” behavior.
The Goal
33 Sticks proposed an A/B test that would leverage a popup on entry pages within the “resources” section of the Renew Financial site that would look to retain a portion of the one and done traffic by pushing this segment of site visitors into Renew’s existing lead flow.
There is nothing wrong with the existing articles within the resources section, they are ranking highly in Google and they are providing valuable tips and insights for homeowners but with a little strategy, a larger portion of the visitors to these pages can be retained, nurtured, and converted into paying customers.
Conversion rates, which are computed by tracking visitors that use an address checker to validate that their home is in a supported area and then going on to complete a lead form, for these pages historically has hovered around 4% — not horrible but we think we can do better.
The Hypothesis
Traffic to the resources section are “one and done” visitors, by targeting a page takeover style popup to first time visitors landing into the resources section of the Renew Financial website we would increase address checks and lead conversions without risking increasing bounce rates.
The 33 Sticks team designed a popup that blended in nicely with the overall look and feel of the Renew Financial site, using colors, copy, and imagery that blended nicely with the existing site.
The popup would be served to 50% of the visitors landing into the resources section, the other 50% would see the original site, no popup, as the control group.
The Results
After 3 weeks of running the test, we observed:
- Google Optimize reporting a “Probability to be Best” of 100% for the Address Popup variation of the A/B test.
- The Address Popup resulted in an 8.27% Conversion Rate compared to the Control’s 4.24% Conversion Rate.
- The Address Popup Conversion rate of 8.27% is a 95% increase from the baseline 4.25% of the current, non-popup, site.
Why Was the Popup A/B Test a Success?
We started with a hypothesis, backed by data. This is one of the most important steps in building an optimization roadmap yet most businesses skip this step in favor of gut instincts of what and where to test. Let the data be your guide to A/B testing ideation!
Once the data informed a potential high value test, we crafted a user experience that while forceful was not overly aggressive and blended in well with the look and feel of the existing site.
And finally, we targeted the A/B test to a select, pre-qualified audience e.g. First time visitors, landing on a page contained within the resources section of the website.
This deliberate approach resulted in a 95% increase in leads generated for Renew Financial for visitors to the Resources section of their website.
Will all tests be this successful from a lift perspective? No. In fact, even the most well thought out tests sometimes don’t beat the original, and that is ok too, running an optimization campaign is as much about learnings as it is about running a test that results in measurable business impact as the popup test did for Renew Financial.
Are We Done?
No. We will learn from the results, continue to refine the experience, and test additional variations.
Your first test is just the beginning….always be testing!
A Note About Testing During the COVID Crisis:
If you are running an optimization program for your company or you are a marketer, user experience expert, or product owner, you should be using this time to test now more than ever.
Excuses I’ve heard about why companies are choosing not to run optimization campaigns right now:
- The traffic we are getting now is not representative of our normal traffic.
- Traffic isn’t converting right now.
- Visitors aren’t buying.
- A/B testing isn’t a priority, we are going into “keep the lights on” mode.
These are things that companies that operate from a state of fear say. Don’t let fear drive your optimization strategy, those companies that are choosing to double-down on data and optimization RIGHT NOW are going emerge years ahead of their competitors.
Keeping up right now is actually putting you further behind. Don’t adopt a losing strategy, take this opportunity to get strong and emerge a real force to be reckoned with and always be testing.
[author] Jason Thompson [author_image timthumb=’on’]http://i2.wp.com/33sticks.com/wp-content/uploads/2015/02/jason_250x250.jpg?zoom=2&w=1080[/author_image] [author_info]Jason is the co-founder and CEO of 33 Sticks. In addition to being an amateur chef and bass player, he can also eat large amounts of sushi. As an analytics and optimization expert, he brings over 15 years of data experience, going back to being part of the original team at Omniture. [/author_info] [/author]