Before rolling out a new nurturing email sequence to our free users, however, we first wanted to test their reactions to the new content. They’d told us during the market research that they’d be interested in learning more, but how would that translate to reality?
We decided to run an email A/B test to measure how our audience responded to this new approach.
First, we set a hypothesis: Our free users were not aware of the possibilities and features included in their free plan and even fewer of those included in paid plans. If we provided more visibility into what they could do with a paid HubSpot plan and made an effort to educate them about how to try those premium features, they would be pleasantly surprised and sign up for trials — and eventually purchase.
We believed this because our free users mentioned they weren’t fully aware of what was included in the paid plan and why they should subscribe.
So, we developed an experiment to test this new approach. For this our A/B test, I didn’t want to target all of our free users — only the ones who’d been actively using the HubSpot free tools in prior few months. I assumed this audience would be interested in learning more about what they could do and achieve with a paid version since they were already so active in the free version.
We divided these active free users into two segments:
- Our control group didn’t receive any nurturing emails, as this was the case before the experiment.
- Our variant group received a series of two emails promoting key paid features, including Attribution Reporting, Workflows, and Sequences, and offering them a trial to test out those features.
Here are some screenshots of the text emails:
Our Email A/B Test Results
We ran our email experiment over the course of four weeks and sent about 3,000 emails.
At the conclusion of the test, we were pleased with the results as they were highly conclusive and confirmed our original hypothesis. When we compared the control and variant groups after one month, we found that the emails sent to the variant group:
- Generated 71% more qualified leads
- Created 125% more deals
- Generated 12% more trials
- Generated 15% more revenue
Learnings and Takeaways
Both the research and experiment we conducted yielded valuable insights and takeaways. Here are my top four:
First, get to know your audience. As marketers, we don’t often have the opportunity to discuss with our audience.
If you can chat with your target market and customers, I highly recommend it! This can be a gamechanger for your marketing strategies and alert you to insights and trends you otherwise wouldn’t have known.
Second, align with Sales. Sales talks to hundreds of prospects and free users per week, so they know what features and components of your product or service are most important for certain leads.
You probably can’t promote your entire product in one email and have to select some features. For this reason, you should better align with Sales to better understand who your audience is and what their goals, needs, and pain points are.
Third, test your content formats and information to know what your audience prefers.
For example, we’ve seen in France that doing short videos or GIFs of HubSpot tools was highly appreciated. For each feature or functionality of the tool, our emails included a text description, and, next to it, a video or GIF of it. This helped our readers see what the tool looked like in action.
Finally, experiment as much as you can.
Even if your experiment is not conclusive or doesn’t yield the results you were looking for, it’s not a waste of time if you’ve learned something new and valuable about your audience, content, or product.
Originally published Sep 3, 2020 7:30:00 AM, updated September 03 2020