How to A/B Test Your Pinterest Ads: A Step-by-Step Guide

Pinterest is ideal for sharing visual content and discovering new ideas. Personally, I’m a huge fan of the platform and use it as a source of inspiration — I make Pinterest boards related to my interests (travel, style, etc.). 

Pinterest is a unique social platform in many ways but one similarity it has with many other platforms is that it has ads located throughout the feed.

Enter: business opportunity.

In a letter to shareholders last year, Pinterest stated that its revenue from shopping ads grew over 100% from the first to second half of 2019. Pinterest’s 335 million active users are always searching for new ideas and your ads can be a great way to pique their interest.

An effective way to determine which types of ads grab the attention of your target audience members on the platform is to use A/B testing.

Free Download: A/B Testing Guide and KitA/B Test Your Pinterest Ads

We have a complete guide to A/B tests here, but I’ll give you the long story short.

The goal of A/B testing — which is also referred to as split testing — is to run simultaneous experiments between two marketing variables to determine which results in a better conversion rate.

When it comes to Pinterest ads, the variables you decide to test may include your copy, call-to-action, and/or visual. A/B testing these types of variables allows you to understand what speaks most to your audience so you’re able to serve ads that speak the right message to your target audience at the right time.

Next, let’s review the steps involved in this process.

How to A/B Test Pinterest Ads

1. Set up a Pinterest business account.

To run ads on Pinterest, you’ll need a business account on the platform. Create one by signing up on Pinterest’s homepage or by adding a business account to your existing personal account under Settings.

Some basic information about your business will be required when you register including your company name, website, and industry.

pinterest home page
2. Create an ad campaign.

To A/B test ads, you’ll need to create an ad campaign to serve as the home base for the experiment. You can get started by going to the Ads drop-down in the top menu and then clicking Create ad.

start creating ads in pinterest

You’ll be taken to Pinterest’s ad creation dashboard where you can create campaigns based on a business goal of awareness, consideration, or conversion. Choose your objective and Pinterest will work hard to help you meet your goals.

create an ad campaign in pinterest

Scroll down a bit more to name your campaign and assign spending limits. While it’s easier to name a campaign something simple like, “Mug 1,” it’ll help you stay organized and focused on your goals if you’re more descriptive. For example, consider a name such as “US – Brand name – Mugs – Ocean Mug” so the ad location, product brand, product category, and product name are included.

As for the spending limit, this is Pinterest’s way of ensuring they stay within your budget. This field is optional to include, but can help to prevent overspending and will not affect your campaign pacing. (More on this in step four.)

name ad campaign in pinterest

3. Define your audience.

Now, it’s time to hone in on your target audience. As you go through these steps, you’ll see Pinterest automatically adjust the potential audience size you might reach on the right-hand side.

define ad target audience in pinterest

Sticking with mugs as a product example, see how Pinterest adjusts my audience size from 85m+ to 158k-175k once I added mugs in the Add interests section. The “Search all interests” search bar is a quick shortcut to help you find your niche. The decrease in potential audience size is not something to worry about.

In fact, you want to focus on a smaller audience to ensure you’re actually targeting people who are likely to be interested in what you have to offer.

define your audience in pinterest ads

As you work through this page, you’ll get several more opportunities to define your audience. In the below section, you can add more detail about audience demographics when you choose the “Pick specific…” criteria option.

defining audience demographics in pinterest ads

4. Craft your ad delivery strategy.

Alright, it’s time for some spending. In the Budget & schedule section, you’ll be asked to assign a budget to your campaign, meaning how much money you’re willing to spend for Pinterest to distribute your ads.

You can choose how much to spend per day (daily budget) or allocate a total budget and let Pinterest calculate how to best spend the money (lifetime budget). You also have the freedom to decide when to begin and end your campaign down to the exact time and date.

In Optimization & delivery, select how you would like to bid for your ads, since ads are served based on which advertiser wins the highest bid. The choice is up to you, but it’s worth noting Pinterest optimizes your budget in automatic mode.

budget, schedule, and deliver pinterest ads

5. Create and upload the ads you’ll A/B test.

Here’s your opportunity to get a little creative. Design two versions of your ad with a difference in the variable you’re testing. When creating the ads, only focus on one variable you want to test (e.g. product picture, ad copy, ad layout) for best results. Services such as Canva or PicMonkey are simple online design tools to help you get started.

In the mug example below, the product picture is the single variable and it’s the only difference between the two versions. This allows for clear and actionable results.

pinterest a/b testing ad examples

Next, upload your ads into the Ads section. The ads will be formatted like any other pins on the platform, so fill in all the necessary information. You definitely don’t want to miss the destination link, which takes users to your site!

pinterest a/b testing ad example (a)

pinterest a/b testing ad example (b)

6. Launch your ad campaign.

Now we’re ready for takeoff! Select your A/B ads and click that red Launch button — which I know you’ve been eyeing throughout this whole process — to send these ads out into Pinterestverse.

Kick back, relax, and check this task off your to-do list!

launch pinterest a/b test campaign
7. Observe and gather ad results.

When the ad campaign is underway, you might be eager to constantly check its progress in the ads overview dashboard.

pinterest ad performance dashboard

I’m there with you, but be wary of jumping to conclusions. You’ll want to give your test enough time to produce statistically significant results before analyzing, which can take anywhere from hours to weeks. In simpler terms, you’ll want to get your ads in front of as many people as you can before parsing the data.

We created a free significant test calculator to help you out, along with all other tools you’ll need to run effective A/B tests.

You did it! Kudos on your Pinterest A/B test, but the work doesn’t stop there. Ads are a continuous game of experimentation. Build on your test results and consider other variables you can test in the future. Your consistent effort will improve your overall ad performance and increase conversions. Good luck!

The Ultimate A/B Testing Kit

Originally published Aug 27, 2020 7:00:00 AM, updated August 27 2020

Topics:

A/B Testing

How to A/B Test CTAs Like HubSpot Experts

The call to action is a put up or shut up moment that can turn curiosity into a legitimate business opportunity. It’s the bridge between visitors and leads — a point of entry that allows you to capitalize on the web traffic you generate.

Free Download: A/B Testing Guide and Kit

Despite its potentially massive business implications, your average CTA is generally pretty small and unassuming — maybe a button or a line of text. It might seem like the blog post or product page the CTA is placed on is all that matters when it comes to how many leads it ultimately generates

It’s easy to think that the copy, color, or placement of a call to action doesn’t really matter, but that’s not the case. A lot goes into creating an effective CTA. The best ones have been carefully crafted and consistently tested. Here, we’ll focus on that process.

We’ll learn how to A/B test CTAs successfully and get some insight on the subject from some expert HubSpotters.

1. Decide on a specific factor you want to test.

A/B testing isn’t supposed to be some indiscriminately applied process. In this context, it’s designed to help you best optimize specific aspects of your CTAs. That means conducting a thoughtful, effective A/B test rests on you definitively selecting the individual variables you want to better understand.

Are you trying to zero in on the best word-choice for your CTA? Are you looking for its optimum visual characteristics? Do you want to identify where its most effective placement on your blog posts might be?

Try to lock in on one factor — above all others. You don’t want to A/B test multiple CTAs with different shapes, colors, and copy all at once.

2. Create multiple, distinct CTAs.

There’s a reason A/B testing is called “A/B” testing — not just “A” testing. The concept itself rests on comparing multiple options, so naturally, if you want to A/B test CTAs, you need more than one.

There are a variety of variables you can use to create unique, distinguishable CTAs, including color, position, size, shape, and wording.

A/B tests should take fundamentally similar CTAs that convey the same information in unique ways, leveraging separate — but not outrageously different — options to help hone in on the slight distinctions between choices your audience would be most receptive to.

3. Measure your Results

Identify a date range you’d like to use as your reference point for your experiment. Once you’ve carried out your testing across that period, take the time to analyze and understand your results.

Get a feel for how your various choices performed in terms of metrics like overall views, clicks, and submissions to see which of your CTAs are garnering the most attention and converting the best. 

Once you have your results, you can start to identify the more effective aspects of each option and start to piece together CTAs that will consistently deliver the results you need.

How to A/B Test CTAs like HubSpot Marketers

Avoid testing multiple variables at the same time.

It’s recommended that you don’t employ radically different variations of the same CTA when A/B testing. The hope in carrying out these tests is to identify the specific factors that make your CTAs most effective.

If you’re comparing CTAs with various aspects when A/B testing, your results might be muddled. You won’t have a clear idea of which aspects are most effective — you’ll lose out on the ability to identify the factors worth applying to your CTAs going forward.

As Carly Stec, Team Manager for Blog and Academy Acquisition at HubSpot, suggests you “avoid testing multiple variables at the same time.” She stresses the value of keeping things straightforward.

She discussed a recent series of A/B tests her team ran for HubSpot’s thank-you pages that involved changing the messaging, placement, and visuals of a CTA. She noticed strong increases in conversion but didn’t have a firm picture of the key factor at play, “While we saw a strong increase in conversion, we were left wondering, what exactly did the trick? Was it one of the elements or all three?”

Her team opted to partition the tests to identify what was making the difference. As she put it, “We reran the test as three separate single variable tests and found that placement actually had the most influence on conversion. The lesson? Keep it simple so success is easier to trace.”

Treat it as a marathon — not a sprint.

Slow and steady — that’s the name of the game when it comes to A/B testing CTAs. It’s a piece-by-piece process. Your most successful CTAs will be the sum of several minor adjustments, often carried out over somewhat substantial periods of time.

As AJ Beltis, HubSpot Marketing Manager for Content and Acquisition, put it, “Don’t be afraid to start testing incremental changes.” Minor tweaks to language, visual characteristics, placement, and other factors wind up making for the most effective CTAs. If every A/B test you run for your CTAs is based on a major overhaul, you might pass over the aspects that were working best for you.

Beltis went on to discuss an experience with some tests he and his team ran for the HubSpot blog, “For example, we did an entire series on testing our anchor text CTAs. While we saw marginal increases from every test, it was the combination of these results that resulted in the optimal version of this CTA, which has resulted in a significant increase in annual leads from the blog.”

What HubSpot Learned From A/B Testing CTAs

If there’s one lesson that HubSpot has taken away from A/B testing CTAs, it’s that there’s always room for improvement. A/B testing CTAs is an ongoing process. As Stec put it, “At HubSpot, we constantly iterate on our tests to make sure we’re not settling for the winner. Just because something won once, doesn’t mean it can’t be beat. Keep going.”

A/B testing CTAs is about zeroing in on the perfect option — the caveat here is that perfection isn’t real. Always aspire to consistently improve your CTAs, and A/B testing is central to that process.

It’s important to maintain focus and effort when it comes to this process, and Stec captured the essence of that mindset when she said, “Don’t underestimate the importance of persistence in A/B testing.”

The Ultimate A/B Testing Kit

Originally published Jul 14, 2020 8:00:00 AM, updated July 14 2020

Topics:

A/B Testing

The Ultimate Guide to Social Testing

As marketers, we know the importance of making data-driven decisions.

The more data we know about our audience — from how many are being reached to how many engage with our content — the more we’re able to make effective marketing moves.

Having the numbers to back up marketing strategy is almost as important as the strategy itself.

If numbers aren’t your favorite thing to calculate, that’s not a problem (thankfully). There are so many strategies and automation tools to back up those tough marketing choices effectively — no math needed.

Free Download: A/B Testing Guide and Kit

For instance, let’s say you’re running a social media campaign, and one of your Facebook posts includes a video. You’ve never run a video ad, so you need data to prove it’s the right move for engaging audiences.

To get that data, you decide to run a social test for engagement. In your test, one Facebook ad will show a short cut of the video and the other, a longer cut. If one of the videos reaches 50% of your benchmark engagement goal, you’ll know the length is a good choice to capture the attention of users.

Social media tests, like the one explained above, let you run an experimental campaign before investing in the real thing. They show different versions of the same ad to the desired audience and allow you to pick the winning ad based on the campaign’s objectives and what you know your audience wants to see. At the end of the social media test, you’ll receive a report that provides the data breakdown of performance.

This strategy could be the answer to questions you have about your audience’s behavior on social media. Sure, you can read industry standards and benchmarks as a jumping off point, but these tests are timely, accurate, and experiment for your specific audience.

Social media testing provides data-driven insight about your social media marketing activities. It allows you to analyze how different variables, like photo and video, affect performance. You might run a social media test, for example, to see if adding emojis to a post’s caption increases engagement.

Ultimately, social media tests provide data about how audience behavior can influence the structure of your campaigns.

For example, you might run a social test to learn if video campaigns are worth investing in on Facebook. So, you create one that measures the impressions of an ad with and without a video attached. After the campaign, look at impressions to see if campaigns that have video are worth it for your brand.

Social media tests paint a picture of what messaging is successful for your brand. Instead of researching countless industry benchmarks about what works and what doesn’t, you’ll have the concrete data from testing.

There are three types of social media testing, and we’ll discuss them next.

Types of Social Tests

Let’s say you want to know how copy affects an international audience on LinkedIn. Or, that you want evidence of a landing page performing better with a different image. Maybe you’re trying to identify if changing the tone of Instagram captions will lead to more audience engagement.

All of these scenarios are prime for social tests. They point out a problem that can be answered with data. This data would give insight about audiences interactions with brands on social media.

After identifying the goal, it’s time to pick the type of test. Let’s go through the different types and when you might use them.

A/B Test

A/B tests are likely the most common form of social testing. They look at a variable between two content types, measure the outlined goal, and provide results. Run an A/B test if you want to test a single, small variable that may alter audience behavior.

For example, if you want to test out different CTA buttons on a Facebook ad or experiment a post’s copy with/without emojis, run an A/B test.

Split Test

Split tests are often confused with A/B tests. In essence, they do the same thing: test two content types based on a goal. The difference is that a split test is more general than an A/B test; they’re used to determine big changes, and the two variants are often completely different.

For example, use split testing if you want to know which layout of your Facebook Business page performs better. You can also run split tests to determine A/B test factors. For example, you can run a split test to determine which ad video cut you’re going to use for a Sponsored Tweet, then A/B test different sections of the winning cut.

Multivariable Test

Multivariable tests work differently than the two previously mentioned types. They work with multiple variables instead of one or two. You can run a multivariable test to determine which of four ads works best. Alternatively, one can be run to assess the different makings of a post, like images vs. copy vs. captions.

Run a multivariate test if you want to see the results of more than two different elements. For example, compare the caption, image, and CTA of a LinkedIn post to see what audiences are responding to, or look at three versions of a GIF to learn how audiences react to them.

When you’ve picked out your social test experiment, make sure you’re imploring best practices so the results are useful.

Social Testing Best Practices

Social media testing can be extremely helpful — if you’re executing correctly. Otherwise, your test could be inaccurate, immeasurable, and ultimately, a waste of time.

The good news is that social tests aren’t hard to create or run. The bad news is that if you aren’t prepared to run one, your results won’t be usable. Make sure that when you make your test, you follow these best practices:

1. Have one specific goal.

When your social test has one identifiable goal, everything else falls into place: variables, unit of measurement, and time frame. To illustrate this, let’s say your goal is to improve international engagement with your next Facebook ad.

With that goal decided, you can create the variables, so you decide to run an A/B test to determine which copy earns the most impressions. You estimate that because your impression ads usually run for a month, half that time would give measurable results.

You also know the direction of the ad. When your goal is focused, you’ll know what to look for during analysis. Impressions, for example, would be the metric to look at for the Facebook ad mentioned earlier.

2. Know who your audiences are.

If you decide to conduct a social test, results will reflect the behaviors of a specific target market. They can serve as a reference point of data about how certain customers engage with your messaging.

Social testing is a great way to learn about the social media behavior of an audience segment. If you had little to no information about how your millennial audience would react to a new Instagram Story Ad, running a test would give you a data-driven answer.

Without knowing your audience, your data wouldn’t be applicable to a defined set of your target market. You’d have insight, but without a segmented audience, it’s not clear how the results relate to your goal.

3. Take note of your current performance.

Before you run the test, know how your current campaign is running or make note of previous results. This is so, at the end of your experiment, you can compare results and make informed decisions. The previous report will give the background information and context needed to analyze the social test findings.

Even if your marketing goals for your social test are different than previous campaigns, it’s still a good idea to refer to them for context. For instance, you may be testing for conversions rather than retargeting, but having an idea of what audiences prefer helps you structure your campaign.

4. Check on your test periodically.

Don’t leave your test as soon as it begins — check on it so you can adjust things accordingly. A test that’s running for a month, for example, should be checked regularly for performance benchmarks.

If you use social test software or tools on social media channels, the report starts aggregating when your test starts. When you check in, look at how you’re tracking for your #1 goal. Additionally, take note of what other metrics you see, and how they’re performing.

You might find, for instance, that your conversion rate is low. You can monitor conversions for the duration of the test, or make a small tweak to try to improve performance. For the next check-ins, you’ll have an extra metric to take into account.

When the test concludes, you’ll have the knowledge from previous checks to round out your perception of the completed report. Along with your intended goal, identify supporting metrics to understand how they work together.

5. Make your test timely.

How long should you run your test? Long enough to get the answer to your hypothesis. That doesn’t tell you much, though, so let’s add to that.

Ideally, your test should run for at least seven days. A week is enough time for your social testing software to compile a basis of data. It won’t be as concrete as a longer test, but it’s a starting point.

After seven days, look at your performance and decide if you’ve gathered enough data to answer your hypothesis. If not, run the test for a few more days. Based on the nature of your campaign, fix the time frame to fit your business and your audience — but make sure you give yourself enough room for an actionable report.

Picking a test duration period ensures you won’t be wasting money and time. Instead of having a test run for too long or too little, figure out your time frame and budget during the planning process.

So, with these best practices in mind, you’re almost ready to run a successful social test. Before you get going though, let’s talk about where — and how — to do that.

Where can I run a social test?

There’s a couple of avenues to explore when choosing where to run a social test. If you’re running a test specifically for social media, the channel you’re using might have testing tools in their business software.

Running social tests on the corresponding media platforms is helpful because you don’t have to keep track of different channels during the test. Additionally, you won’t have to worry about misconstrued data a third party might provide.

You can also run a social test using a CRM, like HubSpot. CRMs are a great choice if you want to test functions outside of social media — landing pages, emails, or other marketing activities. First, though, let’s talk about social media offerings.

Facebook Social Testing

Facebook offers tools to run A/B and multivariate tests for ads. You can access these tests via Ad Manager. Ad Manager tells you which ads (or ad tests) are running and their status. When you click on a certain test, details and metrics open.

Facebook A/B testing

Multivariate tests on Facebook are similar, except they can be found in the Experiments section of your Facebook Business account. Though the social tests are located in different places, the process for setting them up is largely the same.

Social testing on Facebook is intuitive, especially if you’re used to running Facebook Ads. After clicking “Create Ad,” you’ll be taken through a series of prompts to create, set metrics, and pick an audience. When you fill in the details, you publish it, and wait for the results.

Start small if you’re getting the hang of things. Its features are customizable, so the choice is yours in terms of time frame, audience, copy, testing options, and metrics. As your familiarity grows, tests can be scaled.

As an example, after running a social test on Facebook, bone broth brand Kettle & Fire found a 14 point increase in brand awareness. The marketing team wanted to raise online sales with a video campaign, and wanted to find out which video length was favored by customers.

Kettle & Fire Facebook A/B Test example

Image Source

In a little less than a month, a 1.5 lift in purchasing consideration and sales conversions from the short video led to the answer. A Facebook test saved the team ad spend and informed them about their audience’s Facebook preferences.

Facebook delivers social testing results in a downloadable report. The software determines the winner based on the metric(s) you chose when creating the test. For information about running a Facebook social testing, check out this post about testing on the platform.

Twitter Social Testing

Social testing on Twitter leads to creating tweets audiences will interact with and enjoy. You’ll identify how they use the platform and know how to cater to their needs. Twitter’s testing tools are best fit for creative, targeting, and brand awareness campaigns.

A Twitter A/B test example

Image Source

Twitter lets you social test Ads in an A/B format. To do this, after publishing the ad, go back into Ads, duplicate it, make the change, and publish the new variable. Track performance from the Tweet Activity Dashboard and campaign dashboard.

Note that there’s no way to set an official test using Twitter, so you’ll have to analyze the data yourself. For instance, if you want to see if your new brand voice is taking flight, you’ll probably want to look at the amount of clicks and impressions your Ad earned.

LinkedIn Social Testing

If you’re not used to LinkedIn Ads, there’s several different types. They’re sorted based on campaign goal, like Conversion, or type of ad, like Carousel, Text, or Sponsored. It’s a great idea to test LinkedIn Ads for promoting — your brand, event, or job listing.

Dynamic Ads, which show up on the ride side of a user’s main feed are highly personalized. Under Dynamic Ads, there’s several versions: Follower, Job, Content, and Spotlight. These versions specify what the goal of your ad is — so if you want to let audiences know your company is hiring, you would run a Dynamic Job Ad.

If you wanted to make a similar Dynamic Job Ad on LinkedIn, you can run a test to see if your listing ad is compelling to job seekers. You’ll have the option to test image layouts, such as the company photo, copy, and central images, if applicable.

LinkedIn’s marketing team uses social testing for content and event promotion, as well as account-based marketing. They ran a test for a webinar Dynamic Ad to gauge if audiences preferred an image of the speaker or the company’s logo. According to one of LinkedIn’s senior marketing managers, Cassandra Clark, results included a 326% lift in click-through rate in the ad with the speaker.

To access ad results, check your Conversion Tracking dashboard, which will show you page and audience activity from your ads. While LinkedIn doesn’t have a formal testing feature, they do have an option for you to duplicate and tweak a portion of an ad, like Twitter.

CRM Social Testing

If you’re not running a test on a social media channel, you can use software to help. If you use a CRM, you can set up tests for website content, like landing pages.

In HubSpot, you can run social tests for web pages simply by accessing your dashboard and going to one of your web pages. Click the “Actions” hyperlink next to one of your pages, and you’ll find the option to “Run a test.”

HubSpot lets you run an A/B or multivariate (Adaptive) test. As you’re creating your test, you’ll have the option to look at testing tips, like figuring out what to test. Since HubSpot has a drag-and-drop editor, and tons of modules to choose from, you’ll have plenty of options. In the past, I’ve run tests for CTA buttons, images, headline copy, body text, and landing page layout through social media.

It’s commonplace for CRMs to have tools for running social testing. Some offer a niche version of social testing; for instance, Mailchimp’s are for emails. When you decide to use a CRM for testing, do some research to determine which one will offer you the tools you need to succeed.

But, if you don’t know where to begin looking for software that offers social testing, here’s a list of tools for your benefit.

1. HubSpot

Price: Free plan, or $800 for Marketing Hub Professional

With HubSpot’s CRM, you can run A/B and multivariate tests. The tools are part of HubSpot’s Marketing Software and are included in the Marketing Hub Professional and Enterprise plans. You can use HubSpot to test landing pages that are promoted through social media.

HubSpot testing screens

The CRM’s drag-and-drop editor makes it easy to configure variables for tests. You’ll have optimization features to make your page shareable and user-friendly, like social media badges. That way, when you promote the page on socials, your followers can share them with one click.

HubSpot’s testing tool is accessible from the landing page dashboard via “Run a test” from the drop-down menu. From there, you can choose your test type and get started. If you want an easy-to-use platform that lets you create and analyze social tests, HubSpot is a great choice.

2. OptinMonster

Price: Free plan, or $19-49/mo.

OptinMonster is a CRM that also offers built-in testing software. You can run A/B or split tests to increase conversions over time with tools that let you experiment with content, headlines, campaign triggers, styles, and layouts.

Once you set up the test, you can leave the software to do the work. It’ll show your test content to website visitors and collect data on conversion rates. Your report will determine the number of impressions, acquisition, and page visits.

Optinmonster A/B test

Image Source

You can use OptinMonster’s testing tool to test social media landing pages. Test the effectiveness of the landing page for your next ebook offer before it’s published on LinkedIn, first. If you want a platform that offers intuitive tools for social testing, try OptinMonster.

3. Optimizely

Price: Contact sales for pricing

If you’re heavily promoting your website on social media, test it to learn how it performs among your audience. One way you can do that is with Optimizely, which provides marketing software solutions.

Comscore social testing variations

Image Source

Optimzely’s tools let you perform split and multivariate tests. Optimizely will count the number of website visitors and the metrics you pick, such as engagement, goals, conversions, and clicks. You’ll be able to apply custom audience segments that can be filtered based on social proof performance.

4. Leadpages

Price: Free, or $48-$199/mo.

If you want a tool specifically for testing landing pages you promote on social media, try Leadpages. This is software that was made just for building professional landing pages. You can use the split testing feature with the tool.

When you run split tests with Leadpages, you’ll be able to access its analytics the minute it aggregates traffic and engagements. At the conclusion, you’ll get a report that includes the total and unique visits, conversions and total conversion rate. Your report will also include specific insights about test variations.

Leadpages A/B test

Image Source

Keep in mind that you can’t set a time frame for your tests. From the analytics dashboard, you’ll be able to see how many days your test has been running. When you’re ready for the test to be complete, you’ll have to end it manually.

5. Inspectlet

Price: Free, or $39-$499/mo.

Inspectlet was created just for testing. It’s a software that lets you test multiple different content types, including web pages. You can also create content variations for future social tests, like edited graphics.

The visual editor in Inspectlet makes it easy to code or create different pages. You can change your page’s design or code in real-time and access analytics as soon as the test begins. Inspectlet lets you track views, page elements, unique visitors, and custom events.

When you access your performance report, you’ll get the metrics based on the goals you set. This can be anything from tracking clicks, user engagement, custom events, or URL views. Basically, anything that you can track with the software can be measured.

Now, you have a couple of options separate from social media sites that you can social test with. With this in mind, you’re ready to run the test.

Think back to the example in the beginning of the post. Knowing all you do about social testing now, you’re fully equipped to run a video engagement experiment of your own. Will it be on a social channel or with testing software?

One of the many great things about social testing is how creative they can be — you can social test just about any piece of content, and social media sites are starting to make that process easy. Testing your social ads ensures you’re providing the most value to your customers, and now, you can.

The Ultimate A/B Testing Kit

Originally published Jul 1, 2020 4:00:00 AM, updated July 01 2020

Topics:

A/B Testing

Forms Testing: What It Is & How to Do It

The other day I decided that I wanted to buy some comfortable outfits to wear around the house.

As I was online shopping, I started to fill out my shipping information in the online form.

However, the form had a requirement that the “City” field could only be 20 characters long. Now, the city I live in is three words, and 22 characters typed out.

Having faced this problem before, I tried to shorten the city name, but my phone was autocorrecting.

Ultimately, I couldn’t get past this form on my phone. I had to go on my desktop to make the order so autocorrect wouldn’t impact the length of the city name.

Predictably, this was a frustrating user experience.

As marketers, we use online forms in our campaigns all the time. To see success, it’s important to test your online forms for user experience and conversion rate.

Below, let’s review what form testing is and how to implement it in your online form strategy.

Free Download: A/B Testing Guide and Kit

Now that we’ve covered what forms testing is, let’s dive into the components of your online form that you should be testing.

Online Form Elements to Test

Once you’ve created an online form on your landing page, it’s important that you test it for both functionality and effectiveness.

Below, we’ve listed a few elements that you should test depending on your goal.

Functionality

  • Validation: Validation is how you’ll ensure the information put on a form is accurate. For example, you can set requirements on your form for the email and phone number fields so users can only input correctly formatted emails and phone numbers. By setting up those requirements, users won’t be allowed to submit the form if there’s an error. When an error occurs, a validation message will appear and let users know what’s wrong. When you set up an online form, you need to check that the requirements and validation messages you set up are working correctly.
  • Notifications: When a user submits a form, you should be notified. After setting up an online form, double check that you’re receiving notifications. Additionally, typically an online form will trigger an email or confirmation message to the user. You’ll want to test and confirm that’s working as well.
  • Thank You Page: After a user submits an online form, usually they’re directed to a thank you page. This is one of the most important elements to test. Make sure that the entire process of landing page to form submission to thank you page is working seamlessly.
  • Payments: If your form requires a payment, you’ll want to test and make sure your payment processor is working.
  • Mobile Devices: When I went through the form submission process above, the mobile experience wasn’t working well. The issue I experienced couldn’t have been found if the company only looked at the desktop user experience. That’s why you should test your form on different devices. Make sure there aren’t any friction points in the process on your mobile device or tablets.

After you’ve tested the functionality of your form, you should begin A/B testing style choices to see what works for your audience. Below are some of the top elements you should test for effectiveness.

Effectiveness

  • Design: The overall design of your form is important to consider. It should look modern and be easy to understand. You can test elements such as form length or form requirements to see what your audience responds to.
  • Copy: Form copy should set the expectations. For example, is there a clear purpose and instructions? Does your CTA copy clearly express what users will receive?
  • CTA: CTAs are one of the most important elements of your online form. Test and see what colors, sizes, and copy converts best.
  • Requirements: When you build a form, you might want to make every element required. However, this might not work with your audience. Test what requirements or pre-filled fields work for your audience.

Now that you know how to test your forms conceptually, let’s review some best practices to consider when creating your online form.

Web Form Testing Best Practices

1. Keep form length short, if you can.

Generally, keeping your forms short is a good rule of thumb. However, it’s important to consider that there isn’t a “one size fits all” approach to form building. You’ll want to test your forms to see what works best for your audience.

Kristen Baker, a marketing manager at HubSpot, says “Web forms vary in length, format, content type and appearance. They should simply fit your business’s needs and help you gather the information you want from your leads.”

2. Don’t overuse validations and requirements.

Not to be redundant, but the user experience I had when I was online shopping the other day wasn’t great.

That’s because the company was overusing requirements by limiting the characters I could use in the “City” field.

Overall, the best practice here is to not overuse requirements or validations. You should only make decisions that improve the user experience.

3. Set expectations with your CTA.

Your CTA is one of the most important elements on your landing page because some people might skip right over the form and look at the CTA first.

That’s why your copy should set expectations, while also catching the eye. Consider the color, shape, size, and copy on your button.

Ultimately, your CTA should be actionable and visually appealing.

Once you’ve built your online form, you’ll want to track your efforts. That’s why you should look at these form analytics tools to measure success.

By using form testing, you can adjust your forms on the go and make adjustments to help improve your conversions.

The Ultimate A/B Testing Kit

Your Email Testing Playbook for 2020 (& the Tools You’ll Need)

A/B testing is one of those techniques that, if you have enough volume to give you significant results, is pretty much guaranteed to generate better results from your marketing.

Email marketers have known this for ages, but what drives me nuts is that they waste their time on tiny little tests — instead of tackling some of the bigger, more exciting tests that yield real insights and improvements.

Click here to download our free lookbook that's packed with our favorite email newsletters.

In fact, MarketingSherpa’s email survey found that subject lines are still the most commonly tested element in email marketing. Meaning that those few words that get your subscribers to open your emails and see your wonderful offers are what marketers focus on most in their attempts to optimize their email marketing.

While I’m sure this strategy can end up getting you the most tested, optimized subject line that will ever reach an inbox, the impact of these tests are minimal compared to all the other things an email marketer could be testing.

marketing sherpa email survey chart2 resized 600

So … are you ready to run some big, exciting tests? In this blog post, we’ll highlight what you should be experimenting with and which tools can help you. But, first, we’ll explain the importance of A/B testing. 

A/B testing is a great way to test two different newsletter formats that promote the same content or two newsletters with slightly different design elements, such as different images or types of CTAs.

Rather than testing one template repeatedly for a few weeks, followed by another email format test, this testing phase quickly allows you to test two styles and pick a winning template on a limited schedule.

As you consider A/B testing or other email experimentation, here are a few vital things you’ll want to test when building out your email marketing strategy.

Email Testing

1. Test different types of offers in your messages.

Possibly the biggest lever you have in your email marketing is not the few words you use to describe your offer, but rather, the offer itself.

Whether you’re testing two ebooks against each other, or an ebook versus a webinar, this test is bound to get you better results overall. The reason this is particularly important is, while you may think your offer is the best thing since the iPod, you may also be wrong.

We started doing this sort of testing religiously back in the summer of 2010 and saw dramatic results. Instead of taking our email list and sending them all our latest ebook, we would take a smaller portion of the list, split it in half, send them each two different offers, and then send the better performing offer to the (larger) remainder of the list. This testing alone increased our monthly email leads 4-8x instantly.

Email leads increase

Here are some more specific offer elements you can consider testing:

  • Topic: Do certain offer topics resonate better with your audience? For example, we might test one of our ebooks on Facebook against one of our ebooks on Twitter.
  • Format: Which offer format does your list prefer? Do they love webinars? How does that compare to their interest in ebooks, kits, free trials, etc.?
  • Length/Size: Does your audience prefer smaller, bite-sized offers like tip sheets, or are they hungry for more, like an 80-page ebook? Try testing longer forms of content vs. shorter offers, or one offer vs. a set of offers.
  • Name of Offer: Sometimes the way you position your offer can make a difference with your audience. Think ebook vs. guide vs. whitepaper, or factbook vs. slideshow vs. download.

2. Analyze the landing page you’ll be linking to.

The goal of your email is not just to get someone to open or click through; it’s also to take some action. For example, to download your offer. So don’t think of your email in a vacuum. Think of it in the context of driving that particular action, which means optimizing where the action takes place: the landing page.

After all, if you create this great email that drives lots of clicks to your website but then you lose those potential leads at the last stage, it’s like you’ve run the first leg of a marathon but then decided to drop out of the race during the very last mile.

Here are some important landing page elements to test:

  • Description of Offer: The way you position your offer may have an impact. Calling out that a consultation is free, or referencing testimonials of people who have downloaded that offer, for example, can be interesting variables to test.
  • Length of Description: Do you go on and on about your offer, providing testimonials and screenshots, or do you keep things short and sweet in bullet point form?
  • Image/Preview of Offer: Using a supporting image is great, but what do you show? An image of the ebook cover, a sample page of the ebook so people can see what’s inside, or a preview of the first few pages?
  • Form Placement: Do you put the form on the left? The right? Below a block of text? Best practices say make it visible on immediate page load (above the fold), but feel free to play around with the placement.
  • Number of Form Fields: What data do you really need from your prospects? Fewer form fields usually leads to a higher conversion rate, but you should always test asking the bare minimum versus asking for every personal detail — and somewhere in between. We’ve also published some great advice about this debate here.
  • Which Form Questions to Ask: In addition to the number of form fields, which questions you ask on your form can have a big impact. Asking for Social Security Numbers or visitors’ first born child’s name is very different from asking for size of company or industry.
  • Form “Submit” Button Text: Do you use a straightforward, action-oriented phrase like “Download Ebook Now,” a fun option like “Let’s Go!” or a standard “Download” button? Test out the text of the button you know each lead is clicking on.

3. Leverage audience segmentation tests.

The success of your email is not just dependent on what you’re emailing or how you’re emailing it, but also *who* you’re emailing.

For HubSpot, an offer called, Agency Kit: How to Create Effective Ebooks for your Clients may get a great response from marketing agency owners, but it’d probably get a terrible response from the nonprofit marketers interested in our content.

The simple act of segmenting your email list to narrow your audience down to one that would find your content more relevant can have an amazing impact on your results.

Here are some audience segmentation tests you can run:

  • Interest: Has someone downloaded an ebook on this topic before? Do you know they have a particular challenge based on their website browsing history? Target the offers around those interests for a boost in response rate.
  • Persona: Identify your main business personas, and target your content to each one. At HubSpot, this means we send different content to small business owners than what we send to nonprofit marketers, for example.
  • Recency or Level of Engagement: Did this subscriber come to your site recently, or has it been a few months? Did they download a dozen ebooks, or just one?
  • Other Demographics: Try segmenting on other demographics collected by marketing or sales – things like industry or role or company size.
  • Lifecycle Stage: Where is this person in the sales and marketing funnel? Did they just start engaging with you, or are they in the last stages of the sales process? This article provides suggestions on what to send at each stage of the funnel.

Check out this blog post for even more examples of how you can slice and dice your email list for better segmentation.

4. Test different newsletter formats.

Changing up the format of your email can also have a surprising effect on your response rate. This could mean everything from the length of the email, to including a lot of images, to creating a simple, plain text email. Keep in mind that your results may differ depending on the type of offer.

For example, our new ebooks perform best when sent in a nicely formatted html email, while our free consultation offers perform better when sent as a simple, plain text email.

Here are some formatting elements you can test in your email marketing:

  • Plain Text vs. HTML: Simply try changing your pretty HTML email into a plain, personal-looking email to see how that changes your response rates. You might be surprised at the results!
  • Content in Text Only vs. Text and Images: At HubSpot, for example, we tend not to rely too much on images because many subscribers don’t enable or download images in their emails. That being said, some companies have had great success with using visuals to tell stories that you simply can’t convey through words alone.
  • Number of Calls-to-Action: Do you go with a newsletter style with a lot of calls-to-action, or zone in on one single offer?
  • Length of Email: Do you go short and sweet, include meaty content, or go on and on about the value of the offer?

If you have a number of different email templates or design tweaks you want to test in a limited amount of time, you could consider A/B testing. 

5. Send newsletters at different times and frequencies.

Timing is one of the most popular things marketers try to optimize. But it seems like there’s more talk about the best time to send in general, and not enough testing going on to determine the best time to send email to your own subscribers — or even a specific segment of your subscribers.

Even within HubSpot, we have segments of subscribers who respond more to emails on Mondays, Saturdays, mornings, afternoons — on top of that, all in their own timezones.

Instead of sending email at every marketer’s favorite time (Tuesdays at 10 a.m.), break away from the pack and see what works specifically for your audience in order to optimize for your particular business — and to have a better chance of breaking through the clutter of other businesses’ emails.

Consider conducting the following timing/frequency tests in your email marketing:

  • Day of the Week: If you always email on Tuesdays, try mixing it up and sending on a Monday or Saturday.
  • Time of Day: Do you always send emails in the mornings on the East Coast? Try an afternoon send — or even go for after work hours.
  • Triggered by Specific Behavior: It’s not just about when you want to send an email, it’s about when your subscriber has taken some interesting action. Try targeting your follow-up around when they take an action using marketing automation.
  • Timing Around Trigger Event: How soon after the triggering event should you send that email? Immediately? An hour later? A day later? Longer?
  • Frequency: How much should you email someone, and how much time should you leave in between? Once a month, once a week, once a day? Check out this article to help you determine your optimal email frequency.

6. Determine if your sender name or address impacts your email numbers.

If you haven’t tested a different sender name or address yet, definitely add this to your list. While best practices still apply (in other words, using a name that recipients will recognize as well as a real email address that your prospects can respond to), you can always try out different names to see how it affects your open and clickthrough rates.

Here are some sender name tests to try out:

  • Consistency vs. Change: Should you use the same name for consistency, or try changing it up email to email to garner more attention?
  • Personal vs. Company: Should you use an individual’s name, your company name, or some combination? (e.g. ‘Ellie Mirman,’ ‘HubSpot,’ or ‘Ellie Mirman, HubSpot’)
  • Category-Related Name: If you have a subscriber in a particular segment of your business, you can try sending an email from the name of that segment (e.g. ‘Small Business Team’). If your subscriber signed up for a particular type of content, try using a name related to that specific content type (e.g. ‘HubSpot Webinars)’.

HubSpot’s Free Email Software

HubSpot’s free email tool allows you to create email campaigns that can be ent to email subscribers or contacts in your CRM.

hubspot-marketing-email-drag and drop layout

Aside from providing an easy-to-use drag-and-drop software, HubSpot also allows you to test email designs through an A/B test feature. The software also provides tips related to length in the subject line and preview text area which can help you write captions and instantly verify that they won’t get cut off.

On top of all these features, HubSpot’s software will notify you if it can’t find a link in your email. It will also warn you when inboxes like Gmail will trim your message. 

Litmus

Ever wonder if your email will look aesthetically pleasing on different devices and in different email provider inboxes? With Litmus, you can sign up for free and send them the email you want to test. From there, Litmus will automatically review the email and send you screenshots showing what your message will look like to readers using different email providers. 

Litmus email design testing software

Mail-Tester

Worried you’re using phrases or wording that could trigger spam filters to burry your email? With Mail-Tester, you can log in and get a special email address to send your test email to. After you send it, Mail-Tester will send you a report that notes any of the spammy trigger words that were in your message so you can correct your language before sending to your full list.

Mail-tester email testing tool

Sender Score

Sometimes, if your IP address is associated with sending many different email newsletters, email providers might move your email to spam. If you suspect that your IP address could be negatively impacting your email numbers, you can use SenderScore to find out if your IP address is considered “spammy.”

Sender Score IP address tester

SubjectLine

If you need help with writing subject lines, you can test a few before sending your email with SubjectLine.com. When you go to the website, you simply type in a subject line and click the submit button. Then, you’ll receive a grade out of 100 points as well as pointers for improvement. 

SubjectLine.com subject line scorecard

Reminder: Test only one thing at a time. 

The key with any of these tests is to test just one element at a time so you can isolate your variables and thus tie the difference in results is to that particular change. And if you crank through this list of BIG email tests, here are some great ideas for quick, smaller tests to; always be optimizing.

Happy testing!

Editor’s note: This blog post was originally published in 2012 but was updated for comprehensiveness and freshness in March 2020. 

New Call-to-action