If you want a play-by-play guide on how to use Google Optimize, Google has a whole help center for that. But if you’ve been using Google Optimize for any time at all, you’ve probably noticed there are a lot of lessons that fall outside the purview of that help center. And you have to learn most of them the hard way.
The two of us have run a lot of tests with Google’s Optimize, and we’ve probably made a mistake with every single one of those tests. Excuse me, I mean we learned something new with every single one of those tests.
If you’re new to Google Optimize, here are 13 tips and tricks we’ve gathered from all those learning experiences. Hopefully this blog post can help you start running fast, effective, and revealing A/B tests in half the time it took us to learn these lessons.
1. Setting up Google Tag Manager (GTM)
First, the very technical groundwork.
To use Google Optimize you will first need to install the Optimize tag on your website. There are several different ways you could go about the installation. The good news for you is that we’ve tried them all and one approach stands out as the clear winner.
We strongly recommend that you set up Google Optimize via Google Tag Manager (GTM). It allows for the fastest, most consistent deployment across your entire site, and taking the time to set up Tag Manager will pay dividends in your productivity and flexibility down the road. If you’re new to tag management, on the fence, or you just want to get ahead of the game, check out the Portent post on common mistakes in Google Tag Manager.
Once you’re up and running, the good news is that GTM comes with a built-in tag for Google Optimize.
To get started, you’ll create a new tag in your account, then specify that you want the Google Optimize tag.
Next, you’ll need to paste in your Google Analytics ID into the corresponding box as your tracking ID. You’ll also paste the container ID into the Optimize Container ID box.
Finally, you’ll want to set up a trigger to fire on every page of your site (or just the pages you want to test with Google Optimize). This trigger should already be offered as an option if you follow this install method, so you’ll simply select it.
Once you have everything set up as described you will want to go into preview mode in GTM make sure your tag works where and how you intended. It should look something like this.
Once you confirm that the Optimize tag is firing correctly in preview mode you’ll publish the tag to your live site. Once done, you’re set to create your first experiment in Google Optimize. (But read the next few sections before you do.)
A quick warning: Unlike Google’s guide to deploying Optimize with Google Tag Manger, we do not recommend that you set up the tag to fire before your Google Analytics tag. Doing this can actually cause your Analytics tag not to fire on the page you are testing via Optimize. Translation: the order in which you run the test as the page loads could leave you blind to your actual traffic in some cases.
2. Look at your traffic to set timeline expectations
Before you ever run an experiment in Google Optimize, one of the first things you need to do is make sure that pages receives enough traffic to run the test in a timely fashion.
We’ve had plenty of occasions where a client has a hunch they’d love to test for an important page, often near the bottom of their conversion funnel. But on inspection we see that there’s so little traffic that declaring a statistically significant winner would take over a month. That may be fine depending on your situation, but it’s incredibly important to set expectations if your boss or your executive team are expecting answers quickly.
As a rule, we recommend you choose pages for tests that average at least 1,000 pageviews a month over the last 3 to 6 months. This will ensure that your experiment gets steady traffic and the experiment is able to complete in a matter of days or weeks, not months.
Semi-Pro Tip: If there’s a page that absolutely positively must be tested quickly but it’s low traffic, consider whether you could use paid ads to drive the right kind of prospects to this page at a higher pace.
Create a situation that allows you to say “Yes, and here’s what we would need to do to accelerate the timeline.” Again, work with your paid search and/or remarketing experts to work out what kind of audience and approach would drive the right traffic to this page. Translation: prospects that are at a similar stage of their buyer’s journey to those who will see the page in the future. Few things are worse than explaining an experiment led to the wrong conclusion because of a self-inflicted anomaly with your trafficRule of thumb: target pages with 1,000 or more monthly pageviews
See Section 5 for some related takeaways and a little more on this topic.
3. Create a test plan
One drawback of using the basic, free version of Google Optimize is that you’re unable to run more than three simultaneous tests. Three parallel tests should still give most marketers plenty of horsepower, however, and the best way to stretch this for maximum insight is to create a real test plan.
What’s in a test plan? Whatever you want, as long as it gives you direction, project scope, and a clear path to answering your bottom line business question. But here are some ideas to get you started:
- Page title and URL
- Test type
- Hypothesis
- Targeting
- Description of changes or screenshot of mockup
- Page traffic and time estimate
- Funnel or persona target of page
A good test plan looks and acts kind of like an editorial calendar, or Agile project management roadmap. Keep a shared document to which your team can add ideas for future tests (icebox), and work at least two weeks ahead. Tests take about two weeks to complete, depending on traffic, and a deliberate schedule of rolling launches will make those three simultaneous tests feel like an army.
4. Use custom objectives
Data might not “lie,” but it can definitely mislead. Make sure you’re asking the right questions.
Here’s an example of two goals set up on a test: session duration, and Marketing Qualified Leads (MQLs). Session duration is a metric option baked into Optimize; our MQL data was pulled from a custom objective we set in Google Analytics. As you can see, the new variant wildly outperformed the original page in session duration, but interestingly drove far fewer MQLs
If we’d just been looking at session duration, we might think our new variant won. But because we had a critical custom objective in place, we could tell that the original was actually converting more effectively. In this case, taking the time to get our real business goals and KPIs into the test saved us from pushing a “winning” variant to our whole site, which ultimately would have meant losing business.
5. Optimize is perfect for testing PPC landing pages
One of the coolest things about Google Optimize is that you’re able to use your experiments for PPC landing pages.
Using Optimize for PPC landers is might be the fastest way to see quantifiable ROI from A/B testing. Landers are typically high-converting pages with a clear, easy-to-measure goal. You can run multivariate tests with different form lengths, CTAs, titles, and imagery to determine the perfect formula for a high-performing landing page.
Pro-Tip: Extend the value of your newfound knowledge about what’s most effective for your target audience by crafting a set of landing-page best practices specific to your customers and your business. Make sure the learnings hold up, and keep iterating through new tests to drive your conversion rates even higher.
6. Consider starting on desktop-only
At this point, it’s a given for most marketers that you need to optimize your site for mobile traffic if you have any meaningful component of mobile visitors, let alone Google’s Mobile-first Index. This absolutely extends to testing messaging or layout changes to make sure that results do not differ by context.
That said, when you’re unsure how a change in Google Optimize will affect your CSS or breakpoints for mobile devices, you absolutely need to check. In fact, you need to check the mobile view no matter what. Trust us on this.
Although services like Unbounce or Google Optimize do have tools that allow you some control over a test appearance in mobile, it can definitely be time-consuming to get them up to snuff. In some cases with Google Optimize, we’ve simply hit a brick wall with certain complex or highly customized client sites.
For example, even on our own site getting the mobile version of a test right takes some elbow grease. Here’s a visual of a test from Portent’s Content Services page in Optimize on desktop view:
A mess, right? This is really common with Optimize. When you start moving containers around and changing the padding, your test might look fine on desktop but the mobile or tablet version will be borderline unreadable.
You can either work on your design until it works well for all devices or change the device targeting to only display to desktop users, traffic permitting
See Section 11 for more on the importance of doing your own quality assurance.
7. Make the most of targeting options
Of course, you can (and should) target users and test based on more than just device-type. You can target users only on specific browsers, in specific countries, or even specific cities. (Or you can exclude these localities—for example, how well does your Space Needle picture and Seattle branding test with audiences outside Washington State?)
With “Page referrer” (located under “Behavior” in Google Optimize), you can target users arriving to your page through a specific source. For example, if you’re trying to test a new landing page for folks arriving via your blog, you can do that without creating a new URL; you could also serve the test only to folks coming in from a different website entirely.
With “Time since first arrival,” you can define how long a user must be on the website before the test fires in order to eliminate high-bounce sessions. And you can use JavaScript or information from your data layer to get even more granular with customers and their history on your site.
And if you’re lucky enough to have budget and need that justifies upgrading to Optimize 360, you can directly access all your Google Analytics audiences as a targeting segment.
8. Save your work
Seriously. Just do.
Optimize doesn’t have the same handy auto-save features as some of Google’s other products. If you close that window too early, kiss your test goodbye. You’ll be prompted to save if you’re trying to navigate back, but you’re SOL if you close all your tabs by accident.
Not that this has ever happened to us. Y’know, it just seems like it could be incredibly frustrating.
9. Use a script editor with script from the original website
Getting a little further into the nerdy technical best practices: When you create an experiment in Google Optimize you will have the option to work with a lot of code, and will frequently need to use it. When I first did an experiment through Optimize I had to start over at least three times because I forgot to use a script editor. Doing so keeps the code clean and accurate as you move it around.
We recommend inspecting the elements of normal, live, non-test pages on the site you’re working with and copying the code of elements you want to mimic into Atom or a similar editor.
At Portent we frequently use the script editor Atom, which is for Macs.
This means you don’t have to guess at the HTML for an element you want to create in your new design, and Atom keeps that code organized for you so you don’t make a mess of your Optimize layout. You can then paste that code right into your test design with the HTML.
Handy for not screwing up page layout, tables, image handling, etc.
10. Image hosting
Want to add an image to your test that doesn’t exist on the current website? There are two ways to do this.
Skipping ahead to the answer: The better way is to upload the image directly into the CMS of the site and link to that internal file. If you’re working for a client or you don’t have access to upload files to your site for whatever reason, an easy workaround is hosting your images on a free image hosting site. I like to use Imgur.com.
11. Quadruple check your work
This might go without saying, but it’s surprisingly easy to have errors in your test, especially when you’re testing on all platforms or devices. If you’re editing text in HTML, no handy red line appears to alert you to spelling errors, unless you use something like Grammarly religiously. It’s also very easy to overlook design errors until you see your variant on a new browser size for the first time.
Pro-Tip: there are often a lot of people who have spent significant time polishing the heck out of your site. If you’d like to keep the latitude or freedom to do things like dynamically test pages of the site, pay attention to the details. Being proactive by driving new tests and insights with self-managed marketing tools like GTM and Google Optimize is one thing. Being careless with that freedom will get you smacked, and sent to the back of the ol’ dev and quality assurance queue.
12. Keep the winning variant running
Did your test finish? Did the new variant win? Don’t stop the test!
We often keep our tests running, even after Optimize has declared a winner. Depending on your development bandwidth and site complexity, there’s usually a lag between when your test finishes and when the dev team can implement the findings from your brilliant experiment. If your new variant has increased conversions by 25%, you’re going to be losing that 25% if you end your test and wait.
Better yet, re-weight the percentage of your visitors so that 100% of sessions are shown the winning variant. Not only will this mean everyone is seeing the winning page, but will provide an experience that’s consistent for all users.
13. Expect failure. Embrace it.
Not every test is going to be a slam dunk. “Growth hacking” is frankly a cop-out term for describing a process that involves a lot of trial, error, and guesswork. An A/B test isn’t so much a “hack” as it is an iterative optimization driven by audience knowledge and intuition that gives way to data. To move forward with a winning variant, something else has to be a loser or less effective by definition. Sometimes that’s the challenger, sometimes it’s the incumbent.
If you’re thinking, Well, my CRO team would never say that, I beg to differ. AppSumo founder Noah Kagan candidly wrote that only about 1 in 8 tests he runs produce significant results. Kaiser Fung said the same thing in Harvard Business Review: “80 to 90 percent of the A/B tests I’ve overseen would count as ‘failures’ to executives.”
Of course, what those executives see as a failure is usually just a test where the variant doesn’t win, or wins by an insignificant margin. Even these “failed” tests tell you something, and that’s the beauty of Optimize and A/B testing. It’s not about learning how to ignore your intuition and blindly follow the data, but about learning how to test your theories, refine your expectations, and make that intuition even keener.
And I hope it goes without saying that we’re a lot better at this now.
