Ad Testing: A Glossary of Important Terms

Ad testing has long been a staple of market research. But as the media landscape has transformed, ad testing methodologies have changed along with it, incorporating more and more tools from a growing ecosystem of digital media and advertising solutions. As a result, today’s popular digital media platforms include features explicitly designed for optimization and testing, enabling advanced marketing teams to conduct some of their own ad testing.

As marketing teams partner with consumer insights professionals to create effective campaigns, it’s vital that they speak the same language. Below, our digital analytics team offers 27 important terms to know about ad testing, with many terms extending beyond digital to the broader advertising space.

(terms listed alphabetically and in groups of ad testing categories)

Digital Advertising Components

Advertising Assets

Content in digital form created by a brand.

To promote products on their site, a brand may need to create assets in the form of logos, photos and videos.

Ad Unit

A specific digital ad format, with the size and location of an ad defined.

A campaign with digital elements will have to determine how to activate across many different ad units. For example, a banner ad unit is a rectangular display ad.

Ad Variations

Multiple versions of ad creative to be tested.

Ad variations may differ by headline copy, images, whether the ad is still or moving, calls to action and more. The key thing is to focus on potential changes and why they might impact the business outcome.

Target Audience

Group of people to whom advertising is directed.

An automative ad may have a general audience of people watching a live football game as well as a more specific audience the brand wishes to target, such as men 18-24 who like trucks. The brand  may further split their target into two test audiences that each receive different creative on digital channels.

Audience Segments

Subgroups of the population defined by distinct attributes such as behaviors or demographics.

Digital ad testing commonly involves analyzing performance by segments to understand if their performance is different from the average. The ability to communicate differently to various audiences is key for digital marketing to drive growth and the foundation of targeting and personalization.

Call to Action (CTA)

The text intended to persuade users to perform a specific act from a digital asset, such as clicking to learn more.

“Learn More,” Subscribe Now,” “Sign Up” and “Get Started” are all common CTAs. And CTA engagement could be a success metric for a digital ad test.

Measuring Advertising Effectiveness

Clickthrough Rate (CTR)

The rate at which users clicked, out of everyone who saw the ad.

CTR = clicks / impressions

An ad that receives 50 clicks out of 1000 impressions has a 5% CTR.

Conversion

When a person completes an action valuable to the business, such as signing up for emails or completing a purchase online.

Conversion is typically the most important metric when evaluating ad performance. While ad engagement may be desirable, conversion directly ties the ad to business outcomes.

Cross-Device Tracking

Tracking and measurement of ad performance across the different devices a single user may own, such as mobile, tablet, smart TV and laptop.

The ability to reconcile one user’s data from different sources is technically difficult, but an important goal to pursue. One solution is to require logging in, so you can identify activity linked to one account.

Cost per Acquisition (CPA)

A measure of aggregate cost of a customer taking an action that leads to conversion.

Cost per Click (CPC)

A measure of cost incurred when an ad is clicked.

Cost per Mille (CPM)

A measure of cost when an ad is shown 1,000 times.

CPM, CPC and CPA are three primary ways digital media is charged to advertisers. Which method is used depends on the platform and campaign goals. CPM is generally for brand building, when you want to get in front of a lot of people. CPC is a more efficient, performance-based metric. CPA is a more robust metric that determines ROI but is rarely a pricing option.

Engagement

An overarching term that includes most trackable behaviors related to an ad, social platform or website.

Engagement boils down to how and how much an audience is interacting with your ad content. It could mean clicks on an ecommerce website that demonstrate shopping actions, or it could mean likes and shares on a social platform.

Impressions

The number of times an ad has been served, regardless of whether the user has consciously interacted with the ad.

Typically, in a digital testing environment, test variants are split up so that each has the same number of impressions.

Unique Visitors

The number of distinct individuals, most commonly identified with a system or device ID, who have visited a website or other property.

To better understand the impact of an ad variation and to more accurately track test results, it’s recommended to design tests to the unique visitor-level, rather than the impression-level. This prevents a user from being served multiple test variations, among other complications.

Digital Advertising Methods

A/B Testing

A randomized experiment where two or more variants of a user experience are shown, and a winner is determined using statistical analysis of the performance of each variant.

An e-commerce site, for example, could test two versions of a product page to understand which drives more online sales. Google, Facebook and Amazon feature testing functionality in their ad platforms and encourage its use to improve ROI.

Hypothesis

A prediction created prior to running a test that states what is being changed, what you believe the outcome will be and why.

A strong hypothesis is just as important as statistical significance in determining and understanding the winner in a test. Many creative tests make the mistake of testing wildly different variants, which prevents conclusive results.

Multivariate Testing (MVT)

Designing a test with multiple variations, typically requiring more complexity in the hypothesis generation and test set-up.

Many testing programs advance to running multiple MVT tests at a time after they’ve developed robust test roadmaps, strengthened their testing processes and learned to manage sampling to maximize their traffic.

Sample Size

The number of people from a population that will be included in the test.

Sampling from a larger population to determine how that population behaves or thinks is familiar territory for consumer insights, but now it’s now the purview of digital advertising as platforms enable A/B testing. Determining an adequate sample size that will reach significance in a relatively short amount of time is a key decision in an ad test.

Statistical Significance

The probability that the difference in performance between test variants isn’t due to random chance.

Ad testing is typically run until data reaches 90%-95% significance. This helps ensure the test doesn’t have sampling errors and that the winner will perform with a wider audience.

Video Advertising Terms

Addressable TV

Technology that enables advertisers to select target audiences for TV ads rather than buying based on programming.

Addressable TV permits an advertiser to show different ads to different households during the same program based on targeting data. Showing a dog food ad to a dog owner, and a cat food ad to a cat owner, are examples of addressable TV advertising.

Average Completion Rate

The percentage of the video watched by your audience, divided by the number of people who watched.

This measures your video’s ability to hold attention.

Average View Duration

The amount of time a video has been watched, divided by the number of times the video has been played.

This indicates the attention span users have for your content and can be helpful for guiding ad creative. For example, if videos have an average view duration of 30 seconds, a 30-second ad would be best for this topic.

Pre-Roll Ads

An online video ad that plays before a video the user selected to watch.

These videos are commonly 15-, 30- or 60-seconds long and run before the user’s desired content, meaning the audience is likely still engaged. Like most other digital media, these ads can be tested and optimized for performance. Similarly, mid-roll ads run in the middle of a video.

Skip Rate

The percentage of skips for skippable video ads.

The most effective pre-roll ads capture the user’s attention immediately, as many video ad formats offer them the chance to skip the ad entirely and continue to their desired video.

TrueView

YouTube’s advertising platform. It enables targeted ads, where advertisers only pay when someone watches the ad for its entire duration.

TrueView ads are the skippable ads that play before a video or during a longer video. They also appear in the search results and sidebar. People are now watching a billion hours of YouTube every day, making this a go-to platform for video ads.

Video Drop-Off Rate

The rate of people who navigate away or close a video as it progresses in time.

This shows which moments in the video that people tend to lose interest and leave. Pay particular attention if you’re seeing a high drop-off in the first 10-15 seconds of the ad, as it’s an indication of whether you have an immediate, compelling hook.

Want help getting the most out of your digital advertising? We’re here to help.