Blake Kidd

Internet Marketing Consultant, Paid Search

Quality Score - Breaking Open Google’s Infamous Black Box

Ah, quality score. Whether you hate it or love it, every search marketer learns early on that having a high quality score (QS) for their keywords is essential to lowering cost per acquisition and increasing return on ad spend. In a recent post, we explained how quality scores are calculated and the impact they have on campaigns. At first glance, QS appears simple. It is a score out of 10 for each keyword, further broken down to three categories: expected click-through rate, ad relevance, and landing page experience. But digging deeper into the factors that determine a keyword’s quality score, we learn relatively little. Google is characteristically tight-lipped about the exact algorithms governing the system. It’s the proverbial black-box metric: at best vague and inconsistent, at worst hair-pullingly frustrating.

When dealing with a metric as fickle and opaque as quality score, data has to inform every decision. Realizing there was a lack of data to justify hours of work re-writing ad copy, we set out to quantify and answer this question: Does including a keyword in ad copy improve ad relevance and quality score?

Searching for a Relationship

Before starting the experiment, we compiled keyword data from existing accounts to see what they revealed about the relationship between quality score, ad relevance, and ad copy. Our goal was to determine if keywords included in ad copy text had consistently better ad relevance and quality scores. We examined 50 exact-match keywords across ten accounts. Each of these keywords had more than 200 impressions per month and included a good mix of branded and non-branded keywords, as well as a wide range of quality scores.

From there, we divided the keywords into groups. The two components of quality score that were not analyzed, expected CTR and landing page experience, were held constant within each group. This ensured any QS differences between keywords were due to ad relevance alone. Within each of these groups, the quality scores were divided based on whether or not the keyword was included in the ad copy and then compared against each other.

These ad changes had a significant impact. While holding the other components of QS constant, including a keyword in the ad copy, nearly a whole point was added to its quality score. Crucially, this increase was observed evenly across branded and non-branded divisions, and all levels of QS - meaning that non-branded keywords increased just as much as branded ones, and keywords with a quality score of eight improved just as much as those with a quality score of three.

Putting It Into Practice

It’s one thing to observe a relationship, but another to put it into practice. To prove the viability of keyword insertion, our team conducted a series of concurrent A/B tests in AdWords. We took 14 keywords which were not present in ad copy, then duplicated them and split the pairs into test and control ad groups. The test group ran an altered ad containing the keyword, while an unaltered ad ran in the control group. As you can see below, these changes were held to a minimum to keep the ads as similar as possible. Traffic was divided evenly for each pair of ad groups with the campaign experiments tool. The experiments ran for 30 days; then data was collected and analyzed for each pair of keywords.

The success here was even more resounding. None of the keywords included in their ad copy had a lower quality score than their unaltered counterparts without the keywords, and nine of the 14 keywords increased by an average of two points! The keywords which did not see a jump had a starting quality score of eight or ten, to begin with. Ultimately, these results confirmed keyword insertion’s effect on improving ad relevance and quality score.

How Marketers Can Use This to Increase Quality Score

Based on these tests, we determined a direct correlation between keyword insertion in ad copy and increased ad relevance and quality score. It’s important to note that a few keywords with a quality score of eight and below average ad relevance did not improve with keyword insertion. Even more perplexing is that several keywords were not present in their ad copy, yet had perfect 10's. These outliers demonstrate that quality score has much more going on under the hood, and that every tactic should therefore be applied under careful observation.

Ultimately, this research provides a strong, quantitative foundation for improving quality score. Incorporating keywords into your ad copy has a high likelihood to raise quality scores by 1-2 points, and has never been observed to decrease it. Digital marketers can leverage these results to not only take simple steps to improve their keywords’ quality scores but more importantly, to justify that work beforehand!

If you’d like to learn more about quality score and how to best optimize your search strategy, reach out to Rise.

09/16/2016 at 02:02

Subscribe to
the Rise Blog

Sign up to receive valuable industry content, delivered straight to your inbox.

or Close this form


Want More?

Subscribe to the Rise blog and we'll let you know when new articles are available.

You might be interested in:

Blog / Paid Search

The Importance of a Granular Paid Search Account

Read More

As the Google landscape continues to get more competitive, it’s imperative for brands to [...]

Blog / Trends

Digital Marketing Love Notes

Read More

Get ready to feel the love. According to the National Retail Federation, the average person [...]

Blog / Trends

Rise Roundup: Around the Web in February

Read More

This month, we’ve rounded up five valuable articles packed with tips and insight for [...]