Ronny Sage

Associate Director, Digital Strategy

Fashion Digital New York: Creating a Testing Culture

Hundreds of e-commerce's most fashionable marketing and tech gurus gathered in the Big Apple last month for Fashion Digital™ New York. The three-day conference covered a number of digital marketing topics ranging from omni-channel tactics to customer retention strategies and online and offline integration.

I had the pleasure of speaking on a panel alongside digital experts from Optimizely, Groupon, and Boll and Branch, to discuss growing your business through A/B testing.

As today’s most successful marketers know, gone are the days of product teams, designers, and marketers fighting over what “looks better” when launching a new website. Marketers have an enormous opportunity to incorporate testing into their design processes and make well-informed, data-guided decisions. A/B testing (or multi-variant testing) is no longer strictly an analyst function, it should be an expectation at all levels from the top-down. While increased conversion rates are a primary goal of A/B testing, the benefits also include providing a better user experience and increasing customer loyalty, among others.

Here are five key takeaways from our panel discussion, “The Sky’s the Limit: How to Grow Your Business Fast Through Effective A/B and User Testing”:

Kill the HiPPO.

The term HiPPO refers to the "Highest Paid Person's Opinion." With the ability to gather robust data on your visitors, it’s important to let these insights guide your website decisions from color selection to button placement and more. Personal preferences and instincts should no longer dictate decisions -- no matter the person’s level within the organization.

Set priorities based on a "PIT" system.

“PIT” stands for potential, importance, and technical. By weighing the following elements and asking the right questions, you can better guide decisions and priorities.

  • Potential - How much traffic does a page get and how many of those visits result in conversions? Is there opportunity to increase the number of conversions? What feedback are you receiving from your customers?
  • Importance - Consider a page by page comparison of traffic volume. Does this cross over with future feature development?
  • Technical - How much developer time will be needed to make certain changes? What are the cost implications? And most importantly, is this change going to create a political frenzy due to varying opinions?


Test all shapes and sizes.

Don't spend all your time building out new pages for each customer segment; test the small things. Testing can be as small as changing the color of a button or as large as creating different messaging for individual user segments. 

Recognize that different segments respond differently.

Just because one user segment responds positively or negatively during a test, you shouldn’t necessarily expect the same response from other segments. The worst thing you can do is implement a site-wide change based on the results from one user segment. Try testing against both the macro and micro, such as location, referral source, audience persona, and new vs. returning customer.

Set a testing budget.

No more code freezes -- set a testing budget. During the holidays, many brands refuse to make changes on their websites out of fear of potential site breaks and the corresponding loss in revenue. However, the holiday season can be a great time to set and use a testing budget. This year, choose a channel or segment and run some small low-risk tests on product placement or seasonal color. Use the winning results next year and have the competitive advantage.

The running theme of the discussion was that your website should no longer be a static place to hold information. It should be a living and breathing extension of your brand that speaks to your customers with the right message at the right time. How can you start? Develop a testing culture, start small and think big, and let data drive your decisions.

For more information on how to incorporate A/B testing into your website development and ongoing optimization, reach out to Rise.

Want to learn more? You may also enjoy this article on the role of SEO and usability testing in website development.

11/13/2015 at 12:00

Subscribe to
the Rise Blog

Sign up to receive valuable industry content, delivered straight to your inbox.

or Close this form


Want More?

Subscribe to the Rise blog and we'll let you know when new articles are available.

You might be interested in:

Blog / Analytics

How to Define and Measure Your Customers

Read More

Having clearly defined goals and knowing your budget are key to developing a successful [...]

Blog / Paid Search, Analytics, Connex

Moving from Cost Per Lead to ROI

Read More

Advertisers who practice closed-loop reporting are gaining a substantial competitive edge by [...]

Blog / Web & Mobile Development

Mastering a Website Redesign: Key Aspects to Take into Consideration

Read More

What does it take to launch a successful website redesign? In this on-demand virtual [...]