Three Tips for A/B Testing Success
“Right,” Marka agreed. [Again turning to the placemat.]
3. Tests a Specific “Hypothesis”
“Good A/B tests should contain a ‘B’ element that’s well thought out and will help illuminate a question we have about our current direct marketing activities,” Marka explained. “In this case, the question we seek to answer is, ‘Will the offer of a new, high-value, but low-cost (to us), FEI product generate more recipient interest than a relatively more pricey non-FEI incentive?’ Our working hypothesis, then, is that offering matches will lead to not only better response rates but, eventually, better brand awareness and sales for this item.”
“Sounds like we’re doing everything right,” Org observed, shoveling a forkful of Greek Salad into his mouth.
“I’d say so,” Marka said. “But Org...” [She looked at the FEI leader and rubbed her face..] “You might want to get out a napkin.”
Today’s FIRE! Point
Like almost any experiment, an A/B test has to be done according to a certain procedure. The most successful A/B tests will only test one variable, be focused on a random sample, and contain an intelligent hypothesis.
Campaign Monitor Improves Click-through Results with A/B Testing
The e-mail marketing software company sent two versions of an e-mail with a link to a survey that they wanted recipients to complete. CM changed only one line—the call-to-action in the “B” e-mail. The results: their original version “A” actually received 51 percent more click-throughs.
Next week: The FEI tribe shifts gears to product strategy and discusses when to enter a new product category vs. extend an existing product line.