Farquharson/Tedesco on Business Development: Learn from A/B Testing Results
Here's a little-known fact for everyone out there in Printing Impressions-land: In our past lives, your authors were both Nobel Prize-winning chemists. Don't look at us like that; it's true! We both lived to the ripe old age of 106 and died surrounded by awards and Bunsen burners. Then, we were reincarnated as print sales consultants (though there was some initial confusion as to whether we were in heaven or hell).
Admittedly, in our current forms, we haven't set foot in a chemistry lab since college. But the spirit of scientific experimentation remains! In fact, we recently concocted a series of A/B tests to see which marketing channel—direct mail or broadcast e-mail—would perform better in the B2B space.
For all of you non-scientists out there: In A/B testing, an A "control" group is tested against a B "variable" group with one element changed, to see which one performs better. That element could be an offer, a specific design aspect or, in our case, a marketing channel.
The results of our investigations are in and they're, well, complicated. Direct mail sometimes bested e-mail in the categories that matter, but it didn't achieve the runaway victory we'd expected. And which channel offers a better bang for your buck? You'll just have to read on to find out.
First, let's briefly go over the testing data. Then, we'll share our insights from these assays and what they mean for your printing business. Boy, it feels like we're back in the old lab again!
A quick caveat before we get into the analysis. We admit that most legitimate statisticians wouldn't touch sample sizes as small as ours to draw any firm conclusions. But one perk of not being chemists anymore is that we don't always have to adhere to the scientific method! We have tried, whenever possible, to support our analysis by pointing to outside studies that match our data.