Farquharson/Tedesco on Business Development: Learn from A/B Testing Results
Here's a little-known fact for everyone out there in Printing Impressions-land: In our past lives, your authors were both Nobel Prize-winning chemists. Don't look at us like that; it's true! We both lived to the ripe old age of 106 and died surrounded by awards and Bunsen burners. Then, we were reincarnated as print sales consultants (though there was some initial confusion as to whether we were in heaven or hell).
Admittedly, in our current forms, we haven't set foot in a chemistry lab since college. But the spirit of scientific experimentation remains! In fact, we recently concocted a series of A/B tests to see which marketing channel—direct mail or broadcast e-mail—would perform better in the B2B space.
For all of you non-scientists out there: In A/B testing, an A "control" group is tested against a B "variable" group with one element changed, to see which one performs better. That element could be an offer, a specific design aspect or, in our case, a marketing channel.
The results of our investigations are in and they're, well, complicated. Direct mail sometimes bested e-mail in the categories that matter, but it didn't achieve the runaway victory we'd expected. And which channel offers a better bang for your buck? You'll just have to read on to find out.
First, let's briefly go over the testing data. Then, we'll share our insights from these assays and what they mean for your printing business. Boy, it feels like we're back in the old lab again!
A quick caveat before we get into the analysis. We admit that most legitimate statisticians wouldn't touch sample sizes as small as ours to draw any firm conclusions. But one perk of not being chemists anymore is that we don't always have to adhere to the scientific method! We have tried, whenever possible, to support our analysis by pointing to outside studies that match our data.
In all three tests, the direct mail postcard demolished the e-mail as far as overall response rates were concerned. A recent study by the Direct Marketing Association found something similar: that direct mail averages a 4.4 percent response rate compared to e-mail's measly 0.12 percent.
Direct mail usually pulls higher response rates than e-mail because it usually gets higher read rates, too. According to the USPS, about 80 percent of U.S. consumer households read or scan advertising mail (we're not aware of any published data on B2B direct mail read rates. If you are, please send it along.)
We estimate that B2B direct mail read rates average about 66 percent. Why are we 14 percent lower? At most businesses, there is often a formidable barrier between a direct mail piece and its intended recipient, i.e., a receptionist or other front office employee instructed to dump anything that looks like "junk mail" into the trash before it reaches the right person. This is especially true if you're targeting larger organizations.
Anyway, the fact that we're only estimating postcard read rates hardly matters. Even if we go way conservative and say direct mail read rates are just 55 percent, that's still more than twice what e-mail gets (24 percent in 2012, according to Circle Research, though it varies a lot by industry. For example, computer software averaged 24.7 percent and health care IT just 13.7% percent in 2012, according to a report by Silverpop.) It makes sense that higher read rates would lead to higher response rates, and we think that's the case with both channels.
So, if you're looking for the highest read and response rates per campaign, direct mail can deliver (no pun intended). Is this new information? Not really. Is it nice to have firsthand proof? You bet. Reporting favorable results like these is one of our favorite parts of being scientists...er, print sales consultants.
But, can direct mail compete with e-mail from a cost-effectiveness standpoint? The answer is a resounding...kind of. We measured three cost-effectiveness metrics in our A/B testing: cost-per-read, cost-per-response and cost-per-form-submission. Direct mail achieved a lower cost-per-read in two tests, a lower cost-per-response in one test and a lower cost-per-form-submission in one test. We're sorry for the inconclusive results.
Direct mail slightly edged out e-mail in the cost-per-read battle because its much higher read rates made up for its higher program costs. It's obvious why e-mail eked out a victory in the two other cost-effectiveness categories. (Hint: it rhymes with "toastage.")
Is direct mail a good value for your customer's business? That depends on what they're looking to achieve. If they're after increased brand visibility, direct mail—with its lower cost-per-read—may be a better choice. Does your client want the most responses for the lowest cost? Then, they might want to stick with e-mail.
That said, we're also comfortable saying that statements like, "My marketing budget is too tight for direct mail" no longer cut the the mustard. For many of our cost metrics, direct mail either outperformed, or came within spitting distance of, e-mail.
Are QR Codes Scanned?
Recipients could respond to our postcards either by entering a URL or scanning a QR code. In all three campaigns, we observed that more respondents—in one test, more than three times as many—typed in the URL. Your authors were initially baffled by this data and, for a moment, we thought Tedesco had accidentally bungled the lab results again. But we soon realized these results made perfect sense.
Here's why: Postcards that arrive at businesses are more likely to be read at the recipient's desk, usually in the middle of the day. For the typical print buyer sitting at her computer, it's actually less work to type in a URL than to get out her smart phone and scan a code. When your customer has a computer and smart phone nearby, chances are she'd prefer to access your landing page by typing it in on the computer.
QR codes draw the eye, drive customer engagement and make trackability possible. Include them in your customers' print marketing materials whenever you can.
No matter what, every printed promotional piece your company produces needs analytics—whether through a QR code or just a URL. Otherwise, how the heck will you and your client know if it worked? Imagine if Isaac Newton had failed to record the results of his apple-headache experiment because it was "too much of a hassle."
One final note in the interest of transparency...we admit it. We're not perfect scientists. Technically, our A/B testing was not 100 percent scientifically rigorous. With pure A/B testing, the "A" and "B" communications must go to completely separate lists. But all of our A/B test campaigns were sent to lists that shared some of the same contacts.
Were there people who saw the e-mail first, ignored it, then interacted with the similar-looking postcard because they recognized the design (or vice versa)? No doubt. Does it matter? For practical purposes, probably not. PI
—T.J. Tedesco, Bill Farquharson
About the Columnists
T.J. Tedesco is team leader of Grow Sales, a marketing and PR services company that has served graphic arts companies since 1996. He wrote "Direct Mail Pal 2012" and seven other books. Contact Tedesco at (301) 294-9900 or e-mail tj@growsales.com. Bill Farquharson is the president of Aspire For. Through his Sales Challenge and Tuesday eWorkshop training programs, Farquharson can help drive your sales. Visit his Website at www.aspirefor.com or call him at (781) 934-7036.
