Social Media Case Studies

Our Dangerous Addiction to Social Media Case Studies

The two most important words in social media should be “so what?”

As Tom Webster eloquently put it in his recent Blogworld keynote, when research wears the cloak of content marketing, it’s a recipe for the incurious to pull a fast one on the masses, disguising pointless data as gospel.

Tom was referring to data-dredging and the spate of statistical “best practices” that bubble up in social media like something in Jed Clampett’s back yard. But it applies equally to the basket of case studies that are left for us each day by the sites that chronicle social’s advance. It’s like Moses in the bullrushes but with “math” instead of blankets.

The Recalcitrant Social Media Cheerleader

 

Point Less

Too often, we try to wield these case studies as a crude evidentiary weapon, trying to persuade ourselves or our superiors that our belief system around social media, social business, social CRM and all that jazz is in fact correct. But in almost every case, you’re bringing a knife to a gunfight. Or maybe even a limp noodle.

Case studies should be used for ideation, not ratification. Even in the best possible scenario, where the case study in question is extraordinarily applicable to your business goals, social media situation, KPIs, budget, timeline, customer personas and more (which is a rare alignment indeed), you are placing significant influential value on ONE outcome. That’s anecdote-driven decision making at its worst.

If you only need to hear one story that you find persuasive, I can convince you of ANYTHING if I get to select the parable.

Beyond the fact that case studies are often strategically irrelevant because the company profiled is in a different industry, with different goals, competitors, and customer expectations (among other variances), perhaps the biggest problem with most social media success stories is that the measures of that success are largely without real merit.

Social Media Case Studies – The Crack of the Stat

My favorite example last week was the glowing report on ClickZ of Louisville Slugger‘s scavenger hunt promotion. Evidently, the erstwhile manufacturer of wood and aluminum bats launched (with agency help) a one-day promotion on October 29 in St. Louis. They hid 45 commemorative World Series bats around STL and posted clues on their Facebook and Twitter accounts as to their whereabouts.

I’m no baseball manufacturing expert (but I do adore the Louisville Slugger Museum in downtown L-ville), but I’m guessing they ultimately need to sell bats to kids and weekend warriors to maintain/expand their business. The MLB buys what they buy, and the Twitter acumen of the company has a zero percent impact on whether Derek Jeter swings their wood.

Strategically then, the question becomes “does hiding and giving away 45 bats and talking about it in social media ultimately sell more bats to amateurs (either current or new customers)?”

And the simple answer is that we do not know. This case study appeared 10 days after the conclusion of the promotion. Is 240 hours enough time to determine the true impact of this promotion? Of course not. November is hardly peak season for bat sales anyway, and any bottom-line impact of this campaign would need a while to germinate and flower.

But that doesn’t stop the agency (presumably), the company, and ClickZ from proclaiming the unmitigated success of the effort. On what basis? Evidently, Louisville Slugger saw a 834% increase in the “talking about this” metric on Facebook, which purports to measure total number of Facebook users referencing the brand in any capacity whatsoever.

Well, if you go to 45 locations around town in one day and give stuff away, a goodly portion of the people angling to be the recipient of free goods will be updating status, location, photos and so forth. It’s what we do now, and our proclivity to shout from the status rooftops about the awesome thing we’re engaged in at this very moment is exacerbated by three factors: deviation from the hum drum norm; a crowd; and free stuff. Wisely, Louisville Slugger designed a program that hit for the cycle, and St. Louis residents choosing to spend a portion of their Saturday chasing after free baseball bats responded on Facebook en mass.

But does a 834% increase in people “talking about you” on Facebook achieve business goals like sales, retention, average order size, advocacy, etc? Again, in this instance we do not know yet. Before this initiative (and I give them credit for divulging real numbers) Louisville Slugger had 755 people talking about them for the week. Afterwards, it was 7,049.

The brand spent considerable time and effort to drive a short-term spike in conversations about it on Facebook. So what? You might argue that because they also saw 100%+ week over week increases in Facebook fans and Twitter followers, that this campaign was a smash. Again, so what? They have 21,000 Facebook fans, and their best posts (announcing a second giveaway) show an engagement rate of .03.  They have 3,200 Twitter followers, and the account essentially parrots their Facebook status updates.

I’m not suggesting this is a flawed campaign or wholly without merit. We simply do not know whether it has strategic or financial merit, and I’d love to see a case study that tracks sales and customer retention in the St. Louis market on a 60-day trailing basis. Now THAT would be a case study.

I am not irked by the idea, or the effort behind it. I am irked by the near-instant reporting and self-congratulatory reach around that content repositories, agencies, and corporations are engaged in every time they find some statistic that allegedly proves that their idea was a big success. And I’m irked that we fall for it every time, retweeting and sharing and printing it out for our boss to read on Monday.

We can do better than that, can’t we?

The poll mechanism above is a Quipol. Quipol is a brand-new, supremely easy-to-use polling tool created by my friend Max Yoder. It includes embedded comments (maybe you left one) and a handy thumb up/down function. It’s built for bloggers (easy to embed), and you can also drop Quipol’s on regular Web pages and elsewhere. The comments can also be shared to Facebook or Twitter.

Quipol is totally free, with advanced functions rolling out for small add-on fees. Check it out.

Max is an awesome guy, and an inspiration. He’s 23 and already paid off all his student loans and is bootstrapping Quipol on his own nickel (no VC) by saving 61 percent of everything he makes at his day job (working for Compendium – a former client here at Convince & Convert).