The Social Media Measurement Paradox
By: Amber Naslund
Social media accountability is a hot button for discussion, and rightfully so.
Companies and businesses want to be able to accurately and consistently track specific metrics that can point to their success or failure in social media. They want to be able to justify continued investment in listening and engagement efforts. They want to be able to show the leadership that they’re getting back something of value for the effort and resources they’re putting in.
All of those things are absolutely valid, as they are with any other area of business.
But the challenge that’s emerged in regard to social media measurement specifically has everything to do with our sense of urgency.
The speed of the social web is forcing us to stand up and take note. We know we need to respond quickly. We know that a crisis can boil over in minutes and hours, not days or weeks. We know that expectations are for more, faster, now.
But measurement, collection of data, development of proof points; all of those take time. Sometimes a great deal of time. Gathering enough bits of information in order to draw conclusions, derive insights, spot trends over time that can inform decisions and process about future investment? That requires the patience to collect. To put the processes and platforms in place to capture that information, tie it together. To allocate human resources that can not just aggregate the data, but spend time evaluating it, asking questions, extract not just insights but recommendations.
The very nature of quality analysis runs counter to the immediacy of interaction on the web. We can collect information quickly, but making sense of it takes time, and adjusting our processes and plans to react to what we learn takes longer still.
It’s always been that way. Once upon a time we had to launch an email campaign before we knew if it would work. We build websites before we were sure that they’d pay off in the long run. We wrote sales strategies based on the information we had available, we tracked the ensuing sales, and we adjusted accordingly.
In short, we need make the investment in the efforts first – based on the best information we have and a well-thought hypothesis – knowing that proving out our hypothesis is going to take consistent execution over time in order to have any usable or informative data upon which to assess our efforts.
The real trouble is that we don’t have precedent. What we’re struggling with is not the measurement and analysis, nor even the gathering of the data. (In fact, we could argue that we’ve almost got too much information and knowing which are the valuable bits is the trick, but that for another post).
We’re struggling with creating our hypotheses in the first place, and creating a plan that we have confidence in because it’s based on what we know might be working for others. Hence, our hunger for case studies and examples, as imperfect as they may be today. For all of us, it’s some reassurance that we’re not the only ones taking these risks, and that there are some glimmers of at least superficial success that we can draw from in order to design our own plans and approach.
We’ve shortened that cycle in other areas because we can build our plans and strategies based on whatever we’d like to call “best practices”. We’ve invested the years, the trials and errors, and the fits and starts of nascent measurement there, too, while we were getting our footing. Only now can we look back and say we “know” how to measure success in these areas (though that in itself I’d debate, too).
But regardless, the realization we have to come to is that valuable measurement and accountability analysis for social media takes time and investment in proportion to the value we’re hoping to extract from it. The social web is fluid, rapid, and real-time. Sound measurement and analysis of what we find there is not. And if we’re to succeed in this evolution toward more social business, we have to find a way to reconcile both.
Tags: social media analysis, social media measurement, social media ROI
"The very nature of quality analysis runs counter to the immediacy of interaction on the web." My fave quotable from this article.
You're definitely right, Amber: collecting data is no longer a challenge (thanks to tools like Radian6). I think the real challenge for companies is finding the happy balance between measurement, analysis & engagement. I'd wager that most companies who have jumped into using social media are doing so with limited resources. Measurement is most likely being done manually and that can take away from time spent in active online engagement.
[...] This post was mentioned on Twitter by Amber Naslund, Anthony Coppedge, Dan Dashnaw, Simon McDermott, Mark Oborn and others. Mark Oborn said: RT @AmberCadabra The Social Media Measurement Paradox: http://bit.ly/cIaeZ4 a must read! [...]
[...] very thoughtful article on social media measurement, and how the immediacy of the social web actually runs counter to the time required to develop good [...]
"the realization we have to come to is that valuable measurement and accountability analysis for social media takes time and investment in proportion to the value we’re hoping to extract from it. The social web is fluid, rapid, and real-time. Sound measurement and analysis of what we find there is not."
Oddly, that sounds like an admission of failure to me
Sorry, but I'm with Dave.
This reads like whining from a naive management and development team that cared more about generate sign-ups that their investors could flip than a responsible company generating value for customers.
So YES we're all pissed about it.
The hype machines that accelerated the adoption of many "monitoring" or "listening" tools forgot the golden rule which is that a hypothesis is something that you set-out to prove. And because all the corporations fired all the old farts with PhD's, corporations are struggling with what to do next…
How many of your customers have ANY real objectives any more, let alone hypothesis? (oh, sorry, they all want social media strategies that don't cost money…)
How many of your account managers or developers INSIST that there are measurable objectives or hypothesis in the work they do?
How many developers creating your tools (or anyone's tools for that matter) have any clue what they're actually trying to accomplish or solve? Listening isn't a good enough answer, and it certainly isn't any better than focus groups, or mall intercepts, or market basket analysis because it is only a small percentage of the total audience. Just because a small group can ignite a media storm doesn't mean it's ALL THERE IS!
I know from experiences that the answer is FEW people in this industry really know what they're trying to accomplish. That's because every junior coder thinks he's gonna get rich by writing just a few lines of code that intercepts data from an API or auto-generates a pie chart with Dundas objects. And vendors have really only hired know-it-all kids that did well on a bit.ly test or were first on their block with a blog. Give those of us with serious money at stake a break. Please.
Market Research methodologies have been around a very long time, and for good reason. Corporations have BIG dollars at stake. Dollars that must scale. Dollars that are investments. Of course, we want to see a return. And don't tell us about real-time blah, blah. People's buying behavior and thinking has always been real-time. Ever heard of scanner data? It too water-logged marketers in the beginning. But, not for long — and that's because a respect for personal privacy, a responsible industry association or two, and many hard working research people helped to make sense of it. And the lack of VC money made vendors think long and hard to ensure their product delivered answers. Maybe you all should try that.
Bottom line: the vendors in this space need to grow up. VC's need to go home.
Sorry, but I'm with Dave.
This reads like whining from a naive management and development team that cared more about generate sign-ups that their investors could flip than a responsible company generating value for customers.
So YES we're all pissed about it.
The hype machines that accelerated the adoption of many "monitoring" or "listening" tools forgot the golden rule which is that a hypothesis is something that you set-out to prove. And because all the corporations fired all the old farts with PhD's, corporations are struggling with what to do next…
How many of your customers have ANY real objectives any more, let alone hypothesis? (oh, sorry, they all want social media strategies that don't cost money…)
How many of your account managers or developers INSIST that there are measurable objectives or hypothesis in the work they do?
How many developers creating your tools (or anyone's tools for that matter) have any clue what they're actually trying to accomplish or solve? Listening isn't a good enough answer, and it certainly isn't any better than focus groups, or mall intercepts, or market basket analysis because it is only a small percentage of the total audience. Just because a small group can ignite a media storm doesn't mean it's ALL THERE IS!
I know from experiences that the answer is FEW people in this industry really know what they're trying to accomplish. That's because every junior coder thinks he's gonna get rich by writing just a few lines of code that intercepts data from an API or auto-generates a pie chart with Dundas objects. And vendors have really only hired know-it-all kids that did well on a bit.ly test or were first on their block with a blog. Give those of us with serious money at stake a break. Please.
Market Research methodologies have been around a very long time, and for good reason. Corporations have BIG dollars at stake. Dollars that must scale. Dollars that are investments. Of course, we want to see a return. And don't tell us about real-time blah, blah. People's buying behavior and thinking has always been real-time. Ever heard of scanner data? It too water-logged marketers in the beginning. But, not for long — and that's because a respect for personal privacy, a responsible industry association or two, and many hard working research people helped to make sense of it. And the lack of VC money made vendors think long and hard to ensure their product delivered answers. Maybe you all should try that.
Bottom line: the vendors in this space need to grow up. VC's need to go home.
Thank you both for your input and for dropping by to comment.
I sense that you might perhaps be interpreting Amber's use of "we" differently. She is using the royal "we", as we say, where we = the community of marketers or marketing industry, not we = radian6. She isn't saying that we, Radian6, are struggling with our hypothesis or even creating a plan we have confidence in – not at all. She is reflecting on the state of the industry's thinking.
There is no lazy way to "do social" where you just press a button, make a "buy", measure some clicks and… there!! Social takes work. You could try to approach the social web that way… as a push channel for your messages, but you'd be missing the point. Relationships don't work that way. Trust and reputation aren't built that way. The social web is a relational medium and provides companies with the opportunity to listen and engage with customers on their terms and give them better experiences. It touches every part of the enterprise (marketing, PR, customer service, product insights, etc.). With customer service, we have the opportunity to line up for our customers instead of making them line up for us. That's not to say the other channels (phone, email, etc.) aren't also important, but the reality is that there is a growing base of customers who will ask questions, complain, compliment, etc. via social channels and will expect companies to engage and respond to them there. And, when it comes to research, social listening isn't a direct replacement for traditional research like focus groups, but it most definitely ads tremendous customer insight. In terms of strengths, it is very real time (i.e. issues are heard within seconds), it contains unsolicited feedback (not a response to a specific question you asked) so you might hear things you weren't even looking for, it is easy to distribute the listening throughout the enterprise, but most importantly, it is directly actionable (i.e. the company can respond and build relationships, not just gather data). Enabling a broad base of employees to directly hear the unfiltered voice of the customer is transformative. Listening is not just about one-way gathering of data. It is also an opportunity to let customers know they have been heard and build equity in the relationship.
But that doesn't mean it isn't measurable. The social web is a highly measurable medium. But there is no Universal Magic Metric. You have to do the work to define your business goals and figure out which metrics are important indicators of success.
For instance, let's say you wanted to measure the ROI of your company's investment in an influencer relations program. As a company you use a listening platform to analyze & Identify influencers in social media who discuss your industry – those who have the ear of your potential customers. Let's say you invest one full-time resource who is focused on building relationships, engaging with these influencers on the content they write about, informing them about the company's direction and engaging with them as they start to cover your company's initiatives. How can we measure the ROI of this investment? If you determine, say, that your ultimate business goal is reaching new prospects and driving sales, then the listening platform can directly tag the influencers, measure the volume of articles AND directly measure the total visits, new visitors and conversions that directly came from these influencers' conversations (and do all this automatically). Now you know exactly the number of new customers/sales that arrived directly from articles written by your community of influencers (and you know the cost of your investment so can calculate ROI). Not only does this give you outcomes to measure (#new customers, ROI), but it also gives you metrics to further improve where you are directing your investment (i.e. which sites & influencers result in better conversion rates, new visitors, etc.). So then you have the hard numbers to support your investment.
This is just one example, from the many business processes or initiatives that can be undertaken on the social web, but you can see that there is real value for customers and clear ways to measure it.
For some companies, they might choose to justify their investments on principle alone, "our customers are there asking for help, and so we must respond" and I believe that is quite valid. Others are more guided by metrics. I think it depends on the type and size of investment you are making. If you want to spend $1M on social content marketing, I think you better have a clear measurable ROI. On the other hand, anyone standing on the sidelines and ignoring complaining customers, then asking for a business case in order to respond, in my opinion, is just asking to be featured in a Dilbert cartoon.
Cheers,
Marcel
Thank you both for your input and for dropping by to comment.
I sense that you might perhaps be interpreting Amber's use of "we" differently. She is using the royal "we", as we say, where we = the community of marketers or marketing industry, not we = radian6. She isn't saying that we, Radian6, are struggling with our hypothesis or even creating a plan we have confidence in – not at all. She is reflecting on the state of the industry's thinking.
There is no lazy way to "do social" where you just press a button, make a "buy", measure some clicks and… there!! Social takes work. You could try to approach the social web that way… as a push channel for your messages, but you'd be missing the point. Relationships don't work that way. Trust and reputation aren't built that way. The social web is a relational medium and provides companies with the opportunity to listen and engage with customers on their terms and give them better experiences. It touches every part of the enterprise (marketing, PR, customer service, product insights, etc.). With customer service, we have the opportunity to line up for our customers instead of making them line up for us. That's not to say the other channels (phone, email, etc.) aren't also important, but the reality is that there is a growing base of customers who will ask questions, complain, compliment, etc. via social channels and will expect companies to engage and respond to them there. And, when it comes to research, social listening isn't a direct replacement for traditional research like focus groups, but it most definitely ads tremendous customer insight. In terms of strengths, it is very real time (i.e. issues are heard within seconds), it contains unsolicited feedback (not a response to a specific question you asked) so you might hear things you weren't even looking for, it is easy to distribute the listening throughout the enterprise, but most importantly, it is directly actionable (i.e. the company can respond and build relationships, not just gather data). Enabling a broad base of employees to directly hear the unfiltered voice of the customer is transformative. Listening is not just about one-way gathering of data. It is also an opportunity to let customers know they have been heard and build equity in the relationship.
But that doesn't mean it isn't measurable. The social web is a highly measurable medium. But there is no Universal Magic Metric. You have to do the work to define your business goals and figure out which metrics are important indicators of success.
For instance, let's say you wanted to measure the ROI of your company's investment in an influencer relations program. As a company you use a listening platform to analyze & Identify influencers in social media who discuss your industry – those who have the ear of your potential customers. Let's say you invest one full-time resource who is focused on building relationships, engaging with these influencers on the content they write about, informing them about the company's direction and engaging with them as they start to cover your company's initiatives. How can we measure the ROI of this investment? If you determine, say, that your ultimate business goal is reaching new prospects and driving sales, then the listening platform can directly tag the influencers, measure the volume of articles AND directly measure the total visits, new visitors and conversions that directly came from these influencers' conversations (and do all this automatically). Now you know exactly the number of new customers/sales that arrived directly from articles written by your community of influencers (and you know the cost of your investment so can calculate ROI). Not only does this give you outcomes to measure (#new customers, ROI), but it also gives you metrics to further improve where you are directing your investment (i.e. which sites & influencers result in better conversion rates, new visitors, etc.). So then you have the hard numbers to support your investment.
This is just one example, from the many business processes or initiatives that can be undertaken on the social web, but you can see that there is real value for customers and clear ways to measure it.
For some companies, they might choose to justify their investments on principle alone, "our customers are there asking for help, and so we must respond" and I believe that is quite valid. Others are more guided by metrics. I think it depends on the type and size of investment you are making. If you want to spend $1M on social content marketing, I think you better have a clear measurable ROI. On the other hand, anyone standing on the sidelines and ignoring complaining customers, then asking for a business case in order to respond, in my opinion, is just asking to be featured in a Dilbert cartoon.
Cheers,
Marcel
<a rel="nofollow" href="http://twitter.com/radian6" rel="nofollow">@radian6 The social web is fluid, rapid, and real-time. Sound measurement and analysis of what we find there is not: http://bit.ly/cmF0Ew