Quantifying the Impact of Social Media: Where the Edelman White Paper Got it Right, Got it Wrong and What We Should Do Next
January 31, 2008
If you're new here, and interested in the latest news and insights on social media marketing, you may want to subscribe to our RSS feed. Thanks for visiting Ignite Social Media.
Earlier this month, Jonny Bentwood of Edelman published “Distributed Influence: Quantifying the Impact of Social Media.” Edelman gets picked on a lot (often deservedly so) for the highly visible mistakes they’ve made in the social media space, but there’s no question that they are both deeply involved and pretty darn thoughtful on the topic. This white paper fits the latter; it was a thoughtful analysis by some major players in the game on how you measure online influence.
What they got right:
1. This white paper builds on the concept of a Social Media Index, which is an effort to get all of us in this space to think about a standard way of measuring of influence. The index is not perfect (as they openly admit), but it is thought provoking. It’s a good starting point and a must read.
2. They gave some additional insight into how meme’s get started, concluding that there are:
- meme starters,
- meme spreaders,
- meme adapters,
- meme commentators, and
- meme readers.
This is analogous to Forrester’s Social Technographics Ladder, which measures web participation from low to high as:
- Inactives (52%) –>
- Spectators (33%) –>
- Joiners (19%) –>
- Collectors (15%) –>
- Critics (19%) –>
- Creators (13%).
3. They begin an interesting discussion that influence as measured by the power of an individual isn’t the way to go, but by the power of the idea and how much it actively moves within the space. Credit to Jeff Jarvis for that thinking, which I think is 100% right. This is similar to what Radian6 does in their monitoring tool when they allow you to search content by the activity surrounding it (i.e., total comments, unique commenters, etc.). I firmly believe that this is likely to be more useful than chasing an amorphous, ever-changing group of influencers for reasons I explain below.
What they got wrong:
Early on Bentwood acknowledges that this white paper doesn’t even try to be the definitive answer we’ve all been looking for, but another piece of the conversation. It’s a valuable one at that. These “got it wrong” statements are meant in the same way: to contribute another series of thoughts to this conversation, not to criticize even in a small way. So here goes:
1. For the bulk of this white paper, the roundtable seems to conclude that influencers are influencers by their very nature. I don’t believe this is true.
Relationships (both online and off) are contextual. A powerful influencer in one situation is a meek influencee (or someone who simply doesn’t care) in another situation. Steve Rubel has influence in social media conversations, but likely has no influence in conversations regarding women’s shoes, vitamins, Bollywood, etc. (Steve, if you are, in fact, a player in the women’s shoe world, more power to you, but hopefully the point is well made.
2. Social networks and trend starters are too complicated to ever be identified in advance.
That “influencers” exist and “do their thing” is the premise of the Tipping Point, a favorite book of mine. But we all know (or hopefully should know) that the world is vastly more complicated than that. Duncan Watts of Yahoo has done some groundbreaking research on social network behavior that suggests that the power of the idea is in the context of the moment, and the influencer on a given idea varies widely and unpredictably. As a result, we should not be trying to pin down the “powerful people” as some fixed set (n=what?), but rather build better systems to help us quickly identify powerful people by topic.
3. Personality-type is not the horse we should be betting on.
According to Bentwood, the roundtable concluded that “a system equivalent to Myers Briggs was needed for micro-communications. This would enable people to be able to map target media, meme creation, consumption and sharing habits.” In my view, that sort of effort would miss the boat entirely and be of dubious value. As someone who works in a multi-client social media agency, I have to work within conversations by topic. I believe that people’s passions and influence are not innate qualities of the individual that make them either an influencer or an influencible, but rather vary widely based on their comfort in and knowledge of a particular topic relative to others in the discussion. (Think of the bossy know-it-all around the office who is meek and mild when his big brother is around.)
Whether they ever choose to participate may relate to their personality type, but among those willing to participate, influence-level is not fixed. Max Kalehoff’s statement in the white paper that, “You don’t go for the most influential but the most easily influenced” is interesting, but misdirected and sounds suspiciously like targeting 1.0. (I acknowledge that I don’t know the context of Max’s statement, so I may have missed some nuance he injected.)
What we should do about it:
Based on all this, what do we do about it? What’s the ideal situation for moving forward? Here are my thoughts to add to the conversation:
1) To move the social media index forward, we’ll need a system that serves as the white pages of social media with RSS.
In other words, one of us needs to build an open directory into which people can voluntarily put their various online personas (blog, Facebook ID, Flickr account, LinkedIn, Twitter name, Utterz account, etc. etc. etc.). There’s no question that everyone would not do this, but social media types are openly creating online personalities and want to be found, so active “creators” users would. While Naymz and a few others do something similar to this, the utility comes not from just a “look-up” feature, but the ability to use this data to analyze contribution levels. (The social media index is interesting, but to make it work in the long-tail we have to be able to both automate it and use it as a tool within a monitoring platform.) So we’ve got to be able to allow folks to query this data in an automated fashion as part of our monitoring.
2) With the dataset we’ve created in Step 1, we will be able to improve our online monitoring so can identify both the most virulent ideas that are percolating within each conversation (measured by the activity around the idea including comments, photo tags, unique commentors, length of the comment, etc.) and cross reference it with the most active contributors in each conversation (by volume of contribution, across multiple platforms and media type).
3. If we then build an algorithm that weights the activity around contributions (audience receptivity to a contributor as measured by their engaging with that contribution) with volume of contributions (across multiple media like blogs, Tweets, Flickr tags, etc.), we’ll have a useable, ever-changing snapshot of which individuals are controlling and guiding a particular topic at a particular time. Ideally, this system would allow you to adjust weights like a graphic equalizer to get different outcomes depending on exactly what you’re trying to learn with your search.
Summary
When you work for a social media agency, you end up participating with the “true believers” in the world (like the folks on this roundtable), but you also participate with the openly skeptical and the cautiously optimistic. They neither know nor care who Robert Scoble is and their universe (online or off) is unlikely to intersect with his. But I firmly believe that there are billions of interesting conversations taking place out there, being steered (intentionally and unintentionally) by hundreds of millions of micro-influencers. To chase them all as somehow wired differently than us is not the way to go. To have better tools by which to find them within the conversations just as you need to is the way to go.
With that, I’d like to tag Jeremiah Owyang for his thoughts on the concept of context-based influence versus personality-based influence. I’d also like to tag Duncan Watts for his thoughts (although I can’t seem to find his actual blog, so doubt he’ll see this). I’ve likely bastardized and twisted his brilliant thinking into something he’ll barely recognize. I’m always curious as to what Geoff Livingston thinks. And, finally, of course, since I’m contributing to a roundtable that I did not attend, I’d love to hear from the actual participants, including Jonny Bentwood, Max Kalehoff, Sarah Petersen (who is either too shy to blog or needs some SEO help), Henry Copeland, Jeff Jarvis, Steve Rubel, Keith O’Brien, Richard Edelman, Rick Murray (another hidden/nonexistent blog–irony noted), David Dunne, Peter Kim, Charlene Li, and Dr. Walter Carl (congrats on the baby by the way–she’s very cute).
Now, who wants to build this tool I described? I’d certainly pay to subscribe to it… Heck maybe we’ll build it, who knows…
Comments
23 Responses to “Quantifying the Impact of Social Media: Where the Edelman White Paper Got it Right, Got it Wrong and What We Should Do Next”
Got something to say?
Interesting, let me digest this, and think it over.
I think that opening up the discussion since the roundtable and paper has been great - lots of valuable reactions. Perhaps a wiki is in order to help drive the discussion forward…?
Hey Jim,
You make excellent points. I think your point about context and topic is very central to this whole discussion. This is exactly how we approach influence at Radian6. Influence is context specific and topic specific, as you say. That is why our measurement of influence needs to be context-based (by topic) and not only personality-based. Now, there is a factor (let’s call it popularity instead of influence), where “by their nature” someone has a greater potential to become an on-topic influencer, but this potential does not make them an influencer (yet) until they become active in that topic and they begin to generate the conversational dynamics that demonstrate their influence (i.e. posts/memes, comments, unique commenter, on-topic links, engagement, etc.).
In terms of your proposed ideas, I would love to see this “white pages” directory materialize. There are several attempts underway (Spock, lifestreams/Tumblr, etc.), but none of them have tipped yet. In our monitoring tool, we already analyze contribution levels from multiple sources and measure topic/context-based influence down to a user level but each of your online personas would be identified and ranked independently. If we had such a directory, we could unify them in a snap. We are thinking through various ways extract these persona linkages, but a directory would be great, even if it was just a simple lookup.
Marcel LeBrun
CEO, Radian6
@Jeremiah: Thanks. Would love to hear your thoughts.
@Peter Kim: That’s a good idea. I’ve stumbled across various thoughts on the white paper that are all across the board. Anything from a del.icio.us page of links on it to a wiki could be cool. We could do it, of course, but maybe better coming from the roundtable. Don’t want to steal/monopolize the conversation you started.
@Marcel: Glad you liked it. What you’re doing in the space seemed to fit very naturally and your technology seems like it would adjust nicely to it. I was poking around Naymz last night, and it could be it, but they’ve got it walled off to even see a full profile, so I doubt you could get all that data..
Yeah, I’m not sure dismissing personality based influence is a good idea. The reality is I think there’s little difference between Seth Godin’s Meatball Sundae and a host of other social media books already written. But because Seth wrote i, everyone in the business world is ga ga. The tipping point for business social media has arrived.
Ideas have power. But when the idea is matched with a proven personality, power becomes dynamite.
Similarly, when an influential person propagates bad ideas regularly they lose influence. Rubel is a great example of that.
Also, I thought Edelman’s perception study was not that great. Businesses are not gaining more respect in my opinion, nor in DC pollster’s opinions. Edelman needs to try harder.
[…] It’s only ONE strategy. And not even a really great one. Although it can produce results (but how do you measure influence?), it should not be relied on and certainly shouldn’t replace existing campaigns. Advertising, […]
[…] Ignite Social Media wrote an interesting post today on Quantifying the Impact of Social Media: Where the Edelman White Paper Got it Right, Got it Wrong and What We Should Do NextHere’s a quick excerpt Earlier this month, Jonny Bentwood of Edelman published “Distributed Influence: Quantifying the Impact of Social Media.” Edelman gets picked on a lot (often deservedly so) for the highly visible mistakes they’ve made in the social media space, but there’s no question that they are both deeply involved and pretty darn thoughtful on the topic. This white paper fits the latter; it was a thoughtful analysis by some major players in the game on how you measure online influence. What they got right: […]
Jim,
Thanks for the thoughtful post.
As Jonny wrote, our (the round table’s) intent was always to open this dialogue up to the community. We’re well aware that we don’t have all the answers; we’re not even sure there is “one.” What we all agreed on was this: the current tools and approaches to measuring the impact of social media are incredibly flawed.
We all think a wiki would be a great way to capture the dialogue and move this effort ahead. I presume you’d be moe than willing to contribute your thinking. We’ll build it / launch it next month (as in February). With any luck, we can generate the kind of momentum Chris Brogan got with TwitterPacks earlier this week. .
Finally, irony noted. I read plenty, but have have been doing all my talking on facebook and twitter. Feel free to follow teitter.com/rickmurray. And FWIW, a new blog is being launched within the week.
Cheers,
Rick
At any rate, hoose to believe it or not, we actually hoped for this kind of dialogue.
@Geoff: I agree that personality is important and that alpha’s can be powerful and I also agree that influence waxes and wanes depending on the validity of what you contribute. I’m trying to measure that by this metric.
At the same time, I also think that certain industries (particularly tech) will have their alpha bloggers and the rest of the world that are just using the tools to talk may be having thousands of conversations without an alpha leaders. But those micro-influencers are certainly out there, and if we can find them we can help our clients immensely.
@Rick, thanks for the reply. I completely believe you were trying to stir debate, which is great. I look forward to seeing the wiki…
~Jim
[…] Quantifying the Impact of Social Media: Where the Edelman White Paper Got it Right, Got it Wrong and… […]
Hey Jim…great post!
Been thinking about this topic for a few weeks now after reading several of those white papers you mentioned.
Also been trying to define the success of several social media campaigns.
I also would like to understand and evolve my own social media index.
I have a great opportunity to start fresh with a new social media persona, I am starting a new job next week. It will be in the outdoor gear and clothing industry and I’ll be starting fresh with some new social media campaigns.
So I have started tracking my regular social media and tech activity under one account. And I will start tracking fresh my activities in this new role as well next week.
I am using Google Notebook and Google Docs to track all of my activities in hopes of sharing and publishing the data for analysis. I hope to define some benchmarks for success, rating, and analyzing reach of my social media index.
I invite anyone to collaborate and can have access via API to my META Data and Daily content feeds for analysis and feedback.
I also invite anyone track and share their own data.
Your number 1 got me going Jim: To move the social media index forward, we’ll need a system that serves as the white pages of social media with RSS.
@Kin Good luck with the new gig. The social media index looked interesting, but I couldn’t imagine trying to replicate it over the long tail of thousands of conversations. It seems like it was manually constructed, so to be useful, how can we automate that? Let me know what happens with your project.
~Jim
[…] measurement of distributed influence in Social Media. Jim Tobin of Ignite Social Media added to the discussion this week. My comment on his post is quoted below. If you are interested, the discussion was started […]
It struck me as I was reading, and forgive me if I’ve misinterpreted, but your white pages idea is a striking one, but omits the important part of the equation. Social media influencers, or those influential in the space of social media, have little to do with the greater world around us. (i.e. Steve Rubel has little influence on the women’s shoe market.) We are just talking to ourselves wondering how the rest of the online world is behaving. Sure, you could catalog influencers in other areas, but how? They don’t care to be studied and quantified by us social media trade geeks. They just want to gnosh about who they saw a Spago (okay, lame 90s reference … my bad). Sure, studying the social media trade influencers might give you a sample of the rest of the world, but there’s a big, big Internet out there we are only a small portion of. Knowing how much influence Owyang, Rubel and Scoble have matters not to knowing how influence is spread.
Find the people who are passionate about the topic in question. Give them the tools to spread the news. Viral will happen. Stop looking for the big dogs … they don’t run fast. Gather up a herd of chihuahuas and more people will hear the bark.
Jim; Very nice thinking here. I was an early commenter on the whitepaper.
I like your thoughts about hubs of influence. Over time, it has been my experience that people rise up as an influencer and then ebb and flow as time goes on. Let me explain, where someone may have once been the go-to person on a particular topic in a small community, say useful tips, for example, they may then become a more credible voice on some other aspect, or they may fade from the scene entirely. It changes.
One caveat, if they are one of the top influencers in their niche, their opinion or referral might still carry more weight than others on topics outside their expertise.
Check out what Plaxo announced today, they are trying to build this aggregate network.
@Jason: You’re right in that only the hard core geeks would (at first) list themselves in this “white pages”. There would have to be something in it for the masses to do it. Like making it easier for your friends to find you, or maybe it’s one piece of that social media portability site that everyone keeps dreaming of. I don’t have it worked out yet, but there has to be a good exchange in order for this to tip… I just don’t know exactly what it is yet… But if we can get it to tip, then we can find those 1,000 small dogs much more easily. ~Jim
@Kami: YES! This Plaxo move might just be it… They might be building the social media white pages here, and the “hook” that Jason felt was missing might be their auto-update address book feature.
To see what Kami is referring to, check out: http://blog.plaxo.com/archives/2008/02/introducing_pub.html
Hi Jim
Thank you for linking to the paper. As Rick stated above and as we mentioned in the paper, we never intended to present a fait accompli but rather an aim to try and act as a catalyst for dialogue. I think what you have written is great and has raised a few thoughts in my mind as to how things could progress moving forward.
I think we are a long way of from having a definitive answer but with so many people actively engaged in understanding this, I am sure that we are closer to achieving some consensus. I like the idea of a wiki and look forward to the discussions moving forward.
[…] Quantifying the Impact of Social Media: Where the Edelman White Paper Got it Right, Got it Wrong and… Jim continuing the debate about the Social Media white paper - great points and reasoning (tags: socialmedia) Filed under: del.icio.us links | […]
Very thoughtful. I will need to chew this over some more, but I think you have moved it on again.
The issue of ‘who’ is the influencer has bugged me for some time. Nodes in any network are not of equal value - that is self-evident but here is another complication for you: Is the ‘expert’ really the one who knows best or is the ‘expert’ the one who knows better than the user?
In other words: A user is looking online for experts on photoshop. Who does he trust? Or (an even better question) with whom does he/she feel comfortable enought to ask the question? The photoshop super ‘expert’ (years of experience blah blah blah) or a friend that dabbles in photoshop?
Leads me to think that it’s not just about the nodes themselves but that the quality of the connection changes according to the subject and the way the nodes view each other.
[…] O’Brien and Henry Copeland at 6.4%, Dr. Walter Carl, Sarah Petersen and Charlene Li at 5.5%, Jim Tobin and Rick Murray 4.5%, Kami Huyse at […]