PR Industry: please get it right – outputs do not equal outcomes

Delahaye’s quarterly reputation index came out again this week. I have absolutely nothing against this PR industry metric company, but am baffled and confounded with this specific gauge of corporate reputation.

The Delahaye Index compares and contrasts America’s 100 largest companies’ corporate reputations based on how the media cover them. AT&T is the big winner this time around because of the highly publicized launch of the iPhone.  AT&T received top honors solely because it obtained the best quality media coverage in such categories as stakeholder relations, financial management, products and services, organizational integrity and organizational strength.

What doesn’t measure up is that this Index tries to create a direct bridge between AT&T’s excellent media coverage and having the top reputation among the media. Unfortunately, that bridge simply doesn’t exist because this Index never actually polls the media on their actual perceptions of AT&T versus P&G, GE, Microsoft, etc. Instead, what this research does is simply provide outputs that demonstrate that AT&T did a great job this quarter of building quality media coverage. That’s it. If we truly wanted to understand how these companies’ reputations have changed among a key audience (like consumers or media), then we’d need to conduct detailed qualitative or quantitative research gauging their perceptions and then conduct benchmark surveys each subsequent quarter based on the same variables.

Can one make the indirect assumption that generating positive media coverage will ultimately lead to a better reputation (or a better reputation with the media in this case)? Sure. But, what Delahaye has generated (and our industry continually does) are outputs, not end business or reputation outcomes that most CMOs, and certainly all CEOs, actually care about. Thus, the name of this particular Index is misleading.

Positive outputs in the PR World are quality media “hits,” great speaking platforms, lots of traffic to a web site, etc. Outcomes are typically those business goals such as building a better corporate reputation, sales, obtaining new investors, raising stock price, etc., that business leaders actually care about. I’ve said it before, and will now write it again. Until our industry starts to call a spade a spade by actually treating outputs as positive indicators that can lead to real outcomes, but not actually the end factors themselves, we won’t be taken seriously.

Comments

[Alan Chumley]

Absolutely agree. It's akin to the old claim that reach equals influence. It's a massive leap of (blind) faith in my view. Reputation is a beast and it's driven by so much more than media coverage alone.

[Michael Moed]

Similar to a comment I made in a previous blog - the type of meaningful research that you preach can't be ascertained without expending significant resources, and only the well-trained eye (like yours) pays enough attention to know the difference. So there's little incentive for the company generating the metric to improve. In their mind (and pockets) they are sufficiently serving their customer base.

[ed]

Sadly, your comment is true. But, what makes this all the more appalling is that our trade publications give these studies real credibility by featuring them whenever they are released.

Most Chief marketing officers understand pretty quickly that this information is pretty meaningless. It only makes our disclipline seem superficial and not at all strategic.

Think about this particular one. It is beyond wrong. AT&T has the best reputation among media this quarter? That's absurd. AT&T has been continually ripped to shreds for its lack of coverage vis a vis a the iPhone. Also, AT&T is seen as a slow, outdated company. How could best of class and AT&T ever be used in the same sentence?

[kdpaine]

Since I was still at Delahaye when this index started, I might be able to shed some light on this topic. I thoroughly agree with every comment so far. This is nothing more than a the relative number of times AT & T showed up in the context of certain key words that the Reputation Institute decided had something to do with reputation. I find it particularly amusing that while the trades trump "AT&T's top reputation score" Cingular comes in second in the "sucks" category in the blogs. When people's opinions are counted, it's a amazing how different the results can be, eh? A big factor in what drives this score is the circulation of the publications that cover the brand which is why you consistently see such bizaare results.
On the other hand, you get what you pay for. I'm pretty sure AT&T isn't paying anything to be on the list.

[Deepak Aneja]

Wow! To me this seems to be the most important debate on measurement / reputation rankings. The question is, can companies be ranked according to the media coverage they recieve, if this is not the right measure, then how to rank companies as far as their reputation (in media) is concern? Apollo does the same way!

[ed]

Companies cannot be ranked on their reputation solely by analyzing media coverage. Media coverage (quantity and quality) can indirectly help us see that perceptions might be improving or getting worse because many target audiences could be reading those articles.

But, the only true measure of reputation is to actually poll/ask/gauge audiences as to their thoughts/opinions on those companies' reputations.

A more sophisticated method to see if the media program is working would be to measure media outputs every six months and on parallel tracks compare the outputs to a qualitative reputation study of consumers (or whoever the audience is). If we benchmark both of these studies every six months, then we can see if the media relations program is making a dent.

Of course, there are other important factors that influence reputation such as advertising, sales, customer service, etc. But, at least you could use the media measurement as one real metric for reputation.

[Sam Ford]

Ed, very interesting comments here, and I think you are quite right that we have to be very careful to understand what numbers are saying before we accept them as meaning more than they do. I wrote some response to your comments at http://www.convergenceculture.org/weblog/2007/08/the_problems_with_measuring_re.php

[Glenn O'Neil]

Very good debate - I'm always amazed at how many people believe reputation can be judged alone by media coverage. But the media monitoring firms do not help by offering tools that provide all sorts of ways of drilling down and glamorising what is basic output data.

And many agencies that actually do survey research on reputation use proprietary methods to hide behind a relatively simple approach of attributing variables/values/characteristics to organisations.

[Angela Jeffrey]

Ed - I agree with most of everything you've written, and especially, "Until our industry starts to call a spade a spade by actually treating outputs as positive indicators that can lead to real outcomes, but not actually the end factors themselves, we won’t be taken seriously."

We have seen, though, through hundreds of competitive analysis studies utilizing millions of clips - that a company's "share" of positive unpaid media does correlate extremely well with hard business results like sales, survey preference scores, etc. So we do believe there is a strong link between outputs and outcomes, and in some cases when the samples are particularly large, they can serve as a proxy for survey research. However, we are only using outputs as "linkage" data, not as outcomes in and of themselves. Exciting stuff, though!

[Jennifer Hoffmann]

Amen, Angie. It is a shame that "one number" metrics can sometimes undermine the utility of media measurement for understanding reputation and image. Media measurement is most relevant for media relations teams, where media output is often the desired outcome. Even when the goal is in the coverage itself, however, “one number” metrics are confusing and misleading: management rarely likes them, as they require too much explanation and seem to weight their component parts arbitrarily. Linking simpler metrics with financial or survey outcomes makes a stronger case for the influence of PR and communications on reputation.

[emeliajines]

So Come on? Is that really the best that you can do?