Measuring internal communications: Don’t get drunk on cheap intranet statistics

When measuring internal communications on the intranet, you can’t rely on basic web statistics to demonstrate value. Don’t settle for the number of comments per article and get side-swiped by an underwhelmed executive.

“A page view is not an idea understood, a behaviour changed or a user engaged.”

I wrote those words in a DWG research paper called “Measuring Intranets” published in 2012. It seemed to strike a nerve. Metrics is an area that is always tough for intranet teams. You either don’t have good numbers or, when you do get numbers, you have no idea whether to trust them or even what they actually mean. Today we publish a follow-up briefing paper for DWG members called Measuring Internal Communications: Targeted metrics that demonstrate impact. The full report is exclusively for DWG members but I’ll give a flavour here and you can download the executive summary for free. No longer the postman, Internal Communications teams are attempting to “win hearts and minds” and “increase engagement”. But what does that mean and how can you tell when you’ve succeeded? If you are struggling to understand what this would mean for your business, maybe with the help of a internal communications agency you’ll get a better understanding. As this has to do with communication, at least you know that by finding out more, you could make a positive change to your business.

The theory, as you know, is that by increasing the engagement of its employees, an organization can increase discretionary effort and therefore improve business performance. It is up to a wide array of people within organizations, from senior leaders and HR professionals to line managers and communicators, to choose how to turn that theory into reality. Communicators attempt to achieve this by using a broad range of channels to influence people’s feelings. The intranet is one arrow in their quiver.

We’re not mind readers

But you can’t read the inner emotional state of your colleagues. And, if you tried, they would tell you to hop it pretty quickly: “How do you feel now?” “Go away, I’m busy.” “Watch this motivational video of the CEO. It will increase your engagement.” “I’m going to call security.” So the best we can do is look for ripples in the pond. In the online world there is a mixture of ripples available to us. The classic metrics such as “page views”, “bounce rates” and “time on page” are the foundations. These provide clues as to whether the content passed the first hurdle: Was it consumed at all? If they didn’t see the content, you have no chance of achieving the communication goal. No eyeballs, no brain time. But even if you did get eyeballs, there is no proof that you got the benefit of what you wanted to achieve. It just means they glanced at it. Eyeballs yes, but perhaps nothing more than a glancing blow to the frontal cortex. The second bunch of clues, which are all the rage at the moment, are indications that the reader has “engaged” with the content. Again, the theory is that someone reads or watches the content and is consequently spurred into action. The ripples in the pond we can see here are things like “numbers of comments”, “numbers of likes”, “rating” and “sharing”. Unless you’ve been hiding in a cave in a mountainous area without Internet access for the past five or six years, you’ll recognize these as being à la mode on the web. With social functionality becoming everyday on intranets these clues are yours to pick.

Hold your horses

So that’s that then. Count the various reactions of the audience, pop it into a report and throw it at the CEO, back in time for tea and medals. Ummm… no. The reactions of the audience might not be bravos, they might well be boos. Let’s imagine you have a vague communication goal to “make people feel great about working for this company” and you splash out on an inspirational video. You bat it up on the intranet and half the audience makes it to the page and watches the vid: The reaction of the audience that did see the campaign may span a range from positive (“Wow!”), through neutral (“Meh”), to negative (“What a waste of bloody money!”).

• A small minority of employees at the extremes will demonstrate their reaction with a like, a share or a comment.
• The neutrals will stiffen their sinews, summon up the blood, close the browser window and get on with their day and rapidly forget about it.

Comments can be both positive and negative so a strict count of these to demonstrate “engagement” is a worthless measure of the sought benefit. Positive reception is more likely to be shown by sharing and liking. Negative reactions can also be inferred by having a look at bounce rates and, in the case of video, how soon the viewers abandoned the video after it started playing.

It’s all about the benefit

Ultimately, measuring anything is fairly pointless unless you know why you are doing so. If you have a communication goal as vague as “making people feel great about the company”, it should hardly come as a surprise that this is difficult to measure. If another communication goal is “Make sure everyone has filled in their performance management agreement by December 1st”, measurement should be less about page views of the campaign and more about the proportion of agreements filled in on time. The choices made will depend very much on the intention of what you need the numbers to prove. For me, it all comes back to one of the finest quotes about statistics ever, widely attributed to the poet Andrew Lang: “He uses statistics as a drunken man uses lamp posts – for support rather than illumination.” The best use of measurement is to figure out what you are going to do better next time, rather than as a way of justifying how great you are. But hey, I can’t tell what you need the numbers for. I’m not a mind reader.

Related research

See Also

• Are you measuring your intranet for the wrong reasons?

Categorised in: Collaboration, Content management, Internal communications, Metrics & measurement, ROI & Business Case, Strategy & governance