Digital Workplace Logo

News

You are here: Home » Type of post » Expert blog » The art of rating

Thanks for visiting the Digital Workplace Group (DWG) website. You'll see this post may refer to the "Intranet Benchmarking Forum (IBF)," the "Digital Workplace Forum (DWF)" or "IBF Live." But that doesn't match our website name!

In a nutshell, we merged IBF and DWF into one service and changed our name to "Digital Workplace Group." The new name represents the broader set of services we've grown to offer, beyond an original focus on just intranets. We also changed the name of our monthly webinar from "IBF Live" to "Digital Workplace Live."

Although we've relabelled things, we're proud of our decade+ history and have left this page intact. Enjoy your time on our site and please contact us with any questions or comments.

In this new series of blog posts, Chris Tubb, one of the IBF Benchmarking team will take a look at ways that small elements of the digital workplace can be improved through design.

When we are benchmarking member intranets at IBF, feedback and rating often comes up as an area for improvement. As more intranet teams use content management systems that are “out-of-the-box” I fear there will be less and less design put into crafting thoughtful designs that really add value, without adding much cost. Let’s start the series with rating, which seems to me often thrown into the mix as an afterthought.

The remainder of this article is available to DWG members on the extranet.

 

Chris Tubb is a lead benchmarker for IBF in the Management Benchmarking Areas. He is an intranet and digital workplace consultant with a background in information architecture. He worked with intranets for Orange and France Telecom for 13 years. He occasionally twitters @christubb

8 Comments

  1. Excellent article Chris. You’ve provided really useful ideas about how to improve the implementation of content ratings, as well as offered good arguments.

    “LIKE” VS. RATING

    The topic of “likes” is a sticky one. The benefits are that i) people are familiar with the concept via Facebook and other sites and ii) it is simpler than rating and requires lower cognitive effort.

    With ratings an employee has to take a bit of time to consider how many stars an article deservers. They have to consider what would be five stars, what would be one star and how the article in question fits into that scale. In some cases this effort is quite a turn off or seems erroneous.

    That being said, by offering a more explicit rating system with two different scales you may reduce the cognitive effort by providing more explicit instruction.

    One question: Why include a “send” button rather than letting people click on the stars to submit their ratings, such as in iTunes and on Netflix? I’m really curious to hear your thinking for that.

    Reply
    • I think audience size is important,too. I’m curious if there is a typical rate of participation for various systems. If 1 in 100 take the initiative to rate, then I won’t get decent data on pages with less than 1000 visitors.

      Reply
      • Bert Sandie

        Audience size and composition is extremely important to consider. We need to remember that we have a capped audience size inside the walls of our companies. We all have probably heard of the 1-9-90 (creators, raters/commenters, lurkers) rule – even if we scale this slightly more aggressively to 5-15-80 which is true of some internal communities then we still have a small sample size of people rating content.

        I prefer views and likes for the social aspect of driving content usage inside a company. In terms of providing feedback on content it is much better in my opinion to:

        (a) create templates for employees to start from (i.e., for article)
        (b) provide “gold-standard” examples of real content to aspire to
        (c) ask the employee if they want feedback on their content = a review prior to posting or perhaps after

        Ensuring early on that the early adopters create great content will help create the standards for people to follow.

        Reply
  2. Bert Sandie

    There is a study that YouTube published a few years ago about their own analysis of the use of 5-star ratings they provide – the study showed that only a small percentage of viewers (in many cases statistically too small to be provide any conclusions) rate and those that do predominantly use 5=great followed 1=bad and 2,3 and 4 are a valley. Their conclusion is that the rating serve a small purpose at best.

    Think about how people actually use something like YouTube – they search for a string such as “U2” then they look for the video with the most views, most/many people do not even look at the rating anymore. So what does the rating really buy you?

    I would recommend based on personal experience at EA and other like-minded companies that a better solution is to use:

    (1) View – track and show the number of views for each piece of content
    (2) Allow comments on any piece of content
    (3) If you really feel you need a rating system simply default to the “Like” button system – there is a reason FB does this

    I will leave you with a short story to show you why I feel that a 5-star rating system has issues…imagine a new grad posts a great article that is intended for other new grads but that is not obvious, a number of veterans read the article then rate it as a “1=bad”, the new employee (new grad) may never post again, and the article will probably not get read by its target audience.

    Ratings are a tricky feature set to get right – I suggest walking through some common use cases to get to something that works at your company.

    Bert Sandie

    Reply
  3. We need to be clear that ratings and “like” buttons have two very different functions.

    The rating scale provides feedback for the content provider. You may increase the value to the originator by choosing to measure different dimensions, but there will be a drop-off with the increased complication. I’d question the value of using Likert scales in the context of the action you plan in response. (“Write more 4 content!”?) Anecdotal is more valuable – you’ll get fewer responses, but they’ll give you more insight.

    The “Like” button exists for one reason: to move information through a social network. It is simple, lightweight and easily understood. If I “like” something, I’m signalling to everyone connected to me that I’ve found something useful or interesting and that, dependent on the nature of our relationship, they might also find it useful.

    When I see that a contact in my feed has “liked” something, I may go and check it out. I don’t need to know whether they dislike something – that’s implicit in the fact that they don’t click “like”.

    The “like” button is not primarily a way of evaluating content – it’s a way of sharing content. It’s important not to confuse the two.

    Reply
  4. Very nice post, zooming in on specifics of giving feedback.

    I totally believe that giving feedback will be an increasingly important role in the social economy/ecology as we will need to become better and better at filtering ever increasing info-glut.
    And your kind of articles with very precise tips and tricks comes in very handy there.

    I’m all up for feedback (and fun and focus): http://km4meu.wordpress.com/2012/01/09/a-new-year-of-fun-focus-feedback-and-some-new-ideas/

    Cheers!

    Ewen

    Reply
  5. Cheryl Lesser

    Chris – I only use ratings if I’ll be using the ratings later, for example, I’m looking to buy a new house so in my house-search app I’ve indicated some with 5,4,3,2 & 1 stars for prioritizing my visits.

    However for corp comm articles I don’t want to take the time to analyze them the way I would my own content. I *might* take the time to “like” them if I think they’re particularly useful. But I don’t see myself going in later and filtering out all the articles I’ve rated 5 stars, etc.

    So in my scenario “what’s in it for me?” is a key driver in providing feedback (or maybe I’m more selfish than most :-).

    Reply

Leave a comment

Are you ready for the digital workplace?
Learn and explore the new digital world of work.
Your information will never be shared with any third party.
Are you ready to connect with DWG?
Learn and explore the new digital world of work.
Your information will never be shared with any third party.
I hereby acknowledge that the Digital Workplace 24 Video Library is designated for my professional use alone and may not be shared with any other parties, in whole or in part, without express written permission from the Digital Workplace Group.