I mentioned that I wanted the DH digital communication team to aspire to run the most effective digital communication operation in government.
It's a nice ambition, but it's pretty meaningless unless we can find ways to rate our performance.
We ran a little workshop the other day with the DH digital comms and publishing teams to identify things that we can measure to give us an indication of how we are doing.
The team came up with 13 simple performance indicators. We've set ourselves some targets so that we can measure our performance against them in March and August next year.
Of course, we weren't able to identify absolute measures of success, and some of the KPIs might seem a bit arbitrary. But they should at least give us an indication of how we're doing.
I've listed the KPIs we came up with and some commentary below.
1. Comparison to peers
KPI: Mentions in government blogs
This is pretty easy to measure. Counting mentions for DH digital comms in a sample of the Public Sector bloggers feed should give us an idea of how influential our work is, compared to say BIS or the FCO.
2. Digital hero
KPI: Sentiment of Twitter references for our digital engagement lead
We can use Social Mention to measure this. As I write this, the positive/negative sentiment for @timolloyd is 8:1. To meet our target he needs to outperform @treepixie , @juliac2 and others.
3. Efficiency
KPI: Percentage reduction of cost-per-visit in the 2011 report on cost, quality and usage
Like all central government departments, we published the total cost of our website, and the total number of visitors as part of the review of central government websites last year. Our target is a 50% reduction in cost per visit.
4.Types of digital content
KPI: Number of relevant results for “Department of Health” and “blogs” in first page of google.co.uk search
We're not particularly renowned for blogs at DH, but my team would like us to be. Or at least they would like for us to be renowned for what blogs represent: a culture of personal digital engagement. It's easy to measure - just count the relevant results.
5. Audience engagement
KPI: Volume of referrals to dh.gov.uk
We can use the total volume of referrals to dh.gov.uk as an indication of how interesting our content is - if it's interesting, people will link to it. We reckon we can double the volume of referrals in a year, largely just by producing interesting bespoke web content that people want to reuse.
6. Platform
KPI: Invitations to talk at conferences about our web platform
If we do interesting things with our web platform - either to save money or to deliver innovative solutions to problems - then people will invite us to share the experience. I found out all about the impressive work Defra have done with their web platform at Word Up Whitehall. The number of invitations we get to talk at things like this will be some kind of an indication of our success.
7. Social media engagement
KPI: Volume of retweets/mentions for our main Twitter channel
We want to provide engagement opportunities using the full range of social media, but engagement with our corporate twitter channel is a decent measure of how well we do this stuff. We think we can double the volume of retweets and mentions this year.
8. Personal development
KPI: Number of people in the digital communication team with “a broad range of digital communication skills” on their CV.
My team were clear that that the most marketable skill for a digital communication professional in government is to be a good all-rounder with a broad range of skills, from copywriting to video production. It's fairly easy to measure this through self assessment (loathe as I might be to encourage my team to start writing new CVs).
9. Internal campaign
KPI: Positive answer to the question: “Do you understand the role of the Digital communications team?”
Like most digital comms teams, a large part of our job at DH is still about culture change, and selling the benefits of digital communication to the organisation. We call this our internal campaign, and we have a few prime targets, from the people at the top of the organisation to staff in key policy teams. We can measure this by asking a sample of our targets to complete a 1 question survey.
10. Staff engagement
KPI: Referrals to homepage features (corporate messages) on the staff engagement channel
We want to run an effective internal comms operation, making use of collaborative and social tools, and delivering corporate messages. We want to reach everyone in the organisation with these messages so our target is straightforward and we can measure it by counting the unique visitors to our messages.
11. Solving policy problems
KPI: Number of completed case studies showing how digital communication has solved policy problems
We plan to publish case studies showing how we have used the web to solve policy problems. Our target is at least 5 by March.
12. News and press
KPI: Number of examples of press officers including digital communication in media handling notes
Our relationship with our press officers is really important - our work needs to be properly integrated. So we need to factor journalists into our digital plans, and we want all the media handling notes that press officers draft to cover digital media as well as press.
13. Strategic campaigns
KPI: Sentiment of comments about our priority campaign on target websites
The biggest strategic issue for the department is the transition to a new health and care system, as described in the NHS and public health white papers. We're designing a strategic digital campaign to help deliver this transition, and our success on this will be the real measure of our value to the organisation over the next couple of years. We have another set of KPIs to measure this, but measuring the sentiment of comments from our target groups will give us some indication of how we're doing.
6 comments
Comment by Steph Gray posted on
A great list, Stephen. It's been troubling me for a few days, and I've written a longer form response, full of my own rank hypocrisy, over here: http://www.helpfultechnology.com/helpful-blog/2010/12/how-should-you-measure-the-success-of-a-digital-team/
Comment by Neil Williams posted on
Agree with much of what Steph has written, but also felt this was a pretty great list - and am grateful for your sharing your thinking on this. My team has an awayday next week (austerity away day: it's in the same building, and features lunch in the canteen). I might pick up some of this with them and see what we can feed in, if time.
Comment by Vicky Sargent posted on
This is a really useful post, so thank you. One thing though, I wonder if under efficiency, you had thought of measuring the proportion of visitors able to find/do what they want on your website? The Socitm Website Takeup service (www.socitm.net/info/165/services/31/website_services/5) which meansures usage of and satisfaction with council websites, regards this as the 'killer' measure of efficiency. If people can't do what they want on the website, they will turn to other, significantly more expensive channels, like the phone. That will drive up the department's costs, even if your website cost per visitor falls...be interested to know what you think.
Comment by Bookmarks for December 12th through December 30th | DavePress posted on
[...] 13 measures of success for government digital teams – Good stuff on evaluation of online activity from Stephen Hale [...]
Comment by Will Callaghan posted on
Cheers for the list Stephen, really useful. Obviously some KPIs are more important than others - which are the stand out ones for you?
Comment by Stephen Hale posted on
Thanks Will. I think all of these are just indicators. They'll give us a rough idea about how we're doing as a team over time. The most important KPIs for us though will be more directly related to what we're trying to achieve as a Department - like the specific KPIs around our work on health and care reform, eg around our reach and influence amongst GP pathfinders.