Skip to main content

https://digitalhealth.blog.gov.uk/2021/03/02/nhse-i-calculating-quality-reporting-service-cqrs-local-alpha-assessment/

NHSE/I Calculating Quality Reporting Service (CQRS) Local alpha assessment

Posted by: , Posted on: - Categories: Alpha, Assurance, Service assessments

Text saying "Service Assessment" with DHSC approved brand colours

From: NHSX
Assessment date: 18 January 2021
Stage: Alpha
Result: Met
Service provider: NHSE/I with the North of England Commissioning Support Unit (NECS)

Service description

CQRS (Calculating Quality Reporting Service) Local is being developed to be a flexible, web based, payment claim system to support colleagues in primary care, PCNs (primary care networks), CCGs (clinical commissioning groups) and NHS England with the aim of streamlining the management processes associated with local incentive schemes. 

The CSU Collaborative will develop and operate CQRS Local, providing training and operating a single point of entry service desk and a central programme management office to manage operations, changes and reporting to NHS England.​

Service wrap will include: application management, IT operations management, service desk, technical management and service governance

This service aims to solve these problems.

  • CCGs/practices are using spreadsheets to manage/claim for local incentive scheme (LES) which is time consuming.
  • CCGs are manually uploading invoices to financial systems.
  • Practices spend a lot of time trying to reconcile LES claim values against their bank accounts to confirm payment.
  • No standardised approach across the country. 
  • No national visibility across all local incentive schemes.

Service users

This service is for: 

  • Clinical Commissioning Groups (CCGs)
  • Primary Care Networks (PCNs)
  • Commissioning Support Units (CSUs)
  • General Practice (GPs) – note: GPs would only utilise the system if they are commissioned to deliver a local incentive scheme by a commissioner using CQRS Local
  • NHS England – Regional Teams
  • NHS England – National Team

Report contents

  1. Understand users and their needs
  2. Solve a whole problem for users
  3. Provide a joined-up experience across all channels
  4. Make the service simple to use
  5. Make sure everyone can use the service
  6. Have a multidisciplinary team
  7. Use agile ways of working
  8. Iterate and improve frequently
  9. Create a secure service which protects users’ privacy
  10. Define what success looks like and publish performance data
  11. Choose the right tools and technology
  12. Make new source code open
  13. Use and contribute to open standards, common components and patterns
  14. Operate a reliable service

1. Understand users and their needs

Decision

The service met point 1 of the Standard.

What the team has done well

The panel was impressed that:     

  • the team has carried out UAT testing and taken on board feedback from the user group to establish user needs for functionality
  • they have done this despite the challenges of researching and developing a service that meets the needs of CCGs and GP practices during the COVID pandemic

What the team needs to explore

Before their next assessment, the team needs to:

  • be clear if the research they are carrying out is testing whether the functionality and technology meets business requirements, or if they are exploring interactions with and general usability of the system. This will help them to prioritise iterations that are being made, and clearly identify areas where users of the system may have difficulty even if the business requirement is being met by the functionality
  • carry out more usability research and user testing, including of the design and content, with a variety of end users. The team should keep a record of this testing, and what design solutions have been tested to solve any problems, and record why design decisions were made based on user feedback
  • conduct research to understand what happens in the offline elements of the user journey, for example what happens if system information is found to be incorrect during an audit? The team should then explore how this affects users' use of the system, and what support could be provided in the process of preventing and rectifying errors with data input

2. Solve a whole problem for users

Decision

The service met point 2 of the Standard.

The panel was impressed that:

  • the team was able to clarify the problem that they were trying to solve and what they are doing to try and solve the problem
  • the team has done user research with their different users to make sure that the service is designed based on user needs
  • the team has an end user working group to ensure user input throughout the process
  • the team is engaged with the business and policy to get input into the service

What the team needs to explore

Before their next assessment, the team needs to:

  • make sure that user needs are the primary focus and push back on the business if necessary
  • explore whether the system could integrate better with GP IT systems to improve the user journey

3. Provide a joined-up experience across all channels

Decision

The service met point 3 of the Standard.

What the team has done well

The panel was impressed that:

  • the service team understands the problem they are trying to solve and have worked with users to make sure the service suits everyone
  • the team is confident in the service moving to Beta
  • policy has contributed to the service and is engaged with the team
  • the team plans to monitor digital take up
  • the team has made sure that CQRS Local aligns with the national product and so is familiar to users
  • the team is working with communications teams, trainers and the service desk to join things up for users

What the team needs to explore

Before their next assessment, the team needs to:

  • explore ways to improve digital take up
  • continue user research and end-to-end usability testing with a variety of users
  • avoid being unduly constrained by CQRS National by sharing learnings with the National team, including content and design findings, so that both services can respond to user needs together

4. Make the service simple to use

Decision

The service met point 4 of the Standard.

What the team has done well

The panel was impressed that:

  • the team is committed to developing a simple, intuitive service
  • the team identified their riskiest assumption (tariffs) and have addressed this and tested their solution
  • they have iterated the service to better reflect the way their users work

What the team needs to explore

Before their next assessment, the team needs to:

  • minimise the need for documentation, training and support by testing and iterating the interface and creating something more intuitive
  • test and develop the help documentation with users to make sure it addresses the points where they get stuck
  • review the content in documentation and training materials, as well as the service, against the content style guide in the NHS digital service manual
  • (while recognising that this service is tied up closely with CQRS National) carry out a review of the styles, components and patterns in the NHS digital service manual and identify the extent to which they can follow them - the service should reflect these styles rather than the NHS identity guidelines
  • make sure that elements are consistent across the service and, where possible, bring them (and CQRS National) into line with the NHS digital service manual
  • iterate their designs based on user research

5. Make sure everyone can use the service

Decision

The service met point 5 of the Standard.

What the team has done well

The panel was impressed that:

  • the team has considered and responded to the needs of the groups that will use the service
  • the team is carrying out accessibility testing

What the team needs to explore

Before their next assessment, the team needs to:

  • review the language used in the service and test it to make sure that users understand it
  • continue to test the service with users with poorer digital skills
  • carry out research with users with accessibility needs to make sure they can use the service
  • conduct an accessibility audit

6. Have a multidisciplinary team

Decision

The service met point 6 of the Standard.

What the team has done well

The panel was impressed that:

  • the team has many of the roles required in a typical multidisciplinary team - delivery managers, developers, QA, architects, solutions analysts
  • while the team does not have dedicated user researchers and interaction, content and service designers they have solutions analysts who pick up these tasks
  • the team is engaged with the project management office and stakeholders and keep these parties updated on progress and involve them in the service design

What the team needs to explore

Before their next assessment, the team needs to:

  • consider bringing in dedicated UR/UX roles as having these specialisms in the team would be beneficial
  • consider bringing in a content designer for a content review of the service to ensure it is in line with standards and is easy to understand
  • during a service assessment it would be advisable to bring the other roles in to lead on the different areas, for example whoever did the user research should present and answer questions in that section 

7. Use agile ways of working

Decision

The service met point 7 of the Standard.

What the team has done well

The panel was impressed that:

  • the team have adopted appropriate agile ways of working including working in fortnightly sprints and holding the appropriate agile ceremonies (sprint planning, stand ups, retrospectives)
  • the team has a show and tell at the end of every sprint
  • the team has addressed the challenge of remote working by using Microsoft Teams and Slack
  • the team uses Azure to hold their product backlog and the product owner and delivery manager prioritises the backlog

What the team needs to explore

Before their next assessment, the team needs to:

  • try using Miro for their retrospectives and workshops as this is a great collaborative tool 

8. Iterate and improve frequently

Decision

The service met point 8 of the Standard.

What the team has done well

The panel was impressed that:

  • the team were able to provide examples of where they have iterated based on usability testing and the end user working group 

What the team needs to explore

Before their next assessment, the team needs to:

  • continue usability testing and iterating based on feedback
  • be prepared to demonstrate how they have iterated the design and content based on what they have seen in user research
  • start collecting performance data and analyse this to identify whether any improvements can be made based on that

9. Create a secure service which protects users’ privacy

Decision

The service met point 9 of the Standard.

What the team has done well

The panel was impressed that:

  • the team has considered the potential security risks to the service an taken steps to mitigate them utilising Azure’s security components  
  • the service leverages existing NHS Mail single sign on where possible for authentication
  • potential fraud risks have been identified, role based access and audit logs are in place to mitigate them

What the team needs to explore

Before their next assessment, the team needs to:

  • undergo an external penetration test, which they have booked, and address any issues coming out of the report

10. Define what success looks like and publish performance data

Decision

The service met point 10 of the Standard.

What the team has done well

The panel was impressed that:

  • the team has identified their key performance indicators including the four mandatory GDS KPIs and some of their own 
  • the team plan to use dashboards with Azure to monitor the performance of the service. The Lead Delivery Manager will be monitoring the performance 

What the team needs to explore

Before their next assessment, the team needs to:

  • consider how they can use the performance data to improve the service i.e. how can digital take up be improved
  • consider using Google Analytics or Power BI to monitor performance
  • consider having an exit survey to generate some qualitative data about user satisfaction

11. Choose the right tools and technology

Decision

The service met point 11 of the Standard.

What the team has done well

The panel was impressed that:

  • the team has selected a technology stack that they are experienced with and are confident will meet user needs
  • the team considered alternative options including using angular. However, it was deemed unnecessary for this service  

What the team needs to explore

Before their next assessment, the team needs to:

  • continue to assess that the choses technology stack is suitable for the service

12. Make new source code open

Decision

The service met  point 12 of the Standard.

What the team has done well

The panel was impressed that:

  • the team has a plan in place to open source code as appropriate before public beta, and the repository is already in place

What the team needs to explore

Before their next assessment, the team needs to:

  • host source code in a public repository, minus any sensitive information   

13. Use and contribute to open standards, common components and patterns

Decision

The service met  point 13 of the Standard.

What the team has done well

The panel was impressed that:

  • the team is making use of open source technology where possible

What the team needs to explore

Before their next assessment, the team needs to:

  • the team should consider how they can share any components or patterns they develop with the community, including the NHS digital service manual community

14. Operate a reliable service

Decision

The service met  point 14 of the Standard.

What the team has done well

The panel was impressed that:

  • the team has a disaster recovery plan in place   
  • the service has an active passive architecture utilising data centres in two UK locations     

Sharing and comments

Share this page