From: NHSX
Assessment date: 18 January 2021
Stage: Alpha
Result: Met
Service provider: NHSE/I with the North of England Commissioning Support Unit (NECS)
Service description
CQRS (Calculating Quality Reporting Service) Local is being developed to be a flexible, web based, payment claim system to support colleagues in primary care, PCNs (primary care networks), CCGs (clinical commissioning groups) and NHS England with the aim of streamlining the management processes associated with local incentive schemes.
The CSU Collaborative will develop and operate CQRS Local, providing training and operating a single point of entry service desk and a central programme management office to manage operations, changes and reporting to NHS England.
Service wrap will include: application management, IT operations management, service desk, technical management and service governance
This service aims to solve these problems.
- CCGs/practices are using spreadsheets to manage/claim for local incentive scheme (LES) which is time consuming.
- CCGs are manually uploading invoices to financial systems.
- Practices spend a lot of time trying to reconcile LES claim values against their bank accounts to confirm payment.
- No standardised approach across the country.
- No national visibility across all local incentive schemes.
Service users
This service is for:
- Clinical Commissioning Groups (CCGs)
- Primary Care Networks (PCNs)
- Commissioning Support Units (CSUs)
- General Practice (GPs) – note: GPs would only utilise the system if they are commissioned to deliver a local incentive scheme by a commissioner using CQRS Local
- NHS England – Regional Teams
- NHS England – National Team
Report contents
- Understand users and their needs
- Solve a whole problem for users
- Provide a joined-up experience across all channels
- Make the service simple to use
- Make sure everyone can use the service
- Have a multidisciplinary team
- Use agile ways of working
- Iterate and improve frequently
- Create a secure service which protects users’ privacy
- Define what success looks like and publish performance data
- Choose the right tools and technology
- Make new source code open
- Use and contribute to open standards, common components and patterns
- Operate a reliable service
1. Understand users and their needs
Decision
The service met point 1 of the Standard.
What the team has done well
The panel was impressed that:
- the team has carried out UAT testing and taken on board feedback from the user group to establish user needs for functionality
- they have done this despite the challenges of researching and developing a service that meets the needs of CCGs and GP practices during the COVID pandemic
What the team needs to explore
Before their next assessment, the team needs to:
- be clear if the research they are carrying out is testing whether the functionality and technology meets business requirements, or if they are exploring interactions with and general usability of the system. This will help them to prioritise iterations that are being made, and clearly identify areas where users of the system may have difficulty even if the business requirement is being met by the functionality
- carry out more usability research and user testing, including of the design and content, with a variety of end users. The team should keep a record of this testing, and what design solutions have been tested to solve any problems, and record why design decisions were made based on user feedback
- conduct research to understand what happens in the offline elements of the user journey, for example what happens if system information is found to be incorrect during an audit? The team should then explore how this affects users' use of the system, and what support could be provided in the process of preventing and rectifying errors with data input
2. Solve a whole problem for users
Decision
The service met point 2 of the Standard.
The panel was impressed that:
- the team was able to clarify the problem that they were trying to solve and what they are doing to try and solve the problem
- the team has done user research with their different users to make sure that the service is designed based on user needs
- the team has an end user working group to ensure user input throughout the process
- the team is engaged with the business and policy to get input into the service
What the team needs to explore
Before their next assessment, the team needs to:
- make sure that user needs are the primary focus and push back on the business if necessary
- explore whether the system could integrate better with GP IT systems to improve the user journey
3. Provide a joined-up experience across all channels
Decision
The service met point 3 of the Standard.
What the team has done well
The panel was impressed that:
- the service team understands the problem they are trying to solve and have worked with users to make sure the service suits everyone
- the team is confident in the service moving to Beta
- policy has contributed to the service and is engaged with the team
- the team plans to monitor digital take up
- the team has made sure that CQRS Local aligns with the national product and so is familiar to users
- the team is working with communications teams, trainers and the service desk to join things up for users
What the team needs to explore
Before their next assessment, the team needs to:
- explore ways to improve digital take up
- continue user research and end-to-end usability testing with a variety of users
- avoid being unduly constrained by CQRS National by sharing learnings with the National team, including content and design findings, so that both services can respond to user needs together
4. Make the service simple to use
Decision
The service met point 4 of the Standard.
What the team has done well
The panel was impressed that:
- the team is committed to developing a simple, intuitive service
- the team identified their riskiest assumption (tariffs) and have addressed this and tested their solution
- they have iterated the service to better reflect the way their users work
What the team needs to explore
Before their next assessment, the team needs to:
- minimise the need for documentation, training and support by testing and iterating the interface and creating something more intuitive
- test and develop the help documentation with users to make sure it addresses the points where they get stuck
- review the content in documentation and training materials, as well as the service, against the content style guide in the NHS digital service manual
- (while recognising that this service is tied up closely with CQRS National) carry out a review of the styles, components and patterns in the NHS digital service manual and identify the extent to which they can follow them - the service should reflect these styles rather than the NHS identity guidelines
- make sure that elements are consistent across the service and, where possible, bring them (and CQRS National) into line with the NHS digital service manual
- iterate their designs based on user research
5. Make sure everyone can use the service
Decision
The service met point 5 of the Standard.
What the team has done well
The panel was impressed that:
- the team has considered and responded to the needs of the groups that will use the service
- the team is carrying out accessibility testing
What the team needs to explore
Before their next assessment, the team needs to:
- review the language used in the service and test it to make sure that users understand it
- continue to test the service with users with poorer digital skills
- carry out research with users with accessibility needs to make sure they can use the service
- conduct an accessibility audit
6. Have a multidisciplinary team
Decision
The service met point 6 of the Standard.
What the team has done well
The panel was impressed that:
- the team has many of the roles required in a typical multidisciplinary team - delivery managers, developers, QA, architects, solutions analysts
- while the team does not have dedicated user researchers and interaction, content and service designers they have solutions analysts who pick up these tasks
- the team is engaged with the project management office and stakeholders and keep these parties updated on progress and involve them in the service design
What the team needs to explore
Before their next assessment, the team needs to:
- consider bringing in dedicated UR/UX roles as having these specialisms in the team would be beneficial
- consider bringing in a content designer for a content review of the service to ensure it is in line with standards and is easy to understand
- during a service assessment it would be advisable to bring the other roles in to lead on the different areas, for example whoever did the user research should present and answer questions in that section
7. Use agile ways of working
Decision
The service met point 7 of the Standard.
What the team has done well
The panel was impressed that:
- the team have adopted appropriate agile ways of working including working in fortnightly sprints and holding the appropriate agile ceremonies (sprint planning, stand ups, retrospectives)
- the team has a show and tell at the end of every sprint
- the team has addressed the challenge of remote working by using Microsoft Teams and Slack
- the team uses Azure to hold their product backlog and the product owner and delivery manager prioritises the backlog
What the team needs to explore
Before their next assessment, the team needs to:
- try using Miro for their retrospectives and workshops as this is a great collaborative tool
8. Iterate and improve frequently
Decision
The service met point 8 of the Standard.
What the team has done well
The panel was impressed that:
- the team were able to provide examples of where they have iterated based on usability testing and the end user working group
What the team needs to explore
Before their next assessment, the team needs to:
- continue usability testing and iterating based on feedback
- be prepared to demonstrate how they have iterated the design and content based on what they have seen in user research
- start collecting performance data and analyse this to identify whether any improvements can be made based on that
9. Create a secure service which protects users’ privacy
Decision
The service met point 9 of the Standard.
What the team has done well
The panel was impressed that:
- the team has considered the potential security risks to the service an taken steps to mitigate them utilising Azure’s security components
- the service leverages existing NHS Mail single sign on where possible for authentication
- potential fraud risks have been identified, role based access and audit logs are in place to mitigate them
What the team needs to explore
Before their next assessment, the team needs to:
- undergo an external penetration test, which they have booked, and address any issues coming out of the report
10. Define what success looks like and publish performance data
Decision
The service met point 10 of the Standard.
What the team has done well
The panel was impressed that:
- the team has identified their key performance indicators including the four mandatory GDS KPIs and some of their own
- the team plan to use dashboards with Azure to monitor the performance of the service. The Lead Delivery Manager will be monitoring the performance
What the team needs to explore
Before their next assessment, the team needs to:
- consider how they can use the performance data to improve the service i.e. how can digital take up be improved
- consider using Google Analytics or Power BI to monitor performance
- consider having an exit survey to generate some qualitative data about user satisfaction
11. Choose the right tools and technology
Decision
The service met point 11 of the Standard.
What the team has done well
The panel was impressed that:
- the team has selected a technology stack that they are experienced with and are confident will meet user needs
- the team considered alternative options including using angular. However, it was deemed unnecessary for this service
What the team needs to explore
Before their next assessment, the team needs to:
- continue to assess that the choses technology stack is suitable for the service
12. Make new source code open
Decision
The service met point 12 of the Standard.
What the team has done well
The panel was impressed that:
- the team has a plan in place to open source code as appropriate before public beta, and the repository is already in place
What the team needs to explore
Before their next assessment, the team needs to:
- host source code in a public repository, minus any sensitive information
13. Use and contribute to open standards, common components and patterns
Decision
The service met point 13 of the Standard.
What the team has done well
The panel was impressed that:
- the team is making use of open source technology where possible
What the team needs to explore
Before their next assessment, the team needs to:
- the team should consider how they can share any components or patterns they develop with the community, including the NHS digital service manual community
14. Operate a reliable service
Decision
The service met point 14 of the Standard.
What the team has done well
The panel was impressed that:
- the team has a disaster recovery plan in place
- the service has an active passive architecture utilising data centres in two UK locations