From: NHSX
Assessment date: 26 August 2021
Stage: Beta
Result: Met
Service provider: NHSE/I with BJSS
Previous assessment reports
Service description
The NHS Innovation Service aims to help people with innovative ideas in healthcare (‘innovators’) to understand how they can develop and spread their idea (‘innovation’) in the UK. These ideas can be anything from pharmaceuticals to medical devices, apps, new models of care, and non-clinical services like recruitment and training.
It brings information and support that already exists into one central place, making it easier to find relevant, practical information. Innovators can then create a shared online record of their innovation and be connected with the appropriate organisations for coordinated support.
Service users
The Innovation Service is for innovators and staff that support innovators. The users in each of these groups include the following.
Innovators
Anybody developing an innovative product or service for use in health/social care in the UK, for example:
- Small businesses
- Industry
- NHS Staff
- Academics researchers
- Patients and carers
Support Organisations
Staff at a number of agencies who have an existing role in supporting innovators:
- National Institute for Health Research Office for Clinical Research Infrastructure (NOCRI)
- NHS Supply Chain
- National Institute for Health and Care Excellence (NICE)
- Scottish Health Technology Group (SHTG)
- NHSE Specialised Commissioning
- Life Sciences Hub Wales
- Department for International Trade
- Academic Health Science Networks (AHSNs)
- Health Technology Wales (HTW)
- Medicines and Healthcare products Regulatory Agency (MHRA)
Report contents
- Understand users and their needs
- Solve a whole problem for users
- Provide a joined-up experience across all channels
- Make the service simple to use
- Make sure everyone can use the service
- Have a multidisciplinary team
- Use agile ways of working
- Iterate and improve frequently
- Create a secure service which protects users’ privacy
- Define what success looks like and publish performance data
- Choose the right tools and technology
- Make new source code open
- Use and contribute to open standards, common components and patterns
- Operate a reliable service
1. Understand users and their needs
Decision
The service met point 1 of the Standard.
What the team has done well
The panel was satisfied that:
- a solid user research plan and data collection methods are being used
- the team clearly identified the core users of the service and understands their needs to a good extent, especially the Innovators
- there is a good understanding of the role played by the needs assessment team who are core link to the service between innovators and accessors
- regular workshops, feedback collection, analysis and triangulation take place to iteratively explore user needs
- good baseline research to explore similar or relevant services, for example Health Connect experience review
- early problems are systematically identified with clear actions to take forward
- the use of Digital Accessibility Centre (DAC) for a comprehensive accessibility audit of the entire service was commendable
What the team needs to explore
Before their next assessment, the team needs to:
- use the research to better humanise the service, bringing to life the problems the team are solving. For example, user needs and priority, quotes, specific examples to track end-to-end, personas, and so on
- explore and elaborate the needs of the needs assessor user group. While there is a good understanding of the user groups, the research is too light this group. The needs assessor group is crucial for the routing of the primary users
- take care before going live to fully study the needs assessor group and communicate their role in the process to innovators
- further explore the needs and requirements of supporting organisations to maximise engagement and adoption of the service:
- how are they easily onboarded and offboarded?
- what is the service level agreement (level of commitment end-to-end)?
- what happens if they stop responding to an innovator?
- what are the expected response times?
2. Solve a whole problem for users
Decision
The service met point 2 of the Standard.
What the team has done well
The panel was impressed that:
- the team had explored the different user needs and explored how they could meet the different needs and address the problem across a streamlined service
- the service brings relevant guidance and transactions together in a way that makes sense to users – based on where they are in their journey
- the team have identified a wide network, looking outward to the people and organisations that overlap with their service, and built relationships that will continue throughout the beta phase
What the team needs to explore
Before their next assessment, the team needs to:
- consider how they transition to a new service where an existing service already exists – including making sure orientation is clear to users
- tackle the difficult problem of monitoring this long-term end-to-end journey using the innovative analysis and visualisation skills that they have demonstrated
- continue working with associated organisations to iterate the service - including communications and guidance - for frontstage and backstage users
- make a plan for how training and guidance can be maintained when the service transitions to live
3. Provide a joined-up experience across all channels
Decision
The service met point 3 of the Standard.
What the team has done well
The panel was impressed that:
- the team had good feedback mechanisms in place which were being used to review and improve the service to address problem areas
- the team had considered the support model for the new service, taking account of assisted digital needs
- the team tried to make sure that the different user groups experienced a joined up journey – although this could be documented more clearly
- the team has clearly brought together the core users on the same platform for the same transaction end-to-end
What the team needs to explore
Before their next assessment, the team needs to:
- work to ensure that support for users is embedded in the service and it is clear to users how they access that support if needed
- use data and user research to continually improve the service – both online and offline – and demonstrate how improvements are prioritised
- continue close engagement with operations and other key areas to ensure changes to the service are considered across the full journey
- make a plan for governance/maintenance of the relationship between innovators, assessors and accessors. Should everything be online or are offline relationships allowed? For example, phone calls, presentations, demonstrations and so on
4. Make the service simple to use
Decision
The service met point 4 of the Standard.
What the team has done well
The panel was impressed that:
- the team had the right awareness of the design standards - both NHS-specific and cross-government
- the design had been iterated and the team could clearly articulate the reasons for that iteration - and support it with documentation
- the whole service were being tested and iterated - not just the user interface - including communications, guidance and support
What the team needs to explore
Before their next assessment, the team needs to:
- review where the service adheres or departs from the GOV.UK design community principles of ‘start with what exists’. For styles, components and patterns, ensure that the research findings that led to that departure are clearly documented
- consider where they can contribute evidence-based changes with the community - particularly in the area of administrative interfaces where the service may be able to help fill an existing gap in research-led design
5. Make sure everyone can use the service
Decision
The service met point 5 of the Standard.
What the team has done well
The panel was impressed that:
- the service had considered accessibility standards throughout development, commissioned a DAC report, and had a process for incorporating its findings
- the content had been designed using the principles of the design system, and iterated to ensure that it worked for staff
- the team had investigated the inclusion needs of users, and ensured that there was assisted digital support to cover any gaps, and were satisfied that the support was sufficient
- accessibility testing, using tools such as AXE is baked into the development process
What the team needs to explore
Before their next assessment, the team needs to:
- continue researching and designing for users with different physical, mental health, social, cultural or learning needs
- create a plan for further accessibility assessment and updates before transitioning to live
- consider feeding back their staff-facing content findings to the wider community
6. Have a multidisciplinary team
Decision
The service met point 6 of the Standard.
What the team has done well
The panel was impressed that:
- the team had a ramped up empowered multidisciplinary team in place during beta – with the right skills to be able to develop and test the service based on a considered delivery model
- the team had planned ahead to consider how it needs to adapt for public beta and set out a new proposed structure and way of working, including evidence of onboarding and taking steps to fill a small number of vacancies
- currently a blend of permanent and contractual members the team has already set in place skills transfer to take account of staffing changes, to ensure continuity both of delivery and knowledge
What the team needs to explore
Before their next assessment, the team needs to:
- fill any outstanding vacancies
- demonstrate how the team is empowered to iterate the service, for example have evidence of prioritised changes
7. Use agile ways of working
Decision
The service met point 7 of the Standard.
What the team has done well
The panel was impressed that:
- the service team were delivering using recognised agile ways of working – with regular fortnightly sprint cycles, planned research sessions and retros
- engagement has been well thought through with the right level of detail being shared based on the group and purpose including show and tells and monthly stakeholder showcases
- governance seems proportionate with roles of different groups clearly defined and reviewed regularly
- team had a good plan and approach to promoting the new service to different user groups
What the team needs to explore
Before their next assessment, the team needs to:
- test the service with the minister or relevant senior stakeholder, where appropriate and proportionate
- show how the team prioritises feedback and decide which changes to make to the service
8. Iterate and improve frequently
Decision
The service met point 8 of the Standard.
What the team has done well
The panel was impressed that:
- the user research is prioritised every sprint, and use of hotjar to inform changes to the service, for example guides and additional case studies
- the private beta activities were captured and visualised in a clear and innovative way
What the team needs to explore
Before their next assessment, the team needs to:
- continue researching and iterating features, and adapting their approach as the service users increase
- continue finding innovative ways of visualising the iterations and improvements - particularly thinking about how to do this over a long period of time, as this service’s end-to-end journey could spread over years
- continue capturing specific examples of how the service has iterated based on feedback from users, including some quotes
- continue to ensure the change advisory board adds value and doesn’t slow down the team
9. Create a secure service which protects users’ privacy
Decision
The service met point 9 of the Standard.
What the team has done well
The panel was impressed that:
- the team have carried out both internal and external penetration testing
- OWASP Top 10 testing is used to reduce the risk of introducing defects impacting security of the service in-between external audits
- access to the service and authorisation is utilising Azure AD
What the team needs to explore
Before their next assessment, the team needs to:
- clarify the need and onboard to NHS Digital’s Cyber Defence Operations Centre monitoring service
10. Define what success looks like and publish performance data
Decision
The service met point 10 of the Standard.
What the team has done well
The panel was impressed that:
- the service team had thought about measures of success beyond the standard 4 core ones, including how they would measure them
- the team has identified KPIs that need to be measured both in-service and non-digital channels and has a proposal for how to measure across multiple channels
What the team needs to explore
Before their next assessment, the team needs to:
- use performance data to make decisions about how to fix problems and improve the service
- show examples of how the new service demonstrates improvements that address legacy service problems and show the service is successful (including but not solely related to user satisfaction); this is currently not very clear
- show how the many different measuring methods (and tools) gets brought together to show a comprehensive view of how the service is performing
11. Choose the right tools and technology
Decision
The service met point 11 of the Standard.
What the team has done well
The panel was impressed that:
- the team conducted an assessment of available reusable components and has used those where relevant, but has been able to select the right tool when needed, for example Grav rather than Wagtail CMS
- the service is well architected on Azure, using serverless approached where possible, keeping operational requirements low
- although the team are utilising a front-end framework, they are using server-side rendering to meet progressive enhancement need
What the team needs to explore
Before their next assessment, the team needs to:
- investigate alternatives to Hotjar for feedback due to its inability to meet accessibility standards
12. Make new source code open
Decision
The service met point 12 of the Standard.
What the team has done well
The panel was impressed that:
13. Use and contribute to open standards, common components and patterns
Decision
The service met point 13 of the Standard.
What the team has done well
The panel was impressed that:
- the team has used the NHS Design system, and applying it to internal as well as external interfaces
- making use of a lightweight, open source content management system
- common components are in use where possible such as GOV.UK Notify
What the team needs to explore
Before their next assessment, the team needs to:
- review recommendations in against point 4 about contributing to the design system
- continue to assess the authentication options available - the team noted that existing staff and public services don’t fit the use case of this service
- investigate the use of the emerging supplier and developer portal from NHS Digital
14. Operate a reliable service
Decision
The service met point 14 of the Standard.
What the team has done well
The panel was impressed that:
- urgent help, support desk and training functions are fully integrated into the service design and research planning
- they’ve architected the service to use Azure’s native functionality and components to provide high-availability; Azure AD B2C, Functions and WebApps, as well as CosmoDB and Azure SQL
- the team are making use of monitoring and alerting services to keep track of performance and availability of the service
- even with a low level of user load the team has completed performance testing
What the team needs to explore
Before their next assessment, the team needs to:
- continue monitoring and improving the service design across front and backstage