From: NHSX
Assessment date: 3 December 2020
Stage: Alpha
Result: Met
Service provider: NHSE/I
Service description
The NHS Innovation Service is a service for innovators to access a path to market, and for organisations to review and triage innovations to help identify and accelerate the most promising innovations to market.
The service has identified the current broad problem areas as reducing the time taken for innovations to get into the NHS, the uptake of innovations in the first year compared with international peers and to help the system have a balanced view to risk for innovations.
Broadly, the different user groups will use the service as follows.
Innovators will use it to:
- Get information relevant for their innovation(s) collated in one place
- Find out what their gaps are and get signposted to information they need
- Register their innovation with several organisations once
- Get feedback on what they need to do next
- Update information on their innovation to signal they are ready for more feedback / support
Accessors will use it to:
- Coordinate on the most appropriate bespoke support for an innovation
- Hand-over innovations through different stages of the innovation pipeline
- Make strategic decisions on what to support by having an overview of what’s in the innovation pipeline and what the gaps are
Service users
The users will be a range of innovators looking to develop and sell products or services to the NHS. These include:
- Priority – early stage innovators, for example those developing a product or service
- Priority – late stage innovators, for example those with a product or service ready to promote
- Clinical innovators
- Commercial innovators
- Serial entrepreneurs
As well as accessors from healthtech organisations who support innovators. These include:
- Academic Health Science Networks (AHSNs)
- National Institute for Clinical Excellence (NICE)
- National Institute for Health Research (NIHR)
- NHS Supply Chain (NHSSC)
Report contents
- Understand users and their needs
- Solve a whole problem for users
- Provide a joined-up experience across all channels
- Make the service simple to use
- Make sure everyone can use the service
- Have a multidisciplinary team
- Use agile ways of working
- Iterate and improve frequently
- Create a secure service which protects users’ privacy
- Define what success looks like and publish performance data
- Choose the right tools and technology
- Make new source code open
- Use and contribute to open standards, common components and patterns
- Operate a reliable service
1. Understand users and their needs
Decision
The service met point 1 of the Standard.
What the team has done well
The panel was impressed that:
- the team has built a good understanding of innovators, recognising that needs amongst the user group can vary significantly. In addition, the team has developed a good understanding of accessors, and the support they provide to innovators within their user journey
- the team has mapped the user journeys of the user groups, including pain points and opportunities for improvements
What the team needs to explore
Before their next assessment, the team needs to:
- continue to explore other sources of data (e.g. quantitative insight) to understand their user base, and use this information to continue their development of the service
- test the full end to end service with users. This includes ensuring that innovators have a clear method of accessing the service, and that the journey within the service meets the different levels of support that innovators are likely to need
2. Solve a whole problem for users
Decision
The service met point 2 of the Standard.
What the team has done well
The panel was impressed that:
- the team has mapped channels for users and researched where users can be reached
- the team identified the different focus and strengths of different accessor organisations
- the team developed a high level process view of how the service, triage and review will work
- the team engaged a number of different users, organisations and stakeholders involved in the service
- the team focused the service on the particular cohorts to realise benefits for end users and avoiding “boiling the ocean”. This was expressed through the focus on three archetypes ensuring the service is focused and avoids trying to meet too many divergent needs
- the team listened to feedback about chunking up work into MVPs, with multiple references to MVP development including in the service blueprint, process map, dependencies and looking forward to MVP development for private beta
What the team needs to explore
Before their next assessment, the team needs to:
- be mindful of the impacts of Brexit on the overall service. The team should consider whether any direct or indirect impacts arise out of a deal or no deal Brexit scenario. For example, will the service require any specific Brexit guidance or factor in other potential implications for the service?
- pay close attention to how they transition to a new service where others existing services are decommissioned. The team should be considering how to manage or inform users navigating or reviewing multiple channels how this system fits in the wider innovation ecosystem. What happens if an innovator or accessor are reviewing duplicate innovations with different organisations? These questions should be explored further at the next phase and through further user research
- continually learn from where the existing innovation ecosystem can be improved upon. The research highlights the issues around the length of time it takes innovations to get into the NHS and the associated costs and uptake in the first year. Noting the comparison in reaching international benchmarks the service should continually focus its testing, learning and iterating on how this service can help remedy the identified issues hindering innovation into the system
3. Provide a joined-up experience across all channels
Decision
The service met point 3 of the Standard.
What the team has done well
The panel was impressed that:
- the demonstration of the service showed that research and thought had been applied to constructing the user facing (UI) interfaces
- they had tested with different user groups on the proposed system to get feedback and iterate the service
- the team had mapped out pain points of the existing system to identify with greater detail the issues that are impacting innovation being accessed into the NHS
What the team needs to explore
Before their next assessment, the team needs to:
- continue to develop their understanding of the offline elements of innovators and accessors journeys to establish how the service may be able to support these elements
- continue to refine their measurement of success for innovators and accessors using the service
4. Make the service simple to use
Decision
The service met point 4 of the Standard.
What the team has done well
The panel was impressed that:
- the team have prototyped user journeys within the service, and have tested these with users
What the team needs to explore
Before their next assessment, the team needs to:
- explore how innovators will create an account for the service, and whether it is appropriate or necessary for users accounts to require validation and revalidation if they are unused
- continue to develop their understanding of some of the complex user journeys within the system, particularly those who require longitudinal support
5. Make sure everyone can use the service
Decision
The service met point 5 of the Standard.
What the team has done well
The panel was impressed that:
- the team has undertaken rounds of research with users who have accessibility needs
- the team has created an offline innovators pack to give advice to those who can’t or don’t want to use the service
What the team needs to explore
Before their next assessment, the team needs to:
- test the service end-to-end with all users to ensure it meets each user base’s needs
- ensure that the innovators pack meets the needs of users not using the online service, and iterate where necessary. The team should also explore whether there are alternative methods of offline support beyond this
6. Have a multidisciplinary team
Decision
The service met point 6 of the Standard.
What the team has done well
The panel was impressed that:
- the team has put in place a multidisciplinary team consisting of two product owners, a user researcher, service designer, product designer and architect for the service
- the team has listened to feedback and recruited a user researcher as well as other DDaT roles to support the service. Since the last review the team has incorporated feedback to strengthen the multidisciplinary expertise within the team
What the team needs to explore
Before their next assessment, the team needs to:
- ensure the roles within the team are relevant throughout at each phase of the service. One particular role that will become increasingly important as the team progresses is to get in place a content designer to support the team’s work and ensure that user facing content is written using that expertise and supporting user research insights
- develop an ongoing review plan for the expertise in the team needed to support the service. As the team progresses they should put in place a clear plan of the roles needed as the service iterates through the lifecycle of the project
7. Use agile ways of working
Decision
The service met point 7 of the Standard.
What the team has done well
The panel was impressed that:
- the team use agile ceremonies to deliver their work, including daily standups, retros, project planning, setting direction and kanban
- the team share their work through showcases to share learnings and also learn from those attending
- the team listened to feedback to expand their ceremonies to include planning and retrospectives which shows they have been open to listening and adapting how they incorporate agile working into the service development
What the team needs to explore
Before their next assessment, the team needs to:
- continue engagement with users and stakeholders involved in innovation. The team should build on their current engagement plan to ensure users and organisations are invited and can input to the service development via showcases and show and tells. The complex innovation landscape only adds to the importance of the team engaging widely with users and organisations to ensure this service uniquely adds value
8. Iterate and improve frequently
Decision
The service met point 8 of the Standard.
What the team has done well
The panel was impressed that:
- the team have listened to feedback to test different prototypes with users
- the team demonstrated that they are iterating the service with accessor user groups to refine and iterate the service
- the team has set out that the priority for beta is to iterate the UX in response to feedback from users
What the team needs to explore
Before their next assessment, the team needs to:
- have a clear plan to iterate and test the service against the user stories and acceptance criteria they develop. The team should develop a clearer picture, through testing and iterating, of when specific user needs, linked to user stories and acceptance criteria, have been met by the service as they move into private beta
9. Create a secure service which protects users’ privacy
Decision
The service met point 9 of the Standard.
What the team has done well
The panel was impressed that:
- security acceptance criteria has planned to be included in stories in Beta with a section in every end-to-end report before signing
- the team understands the potential sensitivity around the commercial data the service might hold on innovators
- the team has explored different avenues for identity management, authentication and authorization
- the team has done a rigorous study identifying the potential threats for the service, and data stored
- the team has explored which potential options for consent can be best presented to the innovators
What the team needs to explore
Before their next assessment, the team needs to:
- understand which security controls the service must meet and engage with the relevant governance to ensure the service is compliant
- further explore the options that NHS login offers by engaging with the relevant teams within NHS so they can benefit from their work
- continue the work started in Alpha evaluating potential threats and risks to the service and testing those, and their mitigation plans, in Beta
- test the best approach to ensure innovators consent for their ideas to be shared. This should inform the data governance process and any potential migrations needed within the service
10. Define what success looks like and publish performance data
Decision
The service met point 10 of the Standard.
What the team has done well
The panel was impressed that:
- the team have set out 7 key performance indicators, including completion rate, cost per transaction, digital uptake, user satisfaction, speed of transition through idea to adoption, patient outcomes and savings to the health system
What the team needs to explore
Before their next assessment, the team needs to:
- focus on developing user stories with acceptance criteria for their different users to enable the service to articulate and define KPIs that identify if the service is meeting user needs
- continually test progress against the original problem statement for the service. This would likely include exploring any further KPIs that gauge progress against tackling the specific identified hindrances to innovation, including time, uptake and other identified existing problems
11. Choose the right tools and technology
Decision
The service met point 11 of the Standard.
What the team has done well
The panel was impressed that:
- the team has explored different languages, frameworks and services
- the alpha prototypes were built using the NHS.UK prototype kit
- the front-end development will be based on the NHS.UK Frontend library
- the service will be hosted in the public cloud
- CMS discovery has been carried out thoroughly and logically
What the team needs to explore
Before their next assessment, the team needs to:
- identify the technical governance bodies within the organisation and engage with them as soon as possible to ensure consistency in their technology choices is achieved and team skills are available for future support, maintenance and continuous improvement
- explore which alternatives might be suitable for JavaScript to make the service accessible. If none are found, then ensure there is a suitable offline solution as the users should be able to complete the core tasks of the service even if JavaScript has been disabled
- ensure automated testing and deployment is achieved in Beta
- test the beta service with users and prove whether there is benefit in having HTC data (or from any other existing services) in the new innovation service. If that is the case, define a clear data migration strategy and integration points
- have a clear understanding of where the service fits into the wider innovator landscape and communicate this clearly with users. Additionally, the service should continue to work with other innovator organisations to understand where this service can uniquely add value
12. Make new source code open
Decision
The service met point 12 of the Standard.
What the team has done well
The panel was impressed that:
- the team intends to open source the production code (Infrastructure-as-Code) in an open repository
- the Beta team would be empowered to choose which repository would be used (GitHub or Azure Repos). At the moment the team prefers Azure Repos because it would sit in the same ecosystems as the rest of the CI/CD tooling
What the team needs to explore
Before their next assessment, the team needs to:
- ensure the open repository is kept up to date and explore which option (GitHub or Azure Repos) proves more accessible for the wider government community
13. Use and contribute to open standards, common components and patterns
Decision
The service met point 13 of the Standard.
What the team has done well
The panel was impressed that:
- the alpha prototypes were built using the NHS.UK prototype kit
- the front-end development will be based on the NHS.UK frontend library
- the service team plans to GOV.UK Notify for automatic notifications
- the service will be hosted on the NHS domain (nhs.uk)
What the team needs to explore
Before their next assessment, the team needs to:
- consider making contributions to open source libraries as part of the development process.
14. Operate a reliable service
The service met point 14 of the Standard.
What the team has done well
The panel was impressed that:
- monitoring, diagnosis and alerting have already been considered with Azure Monitor being the preferred option.
- the team has worked on how to make deployment secure and what mitigations might be needed, focussing on having a platform-based secret-as-a-service, automation of testing and deployment, continuous improvement, and cloud hosting.
What the team needs to explore
Before their next assessment, the team needs to:
- formalise a plan for monitoring and alerting that will be shared with and approved by the wider NHS relevant technical support teams and agree SLAs to operate the service in a sustainable way
- regular penetration and load testing should be anticipated to check how secure the system is and how well it can cope with demand
- ensure relevant documentation is accessible and up-to-date
- carry out quality assurance testing as part of the ongoing service development
- develop a plan of how the service will be supported. This would include understanding how the service will be supported in and out of regular hours and fielding queries
- consider what would happen if the service goes down out of hours and how to mitigate the resulting issues to support users