From: NHSX
Assessment date: 28 October 2021
Stage: Beta
Result: Conditionally met
Service provider: NHSE/I with Liquid Light
Condition of meeting the standard: the service will meet the standard on the condition that the team do more accessibility testing. Please provide evidence of this additional accessibility testing to the NHSX assurance team, who will pass it on to the panel for review.
Previous assessment reports
As this is a small and niche service, the NHSX Assurance Team decided that only having one assessment is proportionate. This service did not have an alpha assessment and it is unlikely to need a live assessment.
Service description
Since October 2018, the Test Directory has been published on the NHS England and NHS Improvement website as two Excel spreadsheets and a PDF document.
The Genomics Unit within NHS England and NHS Improvement developed an online, searchable and interactive system to host the full content of the National Genomic Test Directory, which can integrate with existing systems and processes in place within the NHS GMS. It hosts and displays the full content of the National Genomic Test Directory via an interactive website. The website is searchable, enables filters to be applied to identify the appropriate clinical indication, and hosts the full content for rare and inherited disease and cancer, including eligibility criteria, in the same place with a single point of access.
Service users
This service is for:
- Seven NHS Genomic Laboratory Hubs (GLH) NHS England » Genomic Laboratory Hubs and the staff who operate within them, including:
- Clinical scientists
- Transcribers
- Informatics
- Bioinformatics
- Administrative users
- Management and leadership
- Clinical users
- Clinical genetics
- Ordering clinicians
- Healthcare professionals involved in delivering genomic testing and associated care
- Genomics England
- PanelApp curation team
- National Genomics Informatics System (NGIS) development and management team
- NHS England and NHS Improvement Genomics Unit
- Patients and the public
Report contents
- Understand users and their needs
- Solve a whole problem for users
- Provide a joined-up experience across all channels
- Make the service simple to use
- Make sure everyone can use the service
- Have a multidisciplinary team
- Use agile ways of working
- Iterate and improve frequently
- Create a secure service which protects users’ privacy
- Define what success looks like and publish performance data
- Choose the right tools and technology
- Make new source code open
- Use and contribute to open standards, common components and patterns
- Operate a reliable service
1. Understand users and their needs
Decision
The service met point 1 of the Standard.
What the team has done well
The panel was impressed that:
- the team has thought about their user groups widely, from internal users providing the service, to users who interact with the service to complete a task and patients
- the team has a good understanding of their expert user base and is conducting benchmarking exercises to understand future users who might uptake genomic testing in the future
- the team is thinking about diverse and meticulous comms plans for different audiences
What the team needs to explore
Before their next review point, the team needs to:
- increase the variety of their participants' sample, ensuring they expand who they test and do user research with beyond their subject matter experts and expert-users. This is especially important for their priority user groups (clinicians ordering tests), which will present them with the greatest variety of expertise and needs
- consider using high level user journeys to map their riskiest hypothesis, for example will a clinician with low genomic medicine expertise know where to find and how to use my service? Establish a testing protocol that focuses on those risks
- increase the number and variety of one-to-one testing during the next phase, beyond the planned five usability sessions to test the design intuitiveness and the planned ratification workshops. The team could consider doing:
- additional sessions with their priority user groups to test the content and journey, for example one-to-one highlighting testing with clinicians ordering a test
- additional sessions with their most technical user group, for example journey testing and think-aloud sessions with lab technicians providing a test
- consider additional sessions with patients who are undergoing a genomic test, to map their journey making sense of what test they have been asked to undertake
- write user needs in a way that makes it explicit the planned solution is the answer to the need, without containing the solution in them. For example:
- from “As a clinician I need to search the National Genomic Test Directory to identify the genomic test/s to order for a patient”
- to “As a clinician ordering a test I need to find information about the test and its eligibility criteria so that I can order the correct test for the correct patient”
- Solution = free text search function in the National Genomic Test Directory (to be tested with users that it meets that need)
2. Solve a whole problem for users
Decision
The service met point 2 of the Standard.
What the team has done well
The panel was impressed that:
- the team are going to conduct an audit into the types of users they have and their clinical background
- integration with other clinical systems in use is part of their future roadmap
- there is a comprehensive communications plan to advertise the service to users
- there is a comprehensive rollout plan accompanied by support
What the team needs to explore
Before their next review point, the team needs to:
- explore how users get to be aware and use the National Genomic Test Directory and what they need to do after they use it. This might not be providing directly or immediately actionable insights for the team, but it will highlight onboarding and offboarding opportunities for further work
3. Provide a joined-up experience across all channels
Decision
The service met point 3 of the Standard.
What the team has done well
The panel was impressed that:
- although this is primarily an information only service with limited user journeys, consideration has been given to linking regional test directories with the national repository
- the online content can be reproduced and replicated as ‘offline’ content, for example as a spreadsheet to ensure continuity of service
What the team needs to explore
Before their next review point, the team needs to:
- explore a manual governance process for the local and offline content to ensure the most accurate version is being used by all users
- map out the service blueprint so they can identify service interactions and potential areas for improvement or areas of friction
4. Make the service simple to use
Decision
The service met point 4 of the Standard.
What the team has done well
The panel was impressed that:
- the team had leveraged the GOV.UK Design System to build the service
- the team were actively exploring opportunities to improve the user experience by incorporating new components outside of the GOV.UK Design system if necessary, and providing that feedback to the Design System team
- offline options including spreadsheets will still be available for users whose internal processes rely on these being available
What the team needs to explore
Before their next review point, the team needs to:
- test with visually impaired individuals and individuals with other types of impairments (such as dyslexia and dexterity impairments), especially considering that the service’s future potential user base is the entire clinical workforce
- test with users with contextual knowledge. For example members of the clinical or scientific teams within GLHs/GMSAs
5. Make sure everyone can use the service
Decision
The service met point 5 of the Standard.
What the team has done well
The panel was impressed that:
- testing has been conducted and influenced design iterations
- users with accessibility needs had been included in the user testing
- there has been extensive engagement with the existing user base and that this is intended to continue throughout rollout and the service launch
What the team needs to explore
Before their next review point, the team needs to:
- test with users with accessibility needs who have contextual knowledge. For example people who work in or around genetic medicine
- expand the testing to include the end-to-end journey, starting from google and navigating through the NHS pages to discover the service
- review and test the content including labelling on the end-to-end journey and re-design content where necessary to meet the NHS Service Manual standard for content
- expand accessibility manual testing to include the recommended assistive technologies
6. Have a multidisciplinary team
Decision
The service met point 6 of the Standard.
What the team has done well
The panel was impressed that:
- there is a contractual agreement between NHS England and NHS Improvement, and Liquid Light for ongoing service support post launch
- NHS England and NHS Improvement are recruiting for further roles in the future that are dedicated to managing the service
What the team needs to explore
Before their next review point, the team needs to:
- consider the separation of tasks dedicated to user research, and service and content design and allocated dedicated suitably skilled professionals to those
7. Use agile ways of working
Decision
The service met point 7 of the Standard.
What the team has done well
The panel was impressed that:
- both the suppliers and NHS England and NHS Improvement team were communicating well, engaging in short sprints, show and tells and weekly stakeholder meetings
- additionally as the need arose, the team did not feel restricted by the dedicated meetings and collaborated where appropriate outside of these agile ceremonies
- senior management within the Genomics Unit have been engaged in the show and tells
8. Iterate and improve frequently
Decision
The service met point 8 of the Standard.
What the team has done well
The panel was impressed that:
- there is a plan for future service support in place
- the service support contract includes ideation time
- there are feedback methods available to users
What the team needs to explore
Before their next review point, the team needs to:
- consider a plan for how they will prioritise and incorporate feedback received into future releases of the directory
9. Create a secure service which protects users’ privacy
Decision
The service met point 9 of the Standard.
What the team has done well
The panel was impressed that:
- the service does not collect and personal identifiable information
- the team has undertaken a penetration test and has plans to address all issues identified and has a future testing plan in place
What the team needs to explore
Before their next review point, the team needs to:
- address all issues identified in the penetration test report
10. Define what success looks like and publish performance data
Decision
The service met point 10 of the Standard.
What the team has done well
The panel was impressed that:
- although this is an information only service, and it is difficult to identify types of users, they have identified key performance indicators at this stage, and have plan for changing the metrics if needs be once the service goes live
11. Choose the right tools and technology
Decision
The service met point 11 of the Standard.
What the team has done well
The panel was impressed that:
- the team considered a range of options including laravel. They decided to go forward with Typo3 an open source CMS which the team have experience with and know will meet user needs
- the team is using a well established technology stack
What the team needs to explore
Before their next review point, the team needs to:
- continue to evaluate their technology choices to ensure they are suitable
12. Make new source code open
Decision
The service met point 12 of the Standard.
What the team has done well
The panel was impressed that:
- the team is developing open APIs which will be published shortly
- the team will be publishing their code in an open repository shortly
What the team needs to explore
Before their next review point, the team needs to:
- publish their code in an open repository
13. Use and contribute to open standards, common components and patterns
Decision
The service met point 13 of the Standard.
What the team has done well
The panel was impressed that:
- the team has made good use of the NHS design system and feedback on their experience using it
- the team are active contributors to the Typo3 community
14. Operate a reliable service
Decision
The service met point 14 of the Standard.
What the team has done well
The panel was impressed that:
- the team and their hosting supplier have appropriate disaster recovery plans in place
- in the event the service became unavailable for an extended period of time the content can be provided in a spreadsheet
What the team needs to explore
Before their next review point, the team needs to:
- include failure scenarios in the ongoing user testing