From: NHSX
Assessment date: 11th September 2020
Stage: Beta
Result: Met
Service provider: Department of Health and Social Care (DHSC) and NHS Business Services Authority (NHSBSA)
Service context
- The Health Service Products (Provision and Disclosure of Information) Regulations 2018
- Section 28 – In the event of a supply shortage of an NHS medicine, the supplier must provide information about the quantity of that medicine currently available;
- Section 29 – If a supplier believes there is likely to be a supply shortage of a particular medicine, or intends to discontinue manufacture of that medicine, they must inform DHSC;
- DHSC requires this information to allow them to determine the impact of the shortage and take appropriate action.
- This is currently a manual process, based around email and spreadsheets. There is no single source of truth about shortages.
- DHSC decided to explore the development of an online shortage notification management system, where all supply issues can be appropriately reported, logged and managed by DHSC.
Service users
This service is for the DHSC Medicine Supply Team and Marketing Authorisation Holders (MAHs) – external organisations involved in drug manufacture/supply.
Report contents
- Understand users and their needs
- Solve a whole problem for users
- Provide a joined-up experience across all channels
- Make the service simple to use
- Make sure everyone can use the service
- Have a multidisciplinary team
- Use agile ways of working
- Iterate and improve frequently
- Create a secure service which protects users’ privacy
- Define what success looks like and publish performance data
- Choose the right tools and technology
- Make new source code open
- Use and contribute to open standards, common components and patterns
- Operate a reliable service
1. Understand users and their needs
Decision
The service did not meet point 1 of the Standard.
What the team has done well
The panel was impressed that:
- The team is striving to create a simple process in-system.
- The team is enthusiastic about conducting user research, and stakeholders are engaged with the process.
- The team were able to acknowledge pain points in the current design and user journey flow, acknowledging gaps in their knowledge, and have plans in place to iterate.
What the team needs to explore
Before their next assessment, the team needs to:
- Continue to build their understanding of the needs of system users, gathering this through user research with end users. It appears as though the team have relied heavily upon the discovery research and legislative requirements, conducting limited research with end users to understand the wider context of their work and how the platform meets their needs. It is currently unclear if this has led to significant gaps in their understanding of users and their requirements of the system.
- Conduct thorough usability testing of various end-to-end scenarios to ensure users are able to complete the various tasks they need the system to be able do and are able to succeed first time.
- Ensure that there are different avenues for getting user feedback for iterations, e.g. moderated/ unmoderated remote user research, analytics etc.
- Ensure user research feeds into the iteration prioritisation process.
2. Solve a whole problem for users
Decision
The service met point 2 of the Standard.
What the team has done well
The panel was impressed that:
- The team has created a simple in-system process.
What the team needs to explore
Before their next assessment, the team needs to:
- Conduct further research to understand the offline elements of the user journey and iterate the service to ensure that users are able to complete their journey in the system as necessary. For example, this could include giving users the option to save a report part way through if they do not have the necessary information on hand.
- Conduct research to further understand whether in-system messaging (e.g. hint text) is helpful in enabling a system users complete their tasks.
- The team needs to ensure they test the system having entered significant data.
3. Provide a joined-up experience across all channels
Decision
The service met point 3 of the Standard.
What the team needs to explore
Before their next assessment, the team needs to:
- Have a clear offline route documented in case the service goes down or for those have assisted digital needs. The panel understands that this route would just be the one in which the digital service is replacing.
4. Make the service simple to use
Decision
The service met point 4 of the Standard.
What the team has done well
The panel was impressed that:
- The team did an excellent job in showing the service to the panel.
- The service clearly follows GOV.UK Design Patterns. The panel liked that at the end of a journey, the service shows a summary of the key decisions made by the user.
- The team have iterated the user journeys through the service and plans in place to make further improvements to elements of the journeys through the service.
What the team needs to explore
Before their next assessment, the team needs to:
- The team should not use acronyms in their service. For example, do not use ‘API’, use the full name ‘Active Pharmaceutical Ingredient’.
- When created a pre-baked user type, can the authorised creator be a title rather than a label.
- The team should explore the possibility of allowing the user to save where they are in the journey, and to return back.
5. Make sure everyone can use the service
Decision
The service met point 5 of the Standard.
What the team has done well
The panel was impressed that:
- The team has thought about accessibility from the start. They use Wave, ARC toolkit and Lighthouse.
What the team needs to explore
Before their next assessment, the team needs to:
- Test the service with a user with accessibility needs.
- The name of the service on GOV.UK is DHSC DaSH. The team needs to use the full name and not an acronym. The team should use a verb to describe the service and not a noun.
- Have a clear offline route documented in case the service goes down or for those have assisted digital needs. The panel understands that this route would just be the one in which the digital service is replacing.
6. Have a multidisciplinary team
Decision
The service did not meet point 6 of the Standard.
What the team has done well
The panel was impressed that:
- The team is clearly passionate about delivering a service that will improve their users’ day to day lives.
- NHS BSA are building the service. They have 3 developers, a tester, project lead, support lead and project engineer. They have the ability to bring in other experts when needed e.g. platform services, cloud environment team, security specialists, external assessors.
- DHSC lead on policy related matters and will be managing the service. They will also be the key users (DHSC pharmacists) of the service. They have a product owner, project lead and analysts attached to this work.
What the team needs to explore
Before their next assessment, the team needs to:
- Some team members are fulfilling multiple roles. The team needs a dedicated user researcher (at least part-time) to conduct user research especially before progressing into public beta.
- The team should have someone to track metrics to ensure the service is constantly being improved.
- As there is a lot of content on the service, the team should consider bringing in a part-time content designer.
- NHS BSA needs to better describe how the team changes from build to support. The team recognises this as important. DHSC will be bringing in extra people to support the management of the service.
7. Use agile ways of working
Decision
The service met point 7 of the Standard.
What the team has done well
The panel was impressed that:
- The team is working in an agile way. They have show and tells, and retrospectives.
What the team needs to explore
Before their next assessment, the team needs to:
- Demonstrate how they improved their service based on an observed user need through team-wide retrospectives and planning.
8. Iterate and improve frequently
Decision
The service met point 8 of the Standard.
What the team has done well
The panel was impressed that:
- The team uses a weekly meeting to get feedback and raise any issues. The end of the sprint coincides with this team meeting.
- DHSC runs governance meetings. The Chief Pharmacist (SRO) is empowered to make key decisions.
What the team needs to explore
Before their next assessment, the team needs to:
- Demonstrate how they improved their service based on an observed user need through team-wide retrospectives and planning.
9. Create a secure service which protects users’ privacy
Decision
The service met point 9 of the Standard.
What the team has done well
The panel was impressed that:
- The team has implemented customisable role based access control with extremely granular control to ensure that users have appropriate permissions. This is further supported by mandatory two factor authentication.
- Full audit logs of updates on the service are captured and the hosting environment has appropriate monitoring in place.
- The service is hosted in a hardened environment, which undergoes regular penetration tests.
What the team needs to explore
Before their next assessment, the team needs to:
- Complete the security testing which is scheduled to commence 21st September and take steps to resolve or mitigate any issues.
10. Define what success looks like and publish performance data
Decision
The service did not meet point 10 of the Standard.
What the team has done well
The panel was impressed that:
- The team can track support calls, number of emails the policy team receives, the time it takes to hear back from an MHA and time to deal with an issue.
- The ambition is for 100% of users to use the digital service. This is a bold ambition. The panel suggests setting a time-based KPIs, for example ‘by 3 months, 70% of users will use the digital service’.
- The team is using Datadog, which can tell you when a user encounters an error message.
What the team needs to explore
Before their next assessment, the team needs to:
- Consider implementing Google Analytics to understand where users drop off in the journey. The panel does understand that Datadog is being used but Google Analytics is a relatively simple tool to implement. A decision should be made to determine if Google Analytics offers something useful that Datadog doesn’t.
- Consider website specific metrics for your service. For example, user satisfaction rates, completion rate, bounce rate and cost per transaction. The team should track metrics that meaningfully impact their service and leads to changes being made. It can be overwhelming to track multiple metrics. 8-10 is usually enough. The team should also have someone to track metrics to ensure the service is constantly being improved.
Further information:
- Measuring success
- How to set performance metrics for your service
- Evaluating digital health products (PHE guidance)
11. Choose the right tools and technology
Decision
The service met point 11 of the Standard.
What the team has done well
The panel was impressed that:
- The team reviewed a range of potential solutions taking into account the functionality of the tools and the available expertise.
- The team is making use of gov.uk components and open source software where possible.
12. Make new source code open
Decision
The service did not meet point 12 of the Standard.
What the team has done well
The panel was impressed that:
- The team has a plan for open sourcing the code. Components developed for this service are already available for internal BSA developers.
What the team needs to explore
Before their next assessment, the team needs to:
- Host the source code in an open repository.
13. Use and contribute to open standards, common components and patterns
Decision
The service met point 13 of the Standard.
What the team has done well
The panel was impressed that:
- The team has made use of existing resources to enhance the service leveraging the existing drug dictionary and Gov.uk notify and design patterns.
- Components developed for this service are available for use across NHS BSA and will be available to the public in the future.
What the team needs to explore
Before their next assessment, the team needs to:
- Consider where they can contribute to the open source community.
14. Operate a reliable service
Decision
The service met point 14 of the Standard.
What the team has done well
The panel was impressed that:
- The team has taken steps to ensure that the service is resilient by utilising an established resilient hosting platform.
- The team has considered the impacts of the service not being available and have ensured that alternative routes are available.
What the team needs to explore
Before their next assessment, the team needs to:
- Develop a disaster recovery plan for the service.