The report from the alpha assessment for NHS Digital's GP IT Futures Buying Catalogue service.
Assessment by: Department of Health and Social Care and Public Health England
Assessment date: 20 March 2019
Stage: Alpha
Result: Met
Service provider: NHS Digital
Service description
The GP IT Futures Buying Catalogue is a digital marketplace for healthcare staff to purchase assured software and services from suppliers.
Service users
Primary users groups are suppliers who want to sell their services to healthcare staff and buyers of software/IT for general practice and primary care establishments.
1. Understand user needs
Decision
The team met point 1 of the Standard.
What the team has done well
The panel was impressed that:
- The team has a good understanding of their broad users groups (suppliers, buyers, and end-users) and sub-categories of core users (technical group, procurement leads, GPs’ admins, IT and practitioners). They have engaged throughout discovery and alpha with users and iterated based on their needs.
- They have shown several examples where they changed direction based on user feedback and needs. This includes learning that video demonstrations of each product/service may not work as intended.
- The team overcame some limitations/restrictions to test with some of the core users group due to commercial implications. They have compensated this by testing with proxy users and group engagements, such as webinars.
What the team needs to explore
Before their next assessment, the team needs to:
- Test end-to-end with all user groups.
- Make sure they test with those who will be managing the service in the future.
2. Do ongoing user research
Decision
The team met point 2 of the Standard.
What the team has done well
The panel was impressed that:
- The team had dedicated user research resources during discovery and most of alpha. When the user researcher left the team at the end of alpha, they compensated by sharing research duties amongst themselves. This showed great commitment to user research but should be considered a temporary solution that needs to be amended in beta.
What the team needs to explore
Before their next assessment, the team needs to:
- Fill the user researcher vacancy. The team needs to have a dedicated user researcher in place for private beta.
- The team has outlined a high-level plan for testing core capabilities in private beta for a buyers journey. This requires more thought and detail.
- Put a solid plan in place to test the current suppliers’ journey.
- Do more accessibility testing. Further research with users with diverse accessibility needs is required.
3. Have a multidisciplinary team
Decision
The team met point 3 of the Standard.
What the team has done well
The panel was impressed that:
- They had a multidisciplinary team for discovery and alpha.
- The team was full-time on the buying catalogue
- There was a 50/50 gender mix on the team
What the team needs to explore
Before their next assessment, the team needs to:
- Hire user researchers
- Bring more content design expertise into the team
- Have access to a cyber-security specialist during private beta
4. Use agile methods
Decision
The team met point 4 of the Standard.
What the team has done well
The panel was impressed that:
- They work in an agile manner.
- They use the tools one would expect of an agile team.
- We heard of cross-organisation collaboration, especially with the Digital Marketplace team.
5. Iterate and improve frequently
Decision
The team met point 5 of the Standard.
What the team has done well
The panel was impressed that:
- They have the ability to iterate and improve the service.
- They feel empowered to test ideas and cease working on things if the don’t get evidence that they are valuable.
6. Evaluate tools and systems
Decision
The team met point 6 of the Standard.
What the team has done well
The panel was impressed that:
- The team have a good mix of systems that are easily portable.
- There is excellent design for scalability.
- Cloud based tools were used.
- The team did daily updates throughout pre-production and staging environments to live.
- They have already researched alternative CRMs.
What the team needs to explore
Before their next assessment, the team needs to:
- Plan for many simultaneous users.
- Speed up page progress. Ideally to less than 3 seconds.
7. Understand security and privacy issues
Decision
The team met point 7 of the Standard.
What the team has done well
The panel was impressed that:
- The team had knowledge of different fraud vectors
- They seemed confident that their service will be secure and are following the NHS Digital’s cloud security good practice guide
What the team needs to explore
Before their next assessment, the team needs to:
- Regularly test backups.
- Explore whether two-factor authentication (2FA) is needed for users (front-end and back-end).
8. Make all new source code open
Decision
The team met point 8 of the Standard.
What the team has done well
The panel was impressed that:
- They have a Github repository and want to code in the open
- Private assets are separated.
What the team needs to explore
Before their next assessment, the team needs to:
- Continue to code in the open under an appropriate license.
9. Use open standards and common platforms
Decision
The team met point 9 of the Standard.
What the team has done well
The panel was impressed that:
- The team is using HTML5, CSS, JS
- They are using openID Connect for login
What the team needs to explore
Before their next assessment, the team needs to:
- Resolve minor usability issues (checked using HTML checker).
- Not solely use MS Excel files. Offer ODS, CSV and other open document formats.
10. Test the end-to-end service
Decision
The team met point 10 of the Standard.
What the team has done well
The panel was impressed that:
- The current iteration works with all major browsers.
- The team has evidence that users will be using desktops but "reader mode" on mobile worked well nonetheless, indicating good semantic markup.
What the team needs to explore
Before their next assessment, the team needs to:
- Ensure that the service will be upgraded with each major browser release.
11. Make a plan for being offline
Decision
The team met point 11 of the Standard.
What the team has done well
The panel was impressed that:
- The organisation they are part of (NHS Digital) has plans and resources allocated for when services are down.
- The team are using containers so they can replace or add components without stopping or shutting down the service if necessary.
What the team needs to explore
Before their next assessment, the team needs to:
- Test hot-failover plan.
- Test restoring backups
12. Make sure users succeed first time
Decision
The team met point 12 of the Standard.
What the team has done well
The panel was impressed that:
- We saw multiple prototypes that were iterated upon.
- They understand the limitations of the current service and have acted on them.
- The automated tests of current iteration have passed WCAG2.1.
- The team worked within some commercial restrictions (cannot give any competitive advantage to suppliers) but still managed to test with users. In the next phase the commercial restrictions should not be as prevalent, so look forward to more end to end testing showing how suppliers could get on the framework first time.
What the team needs to explore
Before their next assessment, the team needs to:
- Test with more users with diverse accessibility needs.
- Dedicate more time to other elements of the end to end service. We saw the service explored broadly in discovery and alpha but clearly more recent effort has been put into the supplier registration journey. The team needs to make sure they work on the rest of the service.
- Publish a roadmap that relevant stakeholders can engage with.
13. Make the user experience consistent with GOV.UK
Decision
The team met point 13 of the Standard.
What the team has done well
The panel was impressed that:
- The team are using GOV.UK and NHS design patterns and style guides.
What the team needs to explore
Before their next assessment, the team needs to:
- Review that the components they are using are up to date and consistent with current GOV.UK and NHS design systems.
14. Encourage everyone to use the digital service
Decision
The team met point 14 of the Standard.
What the team needs to explore
Before their next assessment, the team needs to:
- Explore further what process could exist if the site is offline.
15. Collect performance data
Decision
The team met point 15 of the Standard.
What the team has done well
The panel was impressed that:
- The team has reviewed lots of data on the current service.
- Privacy concerns were taken into account when choosing an analytics provider.
What the team needs to explore
Before their next assessment, the team needs to:
- Have a more detailed analytics plan for beta before commencing.
- Have the skills in the team to get actionable insights from whichever analytics software they use.
16. Identify performance indicators
Decision
The team met point 16 of the Standard.
What the team has done well
The panel was impressed that:
- The team has a list of KPIs.
- They have benchmarked the current service.
17. Report performance data on the Performance Platform
Decision
The team met point 17 of the Standard.
What the team has done well
The panel was impressed that:
- They are in discussion with the performance platform to report on mandatory KPIs.
18. Test with the minister
Decision
The team met point 18 of the Standard.
What the team has done well
The panel was impressed that:
- They have plans to show the service to senior colleagues in the Department of Health and Social Care and are in contact with relevant stakeholders gathering feedback
What the team needs to explore
Before their next assessment, the team needs to:
- Eventually make plans to test with a minister or relevant senior owner of the service within the NHS.
Conditions before the next assessment
The buying catalogue discovery began in 2017 but this was the teams first assessment. We were shown more than what you would expect in an alpha stage. The team must be more punctual with arranging assessments before they go into public beta.
The team must also submit a spend control form for the next phase (Private Beta) to the Department of Health and Social Care Digital team.
During the assessment the panel clarified that by passing an alpha assessment that would allow them to go into a private beta, not to make the service “live”. The panel would expect the team to restrict access to the service, invite users to use it and get more feedback before the next assessment.