The service is aiming to provide a single, trusted location for learning resources, guidance and educational innovations which can be shared, signposted, evaluated and developed. This could include, but is not limited to, documents, video, audio, e-learning or physical resources.
The primary actions a user will be able to complete for private Beta are - search, access and contribute. In private Beta rating, reviewing and discussing functionality will be developed and tested. A proof of concept for collaboration will be developed in private Beta to determine if this meets a user need.
It has a wide user base and will be open to all health and social care staff. It will also be accessible by those entitled to an NHS OpenAthens accounts. The team have identified two main user groups, Educators (educationalists, trainers, tutors etc) and Learners. The service will replace eWIN (the NHS information network designed to enable improvements in workforce development, efficiency and productivity. eWIN provides a central place to share and access valuable workforce information, best practice, case studies and resources) and the eLearning Repository (an extensible “search and discover platform” to support the discovery and sharing of e-learning objects and learning resources held both within the repository and at external locations.) The team also expect it to support the rationalisation and consolidation of HEE websites.
Department / Agency:
Health Education England (HEE)
Date of assessment:
16 November 2017
Assessment Stage:
Alpha
Result of assessment:
Pass
Lead Assessor:
M.Harrington
Service Manager:
N.Ralph
Digital Leader:
I.O'Neill
Assessment Report
Outcome of service assessment
After considering all the information as part of the Alpha service assessment, the panel are very pleased to conclude that the Health Education England Learning Solution service is ready to move into private beta. The report covers a number of recommendations which will need to form part of the beta.
Overall the team presented a clear vision for the product and a well scoped minimum viable product for release. There are however challenges which the panel believe the team will need to develop their understanding of during beta.
- Contributing content: With users contributing content there is no way to ensure the quality of the description or materials uploaded. The rating process is expected to sort the good from the bad, but what if this functionality is not taken up as expected. Health Education England should plan alternatives to make sure user needs are met, and they find the best content.
- Transfer of existing content: The plan is to transfer all the content from the e-Learning Repository to the Learning Solution. Before doing this, the Health Education England team must ensure that anything being transferred is relevant and meets a user need or mandatory training requirement
- Additional features: Continue to research and truly understand the needs of users before starting work on additional functionality, such as a collaboration area.
- Naming of the service: As part of beta research the team should ensure the name supports user understanding. (https://www.gov.uk/service-manual/design/naming-your-service)
User Needs
The team presented a very clear picture of who the users of the service are and will be. It was evident that research was done to discover needs of different users and what the most challenging needs are. The team has been researching with users who have low digital skills and have plans in place to continue doing this and also it was good to hear how the team were going to attend User Research courses at GDS to bring skills inhouse. The panel were impressed to hear how different research methods have been used, together with cross device testing. Also good to hear that there are plans for accessibility testing.
Team
For the alpha, the team was made up of the following roles
- Service Manager
- Product Manager
- Delivery Manager (Supplier)
- Technical Architect x2 (1x Supplier)
- Assisted Digital Lead (Supplier)
- User Researcher (Supplier)
- Designer (Supplier)
- Developer (Supplier)
From the assessment it was clear that the team works incredibly well together. The relationship between Health Education England and their supplier is a good one and it was clear that this was adding value to the project. The team are not co-located but are working very well from locations across the country, using tools like Trello and having regular virtual meetings including all the agile artefacts.
There is a clear process for the creation and prioritisation of stories and a good working relationship between the Product Manager from Health Education England and the Delivery Manager from the supplier. This is supported by the Service Manager who is empowered to make decisions around scope and budget meaning the team have the space to focus on the product. There are fortnightly programme meetings where the project is discussed as part of the wider TEL programme and a TEL Steering Group which ensures there is good visibility on progress to senior stakeholders.
The use of hackdays/ideathons at the start of alpha is a great example of the supplier and Health Education England working together in innovative ways to make the most of contact time with users. Similarly, the changing of sprint duration to support better working with the supplier is a great example of how the team take action from retrospectives to improve ways of working.
Technology
The panel were impressed by the effort made by the team to assess existing learning platforms and develop a technical architecture that both meets existing internal Health Education England requirements and interoperates with external facing platforms that the new service will integrate with, most notably the e-Learning for Healthcare Hub. The panel were particularly pleased to hear that the need for a single sign on hand been considered and was being planned for even at this early stage in Alpha.
It was also clear that the team had considered where open source or existing tools and services could be made use of with the entire platform being built using the open-source platform Drupal. This has both helped the team ensure that they will be able to easily export content from the NHS e-learning repository, which is also built on Drupal (albeit an older version), and eliminate the need for the team to reinvent the wheel where Drupal has existing modules - such as for Cookies and webforms. This, combined with the team’s use of GitHub’s branching and commit functionality also ensures that changes are reviewed and decided upon in the open giving the overall project a high level of transparency which restricts the risk of vendor lock-in if a different company is used for development during beta, live or continuous improvement by making sure all code and coding decisions are available in a publicly owned space.
In addition the team noted that they had reviewed existing open-source learning platforms, such as Moodle, when looking to build the new Learning Solution but decided the specialised nature of the service in question would require something more custom to be built as a base with the other open-source learning platforms available on the market being used as inspiration for future functionality. The panel agrees that this is a sensible approach but would encourage the team to regularly monitor the online learning market to ensure their developing platform meets user expectation in terms of user experience, user design and range of functionality.
As a result of these decisions made during Alpha, the panel feel reassured that the team have a good foundation for heading into Beta, particularly as Drupal provides the team with a high degree of flexibility through test environments and instantaneous deployment functionality. Furthermore, the decision has been made to host the service on Microsoft Azure along with all other Health Education England services which provides the team with efficiencies and economies of scale with regards to security.
The panel would, however, like to stress that while all this combines to ensure a good foundation is in place - it is just a foundation and the team will need to do considerable amounts of work in private beta before the technical underpinnings of the service are of the standard required to enter public beta. To ensure these gaps in thinking are filled, the panel would recommend that the team pays particular attention to the log-in functionality of the service and associated verification and permissions processes. It is noted that answering the remaining questions with regards to these issues is already on the roadmap as described by the team during the assessment.
It was also recommended during the assessment that the team look at other government services (and gov.uk) that link to external websites or resources that might not meet the same accessibility standards, both in terms of assisted digital and device/browser compatibility, to see how the services alerts users to this fact.
Service Design, Design and Content
The panel were pleased to hear that the service will benefit from the existing assisted digital support provided by Health Education England’s Library and Knowledge services. The exact nature of this support will be decided after testing during Beta to see which of the existing channels, which include live chat and support helpdesk, will be the most appropriate for the new service. The more technical aspects of accessibility will be informed by a 3rd party accessibility audit which will be completed during beta.
As the service will be hosted on existing Azure servers used by Health Education England, there is already 24/7 server monitoring in place. This means that should the service go down the relevant members of staff will be immediately notified and HEE will employ a message cascade protocol to inform all relevant parties - including users through Twitter and a front-facing page on the website. This protocol is already in place for the e-Learning for Healthcare Hub and has been shown to be good practice.
Beta users will be recruited through known networks of users for other HEE services and will be invited through email.
Branding & personality designed to be coherent with HEE and NHS in general which is acceptable.
It was good to see that the service has been through various iterations and there is clear thinking about reflecting on user needs, and the need to review guidance and text, whilst ensuring plain English is the key. In Beta there will need to be more thinking around testing different end-to-end journeys and how these can be iterated - so that it is clear that people are able to succeed first time regardless of where they start as people may not start a particular journey from the assumed point.
Service Performance
Even at this stage of development it was clear that the team have been thinking about performance metrics for the service and how they will measure them. Analytics will be added to the private beta service to measure the digital service and the team will work with the support helpdesk to gain further insight into how the new digital service performs. It was very pleasing to see that the service desk team were already being engaged as part of a joined up service. The team have access to the data team for help with analysis and insight.
Summary
The team presented a clear and coherent vision for the service and the panel were particularly pleased to see the way the whole team worked together to answer questions, talk about user research and discuss the things they had learnt during alpha. It is clear a great amount of effort has gone into the alpha and preceding discovery phase and this will stand the team in good stead for private beta. It is however, a significant jump from alpha to beta and the team themselves acknowledged there is a lot they still need to do. The panel are confident it is within the ability of the team to make this step, but it should not be underestimated and the recommendations in the report will need to be followed for a successful beta assessment. Overall the panel were impressed with the alpha from the Health Education England team and look forward to seeing further iterations.
Recommendations:
- To ensure every round of research is inclusive (i.e. ensure users with low digital skills are included in every round of user research).
- To be mindful of asking any leading questions in usability testing sessions as this can lead to biased research findings.
- Engage with the accessibility team at GDS with regards to accessibility approach as it is imperative that the content is accessible on all assistive tools.
- The alpha prototype would have benefitted from the input of a content designer. The team have said that they are recruiting a content designer for beta and the panel strongly recommend that they do to ensure content is clear, consistent and meets user needs..
- Health Education England staff to attend GDS User Research training.
- The team should design and test pages for unhappy journeys and for instances when the service is unavailable.
- Ensure that the team has dedicated time from a performance analyst for beta to provide insight into the quantitative data obtained from the service.