https://digitalhealth.blog.gov.uk/2022/08/08/dhsc-hospital-fault-reporting-alpha-assessment-and-reassessment/

DHSC Hospital Fault Reporting alpha assessment and reassessment

Sam Bowdery, 8 August 2022 - Alpha, Assurance, Service assessments

From: DHSC
Assessment date: 11 May 2021
Reassessment Date 26 January 2022
Stage: alpha
Result: Conditionally met at reassessment
Service provider: DHSC

Condition of meeting the standard: Please return a plan for your beta phase, for the panel to review. 

Service description

This service aims to solve the problem of allowing staff in NHS Trusts to report faults on their premises to their facilities management (FM) service provider. The service then sends an email to the FM provider’s helpdesk service.

The app will walk the user through several information gathering screens to create a profile of the fault to be sent to the FM provider. 

The app is being tested primarily with Trusts with Private Finance Initiative (PFI) contracts, however is being developed with the intention of a wider application across the NHS.

Service users

This service is for:

• Hospital staff
  • Estates staff
  • Other administrative staff including IT staff
  • Clinical and medical staff
  • Senior management
• Facilities Management company including in receiving faults reported via the app

Report contents

1. Understand users and their needs

Decision

The service met point 1 of the Standard.

What the team has done well

The panel was impressed that:

1. 1. • the team identified 3 to 4 prioritised user groups including hospital staff, PFI companies and maintenance staff
      • the team demonstrated openness to designing solutions that address the users’ actual needs emerging from research drawing on learnings from discovery and changing proposed solutions via prototypes in alpha
      • the team demonstrated some understanding of the users’ contexts and wider problems to be solved including reporting faults as quickly as possible and often within an existing rather than additional workflow
      • the team attempted to get the broadest possible representation of users, notwithstanding challenges occasioned by the pandemic, within the target population to provide feedback on the prototype shown 

What the team needs to explore

Before their next assessment, the team needs to:

1. 1. • consider how this would work for members of the general public who might be inclined to, or encouraged to, interact with the service when it is live   
      • address questions about accessing the service such as whether this will be via single-sign on, account or other means and exact responses, rather than assumed, from users to these various routes   
      • include assisted digital and perhaps non-digital users in further rounds of user research
      • clearly indicate the plan for continuity in user research between alpha and beta as if resourcing plans are changing 
      • secure partnerships from additional trusts to ensure a breadth of insight in subsequent phases
      • refine prototypes to cater for varying user needs including those of estate managers 
      • consider how the online and offline journeys join up from users’ varied perspectives 

2. Solve a whole problem for users

Decision

The service did not meet point 2 of the Standard.

What the team has done well

The panel was impressed that:

1. 1. • the context of where and when users will use this service was well understood.
      • the team had planned for a lack of internet connectivity in the locations where the service will be used
      • the team had understood that different users refer to assets or locations in different ways and had planned the service accordingly
      • the service had also the function to request a service related to the fault such as cleaning so that the user did not have to create a separate request using a separate application

What the team needs to explore

Before their reassessment, the team needs to: 

1. 1. • understand how the fault reporting service compliments or replaces the existing phone service.
      • explore how this service links with existing helpdesk and case management systems that the PFI suppliers will use to log and manage fault reporting
      • consider how to manage monitoring of the resolution of faults reported as this will be a key indicator of service performance

Reassessment

Decision

The service met point 2 of the Standard.

What the team has done well

The panel was impressed that:

1. 1. • the team demonstrated a very good understanding of the benefits to the hospital end users and how they would learn about the service
      • the team had carried out interviews to understand how the fault reporting service would fit in with the existing phone and email service
      • the team had identified that a simple, input-based system based on email submission could provide early understanding of the whole problem to be solved

What the team needs to explore

Before their next assessment, the team needs to: 

1. 1. • show evidence-led, user-focused understanding of how the crossover points will connect between systems, people and processes, and how they come together to find, log, and fix a fault
      • quantify the benefits across the whole service - not only hospital staff and visitors, but also benefits related to logging calls, fixing faults, and continuously improving the service
      • show how the simple, input-based system based on email submission was able to solve a whole problem for users, and how it informed the next steps
      • show evidence of how the service may scale up successfully, in complexity with implementing an API-base integrated solution and in number of organisations using it
      • show how fault resolution is monitored and reported, as this will be a key indicator of service performance

3. Provide a joined-up experience across all channels

Decision

The service did not meet point 3 of the Standard.

What the team has done well

The panel was impressed that:

1. 1. • the team were able to recruit from the prioritised user types despite the constraints of the pandemic

What the team needs to explore

Before their reassessment, the team needs to: 

1. 1. • explore how this service will operate with the existing fault reporting phone lines
      • explore how this service will integrate with the existing systems that the PFI companies use

Before their next assessment, the team needs to:

1. 1. • outline how any data collected from the digital service will be used to improve the existing facilities management service
      • outline how they are going to promote use of the digital service

Reassessment

Decision

The service met point 3 of the Standard.

What the team has done well

The panel was impressed that:

1. 1. • the team had identified that there were different approach options that could be taken, with different complexity, for example full integration with FM systems or a simple email interface
      • the team had explored how the service may link with existing helpdesk and case management systems, and fault reporting lines

What the team needs to explore

Before their next assessment, the team needs to:

1. 1. • map the entire user journey, showing how all users, both front and back of house, become aware of the service, prepare for the service, use the service and what happens after they leave,  for example notifications that a job is complete, or follow up queries
      • clearly articulate a plan for how the integrity of the central service can be maintained, while also allowing it to meet the user needs to be bespoke for each individual trust. For example, ensuring user research on one site or instance is communicated to all instances, and the service is updated as a whole. How will that risk of fragmentation over time be overcome?

4. Make the service simple to use

Decision

The service met point 4 of the Standard.

What the team has done well

The panel was impressed that:

1. 1. • the service uses the NHS.UK prototype kit for their work and based all designs on the correct design system to create a simple interface that was easy to navigate
      • the team has improved their use of patterns as the prototype progressed 
      • the designer has used the networking channels, such as slack and github effectively to find design patterns 

What the team needs to explore

Before their next assessment, the team needs to:

1. 1. • explore exactly how users will enter the service, so that they can design it to work online with a range of devices that reflects users’ behaviour. For example, the team mentioned that a key need was to use the service on the move, yet talked about accessing it via an intranet
      • work with a content designer to review content, for example minor issues in tense, grammar such as referring to ‘you’ in help text and ‘me’ in radio buttons 
      • work with a content designer and researcher to iterate content that follows new or uncommon design patterns, for example does the user understand ‘Remember my answers for next time’? Pay particular attention to the first page of the service. Does the user have enough information to start the journey? 
      • continue to improve their gov.uk, and nhs pattern knowledge, for example, consider using the gov.uk panel component on submission. If common patterns are not used, document evidence to say why
      • consider expanding their pattern research into two-way discussion via the NHS service manual backlog. The wider NHS design team would be interested to hear feedback about the use of the Immediate care card component in fault reporting, and any research evidence that is found

5. Make sure everyone can use the service

Decision

The service met point 5 of the Standard.

What the team has done well

The panel was impressed that:

1. 1. • the team had a good awareness of the need to test with users who have accessibility issues, or are digitally excluded, and could clearly articulate the importance of this
      • the team had taken advantage of the work already done in the design system and toolkit to create a prototype that could accommodate further research findings around accessibility and digital exclusion

What the team needs to explore

Before their next assessment, the team needs to:

1. 1. • plan for and carry out accessibility testing
      • plan for and carry out testing across a range of users with accessibility needs
      • articulate a design that takes into account users who are digitally excluded

6. Have a multidisciplinary team

Decision

The service did not meet point 6 of the Standard.

What the team has done well

The panel was impressed that:

1. 1. • the team had a suitably qualified user researcher and service designer leading on the interaction design of the system 

What the team needs to explore

Before their reassessment, the team needs to: 

1. 1. • consider the separation of the roles of user research and service design during the beta phase, this does not mean that they have separated them necessarily , more that the thinking is in place
      • appoint a product owner to ensure that future development of the service reflects the strategic aims and objectives of the department. This person needs to be thought about, but not a named person as yet, however, they will need to have been appointed by the time beta commences

Before their next assessment, the team needs to:

1. 1. • include a technical lead who will be responsible for leading the technical design of systems and services, and will justify and communicate design decisions

Reassessment

Decision

The service met point 6 of the Standard.

What the team has done well

The panel was impressed that:

1. 1. • the team had clear defined roles
      • they had a plan and resource in place for the team going forward in beta
      • NHSBSA, as the future owners of the service, were integrated as part of the core team

What the team needs to explore

Before their next assessment, the team needs to:

1. 1. • ensure that there is a clear separation in roles between the User Researcher and Service Designer

7. Use agile ways of working

Decision

The service did not meet point 7 of the Standard.

What the team has done well

The panel was impressed that:

1. 1. • the design team were working using sprint methodology
      • the team held frequent show and tells with stakeholders from outside the delivery team outlining what had been developed and why

What the team needs to explore

Before their reassessment, the team needs to: 

1. 1. • conduct a retrospective with the whole team, both policy and delivery, into how their experiences and reflections of delivering the discovery and alpha

Before their next assessment, the team needs to:

1. 1. • ensure that the policy team is more embedded with the delivery team such as  attending stand ups, being part of sprint reviews and planning, prioritisation meetings

Reassessment

Decision

The service met point 7 of the Standard.

What the team has done well

The panel was impressed that:

1. 1. • the team ensured DHSC, NHSBSA and the supplier were participating in the majority of agile ceremonies
      • the teams sprint cadence was being monitored with the appropriate tools

8. Iterate and improve frequently

Decision

The service met point 8 of the Standard.

What the team has done well

The panel was impressed that:

1. 1. • the team developed the service in weekly sprint cycles with iterations occurring after each feedback session
      • the team ensured each prototype developed was being tested with users and iterated based on the feedback received 

What the team needs to explore

Before their next assessment, the team needs to:

1. 1. • outline what the future plan for development of the service is
      • develop a product roadmap for the service that can be used to prioritise features for the service

9. Create a secure service which protects users’ privacy

Decision

The service did not meet point 9 of the Standard.

What the team has done well

The panel was impressed that:

1. 1. • the team understands the sensitivity around the data the service will hold on staff reporting faults

What the team needs to explore

Before their reassessment, the team needs to: 

1. 1. • technically prototype the backend
      • explore different avenues for identity management, user access, authentication and authorization, following the least privilege principle, so that users can only access data relevant to performing their role
      • identify threats and fraud vectors to your service, including potential pathways for hackers, and tested ways of reducing them

Before their next assessment, the team needs to:

1. 1. • continue identifying the potential threats for the service, and data stored, including anti personas
      • explore how feasible a single sign-on solution might be, understanding the existing login mechanisms for all staff reporting faults, as in, NHS staff, FM staff, hospital inpatients and visitors
      • ensure no real data can be accessed outside the production environment
      • speak to the DHSC data protection officer about the decisions they have made to ensure there is consistency in the approach and that data security and privacy is maintained

Reassessment

Decision

The service met point 9 of the Standard.

What the team has done well

The panel was impressed that:

1. 1. • the team acknowledged that user needs have shown the challenge of providing authenticated user experience 
      • the team has worked with the constraint of unauthenticated users 

What the team needs to explore

Before their next assessment, the team needs to:

1. 1. • find ways to prevent bot or malicious user attacks for when the site is public

10. Define what success looks like and publish performance data

Decision

The service met point 10 of the Standard.

What the team has done well

The panel was impressed that:

1. 1. • the team identified a number of Key Performance Indicators (KPI's) and metrics along side the mandatory ones had been to measure the success of the service

What the team needs to explore

Before their next assessment, the team needs to:

1. 1. • outline how they are going to measure and monitor the KPIs indicated
      • consider expanding the definition of ‘completion’ to include the completion of fixing the fault that has been reported. This will give a more accurate measure of the performance of the service and contract

11. Choose the right tools and technology

Decision

Decision

The service did not meet point 11 of the Standard.

What the team has done well

The panel was impressed that:

1. 1. • the team has explored different languages and frameworks
      • the service team has implemented and tested different frontend prototypes
      • the service team is aware of some of the current helpdesk systems used by PFI to manage their fault reporting systems
      • the team has ensured relevant documentation is accessible and up-to-date 

What the team needs to explore

Before their reassessment, the team needs to: 

1. 1. • prototype the backend, assessing the different helpdesk systems in use, their technical requirements and the feasibility of the integration. Those activities should inform the backend architecture design including the need for any API 
      • make sure DHSC to oversee the technical architecture design with the right technical governance in place 

Before their next assessment, the team needs to:

1. 1. • test their integration plan with enough helpdesk systems to ensure that it is fit for purpose as different helpdesk systems will have different technical resources in place and no local trust should find itself at a disadvantage
      • test how often data replication is needed and what impact it will have on the service availability and performance
      • ensure automated testing and deployment is achieved in beta
      • define a disaster recovery plan and build redundancy into their deployments, including restoring data from the backups
      • test that trust users are comfortable using their own helpdesk systems and the new service potentially being used in parallel, ensuring there is data consistency across both systems

Reassessment

Decision

The service met point 11 of the Standard.

What the team has done well

The panel was impressed that:

1. 1. • the team has selected a standard development stack that NHSBSA will be able to support and maintain
      • the team will be using commodity cloud for development and hosting of the service

12. Make new source code open

Decision

The service met point 12 of the Standard.

What the team has done well

The panel was impressed that:

1. 1. • the team has open sourced the code and GitHub is the team’s main code repository
        • https://github.com/mysociety/pfi-fault-reporting  

What the team needs to explore

Before their next assessment, the team needs to:

1. 1. • continue to make any new source code open and reusable and ensure it is kept up-to-date   

13. Use and contribute to open standards, common components and patterns

Decision

The service did not meet point 13 of the Standard.

What the team has done well

The panel was impressed that:

1. 1. • the service team has used the NHS design system and frontend toolkit   

What the team needs to explore

Before their reassessment, the team needs to: 

1. 1. • explore Government Platform services including GOV.UK PaaS and Notify
      • ensure that any decision to use a particular hosting platform will not negatively impact the support the service will receive in the future     

Before their next assessment, the team needs to:

1. 1. • explore whether any architecture components might potentially be reused from other government services to avoid duplication 
      • ensure the development team has the right expertise with the technology choices made in alpha
      • consider making contributions to relevant libraries and frameworks if relevant  

Reassessment

Decision

The service met point 13 of the Standard.

What the team has done well

The panel was impressed that:

1. 1. • the team has looked at possible integrations to FM systems
      • where standards lack in FM integrations, the team is considering forming a standard contract for their service

What the team needs to explore

Before their next assessment, the team needs to:

1. 1. • look at option of an open standard for facility and incident management integration API 

14. Operate a reliable service

Decision

The service did not meet point 14 of the Standard.

What the team has done well

The panel was impressed that:

1. 1. •  the team had identified uptake and up time as a key metric for success

What the team needs to explore

Before their reassessment, the team needs to: 

1. 1. • fully map out the space in which the service will operate including all the pinch points of the current user experience

Before their next assessment, the team needs to:

1. 1. • plan how to make deployment secure and what mitigations might be needed

Reassessment

Decision

The service met point 14 of the Standard.

What the team has done well

The panel was impressed that:

1. 1. • the team has recognised that DHSC team doesn’t have the complete capability to develop the service 
      • the team has worked with NHSBSA to find suitable maintenance and support for the longevity of the service

What the team needs to explore

Before their next assessment, the team needs to:

1. 1. • ensure that user needs and product vision remain at the forefront of the service when the service is in run and maintain with NHSBSA
      • tackle configuration drift between the FM systems and the reporting tool, reducing the burden on FM staff having to make changes in two systems

 

Share this page

• Twitter
• Facebook
• LinkedIn
• Email

Digital health and social care

A blog about digital work in the health and social care area within government and arm's length bodies.

Find out more.

Follow DHSC

• Twitter
• YouTube
• Flickr

Recent Posts

• Join the digital transformation team
• NHSD Apply for Care ID alpha assessment
• NHSBSA Prescription Prepayment Certificate (PPC) for Hormone Replacement Therapy (HRT) patients alpha assessment
• NHSE Standards Directory Beta Assessment
• NHSD GP IT Futures Buying Catalogue beta assessment

Categories

Recent Posts

• Join the digital transformation team 24 August 2023
• NHSD Apply for Care ID alpha assessment 3 March 2023
• NHSBSA Prescription Prepayment Certificate (PPC) for Hormone Replacement Therapy (HRT) patients alpha assessment 27 January 2023
• NHSE Standards Directory Beta Assessment 11 November 2022
• NHSD GP IT Futures Buying Catalogue beta assessment 4 November 2022