The Surgical Site Infection Electronic Patient Post-Discharge questionnaire is part of the Surgical Site Infection (SSI) surveillance used by Public Health England (PHE) to monitor the rate of infections associated with short-stay surgery and create a picture of a hospital’s SSI incidence for benchmarking and remediation of outliers.
The questionnaire is currently paper-based and issued to patients to complete 30 days after discharge from hospital, as the majority of infections occur post-discharge and are treated in the community. However the issuing of questionnaires and their completion rates are too low to provide a complete picture of a hospital’s SSI incidence, with currently only 55% of NHS hospitals using the post-discharge questionnaire (PDQ) versus 88% of private hospitals (target 90%) and a total patient completion rate of 48% (target >70%). The PDQ is used to initiate a conversation with the patient about their wound, with data entered into the national surveillance system once it has been confirmed by a medical professional that the patient is suffering from post-surgical infection.
Following a discovery phase that found inefficiencies in the surveillance process and patient willingness to engage with post-discharge questionnaires via digital channels, PHE have developed and tested an alpha electronic Patient Discharge Questionnaire (ePDQ) and associated caseload management system for hospital staff. This has enabled the team to test the assumption that it is possible to increase the uptake and completion of PDQs.
Department / Agency:
Public Health England (PHE)
Date of Assessment:
18 April 2018
Assessment Stage:
Alpha
Result of Assessment:
Pass
Lead Assessor:
H.Hall
Service Manager:
T.Lamagni
Digital Leader:
I.O'Neill
Assessment Report
Outcome of service assessment
After considering all the information as part of the Alpha service assessment, the panel are pleased to conclude that the PHE Surgical Site Infection Electronic Patient Discharge Questionnaire is ready to move into private Beta.
The full report covers a number of recommendations which will need to form part of the Beta phase.
Team
For the Alpha, the team included the following roles:
- Service manager
- Product manager
- Project manager (Supplier internal)
- Business analyst/user researcher (Supplier)
- Technical architect (Supplier)
- Developer (Supplier)
From the assessment it was clear that the team were working collaboratively and iteratively to develop and test the Alpha service. An inception week was used at the outset, followed by a series of 2 week sprints over a 3 ½ month period.
Apart from 5 week’s full-time software development, the majority of the team were delivering the alpha on a part-time basis. They were also remotely located so they used tools such as Skype and Webex to maintain regular communication and share progress in end of sprint show and tells. Priorities for the prototypes and project tasks were discussed at the outset of each sprint, and changed mid-sprint when required, with the product owner having the ultimate say. Testing took place continually, including testing by the PHE team, but working with hospitals was challenging due to it running the Alpha testing during the winter season.
The team benefitted from having access to an Information Governance advisor and PHE’s digital team to advise on organisational technology strategy. They also worked with a developer at PHE to ensure the Alpha products were compatible with PHE technology.
The Service Manager reports regularly on the overall Surgical Site Infections Surveillance Service (SSISS) and so the work of the team is represented at appropriate boards and committees, including at update sessions with the devolved administrations.
Though the PHE team were new to agile ways of working, they were supported by the Supplier and PHE’s digital team and showed that they had embraced the approach.
The team hope to move into the Beta phase soon and will retain the same team, adding user researchers, UX and data analytics roles. These roles are currently being recruited within PHE and may be available to advise on the next phase.
User needs
The panel were impressed with the way in which the Alpha service was used to test the assumptions of the discovery report; that it would be possible to increase the patient response rate and uptake of post-discharge questionnaires by hospitals by building a better service that meets user needs.
Hampered by the winter flu season, the team were only able to conduct limited research with patients but the panel felt this was sufficient to provide assurance that introducing the ePDQ would be beneficial and a good cross-section of patient users were recruited to test the ePDQ. In order to ensure the maximum response rate, and in response to patient preferences indicated during the discovery and alpha phase, the service will continue to allow other channels (post or phone) for completion of the PDQ.
The panel were pleased that the team intend to expand their user research during the Beta phase but would have liked to see a more coherent plan for the next phase of research. More research should be done into the needs of different patient users, for example ethnographic research exploring whether the ePDQ could be made more accessible to patients with specific needs, for example improvements for patients with limitations on their fine motor skills such as using a numerical date entry in place of a date picker. Research should also explore the role of family members, carers and hospital staff in the questionnaire’s completion, as well as research with patients with English as a second language. Elective and unplanned surgeries have very different demographics, and unplanned surgeries were not tested in this phase so any differences should be taken into account in the Beta phase.
The use of personas would help the PHE team to understand and keep in mind different patients’ needs. These could help to identify users with low IT literacy and find ways to support them to encourage ePDQ completion and channel shift from paper
In order to explore the needs of hospital staff administering and recording PDQs, the team conducted a workshop with 3 hospitals and researched across three hospital sites using wireframes and a prototype caseload management system. Users include administrative staff, surgical ward nurses and pre-administration clinic nurses. The team were able to demonstrate the potential to reduce the workload associated with administering and recording PDQs and the user response was very positive.
In the Beta phase, the team plans to analyse behaviours (from feedback, analytics and response rates) and conduct qualitative research to explore patterns that might help improve the service, for example changes to the user interface, use of channels and to support the variation in hospitals’ processes around PDQ administration. By increasing the number of testing sites, the team will be able to understand what further improvements would be needed to improve the uptake of post-discharge surveillance by hospitals not issuing PDQs, starting with those who previously issued PDQs and stopped due to workload.
Service design
The team’s approach to service design gave confidence to the panel as being appropriate for an Alpha phase. The panel felt it was right to deviate from the discovery report assumptions and consider the entire PDQ surveillance process in order to focus on reducing workload for staff.
It would have been helpful for the panel to see a service journey map of the end-to-end service with pain points for patients and hospital staff overlaid. This would have helped illustrate the problems the team was trying to solve and could be used as a communications tool to encourage take-up in hospitals (i.e. by showing how simple and straightforward the ePDQ process can be).
The panel understood that there are no plans to discontinue the paper PDQ in favour of a digital-only approach and were glad to hear how the team had made sure that users who prefer to complete a paper PDQ get an equivalent level of service (eg reminders by phone or SMS) as those who transact digitally.
Completed paper PDQs are retained by hospitals for 1 year for in case of queries related to the data then destroyed. This adds to the administrative overhead of maintaining the surveillance process. The team should explore storage of electronic PDQs during Beta to ensure compliance with GDPR legislation.
The service name could be improved. The team should ensure that words used to refer to the ePDQ are meaningful to patients. The team has anecdotal/implicit knowledge that hospital staff are familiar with the service and the terms used, but this assumption should be tested at Beta. Good service names are generally verbs, not nouns. Ideally, they should describe the task expected of the user - for example ‘Tell us how your surgical wound is healing’ might be more meaningful to patients than ‘Surgical wound healing post-discharge questionnaire’.
The ePDQ is built on the Forms4Health form builder and Aire Logic stated that this is WCAG AA compliant. The team explained that it has fixed some issues but wasn’t able to say which checking tools had been used to check that the ePDQ can be used by patients using assistive technology.
Because the paper equivalent isn’t being withdrawn, this continues to be an option for patients unable to use the online version. However, an accessible ePDQ could represent a better option for people with partial or no sight, for example. The team will continue to work with PHE’s digital team during Beta phase to ensure the service is accessible to users, and could consider an accessibility audit.
Design and content
The panel were impressed with the way in which design and content issues were reviewed and evaluated during the Alpha. For example, the team explored different wording for notifications to patients and applied the NHS logo to GOV.UK Notify emails to increase trust and likelihood of a response. During the Beta phase, further work should be done to identify ways to increase trust in email notifications by including the patient name, hospital name or less identifiable information about the operation itself. Private hospitals are unlikely to use the NHS logo so other options should be explored, working with GOV.UK Notify to resolve issues.
The content of the questionnaire was not changed in the transition from paper to online, due to the fact it has been validated for data collection and in use since 2008, and also to ensure consistency in data across multiple channels. Consequently, the team’s user research focused on whether the digital channel added value, not on ePDQ or caseload management system content. The panel believes that the team should be open to changes over time on the overall question set based on an increased data set and use of the digital channel.
In the assessment, the team explained that patient-entered PDQ data is not recorded for surveillance but is used to trigger a triage process in which hospital staff identify patients who are suffering from infected wounds and then enter information into the system. Therefore, it’s possible that the patient-facing process could be improved further by revisiting the questions in the PDQ with content designer input.
Technology
The panel were impressed with the project's approach to choosing a tool for the ePDQ. Considering two tools already used in house and Aire Logic’s Forms4Health platform. It was clear the team understood the user need in assessing solutions for the ePDQ. Based on the use case, the project went forward with the Forms4Health platform to support the work in Alpha as the other options did not provide the necessary functionality.
The project has yet to decide on what technology to use to support the ePDQ moving in to Beta. However, Forms4Health supports open standards and the modular architecture of the prototype service will allow for another option to be implemented if necessary. This commitment to a flexible architecture was encouraging.
The team have prototyped the caseload management system as a .net application. The panel was pleased that the project have considered integrating this as a module into the wider Surgical Site Infection Surveillance Service (SSISS) casework solution in the future, and further options for reuse were investigated as part of the Alpha. The code base is hosted on Bit Bucket and will be shared for further development on the casework solution.
During Alpha, to protect users’ personal data it was only held in memory. Moving in to Beta the service’s infrastructure will be managed by PHE’s IT team. From the discussion with the team, the panel were confident that suitable security measures and disaster recovery will be in place.
GOV.UK Notify is being used to communicate with users. The URLs presented to the user are adaptable to specific service providers, which may be used to increased trust with non-NHS service providers. Further customisation such as branding is to be explored as part of Beta.
Analytics
The panel were pleased with the amount of thought that the team had given to analytics. The team were able to clearly articulate the KPIs that will be used to manage the service, which is based on the existing paper process.
Beta – full surveillance pilot
- Before and after comparison of workload and expense for administration and recording of PDQ
- User satisfaction – feedback site, surveys
- By surgical category
- Patients opting for paper/phone v. ePDQ channel preference
- Patients completing paper/phone v. ePDQ
- Change in proportion of patients returning completed PDQ
- SSI risk by completion v. non-completion of PDQ
Live
- Increase in number of hospitals using PDQ
- Impact of ePDQ data on outlier detection v. inpatient and readmission data
The panel encourage the team to consider how these KPIs can be used to improve upon both the digital and analogue channels. The anticipated increased volume of returns may provide data for where all channels of the service could be improved.
Summary
Overall the team presented a clear vision for the service and well-scoped prototypes of the ePDQ and caseload management system. There are however areas that the panel believe the team will need to develop their understanding of during Beta.
The panel concluded that the service met the standard because:
- The Alpha demonstrated the potential for an ePDQ and caseload management system to increase the uptake of PDQs and their completion rate using a digital channel for patient responses and workflow improvements for hospital staff.
- The team showed good practice in iterating and improving their work based on testing with a cross-section of users and applied strategic decision making on technology to agree the best approach for the prototypes, including testing the application of GOV.UK Notify to send notifications and reminders.
- Sufficient user research has been completed to move into a Beta phase where more detailed research, development and testing can take place.