FIREDANSE will provide an easy-to-use and efficient way for human experts (pathologists) to record “ground truth” observations about medical images (in our case, digitised slides).
When Artificial Intelligence (AI) answers a medical question, we want to know whether the “algorithm got it right”. This means we need the “correct answer” independently as determined by human experts.
This is important at every stage of the use of AI in medicine, from initial creation of the algorithm through many steps until it “goes live” in hospitals and then starts making decisions about real clinical questions for real patients.
Up until now, generating “ground truth” has been difficult for three main reasons: (a) the experts whose opinions we need are busy NHS staff, weighed down by heavy clinical workloads; (b) logistically, it is challenging to install the required software on every medical expert’s computer for them to perform review tasks efficiently; (c) the images to be assessed are scattered across multiple Trusted Research Environments (TREs) and are not easily accessible to the staff who need to view them.
These issues are important, because our project survey reveals that, for two-thirds of respondents, it is only if results are checked against what a human specialist says that they would be happy for AI to be used as part of their own healthcare.
Moreover, despite widespread interest in AI, it is proving very challenging for hospitals to introduce new tools and to monitor their performance “in the real world”. Success of the FIREDANSE (Federated Infrastructure for Digital Pathology Reporting and Expert Data Annotation in a Secure Environment) project will make it quicker and easier to achieve this, ultimately benefitting patients by increasing the speed and accuracy of diagnoses and treatment decisions.
FIREDANSE is led by the Institute of Cancer Research, with partners Imperial College and the Royal Marsden Hospital, working together with international collaborators from Erasmus Medical Center in the Netherlands.
What does FIREDANSE aim to achieve?
As a DARE UK Next-Gen Catalyst, FIREDANSE will develop new capabilities for the TRE community. Specifically, we will create web-based image-review system for pathologists featuring the following features:
- A task-driven process – individual pathologists will engage directly with a worklist of well-defined items, with a simple click taking them to the data and making reporting straightforward — hassle minimised and no possibility of loading the wrong images;
- Secure online slide access – collaborating pathologists working in arbitrary locations will use a web-browser to log onto a TRE portal, from where there will be seamless access to slides held in all other collaborating TREs — no additional software to install;
- No access to the data outside of the tool – files cannot be downloaded and the slides can be used only for the specific annotation task — no security issues.
By the end of the project, the FIREDANSE team aims to create:
- A programme of public engagement, involving a survey, an education session and focus group, a project website with resources and a final presentation showcase;
- project infrastructure connecting two TREs (at the Royal Marsden and Imperial)
- a worklist manager — this is the “app” that will distribute the right task to the right pathologist;
- a web-based pathology review app — this will allow the pathologist to look at the slides, annotate them with comments, draw regions-of-interest and fill in forms;
- an information governance review of the newly created platform; and
- an exemplar research study.
Progress so far
In the first six months, the FIREDANSE team has:
- Created and distributed a survey as part of the public engagement programme
- Completed a fully functional “Phase 1” version of the task manager that is ready to be used in the exemplar project
- Created an initial release of the pathology review app, with further refinements coming later in the project
The remaining time will be spent moving forward with the public engagement programme, gaining insight from public representatives on the project steering group. With an outline solution already produced, we will start work on the information governance review and commence the pilot scientific study. Our outputs will include both documents and presentations aimed at a lay audience and an academic publication describing the new solution at a technical level.
FIREDANSE supports the DARE UK vision of enabling better, faster, and more trustworthy sensitive data research by removing key “blockers” that prevent otherwise eager pathologists from taking part in research studies, and it will increase participation from experienced clinicians in NHS Trusts that are not specialist academic research centres.
About the DARE UK Next-Gen Catalysts
The DARE UK Next-Gen Catalysts are early-stage, exploratory projects developing and testing the next generation of capabilities for Trusted Research Environments (TREs) across the UK. Bringing together researchers, technology specialists, public contributors and other partners, these projects explore bold new ideas that could expand how sensitive data is accessed, analysed and governed while maintaining privacy, security and public trust. Their role is to generate evidence, prototypes and practical learning that can help shape the future of secure, trustworthy data research for public benefit.
