As artificial intelligence (AI) and machine learning become increasingly embedded in research using sensitive data, Trusted Research Environments (TREs) face an important challenge: how can we determine whether a trained machine learning model can safely leave a secure environment?
Across the UK and internationally, researchers are using machine learning to help tackle complex challenges. But while traditional research output processes are well-established, there is currently no shared approach to evaluating if a trained machine learning model can be safely released.
TREvolution’s ambition is to support interoperability, federation and trusted research across the UK’s TREs. Through developing tools such as SACRO-ML (Semi-automated Checking of Research Outputs and Support for AI), the programme is bringing the community together to explore practical ways to assess the disclosure risk of these models and build a consensus across organisations.
Bringing the community together
A recent TREvolution workshop brought together experts from across the research and data community to take the first steps towards developing a shared, practical approach.
Around 100 participants joined the online TREvolution workshop, Can the Model Leave the Room? Creating Consensus on the Egress of ML in Trusted Research Environments.
The workshop brought together a diverse community spanning universities, NHS and healthcare organisations, regulators, public agencies, research infrastructures, industry and international organisations. Output checkers, TRE operators, governance specialists and AI and machine learning experts contributed their perspectives to a challenge that increasingly sits at the intersection of technical risk, operational practice and public trust.
Despite growing interest in using machine learning within TREs, there is currently limited consensus on how trained models should be assessed before release. Different organisations may apply different processes, creating uncertainty for researchers and challenges for those responsible for protecting sensitive data. This workshop explored a central question: can we develop a consistent, proportionate and evidence-based approach to model release that works across different TREs?
Why do machine learning models present a different challenge?
TREs provide secure environments in which approved researchers working on approved projects can access and analyse sensitive data without the underlying data leaving the environment. Before research outputs such as tables, graphs or statistical results are released, they undergo check to ensure that confidential information cannot be inadvertently disclosed or remodelled from the outputs.
Trained machine learning models bring new complexities to this. Unlike many conventional statistical outputs, a trained model is a reusable tool whose behaviour has been shaped by the data used during training. Depending on its design, training data and intended use, a model may retain information about individual records it trained on. This can then cause risk as it could technically be attacked to reveal sensitive information about dataset it trained on.
This means that releasing a trained model may create disclosure risks that are not adequately addressed through conventional output-checking approaches alone.
However, not all machine learning models present the same risks, and treating every model as equally sensitive risks creating unnecessary barriers to responsible research.
The workshop introduced an emerging categorisation of models based on how they generate predictions:
- Comparison with stored data: Models such as nearest-neighbour approaches, where predictions depend on comparisons with stored observations – and these stored training records can be directly extracted from the model.
- Sequence-based prediction: for example, generative AI, and also large pre-trained models that have been “fine tuned” to specific tasks. These present important disclosure concerns around memorisation, reconstruction and inference.
- Predictions made independently for each record: Models trained for regression, classification and image-segmentation tasks can present risks such as membership inference, reporting on small groups and class disclosure.
While this categorisation remains under development, it offers a promising starting point for identifying shared risks and applying appropriate assessment techniques without requiring output checkers to become experts in every individual machine learning algorithm.
Growing adoption, but limited confidence
Interactive polling during the workshop offered an insight into how machine learning is already being used within TREs.
Half of respondents said that their service permits machine learning, while around a third reported that permission depends on the circumstances.
These responses suggest that machine learning is already part of TRE operations, but that policies and practices are not always clear or consistently understood. Polling on confidence in assessing machine learning models both before and after the session also showed that participants were apprehensive about doing this. This can create uncertainty for researchers and make it difficult for TREs to develop consistent processes.
The discussions reinforced the need for practical guidance that can support decision-making without requiring every output checker to become a machine learning specialist.
TREs need approaches that are technically robust, but also understandable and usable by the staff making release decisions.
Supporting AI readiness
The workshop forms part of wider work being undertaken through TREvolution, exploring tools, methods and governance approaches for machine learning disclosure-risk assessment as part of the work on SACRO-ML
Rather than focusing only on technical solutions, the work recognises that AI readiness requires a combination of tools, processes, guidance and community learning.
By bringing together technical experts, governance specialists and TRE practitioners, TREvolution aims to support the development of approaches that are robust, transparent and practical for real-world use.
How you can support this work
The workshop is just the beginning of this collaborative effort, and we welcome continued contributions from across the research and TRE community. Whether you are interested in exploring emerging approaches to model egress, accessing practical tools or contributing your expertise to the discussion
By sharing experiences, identifying challenges and contributing practical insights, we can collectively develop guidance that is robust, proportionate and workable across different TRE settings. We encourage everyone working with machine learning in secure research environments to get involved. in the next phase of this work.
About TREvolution
TREvolution is a programme of work shaping the future of sensitive data research in the UK by promoting shared standards and building trustworthy innovations to make research faster and more effective for the public good. SACRO is one of the areas of focus for TREvolution to accelerate research and build trust.
