SECRET Competition
The SECRET Competition which began in March 2023 with the release of the models concluded with a Conference day held on the 23rd October at the University of Glasgow.
At the conference each of the top-ranked participants was invited to give a presentation describing their method. This was followed by the presention of the assessment criteria and then an award ceremony.
The conference day continued with an open discussion about the assessment criteria and lessons learnt from the competition, both in terms of model development and running future competitions.
This was followed by a conference dinner held in the Glasgow Grosvenor Hotel.
The winner of the SECRET Competition wasUniversity of Glasgow PhD student William Ryan.
Runners up were:
Professor Iain Murray from the University of Edinburgh and
the University of Exeter team: Louise Kimpton, Victoria Volodina, Xiaoyu Xiong,Victor Applebaum, Michael Dunne, Jake Hollins and James Salter
Highlights of the day can be viewed on the slide show and video links to the right
Background on the Competition
Statistical Emulation for Computational Reverse Engineering and Translation
SECRET international competition in cardiac modelling
This competition aimed to assess computational tools for accurate, robust and computationally efficient inference of unknown parameters in complex cardiovascular biophysical models from physiological data. Two state-of-the-art computational fluid dynamics (CFD) models of the pulmonary and systemic circulation were provided. The models served as a benchmark to assess competitors' methods for fast model parameter inference and uncertainty quantification (UQ). In order to evaluate the performance of a competitior's method synthetic data was provided simulated from the aforementioned CFD models with pre-set parameter values, subject to additive observational noise. The true parameter values were revealed after all participants have submitted their answers.
Competitors were asked to submit
(i) estimates of the biophysical parameters,
(ii) the corresponding “best” data prediction for the vessels of interest (for which data have been provided),
(iii) a measure of estimation uncertainty: ideally the full posterior distribution of (a) the biophysical parameters and (b) of two points in function space (the minimum and maximum of the time-series signal) for every vessel of interest .
Note: for UQ, if standard deviation only is provided, the organisers will use a Gaussian approximation to draw samples.
The assessment of their method was based on accuracy and UQ in parameter and function space.
The competition consisted of two stages.
Stage one: Familiarisation with the model and construction of a statistical emulator for the quantity of interest well in advance of the data being released.
Stage two: After the data was released, a limited time interval was given (one week) to conduct the parameter estimation and UQ analysis and to submit predictions. This limitation is with clinical translation in mind, to mimic clinical practice and decision support.
The participants with the best three entries for each model have been invited to disseminate their work in a one-day conference hosted at the University of Glasgow. The best three entries for each model will also receive a certificate and a financial award, details of which will be announced later. The final goal of the competition is a research publication in a high-impact journal with the first three ranked submissions, which will provide a platform to disseminate the statistical methods and results obtained.
Timeline:
16th January – Competition announcement call out
20th March – Release of the models and the challenge description
1st September – Release of the data
8th September – Deadline for participants to submit their predictions
18th September – Organisers make the correct answers available and release the best three entries for each model
23rd October – Conference day
23rd October onwards – Work towards a joint publication
This competition is organised as part of the SofTMech Statistical Emulation and Translation Hub: https://gow.epsrc.ukri.org/NGBOViewGrant.aspx?GrantRef=EP/T017899/1.
- Event Organiser: Mihaela Paun from the University of Glasgow
- Co-organisers: Dirk Husmeier, University of Glasgow, Mitchel Coleback, University of California,Mette Olufsen and Alyssa LaPole, North Carolina State University