Meet the winners of MedAI: Transparency in Medical Image Segmentation challenge

The winners of NORA’s first dataset challange were announced in November 2021. MedAI: Transparency in Medical Image Segmentation was a challenge held in connection with the Nordic AI Meet conference and focused on medical image segmentation and transparency in machine learning (ML)-based systems. 

NORA, in association with Simula, proposed three tasks to meet specific gastrointestinal image segmentation challenges collected from experts within the field, including two separate segmentation scenarios and one scenario on transparent ML systems. The latter emphasized the need for explainable and interpretable ML algorithms. NORA provided a development dataset for the participants to train their ML models, tested on a concealed test dataset.

The challenge gathered huge interest among young researchers and a total of 17 papers were submitted, answering the challenge. Read all the submitted papers in the first volume of  the journal Nordic Machine Intelligence.

NORA and Simula would like to thank everyone who participated in the competition. 

Here you can meet the winner and runner up.

The winner

Adrian Galdran

Adrian Galdran, is a researcher on biomedical image analysis. Currently he is enjoying a Marie Sklodowska Curie fellowship that allows him to do research under the supervision of Gustavo Carneiro from the University of Adelaide and Miguel Ángel González-Ballester, from Universitat Pompeu Fabra, in Barcelona. His education is as a mathematician, although his Ph.D. was on image processing, and after that he has been working almost exclusively on computer vision for medical image analysis. We have taken the opportunity to ask Galdran some questions about his experience with the challange. You can read his winning paper here.

 

Q&A with Adrian Galdran

How did you come across the competition?

I had already participated on previous competitions organized by the same team, and I got an email from the organization informing me about MedAI.

 

What triggered you to participate in the competition?

The competition I had participated previously, which was held at the International Conference on Pattern Recognition last year, was about polyp segmentation, which means that I was familiar with the kind of data in MedAI. At the same time, I had been improving my image segmentation private code since then, so I thought I would give it a try in this new competition. In addition, the folks at DataCrunch.io, a Finnish startup that provides cloud computing infrastructure, kindly donated to me some GPU time for this project, which made it possible to run many simulations in order to find the optimal configuration of my model.

 

Have you participated in other such competitions? If yes, which and when?

Yes, in the last years I have been taking part in several medical image analysis competitions, usually associated with conferences I (used to) attend. For instance, I won the ICPR polyp segmentation competition I mentioned above (Endotect), and I got the second position on the EndoCV challenge held with the ISBI conference. I do not participate on Kaggle competitions, though, I find that people there exploit too much “dirty” tricks, and everything is too competitive for my taste.

 

How would you compare this competition to other competitions?

Compared to other challenges and competitions, the main two differences were the lack of intermediate validation and the transparency track. 

You can find intermediate validation very often in other competitions, which means that there is a subset of the test data that is made public, and participants are allowed to try their solution many times against it before submitting their final predictions on the hidden part of the test set. I honestly prefer to have it the MedAI way, it prevents people from exploiting too much the data and developing methods that most likely wouldn’t generalize well outside of the competition. Regarding the transparency task, it was a very nice (and original!) addition to the standard performance-based assessment.

 

What was your biggest take away from the competition?

I would say that with a properly trained modern deep learning model, the tasks of polyp or instrument segmentation from this kind of data is almost well-solved. It does not seem that minor differences in the boundary of an instrument’s segmentation would make any relevant difference, so I think we can safely move on to greater challenges, like understanding better the surgical situation, the actions that take place, etc.

 

This competition was in the field of medical imaging. What is your thought on using AI in the medical field? What are the benefits? 

The technology that we have these days is very mature for solving repetitive tasks and carrying out pattern recognition at an incredibly large scale, in terms of the amount of data that neural network can consume. AI models are a bit dumb, though, meaning that they can amazingly do pattern recognition, but their ability for reasoning in anything similar to a human baby is ultra-limited. So, I would say that the benefits are the tremendous amount of data that we can process at very high speed, not so much the augmented reasoning we get out of AI these days.

 

What is the most exciting part of working with AI for you?

Probably the velocity at which the field is expanding and evolving. My Ph.D. supervisor was an applied mathematician, solving Partial Differential Equations and things like that, and I remember when I used to tell him “this paper is from five years ago, it is outdated”, he would answer “how is that possible? In my field, papers from decades ago are still useful nowadays”. I find it challenging but also very rewarding that one needs to say all the time “checking the news” of computer vision and machine learning research if one wants to stay current.

 

The runner up

Debayan Bhattacharya

The runner up was a team consisting of Debayan Bhattacharya, PhD Candidate, Dr.rer.nat. Dennis Eggert, Dr.med. Christian Betz and Dr.Ing Alexander Schlaefer. You can read their paper here

We have taken the oppurtunity to talk to Bhattacharya, who completed his Bachelors in Electronics and Communication Engineering from Thiagarajar College of Engineering, India in 2016. After his Bachelors, he worked for two years as a software developer in two startups in India. In 2018, he came to Germany to pursue his Masters in Hamburg University of Technology to make a career change towards Artificial Intelligence. He completed his masters in February of 2021 and decided to pursue a PhD under Prof. Dr. Alexander Schlaefer and Prof. Dr. Christian Betz.  His Ph.D. is focused on finding novel Artificial Intelligence based solutions to detect and diagnose cancers occurring in the ear, neck, throat and head.

 

Q&A with the runner up

How did you come across the competition?

Debesh Jha tweeted about this competition. 

 

What triggered you to participate in the competition?

There are multiple answers to this question. Firstly, I admire the works published by the organizers of this competition. Simula Research Laboratory has published noteworthy works in the domain of polyp segmentation. Seeing that the competition was being organized by people whose papers I have read, I decided that it would be best to take part in this challenge.

Secondly, the three tasks were challenging. The transparency task especially was really interesting. It mandated that the work that was submitted to the competition was reproducible. Furthermore, the participants were required to explain the model predictions. This was an eccentric requirement of the competition which I have not come across in any other competition.  

 

Have you participated in other such competitions? If yes, which and when?

No. 

 

How would you compare this competition to other competitions?

The transparency task was one of the highlights of this competition. What made this task the most fun was that it was left entirely up to the participants to submit what they thought made their work more transparent. I enjoyed doing this task! 

 

What was your biggest take away from the competition?

Firstly, I was very happy to see that my model performed better than many ensemble models. Secondly, I gained lot of experience in writing reproducible code. This is necessary especially in the rapidly evolving domain of Artificial Intelligence.  Thirdly, I learned the best ways to perform hyper parameter tuning of my models thanks to this competition. 

 

This competition was in the field of medical imaging. What is your thought on using AI in the medical field? What are the benefits? 

AI has huge potential in the medical field. I do not see AI as a complete substitute for doctors/physicians. However, I do see a future where AI based solutions work side by side with doctors and physicians. Taking the example of polyp segmentation, I believe clinics of the future will have portable AI based Computer Aided Diagnostics Systems installed. These AI CADx will give doctors a secondary opinion on the colonoscopy videos and can surely help in reducing misdetection of polyps. 

 

What is the most exciting part of working with AI for you?

Firstly, the most exciting part of working with AI is that it is still a black box. This gives tons of possibilities to discover new insights into how this black box operates from within. The more we understand the decision making process of AI models, the more likely such solutions will be trusted by people. Secondly, the artificial intelligence community is really great! Papers which would have been blocked behind pay walls are being openly shared as preprints. This makes it relatively easy for a novice to enter and for an expert to progress in this domain. I particularly like that many researchers have a code first approach to their research. Sharing code is one of the most important drivers of progress in this domain and I am glad this community is actively making an effort to publish reproducible code. 

Av Anam Javaid
Publisert 14. des. 2021 09:19 - Sist endret 15. des. 2021 12:23