MedAI: Transparency in Medical Image Segmentation

We propose a task that focuses on medical image segmentation and transparency in machine learning-based systems. We propose three tasks to meet specific gastrointestinal image segmentation challenges collected from experts within the field. This includes two different segmentation scenarios and a task on transparent machine learning systems that emphasizes the need for explainable and interpretable machine learning algorithms.

Motivation

Medical image segmentation is a topic that has garnered a lot of attention over the last few years. Compared to classification and object detection, segmentation gives a more precise region of interest for a given class. This is immensely useful for the doctors as it not only specifies that an image contains something interesting but also where to look at which also provides some kind of inherent explanation. Colonoscopies are a perfect use-case for medical image segmentation as they contain a great variety of different findings that may be easily overlooked during the procedure.

Furthermore, transparent and interpretable machine learning systems are important to explain the whys and the hows of the predictions. This is especially important in medicine, where conclusions based on wrong decisions resulted from either biased or incorrect data, faulty evaluation or simply a bad model could be fatal.

Therefore, the "MedAI: Transparency in Medical Image Segmentation" task aims to develop automatic segmentation systems that are transparent and explainable.

Award 

First Prize: 5000 Euro

Second Prize: 2500 Euro

The winning team will be announced during the NordicAIMeet2021 hosted on the 1-2nd of November at the Oslo Kongressenter. 

If you are based in the Nordic region we also encourage you to register for the Nordic AI Meet 2021. Also submit your abstract if you are interested in giving a talk. 

Important Dates: 

  • Development dataset release: 5th of August, 2021

  • Test dataset release: 23th of September, 2021

  • Participants submission of results: 27th of September, 2021 

  • Evaluation results for participants: 8th of October

  • Methods description paper submission: 15th of October

The winning team will be announced during the NordicAIMeet2021 hosted on the 1-2nd of November at the Oslo Kongressenter.

Registration 

To register for the competition, click here. 

The participants will be invited to submit to the following tasks:

1. Polyp Segmentation Task

The polyp segmentation task asks participants to develop algorithms for segmenting polyps in images taken from endoscopies. The main focus of this task is to achieve high segmentation metrics on the supplied test dataset. Since last year, we have extended the development dataset and created a new testing dataset to which the submissions will be evaluated on.

2. Instrument Segmentation Task

Similar to the polyp segmentation task, the instrument segmentation task asks participants to develop algorithms for segmenting instruments present in colonoscopy videos. The main focus of this task is to achieve high segmentation metrics on the supplied test dataset. 

3. Transparency Task

The transparency task tries to measure the transparency of the systems used for the aforementioned segmentation tasks. The main focus for this task is to evaluate systems from a transparency point of view, meaning for example explanations of how the model was trained, the data that was used, and interpretation of a model's predictions.

To compete for the prize money all three tasks are mandatory. Submissions of only one sub task are allowed, but will not be eligible for winning any of the prizes.

Data

For the polyp segmentation task the development dataset can be downloaded here: https://datasets.simula.no/kvasir-seg/

The development dataset for the instrument segmentation task can be downloaded via: https://datasets.simula.no/kvasir-instrument/

The final test dataset will be released 23rd of September. Details how to obtain the test dataset and what to submit for the final evaluation will be provided directly to the registered participants. The test dataset will be publicly released after the competition is finished. 

The transparent machine learning system task will be based on the previous two tasks and will use each respective dataset.

Evaluation Methodology

Polyp Segmentation Task and Instrument Segmentation Task

For the polyp segmentation task and the instrument segmentation task, we will use the standard metrics commonly used to evaluate segmentation tasks. This includes the Dice coefficient, pixel accuracy, and the Intersection-Over-Union (Jaccard index). The metric which will be used to rank submissions will be the Intersection-Over-Union coefficient.

Transparent Machine Learning Systems Task

For the Transparent Machine Learning Systems Task, we perform a more qualitative evaluation of the submission. Here, a multi-disciplinary team will evaluate the submissions based on how transparent and understandable they are (e.g., availability of the code, were explainable AI methods used, etc.).

Competition Proceedings

All participants are asked to submit a 4 page paper (single column, plus 1 additional page for references) describing their method and results. The submitted papers will be reviewed single blind and will be published. Outstanding submissions will be invited to submit a full length paper to a special issue about the competition.

Innovation and Focus

Medical image segmentation is an application that is immensely useful for medical professionals as it directly ties a classification to a region of interest. As this is primarily a vision problem, we aim to motivate multimedia researchers working in medical image segmentation to develop and validate their approach on a standard dataset.

Addressing the challenges related to polyp segmentation can have a social impact soon and would be a feed-forward step in clinical translation. We believe that this polyp segmentation challenge can help develop a strong benchmark for consistent evaluation.
 

List of Task Organizers

For more information about the competition, please email Michael Riegler at michael@simula.no 

 

References and Recommended Reading

[1] Borgli, H., Thambawita, V., Smedsrud, P.H. et al. HyperKvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy. Sci Data 7, 283 (2020).

[2] Jha D. et al. (2021) Kvasir-Instrument: Diagnostic and Therapeutic Tool Segmentation Dataset in Gastrointestinal Endoscopy. In: Lokoč J. et al. (eds) MultiMedia Modeling. MMM 2021. Lecture Notes in Computer Science, vol 12573.

[3] Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation. In Proceeding of International Conference on Medical image computing and computer-assisted intervention (MICCAI), 234-241, 2015.

[4] Weller A. Transparency: motivations and challenges. InExplainable AI: Interpreting, Explaining and Visualizing Deep Learning 2019 (pp. 23-40). Springer, Cham.

Publisert 2. juli 2021 02:16 - Sist endret 4. aug. 2021 16:07
Del på e-post