Uncertainty Quantification in AI and Machine Learning

Eyke Hüllermeier is a full professor at the Institute of Informatics at LMU Munich, Germany, where he holds the Chair of Artificial Intelligence and Machine Learning. He studied mathematics and business computing, received his PhD in Computer Science from Paderborn University in 1997, and a Habilitation degree in 2002. Before joining LMU, he held professorships at several other German universities (Dortmund, Magdeburg, Marburg, Paderborn) and spent two years as a Marie Curie fellow at the IRIT in Toulouse (France). His research interests are centred around methods and theoretical foundations of artificial intelligence, with a particular focus on machine learning, preference modelling, and reasoning under uncertainty. He has published more than 400 articles on related topics in top-tier journals and major international conferences, and several of his contributions have been recognised with scientific awards. Professor Hüllermeier is Editor-in-Chief of Data Mining and Knowledge Discovery, Associate Editor of the IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), and serves on the editorial boards of several other AI and machine learning journals. He is Fellow of ELLIS and IFSA, and President of EuADS, the European Association for Data Science. He is also a member of the Strategy Board of the Munich Center for Machine Learning (MCML), and a member of the Steering Committee of the Konrad Zuse School of Excellence in Reliable AI (relAI).

He leads the tutorial that anchors the day, introducing uncertainty quantification in a way that is accessible to researchers from every area of AI.

Tutorial abstract

Uncertainty in Machine Learning: From Aleatoric to Epistemic 

Due to safety requirements in practical applications, the notion of uncertainty has recently received increasing attention in machine learning research. This talk will address questions regarding the representation and adequate handling of (predictive) uncertainty in (supervised) machine learning. A particular focus will be put on the distinction between two important types of uncertainty, often referred to as aleatoric and epistemic, and how to quantify these uncertainties in terms of appropriate numerical measures. Roughly speaking, while aleatoric uncertainty is due to the randomness inherent in the data generating process, epistemic uncertainty is caused by the learner's ignorance of the true underlying model. Some conceptual and theoretical issues of existing methods will be identified, showing the challenging nature of uncertainty quantification in general and the disentanglement of aleatoric and epistemic uncertainty in particular.

Camilla Lingjærde is a Postdoctoral Fellow in Statistics and Machine Learning at the University of Oslo. She received her PhD in Statistics from the University of Cambridge in 2023. Her current research is focused on advancing the field of machine learning by developing new supervised learning methods that, by integrating causal inference principles, exhibit enhanced robustness, particularly in their ability to generalise to new domains.

Hands-on session abstract: Building on the morning tutorial, this hands-on session explores uncertainty quantification in machine learning through a real-data use case. Through guided exercises in R, participants will investigate different sources of uncertainty, assess the accuracy of uncertainty estimates, and reflect on their implications for prediction and decision-making. The session will focus on interpreting results, identifying limitations, and connecting the concepts introduced in the tutorial to practical modelling.

Uncertainty Quantification: Ideals vs. Practice

The panel turns from the tutorial and hands-on session to a broader question: how far uncertainty quantification actually gets applied once methods leave the seminar room. Panelists will discuss how much UQ genuinely features in real-world systems, whether the methods introduced earlier in the day hold up for industrial deployment, and what tends to happen when uncertainty is poorly estimated or ignored altogether, drawing on perspectives from academia, contracted research, and industry.

About the panel:

Prof. Ingrid Kristine Glad — Moderator 

Professor in Statistics and Data Science, University of Oslo; Director, Integreat Center of Excellence (Norwegian Center for Knowledge-driven Machine Learning)

Professor Ingrid K. Glad is a leading figure in Statistics and Data Science at the University of Oslo, where she directs Integreat, Norway's center of excellence in Machine Learning and AI. Her research spans high-dimensional regression, Bayesian inference, and change- and anomaly detection in data streams, bridging theoretical advances with practical applications in genomics and industrial settings.

Prof. Eyke Hüllermeier, LMU Munich

See keynote section above.

Dr. Camilla Lingjærde, University of Oslo

See hands-on speaker session above.

Dr. Alex Lenkoski, Head of Data Science, Wheelhouse 

Alex Lenkoski is an applied statistician primarily focused on building forecasting systems across a broad range of applications, from power systems, meteorology and complex digital markets. He is currently the Head of Data Science at the dynamic pricing firm Wheelhouse, focusing on the global short-term rental market. In addition, he has an adjunct position at the Norwegian Computing Center (NR), where he investigates the potential for unintended interaction of reinforcement learning algorithms operating in complex markets as part of the TRUST AI center. He was previously a Research Leader at NR, leading the climate and environmental research area.

Dr. Kjetil Olsen Lye, SINTEF Digital

Kjetil Olsen Lye is a Research Scientist at SINTEF Digital in Oslo, working at the intersection of uncertainty quantification, machine learning, and high-performance computing, with a strong focus on industrially relevant applications. Recently, he has also investigated how agentic AI can help steer and manage UQ workflows. He holds a PhD in applied mathematics from ETH Zurich and a master’s degree in pure mathematics from the University of Oslo.