MODUS Seminar Program
Weekly talks in the MODUS Seminar: Wednesday, 12:15-13:45, Room S102, FAN-B
Individual talks may be held via zoom or at different times, see the information, below.
Further talks will be announced as the semester progresses. To receive email notifications about upcoming talks, please subscribe to the MODUS elearning course.
Summer Semester 2025
21.05.2025, 12:15h
Andreas Löhne
Professur Mathematische Optimierung, Universität Jena
Two Applications of Multi-Objective Linear Programming
Abstract:
We explore two less conventional applications of multi-objective linear programming (MOLP).
In the first application, we employ a MOLP solver to compute geometric operations on polyhedra, including the Minkowski sum, intersection, and the convex hull of their union.
The second application addresses multi-objective optimization problems involving two sequential decision makers. The first decision maker controls a subset of variables and acts initially, while the second controls the remaining variables and acts subsequently. This setting gives rise to a multi-stage decision problem, which can be interpreted as an optimization problem with a set-valued objective function—commonly referred to as a set optimization problem. We examine this framework in detail, discussing solution methodologies, the underlying decision-making structure, and practical applications. Furthermore, we highlight the equivalence of such problems to classical multi-objective linear programs.
28.05.2025, 12:15h
Giovanni Fantuzzi
Chair for Dynamics, Control, Machine Learning and Numerics, FAU Erlangen-Nürnberg
Understanding transformers: hardmax attention, clustering, and perfect sequence classification
Abstract:
Transformers are an extremely successful machine learning model, famously known for powering platforms such as ChatGPT. What distinguishes them from classical deep neural networks is the presence of "attention" layers between standard "feed-forward" layers. In this talk, I will discuss how simple geometrical rules can explain the role of the attention layers and, consequently, the outstanding practical performance of transformers. Specifically, by focussing on a simplified class of transformers with "hardmax" attention, I will first show that attention layers induce clustering of the transformer's input data. I will then use this clustering effect to construct transformers that can perfectly classify a given set of input sequences with arbitrary but finite length, modelling, for example, books to be classified by a library. Crucially, the complexity of this construction is independent of the sequence length. This is in stark contrast to classical deep neural networks, explaining (at least in part) the superior performance of transformers for sequence classification tasks.