- UCL Home
- Research institutes
- ICTEAM
- INMA
- Seminars
Seminars
The Departement of Mathematical Engineering organizes a series of seminars. The seminars are usually held on Tuesday from 2:00pm to 3:00pm in the Euler lecture room, Building EULER, av. Georges Lemaître 4-6, Louvain-la-Neuve (Parking 13). Be mindful that exceptions may occur; see the talk annoucements.
If you wish to receive the seminar announcements by email, please send an email to Etienne Huens.
Master students can take this seminar for credit in either of the two semesters; see LINMA2120 for more information.
Seminars to come
Previous seminars
| 21/03/2023 (14h) [Location: Euler building (room A.002)] Pauline Bernard (Mines Paris) Towards a Unified Perspective to Observer Design |
|
We start by reviewing the main techniques of state observer design for continuous-time dynamical systems. Starting from necessary conditions for the existence of such asymptotic observers, we classify the available methods depending on the detectability/observability assumptions they require. We show that each class of observer relies on transforming the system dynamics into a particular normal form which allows the design of an observer, and how each observability condition guarantees the invertibility of its associated transformation and the convergence of the observer. Then, we give a particular focus to the theory of KKL or nonlinear Luenberger observers. This design consists in looking for a change of coordinates changing the system dynamics into a contracting filter of the output. A state estimate is then obtained by implementing this filter from any initial condition and left-inverting the transformation, which is shown to be possible under a very weak backward-distinguishability property. We show how, unlike other methods, this idea is universal and can be extended to any type of dynamical systems (continuous or discrete time, autonomous or discrete, observable or non-observable, etc), thus offering a promising road towards a unified paradigm for observer design. |
| 14/03/2023 (14h) [Location: Euler building (room a.002)] Welcome Seminar |
|
Title: Randomness in Electronic Devices and Circuits Speaker: Léopold Van Brandt (UCLouvain) Abstract: The MOS transistors of smallest dimensions (below 100 nm), universally favoured for the design of digital integrated silicon circuits, are very sensitive to all types of uncertainties, notably statistical process variability. Multiple variability sources of various physical origins introduce random device-to-device fluctuations in the transistor parameters and thereby affect the performance of the circuits. In this presentation, we will mainly focus on the prediction of the failure probability of Static Random Access Memory (SRAM) bitcells. The estimation of probabilities as extremely low as the ppm (i.e. parts-per-million) with limited computational resources and time is a challenge. We will start by emphasising the limitations of the conventional Monte-Carlo simulation approach, before introducing our semi-analytical prediction methodology that turns out to be fast, accurate and insightful. We will close this talk by mentioning some on-going work, where we attempt to apply fundamental physical results to electronic devices and circuits. Title: Lyapunov functions for open multi-agent dynamics Speaker: Renato Vizuete (UCLouvain) Abstract: Open multi-agent dynamics are systems characterized by arrivals, departures, or replacements of agents during the evolution of the dynamics. These types of systems are better approximations of modern networks where agents constantly enter and leave the systems, modifying the behavior of the group (e.g., social networks, epidemics, vehicle platoons, swarm of robots). Unfortunately, due to the complexity of the analysis, most of the works have been focused on linear systems, which are limited to few practical applications. In this talk, we present an approach based on Lyapunov functions for the analysis of more complex open multi-agent dynamics where arrivals, departures and replacements are determined by stochastic processes. In the first part, we introduce the techniques based on the formulation of stochastic differential equations with an application to a SIS epidemic subject to replacements of agents. In the second part, by analyzing the Hegselmann-Krause dynamics, we show that some common Lyapunov functions might not be adequate in open scenarios characterized by arrivals and departures of agents. |
| 07/03/2023 (14h) [Location: Euler building (room A.002)] Jeffrey Larson (Argone National Laboratory, USA) Structure-Aware Methods for Expensive Derivative-Free Nonsmooth Composite Optimization |
|
We present new methods for solving a broad class of bound-constrained nonsmooth composite minimization problems. These methods are specially designed for objectives that are some known mapping of outputs from a computationally expensive function. We provide accompanying implementations of these methods: in particular, a novel manifold sampling algorithm with subproblems that are in a sense primal versions of the dual problems solved by previous manifold sampling methods and a method that employs more difficult optimization subproblems. For these two methods, we provide rigorous convergence analysis and guarantees. We demonstrate extensive testing of these methods. |
| 28/02/2023 (14h) [Location: Euler building (room A.002)] Titouan Vayer (OCKHAM, LIP, ENS Lyon, France) Towards Compressive Recovery of Sparse Precision Matrices |
|
We consider the problem of learning a graph modeling the statistical relations of the "d" variables of a dataset with "n" samples. Standard approaches amount to searching for a precision matrix Θ representative of a Gaussian graphical model that adequately explains the data. However, most maximum likelihood based estimators usually require to store the "d^2" values of the empirical covariance matrix, which is often too costly in a high-dimensional setting. In this work we adopt a “compressive” viewpoint and look for estimating a sparse Θ from a sketch of the data, i.e. a lowdimensional vector of size "n ≪ d^2" carefully designed from the data using nonlinear random features. Under certain assumptions on the spectrum of Θ, we show that it is theoretically possible to estimate it robustly from a sketch of size "m = O((d + 2k) ln(d))" where "k" is the maximal number of edges of the underlying graph and with an error that decreases in "O(n^{-1/2})". These guarantees are inspired from the compressed sensing theory and involve restricted isometric properties and instance optimal decoders. Our estimator requires solving a non-convex inverse problem and we investigate the possibility of achieving practical recovery by a variant of the Davis-Yin three operator splitting algorithm. We compare our approach with “Graphical LASSO” type estimators on synthetic datasets. Finally, we discuss in a last part the limitations and perspectives of this work, which partially answers some questions but also opens many others. |
| 21/02/2023 (14h) [Location: Euler building(room a.002)] Olivier Leblanc (UCLouvain) Interferometric Lensless Imaging: Rank-one Projections of Image Frequencies with Speckle Illuminations |
|
Abstract: Lensless endoscopy with multicore fibers (MCF) is a technology that enables fluorescent imaging of biological samples at cellular scale. In this talk, I will show that under a common far-field approximation, the corresponding speckle imaging process is tantamount to collecting multiple symmetric rank-one projections (SROP) of an Hermitian interferometric matrix---a matrix encoding a subsampling of the Fourier transform of the sample image. Specifically, each SROP of this matrix is achieved with the complex vector shaping the incident wavefront using a spatial light modulator (SLM), and, for specific MCF core arrangements, the interferometric matrix collects as many image frequencies as the square of the core number. Configuring the SLM randomly allows us to characterize the sample complexity of the system through a two-component sensing perspective. In particular, by inspecting the separate dimensional conditions ensuring the specific restricted isometry properties of the two composing sensing models in the observation of sparse images, we show that a basis pursuit denoising (BPDN) program associated with an L1-fidelity cost provably provides a stable and robust image estimate. In parallel, numerical and experimental reconstruction results demonstrate the effectiveness of this imaging procedure for a TV-penalized formulation of the inverse problem. |
| 14/02/2023 (14h) [Location: Euler building(room a.002)] Gisselson Pontus and Updhyaya Manu (Lund University) Tight Lyapunov function existence analysis for first-order methods |
|
Abstract: We present a unifying framework for establishing linear convergence rates for common first-order methods used to solve convex optimization problems. In particular, we consider i) classes of convex optimization problems of finite sum form with (possibly strongly) convex and (possibly) smooth functional components, and ii) first-order methods that can be written in so-called state-space form, i.e., as a linear system in feedback interconnection with the subdifferentials of the functional components of the objective function. The validity of a given target linear convergence rate is established by deriving a necessary and sufficient condition for verifying the existence of a quadratic Lyapunov function for the algorithm and problem class under consideration for the chosen rate, which amounts to the feasibility of a small-sized semidefinite program. This allows us to find the smallest linear convergence rate for which such a quadratic Lyapunov function exists, by bisection search, yielding a tight procedure. The approach is numerically exemplified on several algorithmic schemes. |
| 07/02/2023 (14h) [Location: Euler building(room a.002)] Rodolphe Sepulchre (University of Cambridge and KU Leuven) Feedback system analysis: back to the future |
|
Abstract: Back in 1960, George Zames proposed to ground the theory of feedback systems in incremental input-output analysis. Sixty years later, the theory of nonlinear control is primarily grounded in non-incremental state-space analysis. The talk will examine some consequences of that evolution and describe ongoing efforts to revive Zames proposal in the algorithmic age of control. Joint work with Tom Chaffey, Fulvio Forni, and Henk Van Waarde. |
| 13/12/2022 (14h) [Location: Euler building(room a.002)] Nick Vannieuwenhoven (KU Leuven) Riemannian optimization for the tensor rank decomposition |
|
Abstract: The tensor rank decomposition or canonical polyadic decomposition (CPD) is a generalization of a low-rank matrix factorization from matrices to higher-order tensors. In many applications, multi-dimensional data can be meaningfully approximated by a low-rank CPD. In this talk, I will describe a Riemannian optimization method for approximating a tensor by a low-rank CPD. This is a type of optimization method in which the domain is a smooth manifold, i.e. a curved geometric object. The presented method achieved up to two orders of magnitude improvements in execution time for challenging small-scale dense tensors when compared to state-of-the-art nonlinear least squares solvers. |
| 06/12/2022 (14h) [Location: Core(room B-135)] Enrico Malaguti (University of Bologna) Chance-constrained problems with integer second-stage recourse decisions |
|
Abstract:We study a class of chance-constrained two-stage stochastic optimization problems where the second-stage recourse decisions belong to mixed-integer convex sets. Due to the nonconvexity of the second-stage feasible sets, standard decomposition approaches cannot be applied. We develop a provably convergent branch-and-cut scheme that iteratively generates valid inequalities for the convex hull of the second-stage feasible sets, resorting to spatial branching when cutting no longer suffices. We show that this algorithm attains an approximate notion of convergence, whereby the feasible sets are relaxed by some positive tolerance. Computational results on chance-constrained resource planning problems indicate that our implementation of the proposed algorithm is highly effective in solving this class of problems, compared to a naive reformulation tackled with a state-of-the-art MIP solver. |
| 29/11/2022 (14h) [Location: Euler building (room a.002)] Welcome Seminar |
|
Title: Double descent in deep learning: Exploring the impact of common regularization, compression and optimization strategies Speaker: Bastien Massion(PhD UCLouvain/INMA) Abstract: For a decade, deep neural networks have revolutionized the domain of machine learning and artificial intelligence. The driving force behind this success is their unprecedented generalization abilities. The mechanisms behind this generalization have long been a mystery. Indeed, it defies the classical bias-variance trade-off as deep learning models are highly overparameterized but still avoid overfitting. The double descent phenomenon has been proposed to characterize this behaviour: the generalization error follows the classical U-shape curve in function of the model complexity but decreases again when the number of parameters exceeds the number of training data, reaching the modern learning regime. Many explanations of the phenomenon have since then been proposed, relying on implicit regularization of the model. Yet, a lot of work has still to be done to measure the impact on double descent of several techniques commonly used in deep learning. This work aims to explore 3 aspects of training. First, explicit orthogonal regularization of neural network weights. Then, low-rank matrix factorization of these weights for compression purposes. Finally, the impact of dropout and momentum in model training. Title: Data-driven control of Markovian Jump Linear Systems: a step towards complexity Speaker: Adrien Banse(PhD UCLouvain/INMA) Abstract:Hybrid systems are dynamical systems whose dynamics is characterised by both continuous and discrete behaviours. Their hybrid characteristic makes these systems notoriously extremely hard to control and analyse. However, they often generate rich data (harvested from cameras, lidars, etc.), to which the engineer has access in large quantities. In my thesis, I plan to tackle the following question: how to control such complex systems in a data-driven fashion? It means, from datasets containing observations of a hybrid system, what can we say about its stability, or how can we design a controller to stabilise it? In particular, Markovian Jump Linear Systems (MJLS), while popular tools for modelling complex hybrid systems, have remained untouched in the framework of data-driven control. My goal, in four years from now, is to prove theorems providing firm probabilistic guarantees on the stability and stabilisation of MJLS. I will proceed in an incremental way, starting by considering a simpler system that I have already studied, namely constrained switching linear systems. I also plan to provide a proof of concept by applying my methods to networked controlled systems, a known application of MJLS Title: Investigation of the neural circuitry involved in the control of movements Speaker: Astrid Doyen(PhD UCLouvain/INMA) Abstract: Reaching movements are movements that we perform thousands of times in everyday life. Computationally, the control of these movements can be represented by the Optimal Feedback Control model, which explains most human behaviours. Among those behaviours, one can mention motor adaptation, i.e., the ability of humans to adapt to new tools or environments, and online change in control policy, i.e., the ability of humans to rapidly change the command sent to the arm following a change in task or movement parameters during the movement. In the Optimal Feedback Control model, these behaviours are attributed to different structures, but it is not known yet if it is also the case biologically. Therefore, this research aims to highlight experimentally potential interferences between both mechanisms to give insight into the biological structures involved in these behaviours and to particularly investigate the role in both tasks of the basal ganglia, a group of subcortical nuclei known to be involved in learning and selection of the most appropriate motor behavioural programs. The Optimal Feedback Control model can then be extended by considering the experimental results. |
| 22/11/2022 (14h) [Location: Euler building (room A.002)] Jean-Pierre Raskin (UCLouvain, ICTEAM/ELEN) The fascinating and frightening face of nanoelectronics |
|
Nanotechnology is revolutionizing the way we communicate, consume, and think. The manufacture and characterization of these nanoscale objects are real challenges for scientists and industry. In the field of nanoelectronics the characteristic sizes of transistors are constantly decreasing and with this reduction in size the structural defects in and between the materials as well as the residual mechanical stresses within the thin films, to name a few, greatly limit performance of some basic electronic components. The scientific community is working to find solutions to these problems. However, these same technological limitations can be exploited in an original way and become real sources of innovation in other fields of application. Two examples will be presented: (i) the interest of interface defects in the production of a high performance silicon substrate essential for the integration of the high frequency electronics necessary for any wireless object, (ii) the exploitation of internal stresses in thin films for the development of mechanical test laboratories on silicon chips in order to explore the electromechanical behavior of materials at the nanometer scale. The beauty of these technological innovations cannot hide a much less pleasing environmental and social reality. The use of these advanced, complex, energy-intensive technologies that require a lot of critical and toxic materials must be considered with much more awareness. Engineers and scientists must embrace the complexity of these societal challenges by adopting a holistic approach. This paradigm shift must be taught to future engineers so that they contribute to the development of a more sustainable and fair world. |
| 15/11/2022 (14h) [Location: Euler building (room A.002)] Juan José Salazar (Universidad de la Laguna) Designing optimal masks for a multi-object spectrometer |
|
This paper concerns a new optimization problem arising in the management of a multi-object spectrometer with a configurable slit unit. The field of view of the spectrograph is divided into contiguous and parallel spatial bands, each one associated with two opposite sliding metal bars that can be positioned to observe one astronomical object. Thus several objects can be analyzed simultaneously within a configuration of the bars called a mask. Due to the high demand from astronomers, pointing the spectrograph’s field of view to the sky, rotating it, and selecting the objects to conform a mask is a crucial optimization problem for the efficient use of the spectrometer. The paper describes this problem, presents a Mixed Integer Linear Programming formulation for the case where the rotation angle is fixed, presents a non-convex formulation for the case where the rotation angle is unfixed, describes a heuristic approach for the general problem, and discusses computational results on real-world and randomly-generated instances. |
| 08/11/2022 (14h) [Location: Euler building (room a.002)] Gianluca Bianchin (INMA,UCLouvain) Learning to Optimize Network Systems via Online Optimization and Control Theory with Applications to Traffic Control |
|
Abstract:Operating networked dynamical systems at an optimal setpoint in the presence of humansin- the loop constitutes one of the most central and impactful problems ahead of systems and control theory. Motivated by this, in this talk I will first present my work on the development of feedback-based optimization algorithms as one of the most fundamental tools that enable to simultaneously control and optimize modern cyber-physical-human systems. I will begin by reviewing some of the classical techniques in numerical optimization and by showing how these can be translated to solve complex control problems. I will then demonstrate how these algorithms can be adapted to control dynamical systems whose model is unknown, and how data-driven control techniques can be combined with online optimization methods. In the second part of the talk and motivated by the first, I will focus on applications in traffic control and demonstrate how strategic decisions made by humans negatively impact the stability of modern traffic networks. I will show how control-theoretic tools can be used to prevent oscillatory congestion in traffic systems via proper design of the information that is made available to humans. While specific focus will be given to traffic management problems, these results are relevant for a wide range of applications ranging from energy management to the control of epidemic outbreaks. |
| 25/10/2022 (14h) [Location: Euler building (room a.002)] Tom Claeys (IRMP,UCLouvain) Repulsive particles and random matrix eigenvalues |
|
Abstract:Determinantal point processes are a class of models for repulsive particles with a specific structure. This structure allows one to effectively study relevant probabilistic quantities. I will discuss some examples of them, including random matrix eigenvalues, and show a glimpse of their rich mathematical structure. |
| 18/10/2022 (14h) [Location: Euler building (room a.002)] Julian Tachella (École Normale Supérieure de Lyon) Unsupervised Learning to Solve Linear Inverse Problems |
|
Abstract: In recent years, deep neural networks have obtained state-of-the-art performance in multiple imaging inverse problems ranging from medical imaging to computational photography. Networks are generally trained with supervised pairs of signals and associated measurements. However, in various imaging problems, we often only have access to incomplete measurements of the underlying signals, thus hindering this learning-based approach. In this talk, I will present a new unsupervised learning framework which overcomes this limitation by exploiting the invariance to transformations (translations, rotations, etc.) present in natural signals. I will also present necessary and sufficient conditions for learning without ground truth. Our proposed learning strategy performs as well as fully supervised methods and can handle noisy data. I will show results on various inverse problems, including sparse-view X-ray computed tomography, accelerated magnetic resonance imaging and image inpainting. |
| 11/10/2022 (14h) [Location: Euler building (room a.002)] Foivos Alimisis (Université de Genève) Geometry of the symmetric eigenvalue problem |
|
Abstract:Eigenvalue problems are some of the most characteristic linear algebra applications of optimization techniques over Riemannian manifolds. These techniques are usually well benefited by the presence of a convexity structure. We study the geodesic convexity structure of the standard problem of computing some of the eigenvectors of a symmetric matrix (Rayleigh quotient) on the sphere and on the Grassmann manifold. We prove that, while the problem is only very locally convex, it satisfies in general weaker convexity properties (weak-quasi-convexity, quadratic growth) that are enough to provide convergence guarantees for many important algorithms. Namely, we prove that gradient descent for the Rayleigh quotient on the Grassmann manifold enjoys a linear convergence rate, while if its step-size is chosen via an exact line search, it empirically outperforms the more popular subspace iteration (power method). We also apply our results in the case of accelerated gradient descent (Nesterov's momentum), which turns out to be particularly beneficial when the gaps between the eigenvalues of the original matrix are small. |
| 04/10/2022 (14h) [Location: Euler building (room a.002)] Apostolos Rikos (Division of Decision and Control Systems, KTH Royal Institute of Technology, Sweden) Distributed Coordination and Optimization with Privacy Guarantees and Quantized Communication |
|
Abstract: A distributed system (or network) can be viewed as a set of subsystems that can share information via interconnection links, which form a generally directed communication topology. Distributed systems prove to be of vital importance for the effectiveness of performing various tasks in the areas of cooperative control, distributed coordination, and control of multicomponent systems. This presentation concerns novel distributed algorithms for (i) distributed averaging, (ii) resource allocation, and (iii) privacy preservation. During the operation of the proposed algorithms, each node operates over a directed communication topologies, processes and transmits quantized information and converges in finite time.In the first part of this presentation, we focus on the distributed quantized average consensus problem. In this problem, each node is associated with some initial quantized value. The goal is for the nodes to obtain the average of these initial values (or some value close to the average). We present and analyze a novel distributed averaging algorithm which operates exclusively with quantized values (the information stored, processed and exchanged between neighboring agents is subject to deterministic uniform quantization) and rely on event-driven updates (e.g., to reduce energy consumption, communication bandwidth, network congestion, and/or processor usage).In the second part of this presentation, we focus on the resource allocation problem. In this problem, each node is associated with a convex local cost function. The goal is for the nodes is to cooperatively minimize a common additive cost function. More specifically, nodes aim to to calculate a set of parameters that minimize the global cost function which is the summation of the local cost functions. We present and analyze a novel distributed resource allocation algorithm which operates exclusively with quantized values and rely on event-driven updates.In the third part of this presentation, we focus on privacy preservation strategies for the algorithms presented in the previous two parts. A node is said to preserve the privacy of its initial state if its value cannot be inferred by any curious node in the network and also curious nodes are not able to determine a finite range in which this value belongs. We propose privacy preservation strategies which operate in an event-driven fashion, and are adjusted to the quantized nature of the proposed algorithms in the first two parts of this presentation. |
| 20/09/2022 (14h) [Location: Euler building (room a.002)] Dávid Terjék and Diego González-Sánchez (Renyi Institute Budapest) The Polyak-Lojasiewicz condition and overparameterized learning |
|
Abstract: We study the convergence of gradient descent on a composition (f ∘ F): G -> R of functions F: G -> H and f: H -> R where G, H are Hilbert spaces. We require f to satisfy the Lipschitz gradient and Polyak-Łojasiewicz conditions, and we find sufficient conditions on F that ensure convergence at a linear rate to a global optimum close to the initial point. We then propose a prototype optimization problem, covering a range of machine learning problems including supervised learning. Applying the results gives us sufficient conditions on the neural network and the data that ensure the global convergence of gradient descent, with the Neural Tangent Kernel making a natural appearance. |
| 26/04/2022 (14h) [Location: Euler building (room a.002)] Tijl De Bie (UGent) Automating data exploration |
|
Abstract: Machine learning is increasingly becoming a commodity, available as a building block to programmers with minimal required understanding about detailed modelling aspects. In large part, this is thanks to advances in automating the labor-intensive aspects of machine learning. In contrast, the more exploratory tasks in data science still remain highly labor-intensive, often requiring a deeper understanding of the intricacies of the methods used. In this talk, I will survey some past results on the automation of data exploration tasks, and outline what I view as some of the more important challenges for the future. |
| 19/04/2022 (14h) [Location: Euler building (room a.002)] Raphaël Berthier (École Polytechnique Fédérale de Lausanne) A Continuized View on Nesterov Acceleration for Stochastic Gradient Descent and Randomized Gossip |
|
Abstract: We introduce the continuized Nesterov acceleration, a close variant of Nesterov acceleration whose variables are indexed by a continuous time parameter. The two variables continuously mix following a linear ordinary differential equation and take gradient steps at random times. This continuized variant benefits from the best of the continuous and the discrete frameworks: as a continuous process, one can use differential calculus to analyze convergence and obtain analytical expressions for the parameters; and a discretization of the continuized process can be computed exactly with convergence rates similar to those of Nesterov original acceleration. We show that the discretization has the same structure as Nesterov acceleration, but with random parameters. We provide continuized Nesterov acceleration under deterministic as well as stochastic gradients, with either additive or multiplicative noise. Finally, using our continuized framework and expressing the gossip averaging problem as the stochastic minimization of a certain energy function, we provide the first rigorous acceleration of asynchronous gossip algorithms. |
| 12/04/2022 (14h) [Location: Euler building (room a.002)] Peyman Esfahani (Delft Centre for Systems and Control (TU Delft)) Data-driven Decision-Making in Dynamic Environments |
|
Abstract: In this seminar, we start with a broad class of anomaly detection for large-scale nonlinear dynamical systems. Noting a connection between the diagnosis filter and the so-called behavioral sets of dynamical systems, we leverage tools from the traditional model-based approaches and modern data-driven analytics to address the inherent complexity of the problem. We then shift our attention to the performance guarantees of our proposed solution. In this part, we study this topic in a general context of data-driven decision-making with a particular focus on the distributionally robust optimization framework. We will discuss the role of convexity from the different viewpoints of computational, statistical, and real-time implementation. |
| 29/03/2022 (14h) [Location: Euler building (room a.002)] Hugues Goosse (UCLouvain (ELI)) Reconstructing the climate of the past millennium by combining indirect climate observations and simulation results |
|
Abstract: It is essential to characterize well past natural climate variations to detect the potential anthropogenic contribution in recent changes. Past variations also allow studying the dynamics of the climate system for a range of conditions broader than the one observed recently. The past millennium appears particularly interesting in this framework as the conditions are similar to those observed currently, except that the human impact was much lower. The reconstruction of past climate changes is based on indirect observations as the instrumental data cover less than 150 years in most regions. Different techniques exists. The goal here is to illustrate how climate models results and indirect records derived from natural archives can be combined to obtain estimates of past states of the climate system and of the mechanisms responsible for the past climate changes. |
| 22/03/2022 (14h) [Location: Euler building (room a.002)] Bálint Daróczy (UCLouvain (INMA)) Learning from pairwise comparisons and gradient representations of neural networks |
|
Abstract: During the seminar we will consider two separate problems. First, we study of learning problems in which we would like to learn intrinsic values of objects based on pairwise comparisons. We suggest an algorithm and deterime a minimax rate and show that both the upper and the lower error bounds are connected to the trace of the Moore-Penrose inverse of the weighted Laplacian of the comparison graph. In the second part of the seminar, we consider feed-forward neural networks and investigate how the tangent space of the network can be exploited to refine the decision in case of ReLU (Rectified Linear Unit) activations by forming a similarity function with a Riemannian metric. |
| 15/03/2022 (14h) [Location: Euler building (room a.002)] Alessandro Abate (University of Oxford) Certified learning, or learning for verification? |
|
Abstract: We are witnessing an increased, inter-disciplinary convergence between areas underpinned by model-based reasoning and by data-driven learning. Work across these areas is not only scientifically justified, but also motivated by industrial applications where access to information-rich data has to be traded off with a demand for safety criticality: cyber-physical systems are exemplar applications. In this talk, I will report on ongoing initiatives in this cross-disciplinary domain. According to the dual perspective in the title of this talk, I will sketch, on the one hand, results where formal methods can provide certificates to learning algorithms, and on the other hand, results where learning can bolster formal verification and strategy synthesis objectives. |
| 08/03/2022 (14h) [Location: Euler building (room a.002)] Welcome Seminar |
|
Title: Towards tight convergence rates for optimization methods on hypoconvex functions Speaker: Teodor Rotaru (UCLouvain/KULeuven) Abstract: Using the framework of performance estimation (PEP), we established the first tight convergence rates of the gradient method for smooth hypoconvex (or weakly-convex) functions. These functions’ curvature (i.e., maximum Hessian eigenvalue) belongs to the interval [µ, L], where µ is negative and L is positive. With the help of PEP, we obtained mathematical proofs for a large range of step sizes. As a direct application, we recommend the optimal step size that minimizes the convergence rate. Title: Accelerating large-scale Kernel Support Vector Machines Speaker: Sofiane Tanji (UCLouvain) Abstract:Kernel methods provide an elegant extension to well-known linear statistical learning. Due to their poor scalability in time and memory, they have limited applications in large scale learning. We propose Snacks : a kernel SVM solver which can tackle large-scale datasets. Our approach consists in using an accelerated version of the stochastic subgradient method to solve the primal optimization problem on smaller random subspaces. The computational savings do not lead to any degradation in terms of learning performance and we demonstrate the effectiveness of the proposed algorithm on benchmark datasets. |
| 1/03/2022 (14h) [Location: Euler building (room a.002)] Estelle Massart (UCLouvain,ICTEAM) Exploring geometry for deep learning |
|
Abstract: We propose to use stochastic Riemannian coordinate descent on the orthogonal group for recurrent neural network training. The algorithm rotates successively two columns of the recurrent matrix, an operation that can be efficiently implemented as a multiplication by a Givens matrix. In the case when the coordinate is selected uniformly at random at each iteration, we prove the convergence of the proposed algorithm under standard assumptions on the loss function, stepsize and minibatch noise. In addition, we numerically demonstrate that the Riemannian gradient in recurrent neural network training has an approximately sparse structure. Leveraging this observation, we propose a faster variant of the proposed algorithm that relies on the Gauss-Southwell rule. Experiments on a benchmark recurrent neural network training problem are presented to demonstrate the effectiveness of the proposed algorithm. Time permitting, I will also present another application of geometry in deep learning, namely, exploiting invariant curves in the loss landscape when training Wasserstein generative adversarial neural networks. |
| 22/02/2022 (14h) [Location: Euler building (room a.002)] Renaud Ronsse (UCLouvain,IMMC) Adaptive oscillator as a template for the assistance of cyclical movements. |
|
Abstract:In this lecture, I will review the theory of adaptive oscillators and how this tool has been initially developed by generating primitive locomotion movements in robotics. Then I will explain how this framework can be extended to provide different types of assistance in cyclical movements (mainly for the lower-limb, but not exclusively). I will further illustrate the coding of a simple adaptive oscillator on a haptic device that can be manipulated by the participants. |
| 15/02/2022 (14h) [Location: Euler building (room a.207)] Anne-Katrin Schmuck (Max Plank Institute for Software Systems, Kaiserslautern, Germany) Let's play! - Solving Controller Synthesis Games for Cyber-Physical System Design |
|
Abstract:Cyber-Physical Systems (CPS) are technical systems where a large software stack orchestrates the interaction of physical and digital components. Such systems are omnipresent in our daily life and their correct behavior is crucial. However, developing safe, reliable and performant CPS is challenging. A promising research direction towards this goal is the combination of formal methods from computer science and controller synthesis techniques from automation. In my talk, I will discuss how infinite two-player games over finite graphs, originating from the formal methods community, can be utilized and enhanced for higher layer control of CPS. In particular, I will discuss how the use of environment assumptions - used to model particularities of the system under control within these games - has to be rethought in order to effectively solve controller synthesis tasks for CPS. |
| 08/02/2022 (14h) [Location: Euler building (room a.002)] Welcome Seminar |
|
Title: Extension of the Performance Estimation framework through constraint interpolation. Speaker: Anne Rubbens(PhD UCLouvain/INMA) Abstract: Selecting parameters involved in an optimization algorithm is a major challenge often without satisfactory solution, and can have a dramatic impact on the algorithm performance. For instance, selecting parameters by optimizing performance bounds that are not exact may lead to misguided choices.Recently, an approach called Performance Estimation Problem (PEP) has been developed to automatically compute exact worst-case bounds on the performance of a wide class of optimization algorithms whose input belongs to a given class of functions, for instance convex functions.This project consists in extending the field of application of this approach to classes of functions for which no satisfactory PEP formulation exists yet. PEP formulations rely on necessary and sufficient interpolation conditions, ensuring the existence of a function of a given class interpolating a finite set of data. Hence, we will provide a novel approach to derive interpolation conditions for various classes of functions. Title: Optimization of compositional functions Speaker: Nizar Bousselmi(PhD UCLouvain/INMA) Abstract:We consider functions with compositional structure, i.e., functions that can be expressed using one or several compositions of simple elementary component functions. These functions naturally appear in many practical problems. We would like to analyze the impact of the compositional structure of objective functions on the performance of the existing methods and to develop new methods exploiting this structure. Title: Automatic quality control of weather and climate time series Speaker: Benoît Loucheur(PhD UCLouvain/INMA) Abstract:In Belgium, the Royal Meteorological Institute (RMI) is the national meteorological service that provide weather and climate services based on observations and scientific research. The RMI collects and archives meteorological observations in Belgium since the 19th century. Currently, air temperature is monitored in Belgium in about 30 synoptic automatic weather stations as well as in 110 manual climatological stations. All observations are routinely checked for errors, inconsistencies and missing values by the RMI staff. Misleading data are corrected and gaps are filled by estimations. This quality control tasks require a lot of human intervention. With the forthcoming deployment of low-cost weather stations and the subsequent increase in the volume of data to verify, the process of data quality control and completion should become as automated as much as possible. The aim of this project is to develop algorithmic tools to address this quality control problem in a fully automatic way. |
| 14/12/2021 (14h) [Location: Euler building (room a.002)] Nina Miolane (UC Santa Barbara) Geomstats: a Python Package for Riemannian Geometry in Statistics and Machine Learning |
|
Abstract: We introduce Geomstats, an open-source Python package for computations and statistics on nonlinear manifolds that appear in machine learning applications, such as: hyperbolic spaces, spaces of symmetric positive definite matrices, Lie groups of transformations, and many more. We provide object-oriented and extensively unit-tested implementations. Manifolds come equipped with families of Riemannian metrics with associated exponential and logarithmic maps, geodesics, and parallel transport. Statistics and learning algorithms provide methods for estimation, clustering, and dimension reduction on manifolds. All associated operations provide support for different execution backends --- namely NumPy, Autograd, PyTorch, and TensorFlow. This talk presents the package, compares it with related libraries, and provides relevant examples. We show that Geomstats provides reliable building blocks to both foster research in differential geometry and statistics and democratize the use of Riemannian geometry in statistics and machine learning. The source code is freely available under the MIT license at https://github.com/geomstats/geomstats. |
| 07/12/2021 (14h) [Location: Euler building (room a.002)] Pierre Ablin (CNRS, Université Paris-Dauphine) The symbiotic relationship between optimization and deep learning |
|
Abstract: Optimization is one of the cornerstones of deep learning: most deep neural networks are trained by optimizing a cost function. The purpose of this talk is to cover other fruitful interactions between the fields of optimization and deep learning that are less obvious. First, I will discuss how deep neural networks can be designed to quickly and efficiently solve classical optimization problems, with a focus on the Lasso. I will then argue that the tools of optimization allow us to describe accurately what weights are learned by these networks during the training process. In a second part, I will discuss how classical optimization ideas like momentum acceleration can be translated to deep learning, and allow us to develop novel architectures that perform well and are memory-efficient. |
| 30/11/2021 (14h) [Location: Euler building (room a.002)] Geovani Nunes Grapiglia (UCLouvain/INMA) A Generalized Worst-Case Complexity Analysis for Non-Monotone Line Searches |
|
Abstract: In this talk we discuss the worst-case complexity of a wide class of non-monotone line search methods for non-convex unconstrained minimization problems. For the algorithms in this class, the non monotonicity is controlled by a sequence of nonnegative parameters. We prove complexity bounds to achieve approximate first-order optimality even when this sequence is not summable. As a by-product, we obtain a unified global convergence result. Our generalized results allow more freedom for the development of new non-monotone line search algorithms. As an example, we design a non-monotone scheme related to the Metropolis rule. Preliminary numerical experiments suggest that the new method is suitable to nonconvex problems with many non-global local minimizers. |
| 23/11/2021 (14h) [Location: Euler building (room a.002)] Flavio Abreu Araujo (UCLouvain) Implementing spintronic based neuromorphic computing hardware under the reservoir computing approach |
|
Abstract: The brain displays many signatures of non-linear dynamical behavior, including synchronization and complex transient behavior. These observations have inspired a whole class of neuromorphic concepts based on complex networks of interconnected non-linear nodes. Such non-linearity has been identified as one of the main ingredients to achieve excellent performance in cognitive tasks such as spoken digit recognition. In this work, we have quantified in detail the contribution of the acoustic filtering and the neural network, respectively, for spoken digit recognition task using three different frequency decomposition methods: Cochlear, MFCC and Spectrogram. In a first step, we have demonstrated that Cochlear and MFCC are powerful stand-alone features extractors, and they can achieve for themselves very high recognition level: up to 95.8% and 77.2% for cochlear and MFCC, respectively. We have found that such high recognition level is mainly due to the non-linear character of these frequency decomposition methods. First, we have investigated the non- linear dependence of the Spectrogram showing a huge increase of recognition rate from 10% (linear) to 85.6%. In a second step, we have evaluated the gain of the recognition rate provided by the neural network. For simplicity, we have modeled a neural network based on the non- linear dynamics of oscillators in the framework of the reservoir computing approach. The reservoir is generally composed by a large number of fixed and random interconnected non- linear oscillators which generates very complex non-linear dynamics. The key insight behind reservoir computing is that only the external connections (between the reservoir and the output layer) should be trained to obtain the desirable target. Reservoir computing has been identified to be very suitable for different hardware implementations: optical, photonic and spintronic devices. We have found that the contribution of the neural network is dominant for the linear Spectrogram filter but not in the other two cases, i.e., cochlear and MFCC. Finally, we have carried out experiments using non-linear and tunable spin-torque nano-oscillators exhibiting an excellent agreement with our simulations. |
| 16/11/2021 (14h) [Location: Euler building (room a.002)] Radu Dragomir (UCL/INMA) Gradient methods with Bregman distances |
|
Abstract: Large-scale optimization problems from signal processing and machine learning are typically solved with gradient methods, because of their low cost per iteration and their simplicity. In this talk, we study a generalization of the standard gradient descent, which consists in replacing the Euclidean distance by a more general Bregman divergence induced by some simple convex reference function. This function is chosen to be adapted to the geometry of the problem at hand through the so-called relative smoothness condition. We present some advances in this recent line of work, including the study of worst-case complexity through performance estimation problems (PEPs), as well as applications to low-rank matrix optimization. |
| 09/11/2021 (14h) [Location: Euler building (room a.002)] Renato Vizuete (CentraleSupélec and GIPSA-lab, France ) Graphons for the Estimation of Performance Indices in Large Networks |
|
Abstract: In recent years, the analysis of large networks has received increasing attention in many scientific fields due to the continuous evolution of the world towards a networked environment with a large number of connections. In this type of networks, uncertainties are almost present, and even the graph topology may not be available, which makes the analysis more complicated. One of the most promising tools to address these problems are graphons, defined as the limits of convergent sequences of dense graphs. In this talk, we will present the fundamentals of graphons and their applications in the estimation of performance indices associated with large networks sampled from graphons. In the first part, we will analyze the stability of a SIS epidemic over a network and its robustness to noise using the properties of the graphon operator. For the second application, we will present a study of the spectrum of the Laplacian matrix of a graph using the degree function of the graphon and the estimation of the well known average effective resistance of sampled networks. |
| 02/11/2021 (14h) [Location: Euler building (room a.002)] Welcome Seminar |
|
Title: Using photogrammetry for the objective study of ancient bowed instruments: a machine learning approach Speaker: Philémon Beghin(UCLouvain/INMA) Abstract: The morphology of today’s violin differs greatly from that of the first instruments of the late 16th century. Indeed, in order to meet the standards suggested by famous orchestras and conservatories, many ancient violins have been recut. It is important for musicologists, violin makers and museum curators to analyse the alterations they have undergone. Specialists agree that the instruments have been reshaped, but have difficulty to prove it rigorously. Moreover, the historical testimonies about this process are imprecise. It is therefore necessary to find an objective way to quantify violin geometry. This project aims to develop a set of algorithmic tools in order to compute an adequate mathematical representation that is able to describe the 3D shape of an object (violin) acquired by photogrammetry. Based on that representation, machine learning techniques will be applied in order to perform clustering and classification with the aim of quantifying their geometric characteristics, their possible anomalies and, if applicable, their original morphology. From an engineering point of view, the development of 3D analyses of objects as complex as violins will allow the growth of methods and knowledge potentially applicable to many fields outside of organology. Title: Biomechanics of tactile feedback during dexterous manipulation of objects Speaker: Donatien Doumont(UCLouvain/INMA) Abstract: The biomechanics of the skin and underlying tissues play a fundamental role in the sense of touch. Indeed, when the finger comes in contact with an object, the deformations induced within these living tissues are translated by the mechanoreceptors (or tactile sensors) into neural signals, which are then interpreted by the brain to generate the appropriate motor response. This project is aimed at better understanding the feedback provided by the tactile afferents during interactions with objects. To that end, we will combine passive stimulation tasks mimicking the mechanical interactions during object manipulation and active manipulation tasks while monitoring the activity of tactile afferents (only in the passive case) and several key biomechanical parameters such as external forces, surface skin deformation, and skin moisture. This combination of techniques will enable us to gain insight into the essential aspects of tactile feedback that enable stable and dexterous control of fingertip forces during manipulation. Such insight will have direct implications for the development of sensorized bionic hands. Title: Simulation of the stride-to-stride variability of patients with Parkinson’s disease Speaker: Clémence Vandamme(UCLouvain/INMA) Abstract:Gait is a complex mechanism involving several neural structures such as the motor cortex, the cerebellum and basal ganglia. It also requires the integration of sensorial feedback from visual, vestibular and peripheral receptors. The complex coordination of these structures is not yet fully understood. In particular, recent studies revealed that gait fluctuations exhibit Long Range Autocorrelation (LRA) such that fluctuations at any given moment are statistically related to those that occur over many different time scales. Up to now, the physiological origin of LRA remains unclear. Moreover, this specific correlation structure is altered in elderly and in people suffering from neurodegenerative diseases, such as Parkinson’s disease. Consequently, it is thought that an optimal level of LRA is a marker of stable gait and high adaptive capabilities. Clinically, LRA indicators could complement the diagnostic of Parkinson's disease and constitutes a precautious marker of risk of fall. This project aims to provide a comprehensive model that includes a continuous-time control of the biomechanics plant for both a healthy population and Parkinson’s patients, as it is an important challenge for fundamental understanding as well as for clinical applications. Title: Influence of vestibular, visual and somesthesic inputs on dexterous manipulation Speaker: Simon Vandergooten(UCLouvain/INMA) Abstract:Since birth we evolve in a steady gravitational environment in which our brain has learned to manipulate objects. For example, we are able to unconsciously optimize the force with which we squeeze objects when handling them. In order to better characterize the influence of gravity on upper limb movement kinematics as well as on the dynamics of prehension (i.e. the act of grasping), several parabolic flight campaigns as well as ground-based experiments were performed. These experiments ultimately led to GRIP, an experiment carried out in the International Space Station that focuses on long-term motor adaptation to microgravity, in particular during dexterous object manipulation. On the ground, our current setup comprising a rotating chair, four motion-tracking cameras and the grip-lift manipulandum allows us to perform control experiments that are necessary for the interpretation of and analysis of GRIP data. With this project, we aim to study how visual, vestibular and somatosensory feedbacks interact and affect the planning of movement trajectory, the perception of the verticality and the anticipative force mechanisms underlying finger-arm coordination. |
| 26/10/2021 (14h) [Location: Euler building (room a.002)] Welcome Seminar |
|
Title: Employing neural network based control models to understand how brain generates movements Speaker: Hari Teja Kalidindi (ICTEAM/IoNS) Abstract:Biological agents display impressive abilities to move under dynamic environmental conditions. Distributed regions in the central nervous system coordinate to generate the motor commands suitable for a given behavioral goal. However, the neural computations that underlie even the simplest of the movements are still equivocal. Historically, one of the main sources of the dispute has been due to the emphasis on describing the motor-encoding in isolated brain regions, by ignoring the effect of key control elements such as sensory-feedback, prediction and the physics of the body under control. In this talk, we demonstrate the utility of mechanistic neural networks as normative models to study neural computations underlying movement control. Particularly, we emulate recent observations from the primate motor cortex - the brain region that is implicated for generating movements. Title: Sensing of low-rank plus sparse matrices Speaker: Simon Vary (ICTEAM) Abstract:Expressing a matrix as the sum of a low-rank matrix plus a sparse matrix is a flexible model capturing global and local features in data. This model is the foundation of robust principle component analysis, and popularized by dynamic-foreground/static-background separation amongst other applications. In this talk we develop guarantees showing that rank-r plus sparsity-s matrices can be recovered by computationally tractable methods from p=O(r(m+n-r)+s)log(mn/s) linear measurements. We establish that the restricted isometry constants for the low-rank plus sparse matrix set remain bounded independent of the problem size provided p/mn, s/p, and r(m+n-r)/p remain fixed. The developed theory and algorithms also apply to the fully observed case of Robust PCA. |
| 19/10/2021 (14h) [Location: Euler building (room a.002)] David Wozabal (Technical University of Munich) Multi-Stage Stochastic Programming for AC Optimal Power Flow Problems |
|
Abstract:We propose the first computationally tractable framework to solve multi-stage stochastic optimal power flow (OPF) problems in alternating current (AC) power systems. To this end, we use recent results on convex semi-definite programming (SDP) relaxations of OPF problems in order to adapt the SDDP algorithm for problems with a Markovian structure employing scenario lattices to discretize the underlying randomness. We show that the usual SDDP lower bound remains valid and that the algorithm converges to a globally optimal solution of the stochastic AC-OPF problem as long as the SDP relaxations are tight. In the last part of the paper, we set up an extensive case study demonstrating the practical viability of our approach. In particular, we use the IEEE RTS-GMLC network to set up a storage sitting, sizing, and operation problem under uncertainty about demand and renewable generation. We show that the convex SDP relaxation of the stochastic problem is tight except in very rare cases. Furthermore, we demonstrate that after a reasonable number of iterations the algorithm finds a policy with a relatively small SDDP optimality gap that yields a significant added value over rolling deterministic planning. |
| 12/10/2021 (14h) [Location: Euler building (room a.002)] Vivian De Smedt (PSI Metals Belgium) Optimization in Steel Industry |
|
Abstract:What are some of the optimization problems of the steel industry? We will discuss the differences between two approaches of the optimization problems: black box vs. expert systems. The size of the solution space being very large some heuristic approaches are needed. We will discuss how to select the heuristics and what are the strong and weak points of the different approaches. The optimization project have to be integrated into workflow that less automatic and have its strong points. We will discuss how to take into account this aspect of the problematic in the conception of the solution |
| 05/10/2021 (14h) [Location: Euler building (room a.002)] Guillaume Drion (Université de Liège) Neuromorphic control principles |
|
Abstract:Owing to the recent advances in control engineering and machine learning, and combined with the remarkable improvement of sensors, actuators and computer power, modern high-performance computing systems far surpass human performance in a plethora of complex tasks. However, they consume megawatts of power, are optimized for specific tasks, and are hardly portable. In sharp contrast, biological brains are energy-efficient, are polyvalent, and show impressive adaptive capabilities in uncertain environments. These main differences between brains and current computing architectures are a crucial bottleneck for the expansion of automation in modern society. This situation has led many universities and world-leading companies, which include IBM, Intel, IMEC, or Thales, to investigate novel, brain-inspired computing systems and artificial intelligence technologies, an approach called neuromorphic computing. In this talk, we will approach the design of neuromorphic computing systems from a control engineering approach. We will first study how excitability, a key property of neuronal signaling, can be analyzed and designed following a nonlinear, multiscale feedback approach. Secondly, we will see how such neuromorphic systems can be controlled through loop shaping via a physiological mechanism called neuromodulation. Finally, we will exploit these brain-inspired mechanisms to improve artificial neural network adaptivity and long-term memory on the one hand, and design robust and controllable neuromorphic electronic systems on the other hand |
| 28/9/2021 (14h) [Location: Euler building (room a.002) ] Josh Taylor (University of Toronto) Convex Optimization of Bioprocesses |
|
Abstract:In this talk, we begin by optimizing the gradostat, in which several chemostats are interconnected by mass flow and diffusion. The gradostat is of interest both as a classical nonlinear system and because the basic network structure and nonlinearities appear in a wide variety of bioprocesses, including wastewater treatment. We formulate a convex relaxation of the gradostat. The relaxation is exact under several conditions, for instance, if the gradostat is outflow connected and its flow matrix is irreducible. When the microbial growth in the bioreactors is described by the Monod or Contois functions, the relaxation is a second-order cone program, which can be solved at scales of over 10^5 variables in minutes with industrial software. We also discuss how to extend the work to a general class of bioprocesses, and present an example based on wastewater treatment. |
| 21/9/2021 (14h) [Location: Euler building (room a.002) ] Balázs Gerencser ( Eötvös Loránd University, Hungary) From theoretical to computable convergence rate of push-sum for consensus |
|
Abstract:We know that reaching average consensus using only local communication along a network is a fundamental building block in the area of distributed computing, leading to applications such as sensor fusion and distributed optimization. We currently analyze schemes based on one such protocol, push-sum, and our target is to understand their convergence speed. We prove a bound on the almost sure convergence rate that is also computable for a class of push-sum algorithms. This extends the works of Iutzeler, Ciblat and Hachem (2013) on similar bounds but in a more restrictive setup and conclusion, and complements the results of Gerencsér and Gerencsér (2019) identifying the exact convergence rate but providing no computable access or approximation |