Public Thesis defense - ICTEAM

Towards understanding deep learning with the natural clustering prior by Simon CARBONNELLE

Pour l’obtention du grade de Docteur en sciences de l’ingénieur et technologie

The prior knowledge (a.k.a. priors) integrated into the design of a machine learning system strongly influences its generalization abilities. In the specific context of deep learning, some of these priors are poorly understood as they implicitly emerge from the successful heuristics and tentative approximations of biological brains involved in deep learning design. Through the lens of supervised image classification problems, this thesis investigates the implicit integration of a natural clustering prior composed of three statements: (i) natural images exhibit a rich clustered structure, (ii) image classes are composed of multiple clusters and (iii) each cluster contains examples from a single class. More precisely, this thesis attempts to identify implicit clustering abilities, mechanisms and hyperparameters in deep learning systems and evaluate their relevance for explaining the generalization abilities of these systems. We do so through an extensive empirical study of the training dynamics and the neuron- and layer-level representations of deep neural networks. The resulting collection of experiments provides preliminary evidence for the relevance of the natural clustering prior for understanding deep learning.

Jury members :

• Prof. Christophe De Vleeschouwer (UCLouvain), supervisor
• Prof. David Bol (UCLouvain), chairperson
• Prof. Laurent Jacques (UCLouvain), secretary
• Prof. Marie Van Reybroeck (UCLouvain)
• Prof. Benoit Macq (UCLouvain)
• Prof. Tinne Tuytelaars (KULeuven)
• Prof. Vincent François-Lavet (VU Amsterdam)

Télécharger l'annonce