Public thesis defense Pierre Martou - ICTEAM
sst |
Combinatorial Testing of Dynamically Adaptive Software Systems
Monday September 7th, 2026 - 2pm - Auditorium BARB91 - Place Sainte-Barbe, 1 - 1348 Louvain-la-Neuve
Dynamically adaptive software systems can change their behaviour at runtime, resulting in large configuration spaces. Features and constraints provide a convenient representation of such variability, but the number of possible configurations grows exponentially with the number of features, making exhaustive testing impractical. This thesis investigates how system-level testing can be used to efficiently find and localise faults in dynamically adaptive software systems. To address faults that arise not only from feature interactions within configurations but also from transitions between configurations during runtime, we propose Combinatorial Transition Testing (CTT). It provides a comprehensive testing process that covers the generation of test suites, the automatic detection of transition faults, and their subsequent localisation. To generate test suites efficiently, we propose an optimised greedy row-by-row generation algorithm. Building upon CTT test generation, we introduce a differential test oracle capable of detecting transition faults without prior knowledge of the system's expected behaviour. We further propose a fault localisation approach that combines differential testing, divide-and-conquer, and SMT solving to isolate faulty transitions from a large set of suspects.
Our empirical results show that the size and cost of CTT-generated test suites are linearly correlated with those generated by Combinatorial Interaction Testing (CIT), as is the additional effort required to use the test oracle. Together, these results provide empirical evidence that both test generation and test oracle effort grow logarithmically with the number of features. For fault localisation, the experimental results indicate approximately linear growth in both the number of executed configurations and SMT solver calls on the evaluated feature models, despite the exponential growth of the configuration and suspect spaces. All data and source code are freely available as an open-source research tool to support the reproduction and further development of CTT.
Jury members
Prof. Kim Mens (UCLouvain), Supervisor
Prof. Charles Pêcheur (UCLouvain), Chairperson
Prof. Hélène Verhaeghe (UCLouvain), Secretary
Prof. Xavier Devroey (UNamur)
Prof. Andy Zaidman (TU DELFT, The Netherlands)
Prof. Myra Cohen (Iowa State University, USA)