Stochastic modelling

linma2470  2022-2023  Louvain-la-Neuve

Stochastic modelling
5.00 credits
30.0 h + 22.5 h
Q2
Teacher(s)
Chevalier Philippe; Madani Mehdi (compensates Chevalier Philippe);
Language
Prerequisites
A probability course and a background in mathematical modelling
Main themes
Introduction to stochastic models in operations research. Study of renewal processes, Markov chains, Markov Processes, Markov Decision Processes. Applications to inventory models, queuing models, branching processes, random walks, etc.
Content
  • Poisson processes and their properties
  • Markov chains with a finite number of states
  • Renewal processes and stopping rules
  • Markov chains with an infinite number of states
  • The notion of reveribility
  • Markov processes
  • Birth and death processes
  • Queueing theory and networks of queues
  • Fluid models for queues
  • Various applications, such as inventory management, replacement, reliability and job shop modeling.
  • Markov decision processes and Reinforcement learning
Teaching methods
The course consists in weekly lectures and 11 exercice sessions. Part of the lectures will be presented by student groups.
Evaluation methods
Students will be evaluated through a written exam based on the objectives of the course. The exam consists in exercices applying the concepts viewed in the course. Many examples of questions of previous exams are solved during the exercice sessions.

The students will make a class presentation of some theory chapter or an application of the theory. This presentation is done in groups and counts for 25% of the grade. This presentation cannot be done again for the session in September.
Bibliography
Lecture recommandée :
 "Stochastic Processes: Theory for applications" de R. Gallagher, 2013, disponible en ligne : http://www.rle.mit.edu/rgallager/notes.htm
"Reinforcement Learning: An Introduction" de R. Sutton et A. Barto, disponible en ligne : http://incompleteideas.net/book/RLbook2020.pdf 
Faculty or entity
MAP


Programmes / formations proposant cette unité d'enseignement (UE)

Title of the programme
Sigle
Credits
Prerequisites
Learning outcomes
Master [120] in Mathematics

Master [120] in Computer Science and Engineering

Master [120] in Computer Science

Master [120] in Mathematical Engineering

Master [120] in Data Science Engineering

Master [120] in Data Science: Information Technology