Numerical Methods for Statistics: Optimization, Simulations and the Bootstrap

Enseignants

Pircalabelu Eugen;

Langue
d'enseignement

Anglais
> Facilités pour suivre le cours en français

Préalables

Concepts et outils équivalents à ceux enseignés dans les UEs

LSTAT2020	Logiciels et programmation statistique de base
LSTAT2120	Linear models
LSTAT2100	Modèles linéaires généralisés et données discrêtes

Thèmes abordés

Le cours se concentre sur les méthodes numériques et sur les techniques de calcul intensif et itératif qui permettent (i) l'estimation des paramètres et (ii) l’inférence pour les modèles statistiques. Le cours introduit des concepts tels que l'optimisation contrainte et sans contrainte, les algorithmes numériques populaires tels que Newton, les méthodes quasi-Newton et l'algorithme EM (entre autres), la mise en place d'études de simulation pour évaluer les performances de plusieurs concurrents, ainsi que des techniques inférentielles basées sur le principe du Bootstrap.

Contenu

The course outline is as follows:

• Part I:
  • Basics of one-dimensional function optimization. Special case: the likelihood function.
  • Global vs local optima; numerical convergence and approximation errors.
  • Challenges of optimizing multi-dimensional functions. Special cases: linear and generalized linear models and computing the multivariate normal density (link with LSTAT2120, 2100, 2110, 2040).
  • Newton’s method, Fisher scoring and IRLS.
  • Non-linear functions and numerical differentiation.
  • Case study: ?optim(), ?nlm(), ?deriv() and friends.
  • The EM algorithm. Special cases: missing data, normal mixture models and linear mixed models (link with LSTAT2210).
• Part II:
  • Setting up controlled simulation studies: competitor selection, performance metrics and reproducibility.
  • Sampling from distributions and DGPs.
  • Case study I: Sample mean (mean, median, trimmed mean from normal and skewed distributions); German tank problem (estimators from slide LSTAT 2040) - mean, bias, MSE, RE.
  • Case study II: t-test (size, power, coverage and length for CIs) and Binomial test (Tables from slides LSTAT 2040).
  • Case study III: GLM variable selection (TPR, FPR, FDR).
  • Parallel computing in R: doParallel, foreach, mclapply and friends to illustrate Case study I-III.
• Part III:
  • Bootstrap and resampling methods.
  • Bias and variance approximation based on resampling.
  • Bootstrap confidence intervals and hypothesis testing.
  • Other techniques: Permutation tests and Jacknife

Méthodes d'enseignement

The class consists of lectures (30h) and exercise sessions (15h).
Attendance at lectures and practical exercise sessions is highly encourgared, almost MANDATORY!

Modes d'évaluation
des acquis des étudiants

January session:

• During the semester the student must submit 3 compulsory assignments (short, 2-3 pages maximum per assignment), counting for 1.5 points of the final grade (each assignment = 0.5 points). The assignments are to be solved individually or in groups of 2. A mark will be assigned per group. Assignments arriving after the deadline are not considered.
• A project (written in French/English in min 6 and max 12 pages in the template on Moodle, appendices not included) which will illustrate the methods of the course for 5 points. This (written) project will be submitted before the exam session and discussed with the teacher during the exam session. The evaluation of the project is done on the basis of the written report and on the basis of the answers in an oral discussion (without slides) on the results and methodology used for the report, during the exam session. The project is to be solved individually or in groups of 2. A score will be awarded per group. Projects arriving after the deadline are not considered.
• An oral exam (~45min), in which the teacher will assess knowldge about the material covered in class (13.5 points), the quality of the project and the homework.

Attention: Any usage of artificial intelligence software for producing part of text, code, figures or equations that are included in the final project or homework is strictly forbidden. All projects and homework will be analyzed with specialized software and infringements of this rule can result in failing the class. 
The final grade for the LSTAT2185 course in January is given by the points obtained for the assignments + the points obtained for the project + the points obtained for knowldge about the material covered in class.
To validate the course, the student needs a final mark of 10 or more. 
August session:

• A project (written in French/English in min 6 and max 12 pages in the template on Moodle, appendices not included) which will illustrate the methods of the course for 5 points. This (written) project will be submitted before the exam session and discussed with the teacher during the exam session. The evaluation of the project is done on the basis of the written report and on the basis of the answers in an oral discussion (without slides) on the results and methodology used for the report, during the exam session. The project is to be solved individually or in groups of 2. A score will be awarded per group. Projects arriving after the deadline are not considered.
• An oral exam (~45min), in which the teacher will assess knowledge about the material covered in class (15 points) and the quality of the project.

Attention: Any usage of artificial intelligence software for producing part of text, code, figures or equations that are included in the final project or homework is strictly forbidden. All projects and homework will be analyzed with specialized software and infringements of this rule can result in failing the class. 
The final grade for the LSTAT2185 course in August is given by the points obtained for the project + the points obtained for knowldge about the material covered in class. The points awarded for homework do not count for the August session, as continuous assessment is only planned for work during the semester.
To validate the course, the student needs a final mark of 10 or more. 

Autres infos

Software: R/Python
French friendly class.

Ressources
en ligne

Slides and notes will be distributed during the semester.
Moodle website of the class :LSTAT2185 - Numerical Methods for Statistics: Optimization, Simulations and the Bootstrap
https://moodle.uclouvain.be/course/view.php?id=5785

Bibliographie

Givens, G.H. and Hoeting. J.A. (2013). Computational Statistics (2nd ed). Wiley.
Rizzo, M.L. (2007). Statistical Computing with R (2nd ed). Chapman & Hall /CRC.
Gentle, J.E. (2009). Computational Statistics. Springer.
Lange, K. (2010). Numerical Analysis for Statisticians (2nd ed). Springer.
Peng, R.D. (2020+). Advanced Statistical Computing. Available at https://bookdown.org/rdpeng/advstatcomp/
Chernick, M.R. (2008). Bootstrap methods : a guide for practitioners and researchers, Wiley Series in Probability and Statistics.
Davison, A.C. et Hinkley, D.V. (1997). Bootstrap Methods and their Applications, Cambridge University Press.
Efron, B. et Tibshirani, R.J. (1993). An Introduction to the Bootstrap, Chapman and Hall.
Hall, P. (1992). The Bootstrap and Edgeworth Expansion, Springer.
Mammen, E. (1992). When does bootstrap work ? Springer.

Faculté ou entité
en charge

> LSBA