Computational Linguistics and Generative AI

Teacher(s)

Dupont Pierre;

Language

English
> French-friendly

Prerequisites

Main themes

• Different levels of linguistic analysis
• Probabilistic modeling of language (N-grams and Hidden Markov Models)
• Part-of-speech tagging
• Vector semantics and methods for vector word embeddings
• Deep learning applied to natural language processing
• Attention mechanisms and transformers
• Question-answering systems and conversational agents
• Applications in language engineering such as autocomplete software, automatic tagging, summarization, machine translation, information retrieval, and chatbots

Learning outcomes

At the end of this learning unit, the student is able to :

At the end of this learning unit, the student is able to : Given the learning outcomes of the "Master in Computer Science and Engineering" program, this course contributes to the development, acquisition and evaluation of the following learning outcomes:     INFO1.1-3     INFO2.3-4     INFO5.4, INFO5.5     INFO6.1, INFO6.5 Given the learning outcomes of the "Master [120] in Computer Science" program, this course contributes to the development, acquisition and evaluation of the following learning outcomes:     SINF1.M4     SINF2.3-4     SINF5.3, SINF5.4     SINF6.1, SINF6.5 Students completing successfully this course should be able to:     describe the fundamental concepts of natural language modeling;     master the methodology of using linguistic resources, in particular large scale corpora, possibly annotated or structured;     apply in a relevant way statistical language modeling techniques;     implement recent machine learning methods applied to language processing;     develop linguistic engineering applications. Students will have developed skills and operational methodology. In particular, they have developed their ability to:     integrate a multidisciplinary approach between computer science and linguistics, using wisely the terminology, tools and existing methods;     manage the time available to complete projects of medium size;     manipulate and exploit large amounts of data.

Content

• Various levels of linguistic analysis
• Probabilistic language modeling
• Part-of-speech tagging
• Vector semantics and word embeddings
• Deep Learning applied to natural language processing
• Question answering and conversational agents
• Linguistic engineering applications such as automatic completion software, POS tagging, text summarization, machine translation, information retrieval and chatbots

Teaching methods

• Lectures
• Practical projects implemented in Python on the Inginious platform.

Evaluation methods

Computation of the overall course grade

The continuous assessment grade is based on the 3 projects, which are considered a single learning activity given their thematic connections and progressive nature. The projects cannot be implemented again during the second session. The continuous assessment grade is therefore fixed at the end of the first semester.
The weight of the continuous assessment grade in the overall course grade is calculated as follows:

• 40% if the individual written exam score is higher than 10/20;
• 0% if the individual written exam score is lower than 5/20;
• linearly progressive between 0, if the individual written exam score is 5/20, and 40%, if the individual written exam score is 10/20.

The additional weighting (relative to 100% of the overall grade) corresponds to the individual exam. These weightings are calculated for each exam session in which the student is enrolled, taking into account the exam for that specific session. The final exam is, by default, a closed-book written exam (on paper or, if applicable, on a computer) with no access to any external resources.
The grade submitted to the jury for its deliberation is the student's latest grade for the academic year.

Rules for student collaboration and use of external resources, including generative AIs

Collaborative studying among students is encouraged during project follow-up sessions and via an exchange forum on Moodle.
Each student is expected to submit a personal solution to each project. The use of public resources (e.g. stackoverflow.com), including generative AIs (e.g. chatGPT) is permitted, as long as each (fragment of) code submitted by the student mentions specifically all the resources used. In particular, when generative AI has been used to produce (fragments of) code, a public link to the conversation that took place with the AI must be included as a comment in the code submitted by the student. Such links must be included for each conversation that took place during the course of each project.
The distribution or exchange between students of (fragments of) code is not authorized by any means (GitHub, Facebook, Discord, etc.), even after the project deadlines.
Failure to comply with these rules will be considered as plagiarism and/or cheating and will be sanctioned according to the Academic Regulations and Procedures.
These rules are explained in detail during the first class (see course Moodle site).

Other information

Strong prerequisites 

• Good knowledge of python programming
• Introductory knowledge in data strucures and algorithms

      (see LINFO1101 and LINFO1103, or equivalent courses)

Additional prerequisites

• Basic knowledge in probability theory (e.g. conditional probability, probabilistic independence, Bayes rule, ...

Previous know-how

• Existing knowledge in machine learning is a plus but not a prerequisite

Online resources

moodle.uclouvain.be/course/view.php?id=1182

Bibliography

One recommended textbook - un ouvrage conseillé :

• Speech and Language Processing, D. Jurafsky and J.H. Martin, Prentice Hall.

Teaching materials

• Les supports obligatoires sont constitués de l'ensemble des documents (transparents des cours magistraux, énoncés des travaux pratiques, compléments, ...) disponibles depuis le site Moodle du cours.
• Required teaching material include all documents (lecture slides, project assignments, complements, ...) available from the Moodle website for this course.

Faculty or entity

> INFO