Universität Wien

052316 VU Deep Learning for Natural Language Processing (2023W)

6.00 ECTS (4.00 SWS), SPL 5 - Informatik und Wirtschaftsinformatik
Continuous assessment of course work
Moodle

Registration/Deregistration

Note: The time of your registration within the registration period has no effect on the allocation of places (no first come, first served).

Details

max. 25 participants
Language: English

Lecturers

Classes (iCal) - next class is marked with N

  • Thursday 05.10. 09:45 - 11:15 PC-Seminarraum 3, Kolingasse 14-16, OG02
  • Thursday 05.10. 11:30 - 13:00 PC-Seminarraum 3, Kolingasse 14-16, OG02
  • Thursday 12.10. 09:45 - 11:15 PC-Seminarraum 3, Kolingasse 14-16, OG02
  • Thursday 12.10. 11:30 - 13:00 PC-Seminarraum 3, Kolingasse 14-16, OG02
  • Thursday 19.10. 09:45 - 11:15 PC-Seminarraum 3, Kolingasse 14-16, OG02
  • Thursday 19.10. 11:30 - 13:00 PC-Seminarraum 3, Kolingasse 14-16, OG02
  • Thursday 09.11. 09:45 - 11:15 PC-Seminarraum 3, Kolingasse 14-16, OG02
  • Thursday 09.11. 11:30 - 13:00 PC-Seminarraum 3, Kolingasse 14-16, OG02
  • Thursday 16.11. 09:45 - 11:15 PC-Seminarraum 3, Kolingasse 14-16, OG02
  • Thursday 16.11. 11:30 - 13:00 PC-Seminarraum 3, Kolingasse 14-16, OG02
  • Thursday 23.11. 09:45 - 11:15 PC-Seminarraum 3, Kolingasse 14-16, OG02
  • Thursday 23.11. 11:30 - 13:00 PC-Seminarraum 3, Kolingasse 14-16, OG02
  • Thursday 30.11. 09:45 - 11:15 PC-Seminarraum 3, Kolingasse 14-16, OG02
  • Thursday 30.11. 11:30 - 13:00 PC-Seminarraum 3, Kolingasse 14-16, OG02
  • Thursday 07.12. 09:45 - 11:15 PC-Seminarraum 3, Kolingasse 14-16, OG02
  • Thursday 07.12. 11:30 - 13:00 PC-Seminarraum 3, Kolingasse 14-16, OG02
  • Thursday 14.12. 09:45 - 11:15 PC-Seminarraum 3, Kolingasse 14-16, OG02
  • Thursday 14.12. 11:30 - 13:00 PC-Seminarraum 3, Kolingasse 14-16, OG02
  • Thursday 11.01. 09:45 - 11:15 PC-Seminarraum 3, Kolingasse 14-16, OG02
  • Thursday 11.01. 11:30 - 13:00 PC-Seminarraum 3, Kolingasse 14-16, OG02
  • Thursday 18.01. 09:45 - 11:15 PC-Seminarraum 3, Kolingasse 14-16, OG02
  • Thursday 18.01. 11:30 - 13:00 PC-Seminarraum 3, Kolingasse 14-16, OG02
  • Thursday 25.01. 09:45 - 11:15 PC-Seminarraum 3, Kolingasse 14-16, OG02
  • Thursday 25.01. 11:30 - 13:00 PC-Seminarraum 3, Kolingasse 14-16, OG02

Information

Aims, contents and method of the course

This course will cover topics related to the application of DL (Deep Learning) techniques for solving NLP (Natural Language Processing) tasks. It will start with a comprehensive introduction of the deep learning paradigms, neural probabilistic language models and word embeddings. Next, it will cover concepts like the encoder-decoder architecture, recurrent neural networks, Attention and contextual embeddings. The course will continue with advanced developments of the recent years such as the Transformer architecture, BERT and its derivatives, T5, GPT-* as well as ChatGPT. An integral part will be the implementation of certain concepts using Python frameworks such as Numpy and Pytorch. There will be 5 advanced and involving programming projects. Consequently, a solid background of Python is highly relevant and necessary.
(DH students who want to take this lecture need to have passed the lecture "Practical Machine Learning for Natural Language Processing" with very good success, or have equivalent previous knowledge in programming and machine learning, for successfully participating in this lecture.)

Assessment and permitted materials

- Regular assignments throughout the semester in Moodle 20%
- Programming exercises 20%
- Midterm exam 30%
- Final exam 30%

Minimum requirements and assessment criteria

The participant must attend at least 75 % of the sessions. The grade is calculated from the total points as follows:

>= 90% very good (1)
>= 80% good (2)
>= 65% satisfactory (3)
>= 50% sufficient (4)
< 50% not sufficient (5)

Examination topics

Reading list

Jason Brownlee: "Basics of Linear Algebra for Machine Learning - Discover the Mathematical Language of Data in Python"
https://github.com/balban/Books/tree/master/Linear%20Algebra

Yoav Goldberg: "Neural Network Methods for Natural Language Processing", Morgan & Claypool, 2017
https://github.com/Michael2Tang/ML_Doc

Steven Bird, Ewan Klein, Edward Loper: "Natural Language Processing with Python - Analyzing Text with the Natural Language Toolkit"
https://www.nltk.org/book

Ian Goodfellow and Yoshua Bengio and Aaron Courville: "Deep Learning", MIT Press, 2016.
https://www.deeplearningbook.org

Association in the course directory

Last modified: Fr 10.11.2023 10:47