Lehr- und Forschungseinheit für Datenbanksysteme

Breadcrumb Navigation


Master Seminar "Foundation Models in AI" (WS 2023/24)


Registration: via central allocation

All news will be announced here and on Moodle.



Termine und Ort

Title Time Location Date
Mi, 16:00-18:00 Edmund-Rumpler-Str. 9, A012 18.10.2023




A foundation model is a large artificial intelligence model trained on massive data at scale (usually by self-supervised learning) and can be adapted to a wide range of downstream tasks in fields like healthcare/education and manifest capacities of perception, reasoning, and manipulation[1].
Early examples of foundation models are large-scale pre-trained language models like BERT[2] and GPT-3[3] that show great transferability on linguistic tasks. Subsequently, multimodal foundation models, for example, DALL-E[4] and Flamingo[5], that bridge vision and language draw increasing attention and attempt to understand richer sources of data and interact with multimodalities.
This seminar will cover the different dimensions of foundation models, ranging from foundation model architecture to trending foundation model training paradigms like prompt training to an in-depth understanding of the risk and capacities of current foundation model research.

[1] Bommasani, Rishi, et al. "On the opportunities and risks of foundation models." *arXiv preprint arXiv:2108.07258* (2021).

[2] Devlin, Jacob, et al. "Bert: Pre-training of deep bidirectional transformers for language understanding." *arXiv preprint arXiv:1810.04805* (2018).

[3] Brown, Tom, et al. "Language models are few-shot learners." *Advances in neural information processing systems* 33 (2020): 1877-1901.

[4] Ramesh, Aditya, et al. "Zero-shot text-to-image generation." *International Conference on Machine Learning*. PMLR, 2021.

[5]Alayrac, Jean-Baptiste, et al. "Flamingo: a visual language model for few-shot learning." *Advances in Neural Information Processing Systems* 35 (2022): 23716-23736.