Main content
Top content
The Osnabrück University offers every Semester a wide variety of courses in the field of Data Science. The specific orientation and focus of the course is in the hands of the responsible lecturer and of course also depends on the degree programme for which it is offered.
For example, the courses offered by Cognitive Science, Computer Science, Econometrics, Psychology, Social Research and working groups in the field of Data Science are generally more application-oriented, while the courses offered at the Institute of Mathematics tend to have a more theoretical focus.
Current term
Grounding LLMs into the physical world
8.3600
Dozenten
Beschreibung
In this block course, we explore the intriguing debate about whether Large Language Models (LLMS) , known for learning through next-word prediction, can develop an understanding of the physical world. The course highlights the recent emergence of multimodal LLMs, like GPT-4, which learn from both text and visual inputs, and examines the extent to which this multimodality might enhance language grounding and physical understanding.
A core part of the course involves analyzing the traditional research approach in this domain, which focuses on LLMs performing next-word prediction tasks. We critically assess whether this method truly tests a model's understanding of physical concepts,. The course also proposes a novel evaluation framework for assessing 'genuine' physical understanding in LLMs. This framework involves using realistic physical simulators as proxies for the real world, requiring LLMs to solve tasks through sensorimotor interaction with these simulators. This approach aims to test the models' ability to bridge natural language with simulated environments and their grasp of intuitive physics, offering a more comprehensive evaluation of their cognitive abilities.
Learning objectives:
- Understand the current state and potential of LLMs in simulating human-like cognition.
- Evaluate the effectiveness of multimodal learning in LLMs for physical understanding.
- Critically analyze traditional next-word prediction tasks in assessing physical comprehension.
Weitere Angaben
Ort: nicht angegeben
Zeiten: Termine am Montag, 16.09.2024 12:00 - 18:00, Dienstag, 17.09.2024 09:00 - 12:00, Dienstag, 17.09.2024 13:00 - 16:00, Mittwoch, 18.09.2024 09:00 - 12:00, Mittwoch, 18.09.2024 13:00 - 16:00, Donnerstag, 19.09.2024 09:00 - 12:00, Donnerstag, 19.09.2024 13:00 - 16:00, Freitag, 20.09.2024 09:00 - 12:00, Freitag, 20.09.2024 13:00 - 16:00, Montag, 23.09.2024 12:00 - 18:00, Dienstag, 24.09.2024 09:00 - 12:00, Dienstag, 24.09.2024 13:00 - 16:00, Mittwoch, 25.09.2024 09:00 - 12:00, Mittwoch, 25.09.2024 13:00 - 16:00, Donnerstag, 26.09.2024 09:00 - 12:00, Donnerstag, 26.09.2024 13:00 - 16:00, Freitag, 27.09.2024 09:00 - 12:00, Freitag, 27.09.2024 13:00 - 16:00
Erster Termin: Montag, 16.09.2024 12:00 - 18:00
Veranstaltungsart: Blockseminar (Offizielle Lehrveranstaltungen)
Studienbereiche
- Cognitive Science > Bachelor-Programm
- Cognitive Science > Master-Programm
- Human Sciences (e.g. Cognitive Science, Psychology)
