Embedded Files
Co-hosted by ECCV 2024
Sep 29 , MiCo Milano
Date: Afternoon 29th of September 2024 Location: Suite 5
Invited Talk #1. 16:15 – 16:45
Invited Talk #1. 16:15 – 16:45
UCSD, USA
Title: Towards Scalable Open-World Autonomy
While autonomous driving has made large strides over the past decade to be deployed in practice today, scaling it across a diversity of conditions and behaviors remains expensive. This talk explores the possibility of recent advances in computer vision foundational models, large language models, and generative AI allowing the development of more scalable driving, simulation, and DevOps stacks for autonomous mobility. These include perception and planning that leverage language guidance, photorealistic simulation of safety-critical scenarios with neural rendering and controllable diffusion, automated DevOps based on vision-language understanding, and a look ahead to agentic LLMs that handle complex autonomy workflows.
Invited Talk #2. 16:45 – 17:15
Invited Talk #2. 16:45 – 17:15
TU Delft, Netherlands
Autonomous vehicles (AVs) have come a long way in the past two decades. Yet they are nowhere close to safely carrying our children to an arbitrary location. Until AVs become a net provider of safety, comfort, and reduced carbon emissions, a lot of work remains to be done. The real world is full of challenging corner cases and unexpected behavior. My research focuses on finding more scalable approaches to address these corner cases by collecting, mining, labeling, training from, and evaluating sensor data more cleverly.
Panel Discussion 17:15 – 18:00
Panel Discussion 17:15 – 18:00
Situation-aware Scene Understanding for Autonomous Driving
National Yang Ming Chiao Tung University, Taiwan
Overview
Overview
The accurate detection and anticipation of actions performed by multiple road agents (pedestrians, vehicles, cyclists, and so on) is a crucial problem to solve for enabling autonomous vehicles with the capability to support reliable and safe autonomous decision-making. While the task of teaching an autonomous vehicle how to drive can be tackled in a brute-force, direct reinforcement learning approach, a sensible and attractive alternative is first to provide the vehicle with situation awareness capabilities, to then feed the resulting intermediate representations of road scenarios (in terms of agents, events and scene configuration) to a suitable decision-making strategy. This, in particular, has the advantage of allowing the modeling of the reasoning process of road agents in a theory-of-mind approach, inspired by the behavior of the human mind in similar contexts. We propose a change of paradigm towards action recognition/detection in that the focus is not on the objects/actors themserlves and their appearance, but on what they do and the meaning of their behaviour. Accordingly to our previous successful experience in hosting ROAD workshop in ICCV2021 (link) and ROAD++ workshop in ICCV2023 (link) the goal of the third ROAD++ workshop is to put to the forefront of the research in autonomous driving the topic of situation awareness, intended as the ability to create semantically useful representations of dynamic road scenes in terms of the notion of ‘road event’ with the present of domain adaptation.
This includes the following tasks: • Detecting and modeling ‘atomic’ events, intended as simple actions performed by a single agent. • Detecting and modeling complex activities, contributed to by several agents over an extended period of time. • Predicting agent intentions. • Predicting the trajectory of pedestrians, vehicles, and other road users. • Forecasting future road events (both atomic and complex). • Based on all these elements, deciding what action the autonomous vehicle should perform next. • Modelling the reasoning processes of road agents in terms of goals or mental states. • Realistic simulation to generate training data for road event action detection.
We invite both paper contributions on these topics, as well as submissions of entries to a challenge specifically designed to test situation awareness capabilities in autonomous vehicles.
See Paper submission for more details.
This includes the following tasks: • Detecting and modeling ‘atomic’ events, intended as simple actions performed by a single agent. • Detecting and modeling complex activities, contributed to by several agents over an extended period of time. • Predicting agent intentions. • Predicting the trajectory of pedestrians, vehicles, and other road users. • Forecasting future road events (both atomic and complex). • Based on all these elements, deciding what action the autonomous vehicle should perform next. • Modelling the reasoning processes of road agents in terms of goals or mental states. • Realistic simulation to generate training data for road event action detection.
We invite both paper contributions on these topics, as well as submissions of entries to a challenge specifically designed to test situation awareness capabilities in autonomous vehicles.
See Paper submission for more details.
Accepted papers will get a poster at the main conference, and the very best ones also an oral.
Accepter papers are listed in "Accepted Paper" tab.
The accurate detection and anticipation of actions performed by multiple road agents (pedestrians, vehicles, cyclists and so on) is a crucial task to address for enabling autonomous vehicles to make autonomous decisions in a safe, reliable way. While the task of teaching an autonomous vehicle how to drive can be tackled in a brute-force fashion through direct reinforcement learning, a sensible and attractive alternative is to first provide the vehicle with situation awareness capabilities, to then feed the resulting semantically meaningful representations of road scenarios (in terms of agents, events and scene configuration) to a suitable decision-making strategy. In perspective, this also has the advantage of allowing the modeling of the reasoning process of road agents in a theory-of-mind approach, inspired by the behavior of the human mind in similar contexts.
We also introduce atomic activity recognition, from action recognition and video understanding perspectives. The atomic activities are road-topology-grounded actions of the agents. This enables a more expressive and efficient traffic scenario retrieval, which can facilitate other applications, such as safety-critical scenario generation.
Accordingly, the goal of this Challenge is to put to the forefront of the research in autonomous driving the topic of situation awareness, intended as the ability to create semantically useful representations of dynamic road scenes, in terms of the notion of a road event.
To allow the research community to thoroughly investigate situation awareness for autonomous driving, this workshop introduces ROAD++, an extension of the first ROad event Awareness in Autonomous Driving Dataset. It is the result of annotating: a combination of four datasets from different domains including ROAD (UK), ROAD-Waymo (USA), ROAD-UAE (UAE), and TACO (CARLA simulation) dataset in terms of what we call road events (REs), as seen from the point of view of the autonomous vehicle capturing the video. REs are defined as triplets E = (Ag;Ac; Loc) composed by a moving agent Ag, the action Ac it performs, and the location Loc in which this takes place. Agent, action and location are all classes in a finite list compiled by surveying the content of the videos.
More information about the dataset can be found the the dataset tab.
Organization Committee
Organization Committee
Prof. Fabio Cuzzolin
Prof. Fabio Cuzzolin
Oxford Brookes University
Dr. Salman Khan
Dr. Salman Khan
Oxford Brookes University
Dr. Reza Javanmard Alitappeh
Dr. Reza Javanmard Alitappeh
University of Science and Technology of Mazandaran
Dr. Eleonora Giunchiglia
Dr. Eleonora Giunchiglia
Imperial College London
Oxford Brookes University
Mihaela Catalina Stoian
Mihaela Catalina Stoian
University of Oxford
Dr. Andrew Bradley
Dr. Andrew Bradley
Oxford Brookes University
Dr. Gurkirt Singh
Dr. Gurkirt Singh
Swiss Federal Institute of Technology in Zurich
Prof. Jorge Dias
Prof. Jorge Dias
Khalifa University
Nadya Abdel Madjid
Nadya Abdel Madjid
Khalifa University
Dr. Majid Khonji
Dr. Majid Khonji
Khalifa University
Dr. Bilal Hassan
Dr. Bilal Hassan
Khalifa University
National Yang Ming Chiao Tung University
National Yang Ming Chiao Tung University
Page updated
Google Sites
Report abuse
This site uses cookies from Google to deliver its services and to analyze traffic. Information about your use of this site is shared with Google. By using this site, you agree to its use of cookies.
Learn more
Got it