Abstract We present Ego-Exo4D a diverse large-scale multimodal multiview video dataset
and benchmark challenge. Ego-Exo4D centers around simultaneously-captured egocentric …

We address the problem of online action segmentation for egocentric procedural task
videos. While previous studies have mostly focused on offline action segmentation where …

This work explores the problem of generating task graphs of real-world activities. Different
from prior formulations, we consider a setting where text transcripts of instructional videos …

Online resources such as WikiHow compile a wide range of scripts for performing everyday
tasks, which can assist models in learning to reason about procedures. However, the scripts …

One of the fundamental skills required for an agent acting in an environment to complete
tasks is the ability to understand what actions are plausible at any given point. This work …

Procedural activities are sequences of key-steps aimed at achieving specific goals. They are
crucial to build intelligent agents able to assist users effectively. In this context, task graphs …

Task-Oriented Dialogue (TOD) systems have become crucial components in interactive
artificial intelligence applications. While recent advances have capitalized on pre-trained …

A large amount of procedural videos on the web show how to complete various tasks. These
tasks can often be accomplished in different ways and step orderings, with some steps able …

The primary focus of this thesis is to examine the impact of integrating a structural task graph
into a visual recognition network to accurately identify and segment errors in the assembly of …

Abstract What happened? Who? When? Where? Why? What will happen next? are the
fundamental questions asked to comprehend the overwhelming amount of information …