Back to main page
Publications
A curated record across structured procedural understanding, multimodal grounding,
calibrated uncertainty, and robotics. The common thread is reliability: representing
structure, measuring uncertainty, and making AI systems easier to evaluate.
10selected papers
8venues and workshops
4first/co-first author works
NLP and Multimodal Learning
Procedural structure, semantic parsing, and grounding language into visual or video representations.
LREC-COLING 2024
End-to-end Parsing of Procedural Text into Flow Graphs
Dhaivat J. Bhatt, Seyed Ahmad Abdollahpouri Hosseini, Federico Fancellu, Afsaneh Fazly
An end-to-end multitask framework for parsing procedural recipes into actions, entities, and graph edges.
procedural parsingflow graphsNLP
PDF
Abstract
We focus on parsing procedural text into fine-grained flow graphs that encode actions and entities, as well as their interactions. Unlike prior two-stage approaches, we propose an end-to-end multi-task framework that simultaneously performs tagging and graph generation, and use noisy silver annotations from unlabeled recipes to improve learning with limited annotated data.
ECCV 2022 oral
Flow Graph to Video Grounding for Weakly-Supervised Multi-step Localization
Nikita Dvornik, Isma Hadji, Hai Pham, Dhaivat Bhatt, Brais Martinez, Afsaneh Fazly, Allan D. Jepson
Grounding procedural flow graphs in instructional video without requiring exact step-order annotations at test time.
video groundingweak supervisionflow graphs
PDF
Abstract
We introduce flow graph to video grounding for weakly supervised multi-step localization. Instead of relying on human-provided step order for every video, we use procedural flow graphs to represent valid step orderings and infer the ordering and localization jointly.
CoNLL 2022
Visual Semantic Parsing: From Images to Abstract Meaning Representation
Mohamed Ashraf Abdelsalam, Zhan Shi, Federico Fancellu, Kalliopi Basioti, Dhaivat J. Bhatt, Vladimir Pavlovic, Afsaneh Fazly
Repurposing AMR-style meaning representations for structured visual scene understanding.
visual AMRsemantic parsingmultimodal
PDF
Abstract
We explore how Abstract Meaning Representation can address limitations of scene-graph-like visual abstractions. The work shows that existing text-to-AMR parsers can be repurposed toward image-based AMR representations and points toward richer visual scene understanding.
Robotics, Uncertainty, and Autonomy
Robot perception and autonomy work focused on calibration, risk, localization, and navigation.
ICRA 2022
f-Cal: Aleatoric Uncertainty Quantification for Robot Perception via Calibrated Neural Regression
Dhaivat Bhatt, Kaustubh Mani, Dishank Bansal, Krishna Murthy, Hanju Lee, Liam Paull
A calibration method for uncertainty estimates in safety-critical robot perception tasks.
uncertaintycalibrationrobot perception
PDF
Abstract
f-Cal improves uncertainty calibration by enforcing constraints across batches rather than only per sample. The method produces better calibrated uncertainty estimates for robot perception tasks such as object detection and monocular depth estimation.
ICML AIAD 2020
Probabilistic Object Detection: Strengths, Weaknesses, Opportunities
Dhaivat Bhatt, Dishank Bansal, Gunshi Gupta, Hanju Lee, Krishna Murthy Jatavallabhula, Liam Paull
A position paper on uncertainty-aware object detection for autonomous driving.
object detectionuncertaintyautonomous driving
PDF
Abstract
We survey the landscape of probabilistic object detection, identify key shortcomings, and outline promising directions toward accurate, reliable, and consistent uncertainty estimates in autonomous driving perception systems.
RAL + ICRA 2020
MapLite: Autonomous Intersection Navigation Without a Detailed Prior Map
Teddy Ort, Krishna Murthy, Rohan Banerjee, Sai Krishna Gottipati, Dhaivat Bhatt, Igor Gilitschenski, Liam Paull, Daniela Rus
Sparse-map autonomous navigation for complex intersections using onboard perception and local planning.
navigationautonomyperception
PDF
Abstract
MapLite pilots a vehicle to a destination using sparse public topometric maps, onboard road segmentation, and local planning. The system navigates intersections without GPS-like external localization or detailed prior maps.
RO-MAN 2019
Probabilistic Obstacle Avoidance and Object Following: An Overlap of Gaussians Approach
Dhaivat Bhatt, Akash Garg, Bharath Gopalakrishnan, K. Madhava Krishna
Risk-aware obstacle avoidance using overlap between robot and obstacle state distributions.
motion planningriskrobotics
PDF
Abstract
We propose an obstacle-avoidance and object-following method that accounts for uncertainty in robot and obstacle states by optimizing control with a bounded-risk measure based on overlap between Gaussian distributions.
RAL 2019
Deep Active Localization
Sai Krishna, Keehong Seo, Dhaivat Bhatt, Vincent Mai, Krishna Murthy, Liam Paull
Learning perception and action selection for active localization, trained in simulation and transferred to hardware.
active localizationsim-to-realrobotics
PDF
Abstract
We present an end-to-end differentiable active localization system that learns both perception and planning in simulation, then transfers to real robot hardware with no refinement.
Early Vision and Robotics
Earlier work on road understanding and visual perception for robotics.
IROS 2017
Have I Reached the Intersection? A Deep Learning-Based Approach for Intersection Detection from Monocular Cameras
Dhaivat Bhatt, Danish Sodhi, Arghya Pal, Vineeth Balasubramanian, Madhava Krishna
A recurrent visual model for detecting road intersections from monocular camera sequences.
road understandingmonocular visionLRCN
PDF
Abstract
We pose intersection detection as binary classification over a sequence of frames and use a recurrent convolutional model to capture temporal consistency in monocular road images.
ICVGIP 2016
CRF Based Method for Curb Detection Using Semantic Cues and Stereo Depth
Danish Sodhi, Sarthak Upadhyay, Dhaivat Bhatt, K. Madhava Krishna, Shanti Swarup
Curb detection by combining semantic segmentation, stereo depth, and dense CRFs.
curb detectionsemantic cuesstereo depth
PDF
Abstract
We combine pixel-wise semantic segmentation with depth and color information from stereo cameras, using dense CRFs to improve curb detection under varied road conditions.