MG-VQA: Manipulation Grounded Visual Question Answering with VLMs
arXiv:2608.17129
cuRoboV2: Dynamics-Aware Motion Generation with Depth-Fused Distance Fields for High-DoF Robots
arXiv:2603.05493
RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
Robotics Science and Systems (RSS), Sydney, Australia, July 2026
Press: NVIDIA blog post
CARI4D: Category Agnostic 4D Reconstruction of Human-Object Interaction
Computer Vision and Pattern Recognition (CVPR), Denver, Colorado, June 2026
Also in CVPR Workshop on Agents in Interactions: From Humans to Robots, Denver, Colorado, June 2026
Fast-FoundationStereo: Real-Time Zero-Shot Stereo Matching
Computer Vision and Pattern Recognition (CVPR), Denver, Colorado, June 2026
GraspGen-X: Cross-Embodiment 6-DOF Diffusion-based Grasping
Computer Vision and Pattern Recognition (CVPR), Denver, Colorado, June 2026
Press: NVIDIA blog post
FoundationStereo: Zero-Shot Stereo Matching
Computer Vision and Pattern Recognition (CVPR), Nashville, Tennessee, June 2025
Award: CVPR Best Paper Award Candidate
News: #1 on Middlebury Stereo Evaluation - Version 3 (2025/02/03 – 2025/03/04)
News: #1 on ETH3D Low-Res Two-View Benchmark (2024/11/15 – 2025/05/14)
RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Robotics
Computer Vision and Pattern Recognition (CVPR), Nashville, Tennessee, June 2025
Also in CVPR Workshop on Emergent Visual Abilities and Limits of Foundation Models (EVAL-FoMo-2), Nashville, Tennessee, June 2025
Also in CVPR Workshop on 3D-LLM/VLA, Nashville, Tennessee, June 2025
Also in ICLR Workshop on Robot Learning: Towards Robots with Human-Level Abilities, Singapore, April 2025
Also in CoRL Workshop on GenPriors, Seoul, Korea, September 2025
Towards a VLM Benchmark for Simulated Robotics
RSS Workshop on Large Foundation Model for Interactive Robot Learning (LFM), Los Angeles, June 2025
Pushing Buttons: Aligning Humanoid Real2Sim2Real via Active Perception
2025
FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects
Computer Vision and Pattern Recognition (CVPR), Seattle, June 2024
Award: BOP Challenge 2024 "Early Bird" Award
News: #1 on BOP leaderboard for unseen 6D pose estimation (2023/11/19 – 2024/09/03)
BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objects
Computer Vision and Pattern Recognition (CVPR), Vancouver, June 2023
6-DoF Pose Estimation of Household Objects for Robotic Manipulation: An Accessible Dataset and Benchmark
International Conference on Intelligent Robots and Systems (IROS), Kyoto, Japan, October 2022
Self-Supervised Real-to-Sim Scene Generation
International Conference on Computer Vision (ICCV), Montreal, Canada, October 2021 (virtual)
RMPflow: A Geometric Framework for Generation of Multi-Task Motion Policies
IEEE Transactions on Automation Science and Engineering (T-ASE), 18(3):968-987, July 2021
Contextual Reinforcement Learning of Visuo-tactile Multi-fingered Grasping Policies
NeurIPS Workshop on Robot Learning, December 2020 (virtual)
Improving Deep Stereo Network Generalization with Geometric Priors
arXiv:2008.11098, August 2020
Few-Shot Viewpoint Estimation
British Machine Vision Conference (BMVC), Cardiff, UK, September 2019
CityFlow: A City-Scale Benchmark for Multi-Target Multi-Camera Vehicle Tracking and Re-Identification
Computer Vision and Pattern Recognition (CVPR), Long Beach, California, June 2019
Multiple Hypothesis Tracking with Kinematics and Appearance Models on Traffic Flow for Wide Area Traffic Surveillance
ASCE Journal of Civil Engineering, 33(3), May 2019
Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects
Conference on Robot Learning (CoRL), Zürich, Switzerland, October 2018
Press: NVIDIA, VentureBeat, ZDNet, Robot Report, HardOCP, Design World, rbr, R&A News, Sensors
Falling Things: A Synthetic Dataset for 3D Object Detection and Pose Estimation
CVPR Workshop on Real World Challenges and New Benchmarks for Deep Learning in Robotic Vision, Salt Lake City, Utah, June 2018
Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization
CVPR Workshop on Autonomous Driving (WAD), Salt Lake City, Utah, June 2018
Synthetically Trained Neural Networks for Learning Human-Readable Plans from Real-World Demonstrations
IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia, May 2018
Press: NVIDIA, TechCrunch, VentureBeat, Slash Gear, ZD Net
Automated Traffic Surveillance System with Aerial Camera Arrays Imagery: Macroscopic Data Collection with Vehicle Tracking
Journal of Computing in Civil Engineering, 31(3), 2017
Toward Low-Flying Autonomous MAV Trail Navigation using Deep Neural Networks for Environmental Awareness
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver BC, Canada, September 2017
Reverse-Projection Method for Measuring Camera MTF
IS&T International Symposium on Electronic Imaging: Image Quality and System Performance XIV, January 2017
RGBD Point Cloud Alignment Using Lucas-Kanade Data Association and Automatic Error Metric Selection
IEEE Transactions on Robotics (T-RO), 31(6):1548-1554, December 2015
Program Synthesis by Examples for Object Repositioning Tasks
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Chicago, Illinois, September 2014
Efficient Hierarchical Graph-Based Segmentation of RGBD Videos
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, Ohio, June 2014
Fast and Accurate PoseSLAM by Combining Relative and Global State Spaces
IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China, May 2014
An Inexpensive Method for Evaluating the Localization Performance of a Mobile Robot Navigation System
IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China, May 2014
An Energy Minimization Approach to Automatic Traffic Camera Calibration Using MCMC
IEEE Transactions on Intelligent Transportation Systems, 14(3):1095-1108, September 2013
A Person Follower Mobile Robot System for Indoor Environments
IROS Workshop on Assistance and Service Robotics in a Human Environment, Tokyo, Japan, November 2013
Indoor Navigation for Mobile Robots using Predictive Fields
American Control Conference (ACC), Washington, DC, June 2013
Replacing Projective Data Association with Lucas-Kanade for KinectFusion
IEEE International Conference on Robotics and Automation (ICRA), Karlsruhe, Germany, May 2013
A New Approach to Clothing Classification using Mid-Level Layers
IEEE International Conference on Robotics and Automation (ICRA), Karlsruhe, Germany, May 2013
3D Non-Rigid Deformable Surface Estimation Without Feature Correspondence
IEEE International Conference on Robotics and Automation (ICRA), Karlsruhe, Germany, May 2013
Real-Time Obstacle Detection and Avoidance in the Presence of Specular Surfaces Using an Active 3D Sensor
IEEE Workshop on Robot Vision (WoRV), Clearwater, Florida, January 2013
Interactive Perception of Rigid and Non-Rigid Objects
International Journal of Advanced Robotic Systems, vol. 9, September 2012
An Energy Minimization Approach to 3D Non-Rigid Deformable Surface Estimation Using RGBD Data
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vilamoura, Portugal, October 2012
Accurate On-Line 3D Occupancy Grids Using Manhattan World Constraints
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vilamoura, Portugal, October 2012
Extracting Minimalistic Corridor Geometry from Low-Resolution Images
International Conference on Intelligent Robotics and Applications (ICIRA), Montreal, Quebec, Canada, October 2012
Occlusion-Aware Reconstruction and Manipulation of 3D Articulated Objects
IEEE International Conference on Robotics and Automation (ICRA), St. Paul, Minnesota, May 2012
Development of an Automated Algorithm for Tracking and Quantifying Barnacle Cyprid Settlement Behavior
Journal of Experimental Marine Biology and Ecology, volume 410, pages 21-28, December 2011
Model for Unfolding Laundry using Interactive Perception
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), San Francisco, California, September 2011
Robot Crowd Navigation using Predictive Position Fields in the Potential Function Framework
American Control Conference (ACC), San Francisco, California, July 2011
Classification of Clothing using Interactive Perception
IEEE International Conference on Robotics and Automation (ICRA), Shanghai, China, May 2011
Evaluation of a Computer-Vision Tracking System for Collecting Traffic Data
Transportation Research Board Annual Meeting, Washington, D.C., January 2011
Fast image mosaicking using optical flow
Proceedings of IEEE Southeastcon Conference, March 2011
Iris Segmentation in Non-Ideal Images Using Graph Cuts
Image and Vision Computing, 28(12):1671-1681, December 2010
Traffic Monitoring of Motorcycles During Special Events Using Video Detection
Transportation Research Record: Journal of the Transportation Research Board,
No. 2160, pp. 69-76, 2010
(presented at TRB Annual Meeting, Washington, D.C., January 2010)
Vision-Based Path Following Without Calibration
In Mobile Robots Navigation, ed. Alejandra Barrera, InTech, pages 427-446, March 2010 (ISBN 978-953-307-076-6)
Algorithm for Discriminating Aggregate Gaze Points: Comparison with Salient Regions-Of-Interest
International Workshop on Gaze Sensing and Interactions (IWGSI), Queenstown, New Zealand, November 2010
Image-Based Segmentation of Indoor Corridor Floors for a Mobile Robot
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Taipei, Taiwan, October 2010
Rigid and Non-Rigid Classification Using Interactive Perception
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Taipei, Taiwan, October 2010
Adaptive Fragments-Based Tracking of Non-Rigid Objects Using Level Sets
International Conference on Computer Vision (ICCV), Kyoto, Japan, September 2009
Motion-Based View-Invariant Articulated Motion Detection and Pose Estimation Using Sparse Point Features
International Symposium on Visual Computing, Las Vegas, Nevada, November 2009
Automatic Discrimination of Fine Roots in Minirhizotron Images
New Phytologist, 177(2):549-557, January 2008
Automatic Camera Calibration Using Pattern Detection for Vision-Based Speed Sensing
Transportation Research Record: Journal of the Transportation Research Board,
No. 2086, pp. 30-39, 2008
(presented at TRB Annual Meeting, Washington, D.C., January 2008)
Adapting Starburst for Elliptical Iris Segmentation
IEEE Second International Conference on Biometrics: Theory, Applications and Systems (BTAS), Washington, D.C., September 2008
Joint Tracking of Features and Edges
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Anchorage, Alaska, June 2008
Visual Detection of Lintel-Occluded Doors from a Single Image
CVPR Workshop on Visual Localization for Mobile Platforms, Anchorage, Alaska, June 2008
Autonomous Navigation and Mapping Using Monocular Low-Resolution Grayscale Vision
CVPR Workshop on Visual Localization for Mobile Platforms, Anchorage, Alaska, June 2008
Non-Ideal Iris Segmentation Using Graph Cuts
CVPR Workshop on Biometrics, Anchorage, Alaska, June 2008
Limbus/Pupil Switching for Wearable Eye Tracking Under Variable Lighting Conditions
Symposium on Eye Tracking Research and Applications (ETRA), Savannah, Georgia, pages 61-64, March 2008
Spatial Histograms for Region-Based Tracking
Electronics and Telecommunications Research Institute (ETRI) Journal, 29(5):697-699, October 2007
Person Following with a Mobile Robot Using Binocular Feature-Based Tracking
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), San Diego, California, pages 815-820, October 2007
Isomap Tracking with Particle Filtering
IEEE International Conference on Image Processing (ICIP), San Antonio, Texas, vol. 2, pages 513-516, September 2007
Real-Time Detection and Tracking of Vehicle Base Fronts for Measuring Traffic Counts and Speeds on Highways
Transportation Research Record: Journal of the Transportation Research Board,
No. 1993, pp. 155-164, 2007
(presented at TRB Annual Meeting, Washington, D.C., January 2007)
Motion Segmentation at Any Speed
British Machine Vision Conference (BMVC), Edinburgh, Scotland, pages 427-436, September 2006
Qualitative Vision-Based Mobile Robot Navigation
IEEE International Conference on Robotics and Automation (ICRA), Orlando, Florida, pages 2686-2692, May 2006
Vehicle Segmentation and Tracking in the Presence of Occlusions
Transportation Research Record: Journal of the Transportation Research Board,
No. 1944, pp. 89-97, 2006
(presented at TRB Annual Meeting, Washington, D.C., January 2006)
Automated Patient Facial Image Capture to Reduce Medical Error
American Medical Informatics Association (AMIA) Annual Symposium, Washington DC, November 2006
Automatic Detection of Fine Roots in Minirhizotron Images
American Society for Horticultural Science (ASHS) Annual Conference, New Orleans, Louisiana, July 2006
GIS-Based Roadway Video-logging using a Firewire Camera Combined with Satellite Differentially Corrected GPS
19th Geospatial Information Systems for Transportation Symposium, Columbus, Ohio, March 2006
Spatiograms Versus Histograms for Region-Based Tracking
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), San Diego, California, volume 2, pages 1158-1163, June 2005
Vehicle Segmentation and Tracking from a Low-Angle Off-Axis Camera
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), San Diego, California, volume 2, pages 1152-1157, June 2005
Analysis of Neurite Outgrowth for a Laser Patterned Neuronal Culture
2nd International IEEE EMBS Special Topic Conference on Neural Engineering, Arlington, Virginia, March 2005
Acoustic Localization by Interaural Level Difference
IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Philadelphia, Pennsylvania, vol. 4, pages 1109-1112, March 2005
A Unifying Framework for Acoustic Localization
12th European Signal Processing Conference (EUSIPCO), Vienna, Austria, September 2004
Extension and Evaluation of MDS for Geometric Microphone Array Calibration
12th European Signal Processing Conference (EUSIPCO), Vienna, Austria, September 2004
Geometric Microphone Array Calibration by Multidimensional Scaling
IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Hong Kong, April 2003
Fast Bayesian Acoustic Localization
IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Orlando, Florida, May 2002
Acoustic Source Direction by Hemisphere Sampling
IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Salt Lake City, Utah, May 2001
Depth and Motion Discontinuities
Ph.D. dissertation, Stanford University, June 1999
Multiway Cut for Stereo and Motion with Slanted Surfaces
Seventh IEEE International Conference on Computer Vision (ICCV), Kerkyra, Greece, pages 489-495, September 1999
Elliptical Head Tracking Using Intensity Gradients and Color Histograms
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Santa Barbara, California, pages 232-237, June 1998
Depth Discontinuities by Pixel-to-Pixel Stereo
Sixth IEEE International Conference on Computer Vision (ICCV), Mumbai, India, pages 1073-1080, January 1998
An Elliptical Head Tracker
31st Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, California, pages 1710-1714, November 1997
Depth Discontinuities by Pixel-to-Pixel Stereo
Stanford University Technical Report STAN-CS-TR-96-1573, July 1996
