Bio

I am a Research Scientist at Google DeepMind. Before that, I was a postdoc at Meta AI (FAIR) working on real-time 3D scene understanding for robotics. I graduated from my PhD at Imperial College London in 2023, advised by Prof. Andrew Davison. I completed my undergraduate and Masters in Physics at the University of Oxford.

My research interests are in building efficient scene understanding and planning systems for robotics. Towards this goal, my work has focused on 1) graphical representations and distributed inference algorithms on graphs, and 2) training neural scene representations via continual learning for real-time robotics.

Publications

DMC-VB: A Benchmark for Representation Learning for Control with Visual Distractors
NeurIPS 2024
Joseph Ortiz, Antoine Dedieu, Wolfgang Lehrach, Swaroop Guntupalli, Carter Wendelken, Ahmad Humayun, Guangyao Zhou, Sivaramakrishnan Swaminathan, Miguel Lázaro-Gredilla, Kevin Murphy
Diffusion Model Predictive Control
ArXiv 2024
Guangyao Zhou, Sivaramakrishnan Swaminathan, Rajkumar Vasudeva Raju, J Swaroop Guntupalli, Wolfgang Lehrach, Joseph Ortiz, Antoine Dedieu, Miguel Lázaro-Gredilla, Kevin Murphy
A Touch, Vision, and Language Dataset for Multimodal Alignment
ICML 2024
Letian Fu, Gaurav Datta, Huang Huang, William Chung-Ho Panitch, Jaimyn Drake, Joseph Ortiz, Mustafa Mukadam, Mike Lambeta, Roberto Calandra, Ken Goldberg
Neural feels with neural fields: Visuo-tactile perception for in-hand manipulation
Science Robotics 2024
Sudharshan Suresh, Haozhi Qi, Tingfan Wu, Taosha Fan, Luis Pineda, Mike Lambeta, Jitendra Malik, Mrinal Kalakrishnan, Roberto Calandra, Michael Kaess, Joseph Ortiz, Mustafa Mukadam
Perceiving Extrinsic Contacts from Touch Improves Learning Insertion Policies
ArXiv 2023
Carolina Higuera, Joseph Ortiz, Haozhi Qi, Luis Pineda, Byron Boots, Mustafa Mukadam
Decentralization and Acceleration Enables Large-Scale Bundle Adjustment
RSS 2023
Taosha Fan, Joseph Ortiz, Ming Hsiao, Maurizio Monge, Jing Dong, Todd Murphey, Mustafa Mukadam
Gaussian Belief Propagation for Real-Time Decentralised Inference
PhD Thesis
Joseph Ortiz
Theseus: A Library for Differentiable Nonlinear Optimization
NeurIPS 2022
Luis Pineda, Taosha Fan, Maurizio Monge, Shobha Venkataraman, Paloma Sodhi,
Ricky Chen, Joseph Ortiz, Daniel DeTone, Austin Wang, Stuart Anderson,
Jing Dong, Brandon Amos, Mustafa Mukadam
iSDF: Real-Time Neural Signed Distance Fields for Robot Perception
RSS 2022
Joseph Ortiz, Alexander Clegg, Jing Dong, Edgar Sucar, David Novotny,
Michael Zollhoefer, Mustafa Mukadam
A Robot Web for Distributed Many-Device Localisation
TRO 2022
Riku Murai, Joseph Ortiz, Sajad Saeedi, Paul Kelly, Andrew J. Davison
Incremental Abstraction in Distributed Probabilistic SLAM Graphs
ICRA 2022
Joseph Ortiz, Talfan Evans, Edgar Sucar, Andrew J. Davison
iMAP: Implicit Mapping and Positioning in Real-Time
ICCV 2021
Edgar Sucar, Shikun Liu, Joseph Ortiz, Andrew J. Davison
A visual introduction to Gaussian Belief Propagation
Self published / arXiv 2021
Joseph Ortiz, Talfan Evans, Andrew J. Davison
Bundle Adjustment on a Graph Processor
CVPR 2020
Joseph Ortiz, Mark Pupilli, Stefan Leutenegger, Andrew J. Davison
FutureMapping 2: Gaussian Belief Propagation for Spatial AI
arXiv 2019
Andrew J. Davison, Joseph Ortiz

Highlights

February 2023
I passed my PhD! Thanks to the examiners Frank Dellaert and Mark van der Wilk.
December 2022
I will be co-organising the workshop on Distributed Graph Algorithms for Robotics at ICRA 2023!
July 2021