Joe Ortiz

joeaortiz16@gmail.com
Joe Ortiz

I am a Staff Research Scientist at Google DeepMind. My recent research focus is on generative world models that can be used for reasoning and planning across long horizons, such as Genie 3 and Genie 4.

Before that, I was a postdoc at Meta AI (FAIR) working on perception for robotics. I graduated from my PhD at Imperial College London in 2023, advised by Prof. Andrew Davison. There my work focused on 1) graphical representations and distributed inference algorithms, and 2) training neural scene representations via continual learning. I completed my undergraduate and Masters in Physics at the University of Oxford.

Interns: Huijie Zhang (University of Michigan), Mingyang Deng (MIT)

Publications

2026
SEDRAS: Symbolically Evaluated Deep Research And Science
SEDRAS: Symbolically Evaluated Deep Research And Science
Fredrik Carlsson, Daniel Ward, Joseph Ortiz, Fangyu Liu, Joakim Nivre
ICML 2026 · Paper · Code
2025
Improving Transformer World Models for Data-Efficient RL
Improving Transformer World Models for Data-Efficient RL
Antoine Dedieu*, Joseph Ortiz*, Xinghua Lou, Carter Wendelken, Wolfgang Lehrach, J Swaroop Guntupalli, Miguel Lazaro-Gredilla, Kevin Patrick Murphy
ICML 2025 · Paper
2024
DMC-VB: A Benchmark for Representation Learning for Control with Visual Distractors
DMC-VB: A Benchmark for Representation Learning for Control with Visual Distractors
Joseph Ortiz, Antoine Dedieu, Wolfgang Lehrach, Swaroop Guntupalli, Carter Wendelken, Ahmad Humayun, Guangyao Zhou, Sivaramakrishnan Swaminathan, Miguel Lázaro-Gredilla, Kevin Murphy
NeurIPS 2024 · Paper · Code/Data
Diffusion Model Predictive Control
Diffusion Model Predictive Control
Guangyao Zhou, Sivaramakrishnan Swaminathan, Rajkumar Vasudeva Raju, J Swaroop Guntupalli, Wolfgang Lehrach, Joseph Ortiz, Antoine Dedieu, Miguel Lázaro-Gredilla, Kevin Murphy
ArXiv 2024 · Paper
A Touch, Vision, and Language Dataset for Multimodal Alignment
A Touch, Vision, and Language Dataset for Multimodal Alignment
Letian Fu, Gaurav Datta, Huang Huang, William Chung-Ho Panitch, Jaimyn Drake, Joseph Ortiz, Mustafa Mukadam, Mike Lambeta, Roberto Calandra, Ken Goldberg
ICML 2024 · Project Page · Paper · Code · Dataset · Models
Neural feels with neural fields: Visuo-tactile perception for in-hand manipulation
Neural feels with neural fields: Visuo-tactile perception for in-hand manipulation
Sudharshan Suresh, Haozhi Qi, Tingfan Wu, Taosha Fan, Luis Pineda, Mike Lambeta, Jitendra Malik, Mrinal Kalakrishnan, Roberto Calandra, Michael Kaess, Joseph Ortiz, Mustafa Mukadam
Science Robotics 2024 · Project Page · Paper · ArXiv · Video
2023
Perceiving Extrinsic Contacts from Touch Improves Learning Insertion Policies
Perceiving Extrinsic Contacts from Touch Improves Learning Insertion Policies
Carolina Higuera, Joseph Ortiz, Haozhi Qi, Luis Pineda, Byron Boots, Mustafa Mukadam
ArXiv 2023 · Paper · Video · Code
Decentralization and Acceleration Enables Large-Scale Bundle Adjustment
Decentralization and Acceleration Enables Large-Scale Bundle Adjustment
Taosha Fan, Joseph Ortiz, Ming Hsiao, Maurizio Monge, Jing Dong, Todd Murphey, Mustafa Mukadam
RSS 2023 · Project Page · Paper · Video · Code
Gaussian Belief Propagation for Real-Time Decentralised Inference
Gaussian Belief Propagation for Real-Time Decentralised Inference
Joseph Ortiz
PhD Thesis · PDF
2022
Theseus: A Library for Differentiable Nonlinear Optimization
Theseus: A Library for Differentiable Nonlinear Optimization
Luis Pineda, Taosha Fan, Maurizio Monge, Shobha Venkataraman, Paloma Sodhi, Ricky Chen, Joseph Ortiz, Daniel DeTone, Austin Wang, Stuart Anderson, Jing Dong, Brandon Amos, Mustafa Mukadam
NeurIPS 2022 · Project Page · Paper · Code
iSDF: Real-Time Neural Signed Distance Fields for Robot Perception
iSDF: Real-Time Neural Signed Distance Fields for Robot Perception
Joseph Ortiz, Alexander Clegg, Jing Dong, Edgar Sucar, David Novotny, Michael Zollhoefer, Mustafa Mukadam
RSS 2022 · Project Page · Paper · Code · Video
A Robot Web for Distributed Many-Device Localisation
A Robot Web for Distributed Many-Device Localisation
Riku Murai, Joseph Ortiz, Sajad Saeedi, Paul Kelly, Andrew J. Davison
TRO 2022 · Project Page · Paper · Demo Video
Incremental Abstraction in Distributed Probabilistic SLAM Graphs
Incremental Abstraction in Distributed Probabilistic SLAM Graphs
Joseph Ortiz, Talfan Evans, Edgar Sucar, Andrew J. Davison
ICRA 2022 · Project Page · Paper · Video
2021
iMAP: Implicit Mapping and Positioning in Real-Time
iMAP: Implicit Mapping and Positioning in Real-Time
Edgar Sucar, Shikun Liu, Joseph Ortiz, Andrew J. Davison
ICCV 2021 · Project Page · Paper · Video
A visual introduction to Gaussian Belief Propagation
A visual introduction to Gaussian Belief Propagation
Joseph Ortiz, Talfan Evans, Andrew J. Davison
Self published / arXiv 2021 · Article · Intro Video
2019
FutureMapping 2: Gaussian Belief Propagation for Spatial AI
FutureMapping 2: Gaussian Belief Propagation for Spatial AI
Andrew J. Davison, Joseph Ortiz
arXiv 2019 · Paper