with Embodiment Dreaming
EgoX treats human, simulated, and heterogeneous robot demonstrations as unified egocentric trajectories, pretrains one shared manipulation representation, and specializes it to each target embodiment.
MXT+ Architecture and EgoX Embodiments. Top: embodiment-specific modality tokenizers map each embodiment’s observations into tokens that a shared Transformer encoder-decoder trunk fuses across embodiments. Embodiment action experts decode the trunk output into embodiment-specific action chunks, while embodiment-dreaming experts predict future embodiment latents supervised by EMA copies of the corresponding modality tokenizers. Bottom: the embodiments used in EgoX. Each panel marks the unified egocentric frame ℱego and the end-effector-link keypoints.
Choose an embodiment
Select a model to filter videos. Use its expand icon for a larger 3D view.
G1-Dex5
xArm7-Dex4
G1-Sim-Dex3
LocoMan-Dex2
R1Lite-Gripper
Z1-Tool
Policy rollouts
Across all tasks and embodiments, we show successful rollouts of MXT+ with pretraining, illustrating each task setup and what the method can do. On R1Lite-Gripper Kitchen-Bread, we also compare it with MXT+ without pretraining, π0.5, and ACT, marking each rollout as a success or failure.
Does egocentric cross-embodiment pretraining outperform training from scratch?
We compare MXT+ with and without cross-embodiment pretraining against ACT and π0.5 on 13 task–embodiment pairs. Under ID evaluation, pretraining yields particularly large gains on G1-Dex5 Sort, LocoMan-Dex2 Scoop, and xArm7-Dex4 Scoop. The benefit becomes even more pronounced under OOD evaluation, with substantial improvements on G1-Dex5 Kitchen-Bread, G1-Dex5 Insert, and Z1-Tool Scoop.
Normalized task progress score, mean ± SEM. Real robots: 20 trials per pair (13 ID, 7 OOD). Simulation: 50 trials per pair (32 ID, 18 OOD). Hover over a bar for its value.
Main Results. Cross-embodiment-pretrained MXT+ generally improves task progress score across 13 task–embodiment pairs, with substantially larger gains under OOD evaluation.
Does pretraining benefit from embodiment diversity?
We vary the pretraining mixture: no pretraining, human-only, human plus other robots, and full cross-embodiment data, which also includes the target robot's demonstrations on other tasks. Every variant is fine-tuned on the same target-embodiment, target-task demonstrations.
Normalized task progress score for each pretraining mixture. Hover over a line or a legend entry to trace one setting.
Pretraining Data Composition. Human-only pretraining improves over no pretraining on several task–embodiment pairs, and adding robot data helps further in some settings. Broader embodiment mixtures tend to improve performance, with full cross-embodiment pretraining showing the strongest overall trend, particularly under OOD evaluation.
Does embodiment dreaming improve cross-embodiment learning?
Embodiment dreaming asks the shared trunk to predict future embodiment latents built from egocentric image features, robot joint configurations, and end-effector joint configurations. EMA copies of MXT+'s own modality encoders provide stable targets. The objective encourages embodiment-aware dynamics modeling during pretraining and finetuning; dreaming heads and EMA teachers are discarded at deployment.
We compare MXT+ with MXT, its counterpart without the dreaming objective, on seven task–embodiment settings across five embodiments. The gains hold across tasks and embodiments, and are particularly pronounced on G1-Dex5.
Normalized task progress score pooled over ID and OOD trials, mean ± SEM.
Embodiment Dreaming. The predictive objective improves task progress score across all seven reported task–embodiment settings while adding no deployment-time computation.
Does latent geometry become more similar across embodiments?
We compute Gromov–Wasserstein (GW) distances between embodiment-specific distributions of shared-trunk latents extracted from Scoop demonstrations, and report the percentage change in GW for each embodiment pair. Negative values indicate greater geometric similarity.
mean GW across pairs, lower for 9 of 10 pairs. MXT+ with vs. without cross-embodiment pretraining.
mean GW across pairs, lower for all 10 pairs. MXT+ vs. MXT, both pretrained and finetuned.
Latent Geometry. Pretraining and embodiment dreaming both make the internal distance structure of embodiment-specific latents more similar for the same manipulation task.
Does modular design enhance cross-embodiment transfer?
We compare MXT+ with a coarse variant that merges modality-specific inputs and actions, each with and without cross-embodiment pretraining. The modular design consistently outperforms the coarse variant, with particularly pronounced gains on the higher-dimensional xArm7-Dex4 and G1-Sim-Dex3.
Normalized task progress score pooled over ID and OOD trials, mean ± SEM.
Modular Design. After pretraining, the coarse variant scores zero on Kitchen-Cleanup, whose first step requires a 3D velocity command for locomotion, a small but critical modality within G1's 51-dimensional action space.
@article{egox2026,
title={EgoX: Egocentric Cross-Embodiment Manipulation with Embodiment Dreaming},
author={Anonymous Authors},
year={2026},
note={Under review}
}