EgoX
Egocentric Cross-Embodiment Manipulation
with Embodiment Dreaming
Anonymous Authors Under Review
EgoX framework spanning human and robot egocentric data, cross-embodiment pretraining, target-embodiment finetuning, and manipulation tasks

EgoX treats human, simulated, and heterogeneous robot demonstrations as unified egocentric trajectories, pretrains one shared manipulation representation, and specializes it to each target embodiment.

Abstract
Scaling robot manipulation requires learning from demonstrations collected across humans and heterogeneous robots. These sources cover diverse objects, scenes, tasks, and interaction strategies, but gaps in sensing, kinematics, motion patterns, and action spaces make it difficult to train a shared policy effectively. We introduce EgoX, an egocentric cross-embodiment framework for robot manipulation. Its central design is to share manipulation structure at the level of semantically corresponding modalities, while preserving embodiment-specific sensing and control interfaces. EgoX stores human and robot demonstrations as unified egocentric observation-action trajectories, pretrains a Modularized Cross-Embodiment Transformer with Embodiment Dreaming (MXT+) on the mixture, and fine-tunes it on target-embodiment demonstrations. MXT+ combines embodiment-specific modality tokenizers and action experts with a shared Transformer trunk. An auxiliary embodiment dreaming objective predicts future embodiment latents from egocentric visual and proprioceptive features, using exponential-moving-average copies of the modality encoders; this training-only branch encourages the trunk to encode embodiment-specific temporal structure without adding deployment-time computation. Across 13 task–embodiment pairs spanning five tasks and six robot embodiments, cross-embodiment pretraining consistently improves normalized task progress over target-embodiment training from scratch, with particularly strong gains under out-of-distribution evaluation. Broader pretraining mixtures further strengthen transfer, while embodiment dreaming improves contact-rich tool-use performance and reduces mean cross-embodiment latent-geometry distance by 38.52%.
Method
EgoX has three stages: (i) collect human and robot demonstrations as unified egocentric observation-action trajectories organized by corresponding modality types, (ii) pretrain MXT+ on mixed cross-embodiment data using action prediction and embodiment dreaming, and (iii) finetune the shared representation on target-embodiment tasks. Embodiment-specific modality tokenizers and action experts preserve heterogeneous sensing and control interfaces around a shared Transformer trunk. Dreaming experts predict future egocentric visual and proprioceptive latents during training and are discarded at deployment.
MXT+ architecture with embodiment-specific modality tokenizers, a shared Transformer trunk, action experts, and embodiment-dreaming experts, above the EgoX embodiments

MXT+ Architecture and EgoX Embodiments. Top: embodiment-specific modality tokenizers map each embodiment’s observations into tokens that a shared Transformer encoder-decoder trunk fuses across embodiments. Embodiment action experts decode the trunk output into embodiment-specific action chunks, while embodiment-dreaming experts predict future embodiment latents supervised by EMA copies of the corresponding modality tokenizers. Bottom: the embodiments used in EgoX. Each panel marks the unified egocentric frame ℱego and the end-effector-link keypoints.

Embodiments & Tasks
EgoX is evaluated on five manipulation tasks across six robot embodiments (five real robots and one simulated humanoid). Egocentric human demonstrations provide an additional pretraining embodiment.

Choose an embodiment

Select a model to filter videos. Use its expand icon for a larger 3D view.

Left Right
Human-Dex5

Human-Dex5

G1-Dex5

G1-Dex5

xArm7-Dex4

xArm7-Dex4

G1-Sim-Dex3

G1-Sim-Dex3

LocoMan-Dex2

LocoMan-Dex2

R1Lite-Gripper

R1Lite-Gripper

Z1-Tool

Z1-Tool

Explore in 3D

Drag to rotate · scroll or pinch to zoom

Policy rollouts

Across all tasks and embodiments, we show successful rollouts of MXT+ with pretraining, illustrating each task setup and what the method can do. On R1Lite-Gripper Kitchen-Bread, we also compare it with MXT+ without pretraining, π0.5, and ACT, marking each rollout as a success or failure.

G1-Dex5 | InsertID
MXT+ (w/ pretrain)
G1-Dex5 | InsertID
MXT+ (w/ pretrain)
G1-Dex5 | InsertID
MXT+ (w/ pretrain)
G1-Dex5 | InsertOOD
MXT+ (w/ pretrain)
G1-Dex5 | InsertOOD
MXT+ (w/ pretrain)
G1-Dex5 | InsertOOD
MXT+ (w/ pretrain)
G1-Dex5 | SortID
MXT+ (w/ pretrain)
G1-Dex5 | SortID
MXT+ (w/ pretrain)
G1-Dex5 | SortID
MXT+ (w/ pretrain)
G1-Dex5 | SortOOD
MXT+ (w/ pretrain)
G1-Dex5 | ScoopID
MXT+ (w/ pretrain)
G1-Dex5 | ScoopID
MXT+ (w/ pretrain)
G1-Dex5 | ScoopOOD
MXT+ (w/ pretrain)
G1-Dex5 | ScoopOOD
MXT+ (w/ pretrain)
xArm7-Dex4 | ScoopID
MXT+ (w/ pretrain)
xArm7-Dex4 | ScoopID
MXT+ (w/ pretrain)
xArm7-Dex4 | ScoopOOD
MXT+ (w/ pretrain)
xArm7-Dex4 | ScoopOOD
MXT+ (w/ pretrain)
xArm7-Dex4 | ScoopOOD
MXT+ (w/ pretrain)
R1Lite-Gripper | SortID
MXT+ (w/ pretrain)
R1Lite-Gripper | SortID
MXT+ (w/ pretrain)
R1Lite-Gripper | SortOOD
MXT+ (w/ pretrain)
G1-Dex5 | Kitchen-BreadID
MXT+ (w/ pretrain)
G1-Dex5 | Kitchen-BreadID
MXT+ (w/ pretrain)
G1-Dex5 | Kitchen-BreadID
MXT+ (w/ pretrain)
G1-Dex5 | Kitchen-BreadOOD
MXT+ (w/ pretrain)
xArm7-Dex4 | SortID
MXT+ (w/ pretrain)
xArm7-Dex4 | SortID
MXT+ (w/ pretrain)
xArm7-Dex4 | SortID
MXT+ (w/ pretrain)
xArm7-Dex4 | SortID
MXT+ (w/ pretrain)
xArm7-Dex4 | SortOOD
MXT+ (w/ pretrain)
xArm7-Dex4 | SortOOD
MXT+ (w/ pretrain)
xArm7-Dex4 | InsertID
MXT+ (w/ pretrain)
xArm7-Dex4 | InsertID
MXT+ (w/ pretrain)
xArm7-Dex4 | InsertID
MXT+ (w/ pretrain)
xArm7-Dex4 | InsertOOD
MXT+ (w/ pretrain)
xArm7-Dex4 | InsertOOD
MXT+ (w/ pretrain)
xArm7-Dex4 | InsertOOD
MXT+ (w/ pretrain)
R1Lite-Gripper | Kitchen-BreadID
MXT+ (w/ pretrain)
R1Lite-Gripper | Kitchen-BreadID
MXT+ (w/ pretrain)
R1Lite-Gripper | Kitchen-BreadID
MXT+ (w/ pretrain)
R1Lite-Gripper | Kitchen-BreadID
MXT+ (w/ pretrain)
R1Lite-Gripper | Kitchen-BreadID
MXT+ (w/ pretrain)
R1Lite-Gripper | Kitchen-BreadID
MXT+ (w/ pretrain)
R1Lite-Gripper | Kitchen-BreadOOD
MXT+ (w/ pretrain)
R1Lite-Gripper | Kitchen-BreadOOD
MXT+ (w/ pretrain)
R1Lite-Gripper | Kitchen-BreadOOD
MXT+ (w/ pretrain)
R1Lite-Gripper | Kitchen-BreadOOD
MXT+ (w/ pretrain)
R1Lite-Gripper | Kitchen-BreadOOD
MXT+ (w/ pretrain)
R1Lite-Gripper | Kitchen-BreadOOD
MXT+ (w/ pretrain)
R1Lite-Gripper | Kitchen-BreadID
MXT+ (w/o pretrain)
R1Lite-Gripper | Kitchen-BreadID
MXT+ (w/o pretrain)
R1Lite-Gripper | Kitchen-BreadID
MXT+ (w/o pretrain)
R1Lite-Gripper | Kitchen-BreadID
MXT+ (w/o pretrain)
R1Lite-Gripper | Kitchen-BreadID
MXT+ (w/o pretrain)
R1Lite-Gripper | Kitchen-BreadID
MXT+ (w/o pretrain)
R1Lite-Gripper | Kitchen-BreadOOD
MXT+ (w/o pretrain)
R1Lite-Gripper | Kitchen-BreadOOD
MXT+ (w/o pretrain)
R1Lite-Gripper | Kitchen-BreadOOD
MXT+ (w/o pretrain)
R1Lite-Gripper | Kitchen-BreadOOD
MXT+ (w/o pretrain)
R1Lite-Gripper | Kitchen-BreadOOD
MXT+ (w/o pretrain)
R1Lite-Gripper | Kitchen-BreadOOD
MXT+ (w/o pretrain)
R1Lite-Gripper | Kitchen-BreadID
π0.5
R1Lite-Gripper | Kitchen-BreadID
π0.5
R1Lite-Gripper | Kitchen-BreadID
π0.5
R1Lite-Gripper | Kitchen-BreadID
π0.5
R1Lite-Gripper | Kitchen-BreadID
π0.5
R1Lite-Gripper | Kitchen-BreadID
π0.5
R1Lite-Gripper | Kitchen-BreadOOD
π0.5
R1Lite-Gripper | Kitchen-BreadOOD
π0.5
R1Lite-Gripper | Kitchen-BreadOOD
π0.5
R1Lite-Gripper | Kitchen-BreadOOD
π0.5
R1Lite-Gripper | Kitchen-BreadOOD
π0.5
R1Lite-Gripper | Kitchen-BreadOOD
π0.5
R1Lite-Gripper | Kitchen-BreadID
ACT
R1Lite-Gripper | Kitchen-BreadID
ACT
R1Lite-Gripper | Kitchen-BreadID
ACT
R1Lite-Gripper | Kitchen-BreadID
ACT
R1Lite-Gripper | Kitchen-BreadID
ACT
R1Lite-Gripper | Kitchen-BreadID
ACT
R1Lite-Gripper | Kitchen-BreadOOD
ACT
R1Lite-Gripper | Kitchen-BreadOOD
ACT
R1Lite-Gripper | Kitchen-BreadOOD
ACT
R1Lite-Gripper | Kitchen-BreadOOD
ACT
R1Lite-Gripper | Kitchen-BreadOOD
ACT
R1Lite-Gripper | Kitchen-BreadOOD
ACT
Human-Dex5 | Kitchen-Bread
Human demonstration
Human-Dex5 | Kitchen-Bread
Human demonstration
Human-Dex5 | Kitchen-Bread
Human demonstration
Human-Dex5 | Kitchen-Bread
Human demonstration
Human-Dex5 | Sort
Human demonstration
Human-Dex5 | Sort
Human demonstration
Human-Dex5 | Insert
Human demonstration
Human-Dex5 | Insert
Human demonstration
Human-Dex5 | Scoop
Human demonstration
Human-Dex5 | Scoop
Human demonstration
LocoMan-Dex2 | ScoopID
MXT+ (w/ pretrain)
LocoMan-Dex2 | ScoopID
MXT+ (w/ pretrain)
LocoMan-Dex2 | ScoopID
MXT+ (w/ pretrain)
LocoMan-Dex2 | ScoopOOD
MXT+ (w/ pretrain)
LocoMan-Dex2 | ScoopOOD
MXT+ (w/ pretrain)
LocoMan-Dex2 | ScoopOOD
MXT+ (w/ pretrain)
G1-Sim-Dex3 | Kitchen-BreadID
MXT+ (w/ pretrain)
G1-Sim-Dex3 | Kitchen-BreadID
MXT+ (w/ pretrain)
G1-Sim-Dex3 | Kitchen-BreadOOD
MXT+ (w/ pretrain)
G1-Sim-Dex3 | Kitchen-CleanupID
MXT+ (w/ pretrain)
G1-Sim-Dex3 | Kitchen-CleanupID
MXT+ (w/ pretrain)
G1-Sim-Dex3 | Kitchen-CleanupOOD
MXT+ (w/ pretrain)
G1-Sim-Dex3 | Kitchen-CleanupOOD
MXT+ (w/ pretrain)
G1-Sim-Dex3 | Kitchen-CleanupOOD
MXT+ (w/ pretrain)
Z1-Tool | ScoopID
MXT+ (w/ pretrain)
Z1-Tool | ScoopID
MXT+ (w/ pretrain)
Z1-Tool | ScoopID
MXT+ (w/ pretrain)
Z1-Tool | ScoopID
MXT+ (w/ pretrain)
Z1-Tool | ScoopOOD
MXT+ (w/ pretrain)
Z1-Tool | ScoopOOD
MXT+ (w/ pretrain)
Z1-Tool | ScoopOOD
MXT+ (w/ pretrain)
Z1-Tool | ScoopOOD
MXT+ (w/ pretrain)
Z1-Tool | ScoopOOD
MXT+ (w/ pretrain)
Z1-Tool | ScoopOOD
MXT+ (w/ pretrain)
Results
We study five questions: (1) does egocentric cross-embodiment pretraining improve downstream manipulation over training from scratch, (2) how do the diversity and composition of pretraining embodiments affect performance, (3) does embodiment dreaming improve cross-embodiment transfer, (4) does latent geometry become more similar across embodiments, and (5) does modular design enhance cross-embodiment transfer? Evaluations cover ID and OOD settings using the normalized task progress score.
1

Does egocentric cross-embodiment pretraining outperform training from scratch?

We compare MXT+ with and without cross-embodiment pretraining against ACT and π0.5 on 13 task–embodiment pairs. Under ID evaluation, pretraining yields particularly large gains on G1-Dex5 Sort, LocoMan-Dex2 Scoop, and xArm7-Dex4 Scoop. The benefit becomes even more pronounced under OOD evaluation, with substantial improvements on G1-Dex5 Kitchen-Bread, G1-Dex5 Insert, and Z1-Tool Scoop.

Normalized task progress score, mean ± SEM. Real robots: 20 trials per pair (13 ID, 7 OOD). Simulation: 50 trials per pair (32 ID, 18 OOD). Hover over a bar for its value.

Main Results. Cross-embodiment-pretrained MXT+ generally improves task progress score across 13 task–embodiment pairs, with substantially larger gains under OOD evaluation.

2

Does pretraining benefit from embodiment diversity?

We vary the pretraining mixture: no pretraining, human-only, human plus other robots, and full cross-embodiment data, which also includes the target robot's demonstrations on other tasks. Every variant is fine-tuned on the same target-embodiment, target-task demonstrations.

Normalized task progress score for each pretraining mixture. Hover over a line or a legend entry to trace one setting.

Pretraining Data Composition. Human-only pretraining improves over no pretraining on several task–embodiment pairs, and adding robot data helps further in some settings. Broader embodiment mixtures tend to improve performance, with full cross-embodiment pretraining showing the strongest overall trend, particularly under OOD evaluation.

3

Does embodiment dreaming improve cross-embodiment learning?

Embodiment dreaming asks the shared trunk to predict future embodiment latents built from egocentric image features, robot joint configurations, and end-effector joint configurations. EMA copies of MXT+'s own modality encoders provide stable targets. The objective encourages embodiment-aware dynamics modeling during pretraining and finetuning; dreaming heads and EMA teachers are discarded at deployment.

We compare MXT+ with MXT, its counterpart without the dreaming objective, on seven task–embodiment settings across five embodiments. The gains hold across tasks and embodiments, and are particularly pronounced on G1-Dex5.

Normalized task progress score pooled over ID and OOD trials, mean ± SEM.

Embodiment Dreaming. The predictive objective improves task progress score across all seven reported task–embodiment settings while adding no deployment-time computation.

4

Does latent geometry become more similar across embodiments?

We compute Gromov–Wasserstein (GW) distances between embodiment-specific distributions of shared-trunk latents extracted from Scoop demonstrations, and report the percentage change in GW for each embodiment pair. Negative values indicate greater geometric similarity.

Pretraining · lower triangle −21.30%

mean GW across pairs, lower for 9 of 10 pairs. MXT+ with vs. without cross-embodiment pretraining.

Embodiment dreaming · upper triangle −38.52%

mean GW across pairs, lower for all 10 pairs. MXT+ vs. MXT, both pretrained and finetuned.

Latent Geometry. Pretraining and embodiment dreaming both make the internal distance structure of embodiment-specific latents more similar for the same manipulation task.

5

Does modular design enhance cross-embodiment transfer?

We compare MXT+ with a coarse variant that merges modality-specific inputs and actions, each with and without cross-embodiment pretraining. The modular design consistently outperforms the coarse variant, with particularly pronounced gains on the higher-dimensional xArm7-Dex4 and G1-Sim-Dex3.

Normalized task progress score pooled over ID and OOD trials, mean ± SEM.

Modular Design. After pretraining, the coarse variant scores zero on Kitchen-Cleanup, whose first step requires a 3D velocity command for locomotion, a small but critical modality within G1's 51-dimensional action space.

BibTeX
@article{egox2026,
  title={EgoX: Egocentric Cross-Embodiment Manipulation with Embodiment Dreaming},
  author={Anonymous Authors},
  year={2026},
  note={Under review}
}