Managing complexity through structural and temporal hierarchy: predicting both abstract concepts and granular details.
Multi-Level Latent Representations
Physical reality is organized into hierarchies. To build a robust World Model, an agent must simultaneously process information across multiple scales of space and time. **H-JEPA** extends the architecture by adding multiple levels of latent representations.
Higher Levels (Semantic)
Abstract Structure: Focuses on objects, scenes, and identities.
Longer Time Spans: Predicts shifts occurring over seconds or minutes (e.g., "The car is turning").
Lower Levels (Granular)
Fine Detail: Focuses on pixels, edges, and textures.
Short Time Steps: Predicts rapid fluctuations (e.g., "The reflection on the window is flickering").
Interactive Simulation: "The Hierarchical Observer"
Level 3 Analysis: Concept Trajectory
Predicting the abstract identity of the car is highly stable. The model ignores irrelevant pixel noise and wheel rotations, focusing on the overall semantic goal.