An interactive curriculum exploring the evolution of Joint-Embedding Predictive Architectures, world models, embodied perception, and physical common sense — moving from passive observation to active planning in real environments.
Predicting abstract representations rather than raw data. Defining the "Meaning Space" and World Model philosophy.
Module 01
Why pixel-level reconstruction fails. Invariance and efficiency in latent space.
Observer · Noise InvarianceModule 02
The mathematical landscape of compatibility. Minimizing scalar energy.
EBM · MinimizationModule 03
Modeling multiple time scales. Fine details vs. broad semantic concepts.
Hierarchy · Temporal ScalesProving JEPA on static images and dynamic video. Learning motion, persistence, and content.
Module 04
The first milestone. Multi-block masking and asymmetric teacher-student updates.
I-JEPA · MaskingModule 05
Moving from space to time. Tubelet masking and motion/content disentanglement.
V-JEPA · PersistenceInteractive Demo
See how JEPA ignores unpredictable pixel noise by predicting in abstract latent space.
Latent Space · RobustnessApplying predictive architectures to audio, language, and 3D geometry to ensure universal grounding.
Transforming the passive observer into an active agent. Learning the causal effects of actions.
Module 08
Model-Predictive Control (MPC) and latent imagination rollouts.
MPC · RolloutsModule 09
Action-conditioned world models for robotics and zero-shot planning.
Act-JEPA · Embodied AIInteractive Demo
Classic embodied AI task. Use the agent to push a T-block into the target zone.
Embodied AI · InteractionInteractive Demo
Action-conditioned rollouts. Visualize branching futures based on thrust inputs.
Causal AI · RolloutsPrincipled stability, causal reasoning, handling uncertainty, and long-horizon thinking pathways.
Module 10
Mathematical cleanup and understanding cause-and-effect over correlation.
LeGEPA · CausalityModule 11
Predicting multiple sharp outcomes instead of blurry averages using latent variable z.
Stochasticity · Latent zModule 12
Semantic thinking pathways and complex sequence reasoning.
Long-Horizon · ReasoningInteractive Demo
Visualize compounding errors vs. regularized stability in latent world models.
Interactive · StabilityInteractive Demo
Visualizing search trees and long-horizon trajectory optimization through a maze.
Planning · MPCEnd-to-end stability with Gaussian regularization and Recurrent Rollout planning.
Module 13
Replacing complex VICReg with Simple Gaussian Regularization for 48x faster planning.
SIGReg · GaussianModule 14
Actionable intelligence via recurrant rollouts and "The Scrubber of Time".
LeWM · Recurrent RolloutsModule 15
Yann LeCun's argument for autonomous world models, physical common sense, and JEPA-style planning beyond next-token prediction.
Physical Intelligence · World Models| Stage | Focus | Key Milestones |
|---|---|---|
| Foundation | Latent Prediction | JEPA, H-JEPA |
| Perception | Images & Video | I-JEPA, V-JEPA |
| Broadening | Multi-sensory | Audio-JEPA, 3D-JEPA |
| Action | Control & Planning | Act-JEPA, V-JEPA 2 |
| Refinement | Stability & Reason | LeGEPA, Think-JEPA |
| SOTA | LeWorld & Agency | SIGReg, LeWM, Physical Intelligence |
Video Resource: Beyond LLMs: JEPA and the Road to AGI