JEPA is an Energy-Based Model. Instead of probabilities, it calculates compatibility.
JEPA assigns a scalar "Energy" to pairs of inputs. Low energy signifies compatibility (the prediction matches the context), while high energy signifies a lack of physical or semantic relationship.
Where Sx is the Context Encoder, P is the Predictor, and z is a Latent Variable.
The 3D landscape represents the model's internal understanding of the world.
• Valleys (Low Energy): High-compatibility states. These are "physically possible" futures that match the context.
• Peaks (High Energy): Low-compatibility states. These represent semantic contradictions or physical impossibilities.
• The Ball: Represents a candidate prediction. Drag the mouse to test different states and observe how the energy (cost) spikes as you move toward the peaks.
Standard probabilistic models require calculating a Partition Function to normalize probabilities. EBMs avoid this entirely, making them more efficient for complex, high-dimensional spaces like video or 3D environments.
The biggest challenge in EBMs is preventing Representation Collapse, where the model assigns zero energy to everything.