Hey, I was reading about Joint Embedding Predictive Architecture. Based on my understanding, I believe the architecture of JEPA is as so, there are two frames x and y, where y comes after x. The x frame is masked, it has only the important features, that we can understand about the world of x. From that masked x, we are told to predict y, which is called y’.
How is this different from Masked Auto Encoders, if that is the case?
Hmm…?
Your description matches the high-level pattern “hide part of the input, predict something about what’s missing.” Both JEPA-style methods and MAE do that. The difference is what they predict and where the loss is applied.
Goal: reconstruct the original signal (usually pixels) from a heavily masked input.
So MAE is a generative reconstruction objective: it forces the model to preserve enough low-level detail to reproduce the input.
JEPA is a family of self-supervised objectives proposed as part of a broader “world model” direction.
A JEPA trains an encoder so that, given one view/context, it can predict the representation (embedding) of another view/target.
I-JEPA is explicitly described as non-generative and as predicting in representation space rather than pixels.
Mechanically (simplified):
Take one image.
Choose:
Context encoder produces embeddings for the visible context.
Predictor network outputs predicted embeddings for the target blocks.
A separate target encoder produces the target embeddings.
Loss: L2 distance between predicted target embeddings and target encoder embeddings.
The target encoder is updated via exponential moving average (EMA) of the context encoder weights (a teacher-student style stabilizer).
Two details that often clear up confusion:
That framing is more natural for video JEPA variants (past context predicts future embeddings). JEPA is intended to generalize to time as well as space. The “world model” motivation is explicit in LeCun’s position paper.
But the most-cited JEPA implementation in vision, I-JEPA, uses one image and predicts masked regions’ representations, not future frames’ pixels.
In both MAE and I-JEPA, the mask is usually not “importance-based.” It is a training trick to create a prediction task.
“Important features” are what you hope emerges in the learned representation, not what is explicitly kept by the mask.
| Aspect | MAE | JEPA (e.g., I-JEPA) |
|---|---|---|
| Prediction target | Pixels (reconstruct missing patches) (CVF Open Access) | Embeddings of target blocks (latent prediction) |
| Output space | Pixel / patch values | Representation space |
| Needs decoder to pixels | Yes (CVF Open Access) | Not for the SSL loss |
| Typical masking | Random high-ratio masking (CVF Open Access) | Context block + semantic target blocks |
| “Generative?” | Yes (reconstruction) | Non-generative (predict embeddings) |
| Stabilization | Reconstruction provides constraint | Often uses EMA target encoder, etc. |
Notes are above. URLs are here.
I-JEPA paper (arXiv): https://arxiv.org/abs/2301.08243
I-JEPA paper (PDF): https://arxiv.org/pdf/2301.08243
MAE paper (arXiv): https://arxiv.org/abs/2111.06377
MAE paper (CVPR PDF): https://openaccess.thecvf.com/content/CVPR2022/papers/He_Masked_Autoencoders_Are_Scalable_Vision_Learners_CVPR_2022_paper.pdf
LeCun “A Path Towards Autonomous Machine Intelligence” (OpenReview PDF): https://openreview.net/pdf?id=BZ5a1r-kVsf
Official MAE code (Meta): https://github.com/facebookresearch/mae
Official I-JEPA code (Meta): https://github.com/facebookresearch/ijepa
Meta blog on I-JEPA: https://ai.meta.com/blog/yann-lecun-ai-model-i-jepa/
hey, thanks for the answer. I think I will get more clarity after reading I-JEPA, based on your answer. I will reach out to you, if I have more questions. Thank you for your time