Technical question: Lineage of KaLM-Embedding/KaLM-embedding-multilingual-mini-instruct-v2.5
Dear [Developer/Team],
I recently came across KaLM-Embedding/KaLM-embedding-multilingual-mini-instruct-v2.5, and it has been very helpful in my project.
Before building on top of it, I would like to understand its connection with HIT-TMG/KaLM-embedding-multilingual-mini-instruct-v2:
Direct Fine-tuning: Is KaLM-Embedding/KaLM-embedding-multilingual-mini-instruct-v2.5 a direct fine-tuned version of HIT-TMG/KaLM-embedding-multilingual-mini-instruct-v2, or were there intermediate models/checkpoints involved?
Inheritance: Does it strictly inherit the architecture and weights of HIT-TMG/KaLM-embedding-multilingual-mini-instruct-v2 without merging or distilling from other models?
This would help me ensure I use the model correctly.
Thanks a lot for your help!
Hi dqdw,
Thanks for reaching out, and we’re glad KaLM-embedding-multilingual-mini-instruct-v2.5 has been helpful for your project.
To clarify the lineage:
1,.Direct fine-tuning / intermediate checkpoints: v2.5 was obtained by further training on top of HIT-TMG/KaLM-embedding-multilingual-mini-instruct-v2 (i.e., continual training from v2). There were no separate intermediate base models involved beyond this v2 → v2.5 continuation.
2. Inheritance (architecture & weights): v2.5 strictly inherits the same architecture and weights from v2 and continues training from that checkpoint.
3. Model merging/distillation: We did not use model merging in producing v2.5. The training recipe is based on contrastive distillation and contrastive training starting from the v2 model.
For more details on the methodology, please refer to our paper: https://openreview.net/pdf?id=Y7qzhvWhcz
If you have any questions about training settings, feel free to let us know.
Best regards,