doesnt work
either in a colab or kaggle notebook (T4 gpu)
ValueError Traceback (most recent call last)
in <cell line: 0>()
1 from sentence_transformers import SentenceTransformer
2
----> 3 model = SentenceTransformer("lightonai/Reason-ModernColBERT")
4
5 sentences = [
6 frames
/usr/lib/python3.11/enum.py in new(cls, value)
1135 ve_exc = ValueError("%r is not a valid %s" % (value, cls.qualname))
1136 if result is None and exc is None:
-> 1137 raise ve_exc
1138 elif exc is None:
1139 exc = TypeError(
ValueError: 'MaxSim' is not a valid SimilarityFunction
Its because you are treating it like a normal sentence embeddings model (transformer based) and MaxSim isn't working for it.
Try the following way that they recommend in the readme , worked for me :
from pylate import indexes, models, retrieve
# Step 1: Load the ColBERT model
model = models.ColBERT(
model_name_or_path=pylate_model_id,
)
# Step 2: Initialize the Voyager index
index = indexes.Voyager(
index_folder="pylate-index",
index_name="index",
override=True, # This overwrites the existing index if any
)
# Step 3: Encode the documents
documents_ids = ["1", "2", "3"]
documents = ["document 1 text", "document 2 text", "document 3 text"]
documents_embeddings = model.encode(
documents,
batch_size=32,
is_query=False, # Ensure that it is set to False to indicate that these are documents, not queries
show_progress_bar=True,
)
# Step 4: Add document embeddings to the index by providing embeddings and corresponding ids
index.add_documents(
documents_ids=documents_ids,
documents_embeddings=documents_embeddings,
)
hi
thanks for your reply
i used the snippet provided in the 'use this model' section
reading the model card i see the custom implementation "from pylate import indexes, models, retrieve ..."
i dont want custom libraries, prefer to stick to HF libs, there are other rerankers based on modern bert and sentence transormers