Add exported openvino model 'openvino_model_qint8_quantized.xml'
#28
by tomaarsen HF Staff - opened
Hello!
This pull request has been automatically generated from the export_static_quantized_openvino_model function from the Sentence Transformers library.
Config
OVQuantizationConfig(
quant_method=<OVQuantizationMethod.DEFAULT: 'default'>
)
Tip:
Consider testing this pull request before merging by loading the model from this PR with the revision argument:
from sentence_transformers import SentenceTransformer
# TODO: Fill in the PR number
pr_number = 2
model = SentenceTransformer(
"shibing624/text2vec-base-chinese",
revision=f"refs/pr/{pr_number}",
backend="openvino",
model_kwargs={"file_name": "openvino_model_qint8_quantized.xml"},
)
# Verify that everything works as expected
embeddings = model.encode(["The weather is lovely today.", "It's so sunny outside!", "He drove to the stadium."])
print(embeddings.shape)
similarities = model.similarity(embeddings, embeddings)
print(similarities)
效果损失太大，emb模型不建议量化。Quantization is not recommended for embedding models as it results in significant performance degradation.
shibing624 changed pull request status to closed