Meta

• License: MIT License (MIT)
• Author: tien.ngnvan@gmail.com
• Maintainer: tien.ngnvan@gmail.com
• Requires: Python <4.0, >=3.9

Classifiers

• License
  • OSI Approved :: MIT License
• Programming Language
  • Python :: 3
  • Python :: 3.9
  • Python :: 3.10
  • Python :: 3.11
  • Python :: 3.12
  • Python :: 3.13

langchain-jina

This package contains the LangChain integration with Late Chunking

Installation

pip install -U langchain-jina

Environment Variable

Export your logins: export JINA_API_KEY="jina_*

Usage

1. Get Embedings

Here is an example usage of these classes:

fromlangchain_jinaimport LateChunkEmbeddings

text_embeddings = LateChunkEmbeddings(
 jina_api_key=os.environ.get("JINA_API_KEY"),
 model_name="jina-embeddings-v3"
)

text = [
 "Berlin is the capital and largest city of Germany, by both area and population.",
 "With 3.66 million inhabitants, it has the highest population within its city limits of any city in the European Union.",
 "The city is also one of the states of Germany, being the third smallest state in the country by area.",
]

# with late chunking
doc_result = text_embeddings.embed_documents(text, late_chunking=True)
print("With late_chunking")
for doc in doc_result:
 print(doc)

2. Build with Vectorstore

First of all, we need the context length entire input text limit with the model context length. So, we using tokenizer from transformers to check it.

fromtransformersimport AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("jinaai/jina-embeddings-v3")

Next, when the tokenizer is loaded, we can combine it with any text_splitter LangChain. The example below giving the instruction of handle the same method of authors.

fromlangchain_text_splittersimport RecursiveCharacterTextSplitter

text_splitter = RecursiveCharacterTextSplitter(
 chunk_size=200,
 chunk_overlap=0,
 length_function=len,
 is_separator_regex=False,
)

text_splitter.tokenizer = tokenizer 

We create vectorstore embeding, here we use LateChunkQdrant

fromqdrant_clientimport QdrantClient
fromlangchain_community.docstore.documentimport Document
fromlangchain_jinaimport LateChunkQdrant


client = QdrantClient()

vectorstore = LateChunkQdrant(
 client, 
 collection_name="demo",
 embeddings=text_embeddings, 
 text_splitter=text_splitter
)

# load documents
with open("./state_of_the_union.txt") as f:
 state_of_the_union = f.read()

documents = [
 Document(
 page_content=state_of_the_union, 
 metadata={"source": "state_of_the_union.txt"}
 ),
]

vectorstore = vectorstore.from_documents(
 documents=documents, 
 embedding=text_embeddings,
 text_splitter=text_splitter,
 path="test_db", 
 collection_name="demo"
)

Finally, we can combine with any purpose

query = "What did the president say about ketanji brown jackson?" 
results = vectorstore.similarity_search(query, k=3)

for res in results:
 print(f"* {res.page_content} [{res.metadata}]")

License

This project is licensed under the MIT License

Verified details

Maintainers

Unverified details

Project links

• Homepage
• Release Notes
• Repository
• Source Code

• Download URL: langchain_jina-0.0.1.dev0.tar.gz
• Upload date: Apr 6, 2025
• Size: 12.0 kB
• Tags: Source
• Uploaded using Trusted Publishing? No
• Uploaded via: poetry/2.1.2 CPython/3.11.0 Linux/6.5.0-27-generic

See more details on using hashes here.

• Download URL: langchain_jina-0.0.1.dev0-py3-none-any.whl
• Upload date: Apr 6, 2025
• Size: 12.2 kB
• Tags: Python 3
• Uploaded using Trusted Publishing? No
• Uploaded via: poetry/2.1.2 CPython/3.11.0 Linux/6.5.0-27-generic

See more details on using hashes here.