Implement ChromaDB with HttpClient as a MicroService then save and persist embeddings

[manojakondi]

Hi, I found your example very easy to setup and get a fair understanding on how RAG with langchain with Chroma.

Although, I'd be more interested to host chromadb as a standalone microservice and access it in the application to store embeddings and query later. Can you please add that part as well?

I've tried below piece of snippet. But for some reason, I'm not able to get the chunks saved to vector db.

# create chroma db or load db from disk
from langchain.embeddings import OllamaEmbeddings
from langchain.vectorstores import Chroma
import chromadb
from chromadb.config import Settings

client_settings = Settings(
    chroma_api_impl="chromadb.api.fastapi.FastAPI",
    chroma_client_auth_provider="chromadb.auth.token.TokenAuthClientProvider",
    chroma_client_auth_credentials="xxxxxx",
    chroma_client_auth_token_transport_header="X_CHROMA_TOKEN",
    allow_reset=True,
    anonymized_telemetry=False
)

client = chromadb.HttpClient(
    host="localhost",
    port=8000,
    settings=client_settings,
)
collection = client.get_or_create_collection(name="documents")

emb_fn = OllamaEmbeddings(base_url=OLLAMA_URL, model=OLLAMA_MODEL)

def get_chroma(chroma_client):
    chroma_db = Chroma(
        collection_name="documents",
        embedding_function=emb_fn,
        client=chroma_client,
    )
    return chroma_db

chroma_db_client = get_chroma(client)

if init_db:
    chroma_db_client.from_documents(all_document_chunks, emb_fn)
    print(collection.count())
    print(collection.peek())
else:
    chroma_db_client = Chroma(embedding_function=emb_fn)

Output:

0
{'ids': [], 'embeddings': [], 'metadatas': [], 'documents': [], 'data': None, 'uris': None}

The chromadb server is running in a docker container and shows no errors. Also the variable all_document_chunks has several chunks of a local document that I have.

Appreciate your help!