How to add custom LLM?

[nxtkofi]

Hello! I'm trying to add a custom LLM using ollama. My plan is to add an indexed llama3.2
I've managed to do it but I can't seem to format answers properly.

app = FastAPI()

llm = Ollama(model="llama3.2:3b", request_timeout=120.0)
Settings.llm = llm
embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-base-en-v1.5")
Settings.embed_model = embed_model

directory_zettelkasten = os.path.expanduser(
    "~/myfolder/")
index_file = os.path.join(directory_zettelkasten, "index.json")
if not os.path.exists(index_file):
    documents = SimpleDirectoryReader(directory_zettelkasten).load_data()
    index = VectorStoreIndex.from_documents(documents, embed_model=embed_model)
else:
    index = VectorStoreIndex.load_from_disk(index_file)


class Message(BaseModel):
    role: str
    content: str


class GPNvimQuery(BaseModel):
    max_tokens: int
    temperature: float
    top_p: float
    model: str
    messages: List[Message]
    stream: Optional[bool] = False


class QueryResponse(BaseModel):
    response: str


@app.post("/query", response_model=QueryResponse)
async def query(request: GPNvimQuery):
    try:
        user_message = next(
            (msg.content for msg in reversed(request.messages) if msg.role == "user"),
            None,
        )
        if not user_message:
            raise HTTPException(
                status_code=400, detail="No user message found in input.")

        chat_engine = index.as_chat_engine(chat_mode="condense_question")
        response = chat_engine.chat(user_message)
        return response
    except Exception as e:
        raise HTTPException(status_code=500, detail=str(e))

This backend successfully sends an answer to my neovim.

Let's say that I write a message in my neovim chat:

 ## ☕:
Recommend me a few books on procrastination and provide me with a brief summary to each one of them

gp.nvim throws an error with an answer:

Gp.nvim: ollama response is empty:                                                        
'{"response":"Here are three book recommendations on procrastination:\\n\\n1. \\"The Now Habit: A S
trategic Program for Overcoming Procrastination and Enjoying Guilt-Free Play\\" by Neil Fiore - Thi
s book provides a structured approach to overcoming procrastination, focusing on identifying the un
derlying causes of procrastination and developing strategies to overcome them.\\n\\n2. \\"Procrasti
nation: Why You Do It, What to Do About It Now\\" by Jane B. Burka and Lenora M. Yuen - This book e
xplores the psychology behind procrastination and offers practical advice for managing time more ef
fectively, setting realistic goals, and cultivating self-discipline.\\n\\n3. \\"The War of Art: Bre
ak Through the Blocks and Win Your Inner Creative Battles\\" by Steven Pressfield - While not exclu
sively focused on procrastination, this book delves into the resistance that often hinders creative
 work and productivity, offering insights into overcoming self-doubt and staying motivated."}\n'

The answer isn't then wrote into my chat file, an empty string is inserted instead.
Why is that happening?