Pre-configured AI server // How?

[SurvivaLlama]

Bug description

On your demo site, you have a pre-configured AI server.

How did you set this up?

Steps to reproduce

Here are my current docker settings:

Expected behavior

When I load my site https://search.robocanvas.io it should auto load with my pre-configured ollama server AND the user should not have the option to change the configuration.

Device info

No response

Additional context

No response

[felladrin]

Ah, I see! To use the environment variables from the OS, you need to pass the environment variables to the container manually.

So if you're running docker run -p 7860:7860 ghcr.io/felladrin/minisearch:main, pass all the env vars to the command, like this:

docker run -p 7860:7860 -e INTERNAL_OPENAI_COMPATIBLE_API_BASE_URL=$INTERNAL_OPENAI_COMPATIBLE_API_BASE_URL -e INTERNAL_OPENAI_COMPATIBLE_API_KEY =$INTERNAL_OPENAI_COMPATIBLE_API_KEY -e INTERNAL_OPENAI_COMPATIBLE_API_MODEL=$INTERNAL_OPENAI_COMPATIBLE_API_MODEL -e INTERNAL_OPENAI_COMPATIBLE_API_NAME=$INTERNAL_OPENAI_COMPATIBLE_API_NAME ghcr.io/felladrin/minisearch:main

But if you are using docker compose -f docker-compose.production.yml up --build, it means you have access to the machine and have cloned the repository there. In this case, it's preferable to create a .env and set the variables in it. Is it possible? (There's an alternative, but I'll only comment on it if you say it's not possible in your current environment)

Let me know if it helps.

[SurvivaLlama]

I think that is what I have been asking, in regard to mounting the volumes. that way I can create and edit the .env.

I have full control of the machine.

[felladrin]

Ah ok, then I need to clarify it in the Readme.

Please try this:

git clone https://github.com/felladrin/MiniSearch.git
cd MiniSearch
cp .env.example .env

Edit the .env, then run:

docker compose -f docker-compose.production.yml up --build

This will build and run the docker container, already taking all the files in MiniSearch into account, including the .env.

And to update it to the latest version, stop the container, and run:

git pull
docker compose -f docker-compose.production.yml up --build

P.S. I know you know the commands/sequence already, but I'm just writing in detail so I can reference this message if other issues are open about this later.

[SurvivaLlama]

I don't use those commands for my containers. I'm stuck with a GUI for building and maintaining containers (unRaid OS). Are there any flags to turn off the browser or openai interface?

Also, point of confusion, the AI Processing Location has default options of
In the browser (Private)
Remote Server (API)

What option leads to what DEFAULT_INFERENCE_TYPE ? Consider naming them the same thing and graying out options that are not available.

[felladrin]

DEFAULT_INFERENCE_TYPE="internal", as you set, is correct: means whenever a new user visits the app for the first time, this will be the option selected by default. (You can confirm it by visiting the app in a private browser window). (On the first visit, the settings are saved on the localStorage, so it's really only used on the user's first visit).

Ah yes, about the confusion, it's because I've recently renamed those options, but I forgot to update the .env.example to have those new wording.

But here's how they map:

• "browser" -> In the browser (Private)
• "openai" -> Remote Server (API)
• "internal" -> INTERNAL_OPENAI_COMPATIBLE_API_NAME

Are there any flags to turn off the browser or openai interface?

No, as this would go against the main purpose of the app, which was to run models directly from the browser (the OpenAI-Compatible API was only added recently). Could you tell me more about why you think there should be this option?

---

But going back to the topic, if unRaid OS runs docker build . (instead of running the commands mentioned before) and there's a .env there, it will also work.

Please let me know if any of the solutions worked.

[SurvivaLlama]

---

No, as this would go against the main purpose of the app, which was to run models directly from the browser (the OpenAI-Compatible API was only added recently). Could you tell me more about why you think there should be this option?

---

I'm not using the app with browser based LLMs. Ill use it with my ollama server for private search. I view it is a very lite weight webui for llms.

[felladrin]

I'm not using the app with browser based LLMs. Ill use it with my ollama server for private search. I view it is a very lite weight webui for llms.

Ah, so it's to avoid loading the libraries that run the LLMs in the browser, making the app lighter, right?

No need to worry, those libraries are only loaded if you have the "In the browser (Private)" AI Processing Location selected. It was all modularized to use only what users needed.

---

By the way, this discussion gave me great insights, so I thought about simplifying the server configuration, which is becoming complicated with all those environment variables:

• Replace `.env` with a single environment variable holding all the configurations, easily configurable via an HTML Form #694

[felladrin]

Back to the original question, were you able to make it display the custom option to use the internal API?

[SurvivaLlama]

Back to the original question, were you able to make it display the custom option to use the internal API?

still no. I did figure out how to copy and paste the command into the command line to start the container. so thats a start.

[SurvivaLlama]

Still not working. I'm stuck.

[felladrin]

I’m out of ideas on how to address this issue.

In the meantime, as you search for a solution to get the Internal API working, can you use the "Remote Server (API)" option to connect to Ollama directly from the browser?

[SurvivaLlama]

I’m out of ideas on how to address this issue.

In the meantime, as you search for a solution to get the Internal API working, can you use the "Remote Server (API)" option to connect to Ollama directly from the browser?

I can, and thats what I currently have set up. Works great.

[felladrin]

That's good! Then, I'm converting this issue into a discussion to keep it open in case more information related to this arises.

[SurvivaLlama]

Sounds good. I still think this problem has to do with read/write access on my system. Any chance I can get configuration settings so I can mount a volume?