Running from an extracted path

[courrier1]

Hey there
I was wondering if there is a way to make the Koboldcpp.exe extract its files and run from there instead of extracting thousand files to temp folder and deleting them on every use.

[LostRuins]

Yeah that is possible. When running you can copy the temp folder out to somewhere else in your PC, and run it from there (using Koboldcpp.py)

[courrier1]

There is no Koboldcpp.py file unfortunately, I also extracted the exe with pyinstxtractor and that file doesn't exist.

[LostRuins]

Alright no worries, I've added KoboldCpp 1.53 in a zip file. Let me know if it works for you.

https://github.com/LostRuins/koboldcpp/releases/download/v1.53/repacked-koboldcpp-1.53.zip

In future, you can download the source code .zip file from the releases page, and copy+replace in all the *.dll files you see from the pyinstaller folder into the unzipped source folder, that should work too.

[courrier1]

It didn't work, I think this is not a good method, I moved the dlls, and then it said it's missing customtinker and then darkdetect, I moved those folders and another error I got:

[LostRuins]

Did you setup all the dependencies on your machine? If you're not running from the exe you need a local python install with the requirements setup.

pip install -r requirements.txt

then

python koboldcpp.py

[courrier1]

It works now, thank you!

[sinni800]

The koboldcpp.py is still not extracted today. I wish we could do it without such hacks.

My primary idea on why is because koboldcpp will extract a literal gigabyte every time it is run, which is annoying when you run it many times with differrent models. Can't be good for ssd longevity either (though mine has literal petabytes of MTBW, so... okay I guess.)

The ROCM version has a zip file with the python script and the DLLs, could this one maybe not also have one?

EDIT: Wait, the .py is now in the directory on the current version. But using the extracted version with my system python 3.12 also doesnt work, as "ImportError: Module use of python38.dll conflicts with this version of Python." and if I remove python38.dll, I get "ImportError: DLL load failed while importing _ctypes:" instead, huh... Removing the PYDs and trying to get it by requirements.txt gives me trouble with "Getting requirements to build wheel did not run successfully.".

So I guess I need a conda at python 3.8 for this? getting more and more complicated...

EDIT: Extracting a ZIP copy of regular python 3.8 into the same folder just works (tm) lol

EDIT: I have a batch script now. first one to extract the pyinstaller (koboldcpp_cu12.exe) using pyinstxtractor-ng. This needs an extracted verion of python 3.8 (currently) embedded in the subdir python-3.8.0-embed-amd64

set subdir=koboldcpp_cu12
rmdir koboldcpp_cu12 /s /q
pyinstxtractor-ng.exe -d %subdir%.exe
ren %subdir%.exe_extracted %subdir%
robocopy python-3.8.0-embed-amd64\ %subdir% /s /e /r:0 /z

second one to start it

set subdir=koboldcpp_cu12

set TCL_LIBRARY=%cd%\%subdir%\tcl
set TK_LIBRARY=%cd%\%subdir%\tk
%subdir%\python %subdir%\koboldcpp.py %*

I had to also set the TK and TCL library, I'm not sure how this is done by pyinstaller, it probably sets this environment after all, but eh...

[LostRuins]

Not sure if you're aware, in modern KoboldCpp versions there is a self-extract option. You don't have to jump through hoops to get the files anymore.