You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
the format of the models/ directory is the same as described in the docs of reranker deployment.
My current working directory is /apps/data/utils/rerank where under models the openvino compatible model resides.
interestingly, the below command worked perfectly while running from windows.
StackTrace:
[rerank]$ podman run --rm -p 8000:8000 -p 9000:9000 -v $(pwd)/models:/workspace:ro openvino/model_server:latest --port 9000 --rest_port 8000 --config_path /workspace/config.json
Resolved "openvino/model_server" as an alias (/home/user/.cache/containers/short-name-aliases.conf)
Trying to pull docker.io/openvino/model_server:latest...
Getting image source signatures
Copying blob 9a2aafbc5fbe done
Copying blob 79df4d8ba148 done
Copying blob b3a49b36c013 done
Copying blob 7478e0ac0f23 done
Copying blob b52d30ada7d4 done
Copying blob 4f4fb700ef54 skipped: already exists
Copying blob 7fe997937aec done
Copying blob f76e91668d30 done
Copying blob eb4b025d5eeb done
Copying blob 0645e45015e1 done
Copying blob 61134f905a8a done
Copying blob 6c9f59c0a376 done
Copying config 19cf89a012 done
Writing manifest to image destination
[2024-12-23 23:33:12.371][1][serving][info][server.cpp:84] OpenVINO Model Server 2024.5.816f620b6
[2024-12-23 23:33:12.371][1][serving][info][server.cpp:85] OpenVINO backend 2024.5.0.17288.7975fa5da0c
[2024-12-23 23:33:12.371][1][serving][info][pythoninterpretermodule.cpp:35] PythonInterpreterModule starting
[2024-12-23 23:33:12.450][1][serving][info][pythoninterpretermodule.cpp:46] PythonInterpreterModule started
[2024-12-23 23:33:12.451][1][serving][info][modelmanager.cpp:128] Loading tokenizer CPU extension from libopenvino_tokenizers.so
[2024-12-23 23:33:12.470][1][modelmanager][info][modelmanager.cpp:143] Available devices for Open VINO: CPU
[2024-12-23 23:33:12.470][1][serving][info][grpcservermodule.cpp:163] GRPCServerModule starting
[2024-12-23 23:33:12.471][1][serving][info][grpcservermodule.cpp:232] GRPCServerModule started
[2024-12-23 23:33:12.471][1][serving][info][grpcservermodule.cpp:233] Started gRPC server on port 9000
[2024-12-23 23:33:12.471][1][serving][info][httpservermodule.cpp:33] HTTPServerModule starting
[2024-12-23 23:33:12.471][1][serving][info][httpservermodule.cpp:37] Will start 32 REST workers
[2024-12-23 23:33:12.474][1][serving][info][http_server.cpp:276] REST server listening on port 8000 with 32 threads
[2024-12-23 23:33:12.474][1][serving][info][httpservermodule.cpp:47] HTTPServerModule started
[2024-12-23 23:33:12.474][1][serving][info][httpservermodule.cpp:48] Started REST server at 0.0.0.0:8000
[2024-12-23 23:33:12.474][1][serving][info][servablemanagermodule.cpp:51] ServableManagerModule starting
[evhttp_server.cc : 253] NET_LOG: Entering the event loop ...
[2024-12-23 23:33:12.475][1][serving][error][modelmanager.cpp:875] Failed to debug-read fileconfig
[2024-12-23 23:33:12.475][1][modelmanager][error][modelmanager.cpp:907] Configuration file is invalid /workspace/config.json
[2024-12-23 23:33:12.485][1][serving][error][modelmanager.cpp:875] Failed to debug-read fileconfig
[2024-12-23 23:33:12.485][1][modelmanager][error][modelmanager.cpp:907] Configuration file is invalid /workspace/config.json
[2024-12-23 23:33:12.495][1][serving][error][modelmanager.cpp:875] Failed to debug-read fileconfig
[2024-12-23 23:33:12.495][1][modelmanager][error][modelmanager.cpp:907] Configuration file is invalid /workspace/config.json
[2024-12-23 23:33:12.505][1][modelmanager][error][modelmanager.cpp:175] Couldn't start model manager
[2024-12-23 23:33:12.505][1][serving][error][servablemanagermodule.cpp:58] ovms::ModelManager::Start() Error: Configuration file not found or cannot open
[2024-12-23 23:33:12.505][1][serving][info][grpcservermodule.cpp:241] GRPCServerModule shutting down
[2024-12-23 23:33:12.506][1][serving][info][grpcservermodule.cpp:246] Shutdown gRPC server
[2024-12-23 23:33:12.506][1][serving][info][grpcservermodule.cpp:250] GRPCServerModule shutdown
[2024-12-23 23:33:12.506][1][serving][info][httpservermodule.cpp:54] HTTPServerModule shutting down
[evhttp_server.cc : 320] NET_LOG: event_base_loopexit() exits with value 0
[evhttp_server.cc : 255] NET_LOG: event_base_loop() exits with value 0
[2024-12-23 23:33:12.512][1][serving][info][httpservermodule.cpp:59] Shutdown HTTP server
[2024-12-23 23:33:12.512][1][serving][info][servablemanagermodule.cpp:65] ServableManagerModule shutting down
[2024-12-23 23:33:12.513][1][serving][info][servablemanagermodule.cpp:71] ServableManagerModule shutdown
[2024-12-23 23:33:12.513][1][serving][info][pythoninterpretermodule.cpp:57] PythonInterpreterModule shutting down
[2024-12-23 23:33:12.513][1][serving][info][pythoninterpretermodule.cpp:62] PythonInterpreterModule shutdown
Python version
3.10.12 (main, Sep 11 2024, 15:47:36) [GCC 11.4.0]
The text was updated successfully, but these errors were encountered:
the format of the models/ directory is the same as described in the docs of reranker deployment.
My current working directory is /apps/data/utils/rerank where under models the openvino compatible model resides.
interestingly, the below command worked perfectly while running from windows.
StackTrace:
[rerank]$ podman run --rm -p 8000:8000 -p 9000:9000 -v $(pwd)/models:/workspace:ro openvino/model_server:latest --port 9000 --rest_port 8000 --config_path /workspace/config.json
Resolved "openvino/model_server" as an alias (/home/user/.cache/containers/short-name-aliases.conf)
Trying to pull docker.io/openvino/model_server:latest...
Getting image source signatures
Copying blob 9a2aafbc5fbe done
Copying blob 79df4d8ba148 done
Copying blob b3a49b36c013 done
Copying blob 7478e0ac0f23 done
Copying blob b52d30ada7d4 done
Copying blob 4f4fb700ef54 skipped: already exists
Copying blob 7fe997937aec done
Copying blob f76e91668d30 done
Copying blob eb4b025d5eeb done
Copying blob 0645e45015e1 done
Copying blob 61134f905a8a done
Copying blob 6c9f59c0a376 done
Copying config 19cf89a012 done
Writing manifest to image destination
[2024-12-23 23:33:12.371][1][serving][info][server.cpp:84] OpenVINO Model Server 2024.5.816f620b6
[2024-12-23 23:33:12.371][1][serving][info][server.cpp:85] OpenVINO backend 2024.5.0.17288.7975fa5da0c
[2024-12-23 23:33:12.371][1][serving][info][pythoninterpretermodule.cpp:35] PythonInterpreterModule starting
[2024-12-23 23:33:12.450][1][serving][info][pythoninterpretermodule.cpp:46] PythonInterpreterModule started
[2024-12-23 23:33:12.451][1][serving][info][modelmanager.cpp:128] Loading tokenizer CPU extension from libopenvino_tokenizers.so
[2024-12-23 23:33:12.470][1][modelmanager][info][modelmanager.cpp:143] Available devices for Open VINO: CPU
[2024-12-23 23:33:12.470][1][serving][info][grpcservermodule.cpp:163] GRPCServerModule starting
[2024-12-23 23:33:12.471][1][serving][info][grpcservermodule.cpp:232] GRPCServerModule started
[2024-12-23 23:33:12.471][1][serving][info][grpcservermodule.cpp:233] Started gRPC server on port 9000
[2024-12-23 23:33:12.471][1][serving][info][httpservermodule.cpp:33] HTTPServerModule starting
[2024-12-23 23:33:12.471][1][serving][info][httpservermodule.cpp:37] Will start 32 REST workers
[2024-12-23 23:33:12.474][1][serving][info][http_server.cpp:276] REST server listening on port 8000 with 32 threads
[2024-12-23 23:33:12.474][1][serving][info][httpservermodule.cpp:47] HTTPServerModule started
[2024-12-23 23:33:12.474][1][serving][info][httpservermodule.cpp:48] Started REST server at 0.0.0.0:8000
[2024-12-23 23:33:12.474][1][serving][info][servablemanagermodule.cpp:51] ServableManagerModule starting
[evhttp_server.cc : 253] NET_LOG: Entering the event loop ...
[2024-12-23 23:33:12.475][1][serving][error][modelmanager.cpp:875] Failed to debug-read fileconfig
[2024-12-23 23:33:12.475][1][modelmanager][error][modelmanager.cpp:907] Configuration file is invalid /workspace/config.json
[2024-12-23 23:33:12.485][1][serving][error][modelmanager.cpp:875] Failed to debug-read fileconfig
[2024-12-23 23:33:12.485][1][modelmanager][error][modelmanager.cpp:907] Configuration file is invalid /workspace/config.json
[2024-12-23 23:33:12.495][1][serving][error][modelmanager.cpp:875] Failed to debug-read fileconfig
[2024-12-23 23:33:12.495][1][modelmanager][error][modelmanager.cpp:907] Configuration file is invalid /workspace/config.json
[2024-12-23 23:33:12.505][1][modelmanager][error][modelmanager.cpp:175] Couldn't start model manager
[2024-12-23 23:33:12.505][1][serving][error][servablemanagermodule.cpp:58] ovms::ModelManager::Start() Error: Configuration file not found or cannot open
[2024-12-23 23:33:12.505][1][serving][info][grpcservermodule.cpp:241] GRPCServerModule shutting down
[2024-12-23 23:33:12.506][1][serving][info][grpcservermodule.cpp:246] Shutdown gRPC server
[2024-12-23 23:33:12.506][1][serving][info][grpcservermodule.cpp:250] GRPCServerModule shutdown
[2024-12-23 23:33:12.506][1][serving][info][httpservermodule.cpp:54] HTTPServerModule shutting down
[evhttp_server.cc : 320] NET_LOG: event_base_loopexit() exits with value 0
[evhttp_server.cc : 255] NET_LOG: event_base_loop() exits with value 0
[2024-12-23 23:33:12.512][1][serving][info][httpservermodule.cpp:59] Shutdown HTTP server
[2024-12-23 23:33:12.512][1][serving][info][servablemanagermodule.cpp:65] ServableManagerModule shutting down
[2024-12-23 23:33:12.513][1][serving][info][servablemanagermodule.cpp:71] ServableManagerModule shutdown
[2024-12-23 23:33:12.513][1][serving][info][pythoninterpretermodule.cpp:57] PythonInterpreterModule shutting down
[2024-12-23 23:33:12.513][1][serving][info][pythoninterpretermodule.cpp:62] PythonInterpreterModule shutdown
Python version
3.10.12 (main, Sep 11 2024, 15:47:36) [GCC 11.4.0]
The text was updated successfully, but these errors were encountered: