Skip to content

Commit

Permalink
Fix incorrect entrypoint for broken llama stack run (#706)
Browse files Browse the repository at this point in the history
This fixes the issue when using `llama stack run` by correctly
specifying entrypoint:

```
LLAMA_STACK_DIR=. llama stack run /home/yutang/.llama/distributions/llamastack-vllm/vllm-run.yaml
Using config file: /home/yutang/.llama/distributions/llamastack-vllm/vllm-run.yaml
+ command -v selinuxenabled
+ selinuxenabled
+ DOCKER_OPTS=' --security-opt label=disable'
+ mounts=
+ '[' -n . ']'
++ readlink -f .
+ mounts=' -v /home/yutang/repos/llama-stack:/app/llama-stack-source'
+ '[' -n '' ']'
+ version_tag=latest
+ '[' -n '' ']'
+ '[' -n . ']'
+ version_tag=dev
+ podman run --security-opt label=disable -it -p 5000:5000 -v /home/yutang/.llama/distributions/llamastack-vllm/vllm-run.yaml:/app/config.yaml -v /home/yutang/repos/llama-stack:/app/llama-stack-source localhost/distribution-vllm:dev python -m llama_stack.distribution.server.server --yaml-config /app/config.yaml --port 5000
usage: server.py
       [-h]
       [--yaml-config YAML_CONFIG]
       [--template TEMPLATE]
       [--port PORT]
       [--disable-ipv6]
       [--env ENV]
server.py: error: unrecognized arguments: python -m llama_stack.distribution.server.server
++ error_handler 88
++ echo 'Error occurred in script at line: 88'
Error occurred in script at line: 88
++ exit 1

```

---------

Signed-off-by: Yuan Tang <[email protected]>
  • Loading branch information
terrytangyuan authored Jan 3, 2025
1 parent 21357a6 commit 96d8375
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 5 deletions.
7 changes: 6 additions & 1 deletion llama_stack/distribution/server/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -239,7 +239,12 @@ def main():
"--template",
help="One of the template names in llama_stack/templates (e.g., tgi, fireworks, remote-vllm, etc.)",
)
parser.add_argument("--port", type=int, default=5000, help="Port to listen on")
parser.add_argument(
"--port",
type=int,
default=int(os.getenv("LLAMASTACK_PORT", 5000)),
help="Port to listen on",
)
parser.add_argument(
"--disable-ipv6", action="store_true", help="Whether to disable IPv6 support"
)
Expand Down
7 changes: 3 additions & 4 deletions llama_stack/distribution/start_container.sh
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,6 @@ $DOCKER_BINARY run $DOCKER_OPTS -it \
$env_vars \
-v "$yaml_config:/app/config.yaml" \
$mounts \
$docker_image:$version_tag \
python -m llama_stack.distribution.server.server \
--yaml-config /app/config.yaml \
--port "$port"
--env LLAMASTACK_PORT=$port \
--entrypoint='["python", "-m", "llama_stack.distribution.server.server", "--yaml-config", "/app/config.yaml"]' \
$docker_image:$version_tag

0 comments on commit 96d8375

Please sign in to comment.