Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

在dify 中使用 xinference flux-dev step超过20或并发超过1时,生成失败 #2714

Open
1 of 3 tasks
geekidentity opened this issue Dec 28, 2024 · 0 comments
Open
1 of 3 tasks
Labels
Milestone

Comments

@geekidentity
Copy link

System Info / 系統信息

CUDA Version: 12.4
docker 部署

Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece?

  • docker / docker
  • pip install / 通过 pip install 安装
  • installation from source / 从源码安装

Version info / 版本信息

dify version: 0.14.2
xinference version v1.1.1

The command used to start Xinference / 用以启动 xinference 的命令

flux 如下
image

Reproduction / 复现过程

dify中工具配置如下
image
只有一个请求,生成20 step 可以正常生成,20 step多个请求或单请求28 step会生成失败,看xinference没有错误日志。
xinference日志:
image

dify 返回信息
image

Expected behavior / 期待表现

step 28步应该也可以生成,flux模型支持连续批处理,会不会和这个有关系。

@XprobeBot XprobeBot added the gpu label Dec 28, 2024
@XprobeBot XprobeBot added this to the v1.x milestone Dec 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants