Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Exposes TensorRT-LLM finish reason to the server #2818

Draft
wants to merge 30 commits into
base: main
Choose a base branch
from
Draft

Exposes TensorRT-LLM finish reason to the server #2818

wants to merge 30 commits into from

Conversation

mfuntowicz
Copy link
Member

No description provided.

mfuntowicz and others added 30 commits December 3, 2024 09:43
@mfuntowicz
misc(cmake) update dependencies
0f17415
@mfuntowicz
feat(hardware) enable new hardware.hpp and unittests
7a81040
@mfuntowicz
test(ctest) enable address sanitizer
1830fe8
@mfuntowicz
feat(backend): initial rewrite of the backend for simplicity
3a2698f
@mfuntowicz
feat(backend): remove all the logs from hardware.hpp
6d35657
@mfuntowicz
feat(backend): added some logging
9bb6309
@mfuntowicz
feat(backend): enable compiler warning if support for RVO not applying
87272ff
@mfuntowicz
feat(backend): missing return statement
702dc9c
@mfuntowicz
feat(backend): introduce backend_workspace_t to store precomputed inf…
25c6bbe 
…ormation from the engine folder
@mfuntowicz
feat(backend): delete previous backend impl
df99164
@mfuntowicz
feat(backend): more impl
fd7e2b5
@mfuntowicz
feat(backend): use latest trtllm main version to have g++ >= 13 compa…
71e700a 
…tibility
@mfuntowicz
feat(backend): allow overriding which Python to use
879e1a4
@mfuntowicz
feat(backend): fix backend_exception_t -> backend_error_t naming
a7bad25
@mfuntowicz
feat(backend): impl missing generation_step_t as return value of pull…
2f8634e 
…_tokens
@mfuntowicz
feat(backend): make backend_workspace_t::engines_folder constexpr
874bc28
@mfuntowicz
feat(backend): fix main.rs retrieving the tokenizer
16ba2f5
@mfuntowicz
feat(backend): add guard to multiple header definitions
c94b9de
@mfuntowicz
test(backend): add more unittest
ad3ed0d
@mfuntowicz
feat(backend): remove constexpr from par
881527a
@mfuntowicz
feat(backend): remove constexpig
6253064
@mfuntowicz
test(backend): more test coverage
cc6bc33
@mfuntowicz
chore(trtllm): update dependency towards 0.15.0
b6dbf60
@mfuntowicz
effectively cancel the request on the executor
460f290
@mfuntowicz
feat(backend) fix moving backend when pulling
300f6c6
@mfuntowicz
feat(backend): make sure we can easily cancel request on the executor
b3cd5ea
@mfuntowicz
feat(backend): fix missing "0" field access
049f4ac
@mfuntowicz
misc(backend): fix reborrowing Pin<&mut T> as described in the doc ht…
f0cd474 
…tps://doc.rust-lang.org/stable/std/pin/struct.Pin.html#method.as_mut
@mfuntowicz
feat(trtllm): expose finish reason to Rust
60059b6
@mfuntowicz
feat(trtllm): fix logits retrieval
b653605
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@mfuntowicz