You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Also, the documentation states that a JSON is needed with a mapping from token_ids to a number between -100 and 100. But this is misaligned with the code's typing, that asks for a vector of floats.
Motivation
Logit_bias is a very important parameter, the documentation in InferenceClient from huggingface_hub states that it can be used, but it does not work.
Also see: huggingface/huggingface_hub#2720
Your contribution
I can definitely help building a PR but I will need details on what type of solution is expected.
The text was updated successfully, but these errors were encountered:
Feature request
Currently, logit_bias is labeled as unused https://github.com/huggingface/text-generation-inference/blob/main/router/src/lib.rs
Also, the documentation states that a JSON is needed with a mapping from token_ids to a number between -100 and 100. But this is misaligned with the code's typing, that asks for a vector of floats.
Motivation
Logit_bias is a very important parameter, the documentation in InferenceClient from huggingface_hub states that it can be used, but it does not work.
Also see:
huggingface/huggingface_hub#2720
Your contribution
I can definitely help building a PR but I will need details on what type of solution is expected.
The text was updated successfully, but these errors were encountered: