How does the "Target length (tokens)" work?

[GaelicThunder]

Hi everybody, im trying to understand how does it work between endpoints etc.. but i can't find anything about it in the wiki. Can someone help me?

[LostRuins]

Max Length is Amount to Generate = Max number of tokens that are allowed to be generated in 1 request. I assume that's what you're looking for.

Max context length is the maximum total length of the story the AI is allowed to process.

[GaelicThunder]

Hi LostRuins,
Thank you for your response, but I think there might be some confusion, and that's totally on me. What I'm referring to is actually different from the "Max Length" and "Max Context Length" parameters, but is on the SillyTavern frontend ( i wrote in the wrong git... im so ashame of myself). I will still explain it, since maybe you know about it, but feel free to close it asap since it's in the wrong place.
I'm specifically asking about the "Target Length (tokens)" setting that is available in the SillyTavern frontend (in the settings, see the pic). This setting seems to influence the length of the generated responses in a more subtle way compared to the hard limit imposed by "Max Length".

From my observations, when I change the "Target Length (tokens)" value in SillyTavern, the generated responses tend to be shorter or longer accordingly, but without abruptly cutting off the text like "Max Length" does. It appears that SillyTavern is using this setting to guide the generation process towards producing responses of the desired length, while still allowing for some flexibility and natural completion of sentences.
I couldn't find any information in the KoboldCPP documentation or API about a parameter that would provide similar functionality to SillyTavern's "Target Length". That's why I was wondering if you could shed some light on how this setting is being used under the hood and if there's a way to achieve the same effect directly through the KoboldCPP API.
To summarize, I'm looking for a way to softly influence the length of the generated responses, aiming for a target number of tokens, but without strictly enforcing a hard cutoff. This is different from setting a maximum limit on the generation length or the context size.
I appreciate any insights you can provide on this matter. Thank you for your time and assistance!

[LostRuins]

Then I think such a feature does not exist in stock KoboldCpp. I am not sure how the SillyTavern implementation works, but in Kobold a generation will continue until

1. Max length reaches
2. EOS token hit
3. Stopping sequence hit

Normally EOS will be hit long before the max length is hit. Combined with "Trim Sentences" and a decently long max length, it should get you close to what you'd expect.

[GaelicThunder]

I see, thanks for the clarification!

[GaelicThunder]

Just for LYK, the guys on Sillytavern answered me, and that Target Lenghts is only working if you have the "continue to generate" button enable.
Here is the link