Standardize Chat Prompt Templates to Use Jinja Format

[calycekr]

Describe your feature request

Currently, the chatPromptTemplate for each model that can be set in env uses Handlebars format. However, the chat_prompt in the actual model's tokenizer_config.json uses Jinja format. This inconsistency is causing significant inconvenience. Since Jinja is widely used and preferred, it would be beneficial to standardize on Jinja format for both chatPromptTemplate and chat_prompt. This will improve consistency and ease of use for developers.

Screenshots (if relevant)

Implementation idea

To implement this change, the following steps can be taken:

1. Update Codebase: Update the codebase to handle Jinja templates for chatPromptTemplate.

2. Documentation: Update the documentation to reflect this change and provide examples of how to use Jinja templates.

3. Testing: Thoroughly test the changes to ensure compatibility and that all existing templates work correctly with the new format.

[nsarrazin]

This is a good idea! It would be a breaking change though so we could keep handlebars as a fallback if the jinja evaluator fails, wdyt?

[calycekr]

@nsarrazin Sure, I agree with your suggestion. Keeping Handlebars as a fallback in case the Jinja evaluator fails is a good approach to ensure backward compatibility and smooth transition.

[julien-c]

yep, seems like the field has standardized on Jinja in the past year 😁 cc @Rocketknight1 😮

[Rocketknight1]

Please don't fire me, I know you liked Handlebars! 😰

(but yes transitioning to Jinja is definitely the right decision)

[julien-c]

cements that we're a Python-first company (and community) i guess 😁

[varad0309]

hey guys, did you add this feature?

[nsarrazin]

@varad0309 the recommended way for now would be to set "tokenizer": to point to the model's repo on the hub so it can pick up the chat template (using Jinja) there.

[varad0309]

@nsarrazin , thanks for the fast reply. Is there a way to do the same if we have a local model?
also, just tried "tokenizer" : Qwen/Qwen2.5-Coder-14B-Instruct. It is not working.