Securing Specific Routes with Token Authentication in TGI on Kubernetes

[ahadrez]

I'm currently deploying a TGI instance on a Kubernetes cluster, with the service exposed through an Ingress controller. My primary goal is to enhance security by implementing token authentication for two specific routes: /generate and /generate_stream.

Does TGI inherently support token-based protection for individual routes, or will I need to integrate an external API gateway to achieve this level of security? I'm just wondering if there's a built-in mechanism within TGI for this purpose or if a custom implementation is the recommended approach.

Thank you,

[OlivierDehaene]

There is no built-in mechanism within TGI for auth. You need to add a proxy on top of it.