🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of Optimum's hardware optimizations & quantization schemes.
-
Updated
Dec 17, 2024 - Python
#
text-generation-inference
Here are 9 public repositories matching this topic...
☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!
-
Updated
Dec 26, 2024 - Go
Amazon SageMaker Llama 2 Inference via Response Streaming
sagemaker sagemaker-endpoint response-streaming large-language-models text-generation-inference llama2 large-model-inference
-
Updated
Jun 28, 2024 - Jupyter Notebook
This repo introduces MagicData-CLAM, a Chinese SFT dataset, and provides to the community two relevant models that we finetuned. Contact [email protected] for more information.
-
Updated
Aug 3, 2023 - Python
GUI version of text-generation-inference
-
Updated
Sep 1, 2023 - Python
RisuAI backend with python only. TextGen works, need more memory related updates
-
Updated
Jun 12, 2024 - Python
Deploy the Defog sqlcoder2 llm on Modal (https://modal.com) using Hugging Face Text Generation Inference (TGI)
sql code-generation text-to-sql huggingface llm text-generation-inference sqlcoder modal-labs defog-ai
-
Updated
Dec 12, 2023 - Python
Serve the AI Singapore SEA-LION model ⚛ with TGI
-
Updated
Sep 1, 2024 - Shell
Text Generation Interference example in Windows (docker, WSL is needed)
-
Updated
Jun 26, 2023 - Python
Improve this page
Add a description, image, and links to the text-generation-inference topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the text-generation-inference topic, visit your repo's landing page and select "manage topics."