Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature addition: Python backend / grcp backend for ClassifierEngine #449

Open
michaelfeil opened this issue Dec 13, 2024 · 0 comments · May be fixed by #450
Open

Feature addition: Python backend / grcp backend for ClassifierEngine #449

michaelfeil opened this issue Dec 13, 2024 · 0 comments · May be fixed by #450

Comments

@michaelfeil
Copy link
Contributor

Feature request

I'd be interested to implement a Python backend for the classifier. I would be up to work on this.

Motivation

The Python backend allows for prototyping models with trust_remove_code=True faster. Would be great to see the router gauge possible performance impact when performing heavy python operations thanks to low grcp overhead.

Your contribution

I would like to add / modify the following files:

backend/Proto: Add Predict to EmbeddingService in https://github.com/huggingface/text-embeddings-inference/tree/main/backends/proto
backend/grcp-client: Add a pub asyn predict() in https://github.com/huggingface/text-embeddings-inference/tree/main/backends/grpc-client/src
backend/python/libs.rs: Add the logic for fn predict() in https://github.com/huggingface/text-embeddings-inference/blob/main/backends/python/src/lib.rs
Python: https://github.com/huggingface/text-embeddings-inference/blob/main/backends/python/server/text_embeddings_server/server.py

please let me know if you have a preferred choice.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant