Releases: huggingface/inference-benchmarker
Releases · huggingface/inference-benchmarker
v0.2.0
New features
- Results dashboard
- k8s Helm chart
- Slurm example
- Results table shown in console at end of benchmark
Bug fixes
- Fix TTFT average computation
- Handle backend connection drops
- Fix token count in responses
Full Changelog: v0.1.0...v0.2.0
v0.1.0
First release
Features
- Benchmark modes:
sweep
, rate
, throughput
- Custom prompt/decode options
- JSON report output
- Custom prompts via HF dataset