Skip to content

Releases: huggingface/inference-benchmarker

v0.2.0

28 Nov 14:45
754aeb8
Compare
Choose a tag to compare

New features

  • Results dashboard
  • k8s Helm chart
  • Slurm example
  • Results table shown in console at end of benchmark

Bug fixes

  • Fix TTFT average computation
  • Handle backend connection drops
  • Fix token count in responses

Full Changelog: v0.1.0...v0.2.0

v0.1.0

09 Oct 11:25
b7c3347
Compare
Choose a tag to compare

First release

Features

  • Benchmark modes: sweep, rate, throughput
  • Custom prompt/decode options
  • JSON report output
  • Custom prompts via HF dataset