Grafana dashboard for milvus with multi-tenancy strategies

[kish5430]

HI Team,

We have implemented multi-tenancy strategies in milvus, but we can currently view overall cluster resource utilization, we're unable to see the database level utilization. Could you please add a feature that allows us to see database-level stats or a Grafana dashboard? This would enable us to analyze resource usage at the database level.

Thanks

[xiaofan-luan]

feel free to open a github issue for that. This could be a potential feature to improve

[kish5430]

I mean there is no issue , we need this feature to see the resource utilisation at database level.

[xiaofan-luan]

Sure I understand. We just create track features in github issue as well~

[xiaofan-luan]

BTW, I'd like to learn about your use case and feature requirment if you'd like to share.
My email is [email protected] and we definitely want to hep

[kish5430]

We want to implement Multi-tenancy strategies within Milvus. Additionally, it's important for us to track resource utilization individually

[xiaofan-luan]

Sure, we should be working on collection level metrics and db level metrics.

@LoveEachDay

[kish5430]

Also can you plz add collection and database name for metric "milvus_proxy_req_count".

[xiaofan-luan]

I'm not sure how we can display collection level metrics on graphana. Anyone could help on this would be great

[xiaofan-luan]

We need collections,
collection insert, delete, query, search, upsert QPS/latency and entity numbers

[LoveEachDay]

@kish5430 Milvus 2.4 has added database level metrics. And we'll refine the grafana dashboard in the next coming week.

[yhmo]

The work is still in progress, stay tuned.

[kish5430]

The work is still in progress, stay tuned.

@yhmo Thank you for reply....Are you going to add dbname, collection name at cpu and memory usage metrics? if not, can you plz add

Thanks

[kish5430]

@yhmo @LoveEachDay Is there any ETA to release the new grafana dashboard?

[LoveEachDay]

@kish5430 We've refine the milvus dashboard, check it out from https://github.com/milvus-io/milvus/blob/master/deployments/monitor/grafana/milvus-dashboard.json

[kish5430]

@kish5430 We've refine the milvus dashboard, check it out from https://github.com/milvus-io/milvus/blob/master/deployments/monitor/grafana/milvus-dashboard.json

@LoveEachDay I reviewed the new dashboard and noticed a few issues:

1. Collections are not added at the query level.
2. It only shows collections from the default database, which could be problematic for a multi-tenancy cluster.

To address this, please add a database filter before the collections filter and incorporate collections into as many charts as possible.

Thank you.