No data on mimir/reads dashboard

[rosh0450]

I have enabled meta monitoring for my grafana mimir:

values.yaml

metaMonitoring:
  dashboards:
    enabled: false

  serviceMonitor:
    enabled: true

  grafanaAgent:
    enabled: true

    metrics:
      enabled: true
      remote:
        url: "http://mimir-nginx.mimir.svc:80/api/v1/push"

      scrapeK8s:
        enabled: true

and i am able to see components specific metrics like cortex_ingester_active_series, cortex_compactor_block_cleanup_last_successful_run_timestamp_seconds and others.

But when i imported the mimir/reads dashboard, it is completely empty.

Same is the case with other dashboards like mimir/write and others.

But for some odd reason when i remove the rate operation from the queries, i am getting correct data.
Example with Label names queries / sec panel in mimir/reads dashboard:
Before removing:

After removing rate operation:

What can be the reason ?

Thanks.

[dimitarvdimitrov]

i suspect the scrape interval of the datasource and the actual scrape interval of the metamonitoring agent aren't aligned. The metamonitoring agent scrapes at 1m, so if the scrape interval of the datasource is anything lower, then rate functions wouldn't work properly. I recommend checking the configuration of the Grafana datasource.

[rosh0450]

So i changed the scrape duration for the data source(mimir) to 60s. Now I do see some data in the mimir/read dashboard. But the data doesn't seem correct.

For instance here requests/sec for query frontend is very low but it can't be correct because i have few dashboards deployed which are continuously reading from mimir as their data source.

[sojjan1337]

I just enabled the prometheusRule: under metaMonitoring in the mimir values-file.

prometheusRule:
  enabled: true
  mimirRules: true
  mimirAlerts: true

Such a simple thing, yet i didn't find an example how to fix it in the docs.