fix: [2.4] Fix index meta mutex contention

[bigsheeper]

issue: #37630

Reduce the frequency of updateIndexTasksMetrics to avoid holding the mutex for long periods.

pr: #38775

[sre-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: bigsheeper
To complete the pull request process, please assign liliu-z after the PR has been reviewed.
You can assign the PR to them by writing /assign @liliu-z in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• internal/datacoord/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 83.39%. Comparing base (92c7f6a) to head (4eff4db).
Report is 1 commits behind head on 2.4.

Additional details and impacted files

@@            Coverage Diff             @@
##              2.4   #38776      +/-   ##
==========================================
+ Coverage   80.37%   83.39%   +3.01%     
==========================================
  Files        1077      809     -268     
  Lines      169559   143057   -26502     
==========================================
- Hits       136291   119303   -16988     
+ Misses      28781    19277    -9504     
+ Partials     4487     4477      -10     

Files with missing lines	Coverage Δ
internal/datacoord/index_meta.go	97.20% <100.00%> (+1.63%)	⬆️

... and 293 files with indirect coverage changes