List Memory Allocation Issues

[kfrancis]

Getting these notices on AppInsights, thought it worth sharing.

Hangfire.AspNetCore v1.8.14
Hangfire.SqlServer v1.8.14

• FetchJob List.ctor is causing unusually high memory allocations.
• GetStateData List.ctor is causing unusually high memory allocations.

FetchJob List.ctor is causing unusually high memory allocations

Current condition: 40% of your Memory was spent in List.ctor called from Hangfire.SqlServer.SqlServerJobQueue.FetchJob. We expected this value to be <1%.

Recommendations: Consider investigating why List.ctor is causing higher than expected memory allocations.

NOTE: We are currently working on refining the details of this recommendation. In the meantime please click here to get general information about memory performance optimization.

Call Stack:

0. microsoft.applicationinsights.ilrepack!Microsoft.ApplicationInsights.DependencyCollector.Implementation.SqlClientDiagnostics.SqlClientDiagnosticSourceListener.System.IObserver<System.Collections.Generic.KeyValuePair<System.String,System.Object>>.OnNext(value class System.Collections.Generic.KeyValuePair`2<class System.String,class System.Object>)
1. microsoft.data.sqlclient!Microsoft.Data.SqlClient.SqlCommand.ExecuteReader(value class System.Data.CommandBehavior)
2. microsoft.data.sqlclient!Microsoft.Data.SqlClient.SqlCommand.ExecuteReader(value class System.Data.CommandBehavior)
3. hangfire.sqlserver!Dapper.SqlMapper.ExecuteReaderWithFlagsFallback(class System.Data.IDbCommand,bool,value class System.Data.CommandBehavior)
4. hangfire.sqlserver!Dapper.SqlMapper+d__144`1[System.__Canon].MoveNext()
5. system.private.corelib.il!System.Collections.Generic.List1[System.__Canon]..ctor(class System.Collections.Generic.IEnumerable1<!0>)
6. hangfire.sqlserver!Hangfire.SqlServer.SqlServerJobQueue+<>c__DisplayClass14_0.b__0(class System.Data.Common.DbConnection)
7. hangfire.sqlserver!Hangfire.SqlServer.SqlServerStorage.UseConnection(class System.Data.Common.DbConnection,class System.Func`2<class System.Data.Common.DbConnection,!!0>)
8. hangfire.sqlserver!Hangfire.SqlServer.SqlServerJobQueue.DequeueUsingSlidingInvisibilityTimeout(class System.String[],value class System.Threading.CancellationToken)
9. hangfire.core!Hangfire.Server.Worker.Execute(class Hangfire.Server.BackgroundProcessContext)
10. hangfire.core!Hangfire.Server.BackgroundProcessDispatcherBuilder.ExecuteProcess(value class System.Guid,class System.Object)
11. hangfire.core!Hangfire.Processing.BackgroundExecution.Run(class System.Action`2<value class System.Guid,class System.Object>,class System.Object)

GetStateData List.ctor is causing unusually high memory allocations.

Current condition: 32% of your Memory was spent in List.ctor called from Hangfire.SqlServer.SqlServerConnection.GetStateData. We expected this value to be <1%.

Recommendations: Consider investigating why List.ctor is causing higher than expected memory allocations.

NOTE: We are currently working on refining the details of this recommendation. In the meantime please click here to get general information about memory performance optimization.

Call Stack:

0. microsoft.data.sqlclient!Microsoft.Data.SqlClient.SqlStatistics.GetDictionary()
1. microsoft.data.sqlclient!Microsoft.Data.SqlClient.SqlCommand.ExecuteReader(value class System.Data.CommandBehavior)
2. microsoft.data.sqlclient!Microsoft.Data.SqlClient.SqlCommand.ExecuteReader(value class System.Data.CommandBehavior)
3. hangfire.sqlserver!Dapper.SqlMapper.ExecuteReaderWithFlagsFallback(class System.Data.IDbCommand,bool,value class System.Data.CommandBehavior)
4. hangfire.sqlserver!Dapper.SqlMapper+d__144`1[System.__Canon].MoveNext()
5. system.private.corelib.il!System.Collections.Generic.List1[System.__Canon]..ctor(class System.Collections.Generic.IEnumerable1<!0>)
6. hangfire.sqlserver!Hangfire.SqlServer.SqlServerConnection+<>c__DisplayClass13_0.b__0(class System.Data.Common.DbConnection)
7. hangfire.sqlserver!Hangfire.SqlServer.SqlServerStorage.UseConnection(class System.Data.Common.DbConnection,class System.Func`2<class System.Data.Common.DbConnection,!!0>)
8. hangfire.core!Hangfire.Server.ServerJobCancellationToken.TryCheckJobIsAborted(class Hangfire.Storage.IStorageConnection)
9. hangfire.core!Hangfire.Server.ServerJobCancellationToken.CheckAllCancellationTokens(class System.String,class Hangfire.Storage.IStorageConnection,value class System.Threading.CancellationToken)
10. hangfire.core!Hangfire.Server.ServerJobCancellationWatcher.Execute(class Hangfire.Server.BackgroundProcessContext)
11. hangfire.core!Hangfire.Server.BackgroundProcessDispatcherBuilder.ExecuteProcess(value class System.Guid,class System.Object)

[odinserj]

Both of these methods point to allocations in Dapper, not in Hangfire (please see the stack frames on top). The first one is likely solved, since FetchJob now doesn't use Dapper in the most recent versions. But I'm not sure about the List object from the second one, and it's likely from here – https://github.com/dotnet/SqlClient/blob/44cfeddd0543e0ccbcf613e98e5887a6eac67541/src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlStatistics.cs#L89, outside of Hangfire.