Hangfire Dashboard

Program.ExtractJiraDotnetSDK

// Job ID: #441341
using jtl;

await Program.ExtractJiraDotnetSDK(null);

Parameters

CurrentCulture 
""
CurrentUICulture 
""
RecurringJobId 
"daily-jtl"
RetryCount 
10
Time 
1760601605

State

10/16/2025 22:38:34 (+45m 39.033s) Failed

An exception occurred during performance of the job.

Microsoft.Data.SqlClient.SqlException

Execution Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding.

Microsoft.Data.SqlClient.SqlException (0x80131904): Execution Timeout Expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.
 ---> System.ComponentModel.Win32Exception (258): Unknown error 258
   at Microsoft.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at Microsoft.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at Microsoft.Data.SqlClient.SqlCommand.InternalEndExecuteNonQuery(IAsyncResult asyncResult, Boolean isInternal, String endMethod)
   at Microsoft.Data.SqlClient.SqlCommand.EndExecuteNonQueryInternal(IAsyncResult asyncResult)
   at Microsoft.Data.SqlClient.SqlCommand.EndExecuteNonQueryAsync(IAsyncResult asyncResult)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location ---
   at Datadog.Trace.ClrProfiler.CallTarget.Handlers.Continuations.TaskContinuationGenerator`4.SyncCallbackHandler.ContinuationAction(Task`1 previousTask, TTarget target, CallTargetState state) in /project/tracer/src/Datadog.Trace/ClrProfiler/CallTarget/Handlers/Continuations/TaskContinuationGenerator`1.cs:line 143
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.RelationalDatabaseFacadeExtensions.ExecuteSqlRawAsync(DatabaseFacade databaseFacade, String sql, IEnumerable`1 parameters, CancellationToken cancellationToken)
   at jtl.Indexer.ReloadWIPData() in /jtl/Indexer.cs:line 179
   at jtl.Indexer.ReloadWIPData() in /jtl/Indexer.cs:line 181
   at jtl.Program.ExtractJiraDotnetSDK(CancellationToken ct) in /jtl/Program.cs:line 127
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+26m 617ms Processing

Server:
jtl-78fcb778f7-rzmtk:1
Worker:
e97e315c

+13ms Processing

Server:
jtl-78fcb778f7-6pk55:1
Worker:
03ff43e6

+1h 50m Enqueued

Triggered by DelayedJobScheduler

Queue:
default

+3ms Scheduled

Retry attempt 10 of 10: Execution Timeout Expired. The timeout period el…

Enqueue at:
10/16/2025 21:26:50

+45m 26.620s Failed

An exception occurred during performance of the job.

Microsoft.Data.SqlClient.SqlException

Execution Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding.

Microsoft.Data.SqlClient.SqlException (0x80131904): Execution Timeout Expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.
 ---> System.ComponentModel.Win32Exception (258): Unknown error 258
   at Microsoft.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at Microsoft.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at Microsoft.Data.SqlClient.SqlCommand.InternalEndExecuteNonQuery(IAsyncResult asyncResult, Boolean isInternal, String endMethod)
   at Microsoft.Data.SqlClient.SqlCommand.EndExecuteNonQueryInternal(IAsyncResult asyncResult)
   at Microsoft.Data.SqlClient.SqlCommand.EndExecuteNonQueryAsync(IAsyncResult asyncResult)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location ---
   at Datadog.Trace.ClrProfiler.CallTarget.Handlers.Continuations.TaskContinuationGenerator`4.SyncCallbackHandler.ContinuationAction(Task`1 previousTask, TTarget target, CallTargetState state) in /project/tracer/src/Datadog.Trace/ClrProfiler/CallTarget/Handlers/Continuations/TaskContinuationGenerator`1.cs:line 143
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.RelationalDatabaseFacadeExtensions.ExecuteSqlRawAsync(DatabaseFacade databaseFacade, String sql, IEnumerable`1 parameters, CancellationToken cancellationToken)
   at jtl.Indexer.ReloadWIPData() in /jtl/Indexer.cs:line 179
   at jtl.Indexer.ReloadWIPData() in /jtl/Indexer.cs:line 181
   at jtl.Program.ExtractJiraDotnetSDK(CancellationToken ct) in /jtl/Program.cs:line 127
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+30m 1.360s Processing

Server:
jtl-78fcb778f7-z66jn:1
Worker:
59091c4a

+19m 450ms Processing

Server:
jtl-78fcb778f7-f422j:1
Worker:
58b69a9f

+44ms Processing

Server:
jtl-78fcb778f7-77vzl:1
Worker:
fb00f9f6

+1h 9m Enqueued

Triggered by DelayedJobScheduler

Queue:
default

+3ms Scheduled

Retry attempt 9 of 10: Transaction (Process ID 161) was deadlocked on lo…

Enqueue at:
10/16/2025 18:02:03

+17.030s Failed

An exception occurred during performance of the job.

Microsoft.Data.SqlClient.SqlException

Transaction (Process ID 161) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.

Microsoft.Data.SqlClient.SqlException (0x80131904): Transaction (Process ID 161) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
   at Microsoft.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at Microsoft.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at Microsoft.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
   at Microsoft.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString, Boolean isInternal, Boolean forDescribeParameterEncryption, Boolean shouldCacheForAlwaysEncrypted)
   at Microsoft.Data.SqlClient.SqlCommand.CompleteAsyncExecuteReader(Boolean isInternal, Boolean forDescribeParameterEncryption)
   at Microsoft.Data.SqlClient.SqlCommand.InternalEndExecuteNonQuery(IAsyncResult asyncResult, Boolean isInternal, String endMethod)
   at Microsoft.Data.SqlClient.SqlCommand.EndExecuteNonQueryInternal(IAsyncResult asyncResult)
   at Microsoft.Data.SqlClient.SqlCommand.EndExecuteNonQueryAsync(IAsyncResult asyncResult)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location ---
   at Datadog.Trace.ClrProfiler.CallTarget.Handlers.Continuations.TaskContinuationGenerator`4.SyncCallbackHandler.ContinuationAction(Task`1 previousTask, TTarget target, CallTargetState state) in /project/tracer/src/Datadog.Trace/ClrProfiler/CallTarget/Handlers/Continuations/TaskContinuationGenerator`1.cs:line 143
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.RelationalDatabaseFacadeExtensions.ExecuteSqlRawAsync(DatabaseFacade databaseFacade, String sql, IEnumerable`1 parameters, CancellationToken cancellationToken)
   at FlexLabs.EntityFrameworkCore.Upsert.Runners.RelationalUpsertCommandRunner.RunAsync[TEntity](DbContext dbContext, IEntityType entityType, ICollection`1 entities, Expression`1 matchExpression, Expression`1 updateExpression, Expression`1 updateCondition, RunnerQueryOptions queryOptions, CancellationToken cancellationToken) in /_/src/FlexLabs.EntityFrameworkCore.Upsert/Runners/RelationalUpsertCommandRunner.cs:line 402
   at jtl.Indexer.UpsertChangeLogs(IEnumerable`1 changeLogs, String issueKey) in /jtl/Indexer.cs:line 41
   at jtl.Indexer.UpsertChangeLogs(IEnumerable`1 changeLogs, String issueKey) in /jtl/Indexer.cs:line 41
   at jtl.JiraBatchCrawler.<>c__DisplayClass5_0.<<ExtractHistory>b__4>d.MoveNext() in /jtl/JiraBatchCrawler.cs:line 62
--- End of stack trace from previous location ---
   at System.Threading.Tasks.Parallel.<>c__50`1.<<ForEachAsync>b__50_0>d.MoveNext()
--- End of stack trace from previous location ---
   at jtl.JiraBatchCrawler.ExtractHistory() in /jtl/JiraBatchCrawler.cs:line 42
   at jtl.JiraBatchCrawler.ExtractHistory() in /jtl/JiraBatchCrawler.cs:line 42
   at jtl.Program.ExtractJiraDotnetSDK(CancellationToken ct) in /jtl/Program.cs:line 124
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+13ms Processing

Server:
jtl-78fcb778f7-k4wmg:1
Worker:
154554a4

+40m 27.417s Enqueued

Triggered by DelayedJobScheduler

Queue:
default

+ <1ms Scheduled

Retry attempt 8 of 10: Execution Timeout Expired. The timeout period el…

Enqueue at:
10/16/2025 16:52:26

+45m 28.293s Failed

An exception occurred during performance of the job.

Microsoft.Data.SqlClient.SqlException

Execution Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding.

Microsoft.Data.SqlClient.SqlException (0x80131904): Execution Timeout Expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.
 ---> System.ComponentModel.Win32Exception (258): Unknown error 258
   at Microsoft.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at Microsoft.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at Microsoft.Data.SqlClient.SqlCommand.InternalEndExecuteNonQuery(IAsyncResult asyncResult, Boolean isInternal, String endMethod)
   at Microsoft.Data.SqlClient.SqlCommand.EndExecuteNonQueryInternal(IAsyncResult asyncResult)
   at Microsoft.Data.SqlClient.SqlCommand.EndExecuteNonQueryAsync(IAsyncResult asyncResult)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location ---
   at Datadog.Trace.ClrProfiler.CallTarget.Handlers.Continuations.TaskContinuationGenerator`4.SyncCallbackHandler.ContinuationAction(Task`1 previousTask, TTarget target, CallTargetState state) in /project/tracer/src/Datadog.Trace/ClrProfiler/CallTarget/Handlers/Continuations/TaskContinuationGenerator`1.cs:line 143
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.RelationalDatabaseFacadeExtensions.ExecuteSqlRawAsync(DatabaseFacade databaseFacade, String sql, IEnumerable`1 parameters, CancellationToken cancellationToken)
   at jtl.Indexer.ReloadWIPData() in /jtl/Indexer.cs:line 179
   at jtl.Indexer.ReloadWIPData() in /jtl/Indexer.cs:line 181
   at jtl.Program.ExtractJiraDotnetSDK(CancellationToken ct) in /jtl/Program.cs:line 127
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+24ms Processing

Server:
jtl-78fcb778f7-h97qr:1
Worker:
99732b83

+24m 53.630s Enqueued

Triggered by DelayedJobScheduler

Queue:
default

+ <1ms Scheduled

Retry attempt 7 of 10: Execution Timeout Expired. The timeout period el…

Enqueue at:
10/16/2025 15:26:33

+45m 26.846s Failed

An exception occurred during performance of the job.

Microsoft.Data.SqlClient.SqlException

Execution Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding.

Microsoft.Data.SqlClient.SqlException (0x80131904): Execution Timeout Expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.
 ---> System.ComponentModel.Win32Exception (258): Unknown error 258
   at Microsoft.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at Microsoft.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at Microsoft.Data.SqlClient.SqlCommand.InternalEndExecuteNonQuery(IAsyncResult asyncResult, Boolean isInternal, String endMethod)
   at Microsoft.Data.SqlClient.SqlCommand.EndExecuteNonQueryInternal(IAsyncResult asyncResult)
   at Microsoft.Data.SqlClient.SqlCommand.EndExecuteNonQueryAsync(IAsyncResult asyncResult)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location ---
   at Datadog.Trace.ClrProfiler.CallTarget.Handlers.Continuations.TaskContinuationGenerator`4.SyncCallbackHandler.ContinuationAction(Task`1 previousTask, TTarget target, CallTargetState state) in /project/tracer/src/Datadog.Trace/ClrProfiler/CallTarget/Handlers/Continuations/TaskContinuationGenerator`1.cs:line 143
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.RelationalDatabaseFacadeExtensions.ExecuteSqlRawAsync(DatabaseFacade databaseFacade, String sql, IEnumerable`1 parameters, CancellationToken cancellationToken)
   at jtl.Indexer.ReloadWIPData() in /jtl/Indexer.cs:line 179
   at jtl.Indexer.ReloadWIPData() in /jtl/Indexer.cs:line 181
   at jtl.Program.ExtractJiraDotnetSDK(CancellationToken ct) in /jtl/Program.cs:line 127
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+40m 937ms Processing

Server:
jtl-78fcb778f7-cdtg6:1
Worker:
abeac01b

+10ms Processing

Server:
jtl-78fcb778f7-l9nlh:1
Worker:
1a6d0b2d

+13m 137ms Enqueued

Triggered by DelayedJobScheduler

Queue:
default

+3ms Scheduled

Retry attempt 6 of 10: Execution Timeout Expired. The timeout period el…

Enqueue at:
10/16/2025 13:36:10

+45m 29.513s Failed

An exception occurred during performance of the job.

Microsoft.Data.SqlClient.SqlException

Execution Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding.

Microsoft.Data.SqlClient.SqlException (0x80131904): Execution Timeout Expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.
 ---> System.ComponentModel.Win32Exception (258): Unknown error 258
   at Microsoft.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at Microsoft.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at Microsoft.Data.SqlClient.SqlCommand.InternalEndExecuteNonQuery(IAsyncResult asyncResult, Boolean isInternal, String endMethod)
   at Microsoft.Data.SqlClient.SqlCommand.EndExecuteNonQueryInternal(IAsyncResult asyncResult)
   at Microsoft.Data.SqlClient.SqlCommand.EndExecuteNonQueryAsync(IAsyncResult asyncResult)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location ---
   at Datadog.Trace.ClrProfiler.CallTarget.Handlers.Continuations.TaskContinuationGenerator`4.SyncCallbackHandler.ContinuationAction(Task`1 previousTask, TTarget target, CallTargetState state) in /project/tracer/src/Datadog.Trace/ClrProfiler/CallTarget/Handlers/Continuations/TaskContinuationGenerator`1.cs:line 143
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.RelationalDatabaseFacadeExtensions.ExecuteSqlRawAsync(DatabaseFacade databaseFacade, String sql, IEnumerable`1 parameters, CancellationToken cancellationToken)
   at jtl.Indexer.ReloadWIPData() in /jtl/Indexer.cs:line 179
   at jtl.Indexer.ReloadWIPData() in /jtl/Indexer.cs:line 181
   at jtl.Program.ExtractJiraDotnetSDK(CancellationToken ct) in /jtl/Program.cs:line 127
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+8m 574ms Processing

Server:
jtl-78fcb778f7-dcjhc:1
Worker:
359e39d9

+10ms Processing

Server:
jtl-78fcb778f7-zm62d:1
Worker:
96d24f9f

+6m 20.140s Enqueued

Triggered by DelayedJobScheduler

Queue:
default

+ <1ms Scheduled

Retry attempt 5 of 10: Execution Timeout Expired. The timeout period el…

Enqueue at:
10/16/2025 12:29:38

+45m 25.426s Failed

An exception occurred during performance of the job.

Microsoft.Data.SqlClient.SqlException

Execution Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding.

Microsoft.Data.SqlClient.SqlException (0x80131904): Execution Timeout Expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.
 ---> System.ComponentModel.Win32Exception (258): Unknown error 258
   at Microsoft.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at Microsoft.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at Microsoft.Data.SqlClient.SqlCommand.InternalEndExecuteNonQuery(IAsyncResult asyncResult, Boolean isInternal, String endMethod)
   at Microsoft.Data.SqlClient.SqlCommand.EndExecuteNonQueryInternal(IAsyncResult asyncResult)
   at Microsoft.Data.SqlClient.SqlCommand.EndExecuteNonQueryAsync(IAsyncResult asyncResult)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location ---
   at Datadog.Trace.ClrProfiler.CallTarget.Handlers.Continuations.TaskContinuationGenerator`4.SyncCallbackHandler.ContinuationAction(Task`1 previousTask, TTarget target, CallTargetState state) in /project/tracer/src/Datadog.Trace/ClrProfiler/CallTarget/Handlers/Continuations/TaskContinuationGenerator`1.cs:line 143
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.RelationalDatabaseFacadeExtensions.ExecuteSqlRawAsync(DatabaseFacade databaseFacade, String sql, IEnumerable`1 parameters, CancellationToken cancellationToken)
   at jtl.Indexer.ReloadWIPData() in /jtl/Indexer.cs:line 179
   at jtl.Indexer.ReloadWIPData() in /jtl/Indexer.cs:line 181
   at jtl.Program.ExtractJiraDotnetSDK(CancellationToken ct) in /jtl/Program.cs:line 127
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+7ms Processing

Server:
jtl-78fcb778f7-m2j4h:1
Worker:
129d8499

+1m 57.477s Enqueued

Triggered by DelayedJobScheduler

Queue:
default

+ <1ms Scheduled

Retry attempt 4 of 10: Execution Timeout Expired. The timeout period el…

Enqueue at:
10/16/2025 11:37:51

+45m 24.256s Failed

An exception occurred during performance of the job.

Microsoft.Data.SqlClient.SqlException

Execution Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding.

Microsoft.Data.SqlClient.SqlException (0x80131904): Execution Timeout Expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.
 ---> System.ComponentModel.Win32Exception (258): Unknown error 258
   at Microsoft.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at Microsoft.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at Microsoft.Data.SqlClient.SqlCommand.InternalEndExecuteNonQuery(IAsyncResult asyncResult, Boolean isInternal, String endMethod)
   at Microsoft.Data.SqlClient.SqlCommand.EndExecuteNonQueryInternal(IAsyncResult asyncResult)
   at Microsoft.Data.SqlClient.SqlCommand.EndExecuteNonQueryAsync(IAsyncResult asyncResult)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location ---
   at Datadog.Trace.ClrProfiler.CallTarget.Handlers.Continuations.TaskContinuationGenerator`4.SyncCallbackHandler.ContinuationAction(Task`1 previousTask, TTarget target, CallTargetState state) in /project/tracer/src/Datadog.Trace/ClrProfiler/CallTarget/Handlers/Continuations/TaskContinuationGenerator`1.cs:line 143
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.RelationalDatabaseFacadeExtensions.ExecuteSqlRawAsync(DatabaseFacade databaseFacade, String sql, IEnumerable`1 parameters, CancellationToken cancellationToken)
   at jtl.Indexer.ReloadWIPData() in /jtl/Indexer.cs:line 179
   at jtl.Indexer.ReloadWIPData() in /jtl/Indexer.cs:line 181
   at jtl.Program.ExtractJiraDotnetSDK(CancellationToken ct) in /jtl/Program.cs:line 127
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+26m 1.117s Processing

Server:
jtl-78fcb778f7-m2j4h:1
Worker:
dc3e05e8

+7ms Processing

Server:
jtl-78fcb778f7-nddjx:1
Worker:
1ecd958e

+41.406s Enqueued

Triggered by DelayedJobScheduler

Queue:
default

+ <1ms Scheduled

Retry attempt 3 of 10: Execution Timeout Expired. The timeout period el…

Enqueue at:
10/16/2025 10:24:29

+45m 25.470s Failed

An exception occurred during performance of the job.

Microsoft.Data.SqlClient.SqlException

Execution Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding.

Microsoft.Data.SqlClient.SqlException (0x80131904): Execution Timeout Expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.
 ---> System.ComponentModel.Win32Exception (258): Unknown error 258
   at Microsoft.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at Microsoft.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at Microsoft.Data.SqlClient.SqlCommand.InternalEndExecuteNonQuery(IAsyncResult asyncResult, Boolean isInternal, String endMethod)
   at Microsoft.Data.SqlClient.SqlCommand.EndExecuteNonQueryInternal(IAsyncResult asyncResult)
   at Microsoft.Data.SqlClient.SqlCommand.EndExecuteNonQueryAsync(IAsyncResult asyncResult)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location ---
   at Datadog.Trace.ClrProfiler.CallTarget.Handlers.Continuations.TaskContinuationGenerator`4.SyncCallbackHandler.ContinuationAction(Task`1 previousTask, TTarget target, CallTargetState state) in /project/tracer/src/Datadog.Trace/ClrProfiler/CallTarget/Handlers/Continuations/TaskContinuationGenerator`1.cs:line 143
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.RelationalDatabaseFacadeExtensions.ExecuteSqlRawAsync(DatabaseFacade databaseFacade, String sql, IEnumerable`1 parameters, CancellationToken cancellationToken)
   at jtl.Indexer.ReloadWIPData() in /jtl/Indexer.cs:line 179
   at jtl.Indexer.ReloadWIPData() in /jtl/Indexer.cs:line 181
   at jtl.Program.ExtractJiraDotnetSDK(CancellationToken ct) in /jtl/Program.cs:line 127
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+17ms Processing

Server:
jtl-78fcb778f7-zm62d:1
Worker:
04837954

+1m 30ms Enqueued

Triggered by DelayedJobScheduler

Queue:
default

+ <1ms Scheduled

Retry attempt 2 of 10: Execution Timeout Expired. The timeout period el…

Enqueue at:
10/16/2025 09:38:25

+45m 30.067s Failed

An exception occurred during performance of the job.

Microsoft.Data.SqlClient.SqlException

Execution Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding.

Microsoft.Data.SqlClient.SqlException (0x80131904): Execution Timeout Expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.
 ---> System.ComponentModel.Win32Exception (258): Unknown error 258
   at Microsoft.Data.SqlClient.SqlCommand.EndExecuteNonQueryAsync(IAsyncResult asyncResult)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location ---
   at Datadog.Trace.ClrProfiler.CallTarget.Handlers.Continuations.TaskContinuationGenerator`4.SyncCallbackHandler.ContinuationAction(Task`1 previousTask, TTarget target, CallTargetState state) in /project/tracer/src/Datadog.Trace/ClrProfiler/CallTarget/Handlers/Continuations/TaskContinuationGenerator`1.cs:line 143
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.RelationalDatabaseFacadeExtensions.ExecuteSqlRawAsync(DatabaseFacade databaseFacade, String sql, IEnumerable`1 parameters, CancellationToken cancellationToken)
   at jtl.Indexer.ReloadWIPData() in /jtl/Indexer.cs:line 179
   at jtl.Indexer.ReloadWIPData() in /jtl/Indexer.cs:line 181
   at jtl.Program.ExtractJiraDotnetSDK(CancellationToken ct) in /jtl/Program.cs:line 127
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+39m 793ms Processing

Server:
jtl-78fcb778f7-nddjx:1
Worker:
e09f902b

+12m 680ms Processing

Server:
jtl-78fcb778f7-5c5w7:1
Worker:
f2356e3a

+20ms Processing

Server:
jtl-78fcb778f7-s428s:1
Worker:
3a5b3007

+23.853s Enqueued

Triggered by DelayedJobScheduler

Queue:
default

+ <1ms Scheduled

Retry attempt 1 of 10: Transaction (Process ID 63) was deadlocked on loc…

Enqueue at:
10/16/2025 08:00:55

+26.300s Failed

An exception occurred during performance of the job.

Microsoft.Data.SqlClient.SqlException

Transaction (Process ID 63) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.

Microsoft.Data.SqlClient.SqlException (0x80131904): Transaction (Process ID 63) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
   at Microsoft.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at Microsoft.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
   at Microsoft.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString, Boolean isInternal, Boolean forDescribeParameterEncryption, Boolean shouldCacheForAlwaysEncrypted)
   at Microsoft.Data.SqlClient.SqlCommand.CompleteAsyncExecuteReader(Boolean isInternal, Boolean forDescribeParameterEncryption)
   at Microsoft.Data.SqlClient.SqlCommand.InternalEndExecuteNonQuery(IAsyncResult asyncResult, Boolean isInternal, String endMethod)
   at Microsoft.Data.SqlClient.SqlCommand.EndExecuteNonQueryInternal(IAsyncResult asyncResult)
   at Microsoft.Data.SqlClient.SqlCommand.EndExecuteNonQueryAsync(IAsyncResult asyncResult)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location ---
   at Datadog.Trace.ClrProfiler.CallTarget.Handlers.Continuations.TaskContinuationGenerator`4.SyncCallbackHandler.ContinuationAction(Task`1 previousTask, TTarget target, CallTargetState state) in /project/tracer/src/Datadog.Trace/ClrProfiler/CallTarget/Handlers/Continuations/TaskContinuationGenerator`1.cs:line 143
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.RelationalDatabaseFacadeExtensions.ExecuteSqlRawAsync(DatabaseFacade databaseFacade, String sql, IEnumerable`1 parameters, CancellationToken cancellationToken)
   at FlexLabs.EntityFrameworkCore.Upsert.Runners.RelationalUpsertCommandRunner.RunAsync[TEntity](DbContext dbContext, IEntityType entityType, ICollection`1 entities, Expression`1 matchExpression, Expression`1 updateExpression, Expression`1 updateCondition, RunnerQueryOptions queryOptions, CancellationToken cancellationToken) in /_/src/FlexLabs.EntityFrameworkCore.Upsert/Runners/RelationalUpsertCommandRunner.cs:line 402
   at jtl.Indexer.UpsertChangeLogs(IEnumerable`1 changeLogs, String issueKey) in /jtl/Indexer.cs:line 41
   at jtl.Indexer.UpsertChangeLogs(IEnumerable`1 changeLogs, String issueKey) in /jtl/Indexer.cs:line 41
   at jtl.JiraBatchCrawler.<>c__DisplayClass5_0.<<ExtractHistory>b__4>d.MoveNext() in /jtl/JiraBatchCrawler.cs:line 62
--- End of stack trace from previous location ---
   at System.Threading.Tasks.Parallel.<>c__50`1.<<ForEachAsync>b__50_0>d.MoveNext()
--- End of stack trace from previous location ---
   at jtl.JiraBatchCrawler.ExtractHistory() in /jtl/JiraBatchCrawler.cs:line 42
   at jtl.JiraBatchCrawler.ExtractHistory() in /jtl/JiraBatchCrawler.cs:line 42
   at jtl.Program.ExtractJiraDotnetSDK(CancellationToken ct) in /jtl/Program.cs:line 128
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+10ms Processing

Server:
jtl-78fcb778f7-xkh86:1
Worker:
8972a083

+4ms Enqueued

Triggered by recurring job scheduler

Queue:
default

10/16/2025 08:00:05 Created