Solution / Background workers

Monitor Background Workers and Long-Running Jobs

Use different evidence for a worker that is gone, a worker that stopped checking in and a worker that is alive but no longer progressing.

Solution pattern; current heartbeat foundation

Three worker failure classes

  1. Process gone: the worker is no longer running.
  2. Process alive: the worker checks in, but that alone says little about output quality.
  3. Process alive but stuck: the worker checks in while its progress marker remains stale.

Map the evidence to the question

Heartbeat Monitor supplies liveness evidence. Dead Man Monitor is the missing-check-in concept. Dead Hand Monitor is the progress-freshness concept. Logs, metrics, queue depth and application validation fill in details that a heartbeat cannot provide.

Place the signal at a meaningful checkpoint

For a scheduled or long-running worker, report after a meaningful checkpoint rather than at process startup alone. Include only the correlation context needed to connect the report with the worker's own logs.

Limitations

A worker heartbeat does not make a queue durable, prove a job result or supply incident notifications. The current HPH implementation supports authenticated reporting and pending application signals; Dead Man and Dead Hand state engines remain separate implementation work.

Read the background job guide