OpsPulse

OrderFlow is about a small amount of data with a lot of rules. OpsPulse is the mirror image: a large amount of data with a little state machine in the middle of it. Deliberately so — the two together cover most of what a backend architect is asked to reason about.

The core tension is that ingestion wants to be fast and dumb, while incident handling wants to be careful and stateful. Putting both in one write path makes one of them wrong.

Accepting an event does no analysis. It validates shape, writes it down and returns — because the one thing worse than a missed alert is an ingestion endpoint that gets slow enough to start refusing them during exactly the incident it was meant to catch.

A failing service does not emit one event. It emits thousands, and a platform that turns each into an incident is worse than no platform, because it trains people to ignore it. Collapsing a flood into one incident with a count is the feature everything else exists to support.

• Events carry a fingerprint — service, check and failure kind — and events sharing a fingerprint fold into the open incident for it rather than raising a new one.
• Suppression windows stop a flapping service from reopening the same incident repeatedly.
• The raw event stream is kept whole and append-only, so a deduplicated incident can always be expanded back into everything that fed it.

An alerting system is judged by what it does not page you about.

Incident status is a state machine on the aggregate, not a status column. The legal moves are few, and the value of encoding them is that an incident cannot be resolved without having been acknowledged, or acknowledged twice by different people, whatever order the messages happen to arrive in.

1. 1OpenRaised by the detection engine. Escalation timers start here.
2. 2AcknowledgedA human has taken it. Escalation stops; SLA timers continue.
3. 3EscalatedNo acknowledgement inside the policy window — the next on-call is notified.
4. 4ResolvedTerminal. Reopening creates a linked incident rather than mutating a closed one.

Every transition appends to an immutable timeline. Post-incident, the question is always 'when did we know, and when did we act' — which is answerable only if the answer was recorded as it happened.

Escalation is time-driven, which makes it scheduled work rather than event-driven work. Hangfire owns the timers; when one fires, it publishes an event and the escalation itself flows through the same messaging path as everything else.

This is where CQRS stops being a pattern name and starts paying rent. Writes are small, frequent and transactional. Reads are wide, aggregated and constant — an operations dashboard is a permanent query load.

Keyset pagination matters more here than in most systems: rows are being inserted constantly, and offset pagination on a moving table shows the reader duplicates and gaps as they page through it.

A dashboard that needs refreshing is not an operations dashboard. SignalR pushes incident creation, acknowledgement and resolution to connected clients as they happen, and the client applies them to state it already has rather than refetching the world.

• Pushes are diffs, not full dashboard reloads.
• A reconnecting client resyncs from the read model, so a dropped socket cannot leave the UI silently stale.
• Client-facing only — service-to-service traffic stays on RabbitMQ.

• Health probes against registered services, so silence is distinguishable from health.
• Escalation and SLA timers.
• Reprocessing of events that failed detection, off the ingestion path.
• Retention and rollup of the raw event stream.

A monitoring platform that cannot be monitored is a single point of failure with extra steps. The same Serilog and OpenTelemetry setup applies, plus the questions specific to this shape: ingestion lag, detection lag, and how far the read models trail the write side.

• Ingestion lag and queue depth as first-class metrics, not log lines.
• Correlation from inbound event through detection to the incident it fed.
• Liveness and readiness split, so a full queue does not read as a dead process.

The two are deliberately not the same system with different nouns. They stress different parts of the same toolkit, which is the point of building both.