OPS-VIEW-06
Alert rules, incidents & scenarios
A rule detects a condition; an incident organises the response. Acknowledging records ownership and repairs nothing, a cleared condition never erases the incident, and recovery is requested from the existing recovery centre rather than run again here.
OverviewSource & signal monitorFlow runs & deliveryQuality & readinessLogs, traces & changesIncident registerAlert rules & scenariosPerformance & capacityReportsMetric contractDeveloper handover
As of 2026-09-23T06:00Z (virtual scenario clock)Window trailing 720 minTimezone Asia/Kolkata (UTC+05:30), displayed in UTCDataset ops-seed-0.7.0 · run SR-0007-baselineenvironmentMode: simulation
Simulated acquisition operations. No notification is ever sent from this workspace. The templates below are configuration a developer can implement; nothing is emailed, posted or paged.
Alert rule definitions
| Rule | Signal | Predicate | Window / persistence | Hysteresis & cooldown | Suppression | Owner | Runbook |
|---|---|---|---|---|---|---|---|
| OPS-AR-01 Freshness evidence absent process-critical · v0.7.0-proposed | OPS-M-07 | freshness evidence older than the contract rule, or keep-alive-only | 15 min · 3 consecutive evaluations | clears only after 2 evaluations with valid evidence; cooldown 30 min | suppressed during a declared maintenance window, and the suppression stays visible | Operations Lead | RB-FRESHNESS-EVIDENCE |
| OPS-AR-02 Delivery backlog growing standard · v0.7.0-proposed | OPS-M-11 | outstanding obligations increasing for 3 windows and oldest age above 30 min | 10 min · 3 windows | clears below 50% of the trigger level; cooldown 20 min | none | Operations Lead | RB-DELIVERY-BACKLOG |
| OPS-AR-03 Unit or identity unresolved semantic · v0.7.0-proposed | UNIT_INCOMPATIBLE | IDENTITY_UNRESOLVED | any occurrence under an approved contract version | immediate · 1 occurrence | manual clear after a steward decision; cooldown none | none | Semantic Steward | RB-BINDING-REMEDIATION |
| OPS-AR-04 Telemetry lost observability · v0.7.0-proposed | OPS-M-14 | no telemetry receipt for 30 min | 5 min · immediate | clears on 2 consecutive receipts; cooldown none | never suppressed — an unknown monitor is not a healthy monitor | Operations Lead | RB-TELEMETRY-GAP |
| OPS-AR-05 Session lost on a shared engine transport · v0.7.0-proposed | SESSION_LOST | session loss affecting two or more sources on one shared engine | 5 min · 1 occurrence | clears on subscription recovery evidence; cooldown 15 min | dependent alerts are grouped, not deleted | Integration Engineer | RB-SESSION-RECOVERY |
| OPS-AR-06 Expected member missing completeness · v0.7.0-proposed | OPS-M-04 | observed members below the declared manifest or trigger denominator | per batch · 1 batch | clears when the batch reconciles; cooldown none | none | Integration Engineer | RB-MANIFEST-RECONCILIATION |
Incident ledger (14)
| Incident | State | Severity | Assignee | Muted |
|---|---|---|---|---|
Expected member missing — NGD File drop F6 OPSINC-001 | Open | warning | unassigned | no |
Freshness evidence absent — HAL Document library A7 OPSINC-002 | Acknowledged | error | A. Kulkarni (Integration Engineer) | no |
Unit or identity unresolved — HAL OT controller E11 OPSINC-003 | Investigating | error | S. Iyer (Operations Lead) | no |
Expected member missing — SHF File drop D16 OPSINC-004 | Mitigating | warning | unassigned | no |
Freshness evidence absent — HAL Application API E23 OPSINC-005 | Recovered | error | S. Iyer (Operations Lead) | no |
Telemetry lost — HAL Application API E23 OPSINC-006 | Resolved | critical | A. Kulkarni (Integration Engineer) | no |
Freshness evidence absent — NGD File drop B26 OPSINC-007 | Open | error | unassigned | no |
Delivery backlog growing — HAL Application API C3 OPSINC-008 | Acknowledged | warning | A. Kulkarni (Integration Engineer) | no |
Delivery backlog growing — SHF Vision system B8 OPSINC-009 | Investigating | warning | S. Iyer (Operations Lead) | no |
Delivery backlog growing — RGH Application API A13 OPSINC-010 | Mitigating | warning | unassigned | no |
Delivery backlog growing — NGD Vision system F18 OPSINC-011 | Recovered | warning | S. Iyer (Operations Lead) | no |
Delivery backlog growing — HAL Application API E23 OPSINC-012 | Resolved | warning | A. Kulkarni (Integration Engineer) | no |
Delivery backlog growing — SHF Vision system D28 OPSINC-013 | Open | warning | unassigned | no |
Session lost on a shared engine — RGH OT controller A1 OPSINC-014 | Acknowledged | error | A. Kulkarni (Integration Engineer) | no |
Incident OPSINC-001
RuleOPS-AR-06 — Expected member missing — NGD File drop F6
Onset vs detectionOnset 00:48Z, detected 01:00Z
Reopened1 time(s) — the original incident is retained
Grouped withnot grouped
Supporting diagnosticsDIAG-00015, DIAG-00016
Outstanding obligations1 obligation(s)
Resolution evidenceNo resolution evidence recorded; the incident cannot be closed as fixed.
Recovery requestnone
Cause statements
observed factobserved members below the declared manifest or trigger denominator observed on OPSRC-006.
hypothesisA nearby configuration change is temporally close; that alone is not proof of causation.
validated causeValidated by reproducing the condition in the simulation laboratory.
Timeline
01:00Z — → Open · Alert rule OPS-AR-06 — Condition detected by rule evaluation.
Recovery is executed in the existing recovery centre. Requesting it here records intent; it repeats no operational side effect.
Runbook RB-MANIFEST-RECONCILIATION — Expected member missing
- Compare observed members with the declared manifest denominator.
- Identify missing or duplicate members explicitly.
- Request a bounded backfill for the missing window only.
Backfill is authorised in the recovery centre.
Acceptance scenarios (0/13 executed and holding)
No scenario has been executed in this browser session yet.
| Scenario | What it proves | Result | |
|---|---|---|---|
| OPS-SIM-01 Stable OPC UA value under deadband | A healthy subscription with no reportable change must not become a missing-record incident. | not run | |
| OPS-SIM-02 Frozen underlying signal, live server | Recent server contact alone must not satisfy a contract requiring fresh measurement evidence. | not run | |
| OPS-SIM-03 Wrong unit or unresolved identity | A source can stay connected while admission and readiness degrade; every surface must agree. | not run | |
| OPS-SIM-04 Duplicate and fan-out | Retries must not inflate unique inputs; a second consumer creates separate obligations. | not run | |
| OPS-SIM-05 Consumer outage | Acquisition continues while delivery degrades; recovery reconciles the outstanding obligations. | not run | |
| OPS-SIM-06 Delayed historical import | An old event time must not be reported as slow present processing. | not run | |
| OPS-SIM-07 Source clock skew | An invalid cross-clock interval must be unknown, never a performance success. | not run | |
| OPS-SIM-08 Incomplete file or capture group | A declared denominator identifies missing content; incomplete inspection evidence is never PASS. | not run | |
| OPS-SIM-09 WMS / LocateLink conflict | Both conflicting records are preserved and no stock movement is posted. | not run | |
| OPS-SIM-10 Telemetry interruption | Monitoring state becomes unknown; absent metrics never read as zero errors. | not run | |
| OPS-SIM-11 Maintenance and alert correlation | Exclusions stay visible and grouped alerts keep their independent evidence. | not run | |
| OPS-SIM-12 Workspace and report reproducibility | Exports preserve versions, totals and simulation labels; imports refuse unmarked files. | not run | |
| OPS-SIM-13 Confidentiality and safe simulation | No secrets in logs or exports, and no simulated action reaches a live endpoint. | not run |
Runbooks
RB-FRESHNESS-EVIDENCE Freshness evidence absent
- Separate session contact from data-change evidence on the source monitor.
- Confirm whether the subscription is quiet by design (deadband) before suspecting loss.
- Check whether only keep-alive evidence exists; if so, record the source as unknown, not healthy.
- Request a permitted forced read only where the contract allows it, and treat a cached value as not proving a new sample.
- If evidence remains absent, raise a recovery request in the recovery centre.
RB-DELIVERY-BACKLOG Delivery backlog
- Confirm acquisition is still accepting inputs; a consumer outage is not an acquisition failure.
- Inspect outstanding obligations per consumer and their oldest age.
- Check whether retries are amplifying attempts without adding completions.
- Plan a bounded recovery in the recovery centre, pinning the cohort.
- Verify reconciliation against the input-to-delivery report.
RB-BINDING-REMEDIATION Unit or identity unresolved
- Open the binding version referenced by the diagnostic.
- Confirm whether the source remains connected — connection health and admission are separate.
- Record the corrected mapping as a proposed change with a reason.
- Re-run the affected cohort through reprocess, never replay.
RB-TELEMETRY-GAP Telemetry interruption
- Mark the affected sources unknown; absent metrics are not zero errors.
- Confirm whether the monitor or the source is affected.
- Record the coverage limitation on any report generated for the period.
RB-SESSION-RECOVERY Session lost on a shared engine
- Group dependent alerts on the shared engine while preserving each independent condition.
- Check republish evidence for known sequence gaps.
- Confirm whether controller profiles on the same engine are equally affected — a shared engine does not imply a per-profile driver.
RB-MANIFEST-RECONCILIATION Expected member missing
- Compare observed members with the declared manifest denominator.
- Identify missing or duplicate members explicitly.
- Request a bounded backfill for the missing window only.
Notification templates (configuration only — nothing is sent)
OPS-NT-01 Email (configuration only)
Incident {incident} is {state}. Evidence: {diagnostics}. Runbook: {runbook}.Not sent. No outbound service is connected and nothing runs while the browser is closed.
OPS-NT-02 Webhook (configuration only)
{"incident":"{incident}","state":"{state}","environmentMode":"simulation"}Not sent. Demonstrated as routing configuration only.