Failure Alerting

A scheduled forecast that fails silently overnight is the worst failure mode of an operational pipeline. Workspaces can register a webhook that fires whenever a run reaches the failed state — scheduled, manual, or API-triggered.

The same webhook also carries dataset staleness events (dataset.stale / dataset.fresh) when enabled with --on-stale — see Data Quality for freshness SLAs, which catch pipelines that stop producing without ever failing.

Configure the Webhook

rebase workspace notifications set \
  --webhook-url https://hooks.example.com/rebase \
  --webhook-secret <secret> \
  --on-failure

rebase workspace notifications show

Slack and Microsoft Teams incoming-webhook URLs work directly. To stop notifications, run rebase workspace notifications set --no-on-failure, or --clear-webhook to remove the stored URL and secret.

Payload

The webhook receives a POST with Content-Type: application/json and an X-Rebase-Event: run.failed header:

{
  "event": "run.failed",
  "sent_at": "2026-07-11T03:01:02+00:00",
  "run": {
    "id": "8f4c2f9e-...",
    "status": "failed",
    "error": "Cloud Run execution failed: ...",
    "target_type": "workflow",
    "target_id": "1391f0d3-...",
    "project_id": "b7a3c1d4-...",
    "workspace_id": "acme",
    "execution_backend": "prefect_cloud_run_service",
    "trigger_source": "schedule",
    "created_at": "2026-07-11T03:00:00+00:00",
    "started_at": "2026-07-11T03:00:05+00:00",
    "finished_at": "2026-07-11T03:01:00+00:00"
  }
}

trigger_source distinguishes runs fired by a cron schedule ("schedule") from manual or API-triggered runs ("api").

Verify the Signature

When a --webhook-secret is configured, every delivery carries an X-Rebase-Signature header containing an HMAC-SHA256 of the raw request body:

X-Rebase-Signature: sha256=<hex digest>

Verify it before trusting the payload:

import hashlib
import hmac


def verify(body: bytes, signature_header: str, secret: str) -> bool:
    expected = "sha256=" + hmac.new(secret.encode(), body, hashlib.sha256).hexdigest()
    return hmac.compare_digest(expected, signature_header)

Delivery Semantics

• One attempt per failed run, with a 5-second timeout. Delivery failures are logged server-side and never affect the run itself.
• Notifications are workspace-wide. Besides run.failed, the webhook can carry dataset.stale and dataset.fresh events (enable with --on-stale); staleness events are edge-triggered — one per transition, not one per check.