Webhook Patterns

A webhook is an HTTP callback. The producer (Stripe, GitHub, your application) sends an HTTP POST to a consumer-provided URL when something happens. The consumer responds with HTTP 2xx to acknowledge.

Webhooks are the standard pattern for server-to-server event notifications. The mechanics are simple; the operational details — delivery guarantees, retries, signatures — are where the complexity lives.

This page is about the patterns from both sides: designing webhooks that work, and consuming webhooks reliably.

Delivery guarantees

Webhooks are at-least-once. The producer sends; if delivery fails (timeout, 5xx), the producer retries. Eventually most events are delivered; sometimes events are delivered multiple times.

Consumers must handle:

Duplicate deliveries (idempotency)
Out-of-order arrivals
Eventual consistency (events may be delayed)

Senders should not assume:

Delivery order
Exactly-once delivery
Synchronous processing

Retry strategy

Standard retry approach for senders:

Initial delivery: immediate
5xx response: retry with exponential backoff
4xx response: don't retry (consumer error; not transient)
Timeout: retry like 5xx

Backoff schedule example: 1m, 5m, 15m, 1h, 6h, 24h, 48h, 72h. After ~3 days of retries, mark as failed.

Document the retry policy. Consumers need to know how long to expect retries to continue.

Signature verification

Webhooks must be signed; consumers must verify. Without signatures, anyone can POST to the webhook URL and trigger consumer logic.

Standard pattern (Stripe, GitHub, Slack):

1. Producer creates HMAC-SHA256 of payload using shared secret
2. Producer includes signature in HTTP header
3. Consumer recomputes HMAC and verifies equality

POST /webhooks/orders
X-Signature: sha256=<hex-encoded-hmac>
Content-Type: application/json

{ "event": "order.shipped", ... }

Consumer code:

expected = hmac.new(secret, payload, hashlib.sha256).hexdigest()
if not hmac.compare_digest(expected, received):
    return 401

compare_digest is constant-time; == is timing-attack-vulnerable.

Replay attacks

Even with signatures, an attacker who captures a valid webhook can replay it. Defenses:

Timestamp + tolerance: include timestamp in signature; reject events older than ~5 minutes
Nonce tracking: server stores recent event IDs; rejects duplicates within window

Stripe's pattern: signature includes timestamp; consumer rejects if |now - timestamp| > 300s.

Consumer requirements

Reliable webhook consumers need:

Quick acknowledgment

Respond with 2xx as soon as the webhook is received and stored — typically under 5 seconds. Producers often timeout faster than that.

def handle_webhook():
    verify_signature()
    enqueue_for_processing()  # async work
    return 200  # ack now

Don't do the actual work synchronously; enqueue and process asynchronously. Otherwise slow processing causes producer retries (and duplicate work).

Idempotency

Same event delivered twice should not double-process. Use the event ID:

if not seen_event_ids.exists(event_id):
    seen_event_ids.add(event_id, ttl=7days)
    process(event)

See IdempotencyPatterns.

Persistent queue

Don't process events directly from the HTTP handler. Persist to a queue (database, Kafka, SQS) and process from there. If processing fails, the event is preserved for retry.

Logging and monitoring

Log every received event with its ID
Alert on delivery failures (4xx responses, parsing errors)
Track processing latency

Disaster recovery

If your webhook endpoint is down, events are eventually lost (after producer retry exhaustion). Mitigations:

High availability (multiple endpoint instances behind load balancer)
Event log endpoint on the producer side ("get all events since X") for backfill
Manual replay tools

Producer responsibilities

Idempotency keys

Even with at-least-once delivery, include an event ID:

{
    "event_id": "evt_8d4f...",
    "event_type": "order.shipped",
    "data": {...}
}

Consumers use the event ID for deduplication.

Event ordering

Within a single resource (one order's events), events should be sequential. Events to different resources can be parallel.

If strict ordering matters, include a sequence number or rely on retry-with-backoff to maintain order.

Webhook management

Provide a UI or API for:

Subscribing to events (which event types)
Configuring endpoint URLs
Viewing delivery history
Manually retrying failed deliveries
Disabling/re-enabling a webhook

The Stripe/GitHub-style management UI sets the bar.

Versioning

Events may evolve. Version them:

Include a version field in payload
Document what each version means
Maintain compatibility for old subscribers

Or: namespace event types per major version.

Common patterns to avoid

Synchronous processing in the handler. Causes producer retries.
No signature verification. Anyone can POST.
No idempotency. Duplicate processing.
Trusting timestamps from the client without signing them. Attacker forges them.
Single event per webhook. Sometimes batching events is more efficient.
No delivery history. Consumers can't audit; producers can't debug.

Common failure patterns

Slow consumer. Producer retries; duplicates pile up.
No replay protection. Captured webhooks can be replayed.
Drop on signature mismatch silently. Returns 401 to producer; producer retries.
No webhook documentation. Consumers can't anticipate failure modes.
Brittle event schema. Adding fields breaks consumers; document forward-compatibility.