[P0-21] Define distributed target lease and fencing semantics #29

New issue

Closed

opened 2026-07-22 00:13:42 +00:00 by lost-rob0t · 0 comments

lost-rob0t commented

2026-07-22 00:13:42 +00:00

(Migrated from github.com)

Objective

Define the locking system used by HTTP clients, service instances, and actors before implementing a KV backend.

Lease identity

Canonical lock key must include the security/ownership boundary, for example tenant or program, normalized target identity, actor/workflow, and optional operation class. Do not lock on raw unnormalized user input.

Required lease record

lock key and target identity
lease id
owner principal/client and service instance
monotonically increasing fencing token
acquired, renewed, and expiry timestamps
TTL and maximum lifetime
execution/job id, trace id, and metadata
state: active, expired, released, revoked

Required operations

acquire-if-free
renew by matching lease id/owner/fencing token
release by compare-and-delete
inspect
list by owner/target/program
privileged force-release/revoke

Semantics

TTL-based leases, not permanent mutexes
stale holders remain unable to commit work after expiry because downstream writes validate fencing tokens
retries are idempotent by request id
clock assumptions are explicit; backend/server time is authoritative
lease loss is surfaced to the actor/job immediately
no blocking acquisition without a deadline

Acceptance criteria

A design document specifies state transitions, races, failure modes, stale-client behavior, split-brain assumptions, fencing enforcement points, HTTP status/error shapes, and audit events.

## Objective Define the locking system used by HTTP clients, service instances, and actors before implementing a KV backend. ## Lease identity Canonical lock key must include the security/ownership boundary, for example tenant or program, normalized target identity, actor/workflow, and optional operation class. Do not lock on raw unnormalized user input. ## Required lease record - lock key and target identity - lease id - owner principal/client and service instance - monotonically increasing fencing token - acquired, renewed, and expiry timestamps - TTL and maximum lifetime - execution/job id, trace id, and metadata - state: active, expired, released, revoked ## Required operations - acquire-if-free - renew by matching lease id/owner/fencing token - release by compare-and-delete - inspect - list by owner/target/program - privileged force-release/revoke ## Semantics - TTL-based leases, not permanent mutexes - stale holders remain unable to commit work after expiry because downstream writes validate fencing tokens - retries are idempotent by request id - clock assumptions are explicit; backend/server time is authoritative - lease loss is surfaced to the actor/job immediately - no blocking acquisition without a deadline ## Acceptance criteria A design document specifies state transitions, races, failure modes, stale-client behavior, split-brain assumptions, fencing enforcement points, HTTP status/error shapes, and audit events.