Govern writes every change and every enforced decision to one append-only audit ledger. A change and its audit entry are written in the same database transaction: if the entry cannot be written, the change does not happen.
How it is protected
Hash chains per organisation
Each organisation has its own chain; entries without an organisation (platform entries) have one more. Every entry carries an HMAC-SHA256 hash over its content and the previous entry's hash, keyed with NI_AUTH_AUDIT_HMAC_KEY. Removing, reordering or changing an entry breaks the chain.
For throughput, an organisation's chain runs in lanes (8 by default, NI_AUDIT_CHAIN_LANES). Each lane is an HMAC chain of its own: every entry carries its lane and its position in the lane (chain_lane, chain_seq), both inside the signed content. Within a lane entries are totally ordered; across lanes the order is the entry id and time.
An organisation that needs one order for every entry sets one lane:
PUT /api/v1/audit-logs/chain-lanes
{"lanes": 1}- Permission
audit.chain:configure(Owner and Admin), signed-in users only, never an API key.GET /api/v1/audit-logs/chain-lanes(audit:read) shows the count in effect and any pending change. - The change is audited and applied at the next epoch seal. Existing entries are never touched or reordered; a retired lane keeps its entries and still verifies.
- One lane serialises the organisation's writes: choose it only for an organisation that stays well below about 10,000 audited actions per second.
Signed epochs and checkpoints
- Every minute, the heads of the chains that moved are sealed into an epoch: a Merkle root over (chain, sequence, head hash), linked to the previous epoch by hash and signed with the audit signing key (Ed25519). Someone holding the HMAC key therefore cannot rewrite one organisation's chain without breaking the signed record.
- Every five minutes, a signed Merkle checkpoint is made over the hashes of the new entries.
The audit signing key is created at first start, sealed under KMS_MASTER_KEY, and its public half, with every retired key, is published at /.well-known/keystone-audit-keys.json. A super admin rotates it with POST /api/v1/audit/signing-keys/rotate; retired keys stay published so earlier signatures keep verifying.
Immutable rows
A database trigger refuses any update or delete of audit rows. The one exception is the verified move of old entries to the cold tier, inside a transaction that names a sealed cold object holding the row (Retention and tiering). Chain heads only move forward and cannot be deleted. The ledger is partitioned by month.
What each entry records
The organisation, actor, action, resource, outcome, client address, user agent, request id, the region that wrote it, the project, metadata, and the previous and own hash. The client address is the verified end-user address (see Client IP addresses), never a header a client could forge. When the audit PII encryptor is configured and encrypting an entry's PII fails, the entry is not written in plaintext: the write fails, and so does the change.
Read the ledger
| Route | Permission | What |
|---|---|---|
GET /api/v1/audit-logs | audit:read | List and filter the organisation's hot tier: action, action_prefix, resource_type, outcome, project_id, from, to |
GET /api/v1/organizations/{id}/audit-logs | audit:read | The same for a named organisation |
GET /api/v1/audit-logs/query | audit:read | Query both the hot and cold tiers |
GET /api/v1/audit-logs/sessions, /sessions/{id}/replay | audit:read | Agent sessions and their step-by-step replay |
GET /api/v1/audit-logs/endpoint-activity | audit:read | Endpoint agent activity: category awareness, staging or tamper |
GET /api/v1/audit-logs/storage-stats | audit:read | Entries, bytes and time span of each tier |
action_prefix is 1 to 64 characters of [a-z0-9._-] ending with ., for example context. or client., and cannot be combined with action.
Query both tiers
GET /api/v1/audit-logs/query?from=&to=&query= runs a pipeline in the style of CloudWatch Logs Insights over one organisation's entries in [from, to):
fields @timestamp, action, outcome
| filter action like /^models\./ and outcome = "failure"
| stats count(*) as failures by action
| sort failures desc
| limit 20| Element | Supported |
|---|---|
| Commands | fields, filter, stats ... [by ...], sort <field> [asc|desc], limit <n> (at most 10,000) |
| Operators | =, !=, <, <=, >, >=, like "substring", like /RE2/, in [...], and, or, not, parentheses |
| Aggregates | count(*), count_distinct(f), min(f), max(f) |
| Fields | @timestamp, @id, @tier, organization_id, actor_type, actor_user_id, action, resource_type, resource_id, outcome, ip_address, user_agent, request_id, idempotency_key, region, project_id, assistant_proposal_id, previous_hash, entry_hash, metadata.<key> |
A query that would read more than 2,000,000 entries or 1 GiB of cold objects, span more than 366 days or produce more than 100,000 groups is refused whole (422 query_too_wide), never answered partially. A server runs at most 4 queries at once (429 query_busy with Retry-After). A cold object that does not match its manifest refuses the query (409 cold_tier_integrity).
Verify in place
| Route | What |
|---|---|
GET /api/v1/audit-logs/verify?from=&to= | Verify up to 100,000 entries of the caller's organisation now |
POST /api/v1/audit-logs/verifications | {"from", "to"}: verify a longer range as a job |
GET /api/v1/audit-logs/verifications, /verifications/{id} | The organisation's jobs and their results |
from and to are RFC 3339 times; from defaults to the start of the log and to to now. The answer is {"verified", "first_break", "range", "checkpoint", "cold_links"}. A verification reads the organisation's chain lane by lane from its start (or from its last entry in the cold tier) to its recorded head, checks signatures, links, dense sequence numbers and that the end matches the head (so entries cut from the end are found). It reads as of one moment, so a chain verified while appends commit is not reported broken. Every run is audited as audit.chain.verified.
For proof that does not depend on the server, export the chain and verify it offline: Verify offline.
Platform checkpoints
Super admins list the five-minute checkpoints, which span every organisation, with GET /api/v1/admin/audit/checkpoints, and re-check one with GET /api/v1/admin/audit/checkpoints/{checkpoint_id}/verify (signature, Merkle root and event count recomputed from the ledger). The checkpoint signature covers checkpoint_id, merkle_root, start_event_id and event_count; it does not cover end_event_id or the window timestamps.