Retrieval and attribution

Retrieve

POST /api/v1/context/retrieve
Authorization: Bearer <access token>
X-Keystone-Client: neuralinverse-editor/1.4.2
Content-Type: application/json

{
  "query": "how are refunds authorised",
  "spaceId": "<project space id>",
  "tokenBudget": 4096,
  "maxPerLevel": 5,
  "lensId": "<optional>",
  "sessionId": "<optional pair session>",
  "excludeRelated": false,
  "sourceKinds": ["codebase", "issues"]
}

The answer carries chunks, sourcesSummary, tokensUsed, attributionId and requestId (the key under which the attribution record is stored). The caller needs context:space:search on each space that contributes; organisation and user always come from the token, and a body naming another is refused.

GET /api/v1/context/capabilities tells a first-party app, without administrator rights, whether retrieval is served and its maximum and default budgets.

Ranking

Search runs only over the authorised spaces, capped per level, and only over each source's current index version.

• Hybrid score. score = (0.7 × vector + 0.3 × text) × weight, where vector is 1 minus the cosine distance and text a normalised full-text rank. A chunk found by both searches never ranks below one found by either alone. Without an embeddings endpoint, search is full-text only.
• Level weights. The weight of the space level the chunk came from: project 1.20, team 1.00, group 1.00, department 0.92, organisation 0.85, personal 0.85 (subproject 1.40 and business unit 0.88 are defined but unused). An operator changes the defaults with KEYSTONE_CONTEXT_LEVEL_WEIGHTS (level=weight,...). An organisation overrides any level with GET/PUT /api/v1/context/weights ({"overrides": {"project": 1.3}}; context:space:admin; weights in (0, 10] with at most 4 decimals; {} resets).
• Related code. For the top 3 ranked code chunks, up to 2 callers, callees and tests each (at most 6 in total) are appended after the ranked chunks with score 0 and metadata.relatedTo and relatedRelation (calls, called_by, tested_by). They come from the same authorised source, pass DLP and only use budget left by the ranked chunks. Call graphs exist for Go only.
• Cache. The search step is cached for 60 seconds per organisation, authorised space set and query. Authorisation, DLP and the budget run on every request; a cache hit of a source that was deleted or re-indexed is discarded.

Embeddings

Vector search needs an OpenAI-compatible embeddings endpoint (KEYSTONE_CONTEXT_EMBEDDINGS_URL, _MODEL, _DIMENSIONS; see Settings). The default search backend computes exact cosine distance in SQL. The pgvector backend needs the extension installed in your PostgreSQL image and created as a superuser; with KEYSTONE_CONTEXT_SEARCH_BACKEND=pgvector and no extension, retrieval fails closed with 503.

Text sent to an embeddings endpoint can leave your control, so the classification model applies before anything is sent:

• Sources whose DLP action is drop or block_source (confidential, restricted, regulated projects) are never sent to any embeddings endpoint; they are searchable through full-text only.
• Confidential and restricted sources are never sent to an external endpoint. An endpoint is external unless its host is loopback or listed in KEYSTONE_CONTEXT_EMBEDDINGS_INTERNAL_HOSTS, or KEYSTONE_CONTEXT_EMBEDDINGS_EXTERNAL=false says otherwise.
• Every other chunk, and the query text, passes the organisation's DLP redaction first; only the redacted text is embedded.
• A DLP scan failure stops the run and sends nothing.

GET /api/v1/context/status (organisation administrators) reports the search backend, whether embeddings are configured, the model, its dimensions and the endpoint's locality (in_deployment, external, none). It never returns URLs, keys or error text.

Attribution

Every successful retrieval is recorded in the observability module's HMAC-chained evidence chain, as one record of category context_attribution:

• the request id, the user, the space, the lens and the source ids;
• per chunk: chunk id, source id, the SHA-256 of the content delivered and the source's index version;
• the client: clientId (the OAuth client of the caller's token, verified), and clientName and clientVersion from the X-Keystone-Client: name/version header (declared by the app, recorded but never trusted; a malformed header is dropped);
• for team memory chunks, the entry and its accepted revision.

The record is written asynchronously after the retrieval. If the event cannot be queued, the retrieval answers 503 context.attribution.failed.

Each is visible to the person who retrieved, or to a context:space:admin who can read the space; anyone else gets 404 context.attribution.not_found. Because the record is written asynchronously, a call right after a retrieval can answer 404: retry for a few seconds.

The evidence chain is verified daily and on demand with GET /api/v1/observability/chain/verify, and listed with GET /api/v1/observability/compliance-evidence (see Evidence).

Joining a model call to its retrieval

When the Editor, the Agents app or the CLI sends a model request through the Govern model gateway, it names the retrieval it used:

None of these reaches the model provider. When the call is answered, the gateway appends a context_attribution_link record to the same chain, with the gateway request id as the response id, and returns the outcome in X-Keystone-Context-Link:

A failed link never fails or changes the model call; it is audited as models.gateway.context_link. GET /api/v1/models/logs?context_request_id= lists the model calls that used a retrieval.