Neural Inverse Govern / Streams
Event streams that check what is written and who may read it.
Durable, ordered streams on Govern's own partitioned log, with consumer groups, compaction, transactions, DLP, classification and schemas, and endpoints that give producers and consumers their own governed way in.
- Runs on
- Govern's own log, no external system
- Durability
- Local disk, replicated or object storage
- Delivery
- At least once, transactions all or nothing
- Governance
- DLP, classification and schemas on write
- Endpoints
- IAM, JWT, custom, CEL and mTLS authorizers
Overview
Streams is a managed stream service: named, durable logs of records split into partitions, written by producers and read by consumer groups that track their position. Its brokers keep each partition on their own volume and coordinate through PostgreSQL, so it needs no external system and works air-gapped. Records are checked as they are written, against DLP, the stream's schema and its classification, and endpoints give outside producers and consumers a hostname with their own authorizers and protection.
What it does
What Streams provides.
The stream service and its endpoints, as they run today.
Streams and partitionsCreate streams with partitions, retention, a record size limit, classification and tags. Records with the same key keep their order; partitions can be added, with a warning that keys are remapped.
Health and durabilityHealth is computed on the server from the partitions' leaders and lists its reasons. Durability is local disk with fsync, replicated across zones, or acknowledged only once in object storage.
Write and readWrite batches of records and read by offset through the leader broker, with per-minute traffic metrics. Every write action honours an idempotency key, so a retried request is not written twice.
Consumer groupsCommitted offsets per partition with a generation: a reset moves the group on, and a commit from an older generation is refused. Lag per partition is computed on the server; resets go to earliest, latest or a time.
Managed membershipConsumers join a group and are assigned a balanced, sticky share of the partitions. Members that stop heartbeating are removed and the group rebalances; delivery is at least once.
Schema registryA stream can require a JSON Schema subject; records that do not match its latest version are refused. Backward, forward and full compatibility are checked conservatively: an incompatible change is never accepted.
DLP on produceAllow, redact or block per stream, with the account's secret, personal and health data patterns and custom patterns. If the patterns cannot be loaded, nothing is written.
Classification ceilingsEvery record carries the classification it was written under. Reading confidential or restricted records needs its own permission, and records above the reader's level are withheld.
Data explorerBrowse records from the console by offset or time, redacted by DLP before they are shown. Binary values are never sent to the browser.
CompactionDelete, compact or both: keep the latest record of every key, with tombstones to remove a key and a minimum lag before compaction.
TransactionsWrite up to 500 records to up to ten streams as one transaction under a transactional ID. Readers see all of it or none of it, and a newer writer fences an older one.
EndpointsA hostname of its own per endpoint, serving only the data actions, streams and classification it allows. Custom domains need a domain verified for the account and are served by SNI with certificates reloaded without a restart.
Endpoint authorizersGovern IAM, JWT from your identity provider, your own HTTP decision service, CEL expressions or client certificates. An open endpoint is refused unless a rate-limited usage plan covers it.
Usage plans and protectionRates, bursts, request sizes and daily or monthly quotas per caller, held across replicas, plus address allow and deny lists and temporary blocks for request spikes and credential stuffing.
AI governance of payloadsOff, monitor or block per endpoint, with detectors for prompt injection, sensitive data and secrets. In block mode, sensitive data can be replaced with tokens instead; secrets are always refused.
Encryption and receiptsRecord values sealed in segments under the account's managed key or your own key, refused while the key is revoked. Each batch gets a signed receipt in a Merkle proof log that can be verified offline.
How it works
How a record written through an endpoint is checked.
Each check runs before the record is stored; any refusal stops it.
- AuthorizeThe endpoint's authorizer decides the caller; the decision is audited.
- LimitThe caller's usage plan and the endpoint's protection apply.
- GovernAI governance inspects the payload, then the stream's DLP and schema check it.
- SealThe value is sealed under the account's key when the endpoint encrypts.
- StoreThe leader writes it with the stream's durability.
- ReceiptThe batch receives a signed receipt in the proof log.
Who uses it
Built for every role that answers for AI.
Platform engineersRun the brokers, create streams and set durability, retention and quotas.
Application developersProduce and consume through the API, the Go client or an endpoint, with consumer groups and transactions.
Security teamsSet classification, DLP and AI governance, and choose authorizers and encryption for endpoints.
AuditorsVerify batch receipts offline and read every control-plane change and authorizer decision.
Enforced and recorded
Checked as records are written and read.
Governance is enforced on the server and fails closed.
Enforced
DLP and schemasA record that breaks a block policy or its schema is refused; when DLP cannot scan, nothing of the request is written.
ReadersConfidential and restricted records need their own read permission; records above the reader's level are withheld.
EndpointsEvery request passes an authorizer, a usage plan and protection; non-IAM callers stay within the endpoint's actions, streams and classification.
KeysWhile the account's stream key is revoked, writing and reading encrypted records is refused on every path.
Recorded
Control planeStream, consumer group, schema, tag and endpoint changes, in the transaction of the change.
Authorizer decisionsEach denial with its reason and source address; allows aggregated per caller and minute.
Data planeRecords are counted per minute and metered rather than audited one by one, as stream services do.
Self-hostedBrokers run as part of your installation and coordinate through its PostgreSQL. They are off until you deploy them.
Air-gappedObject durability and tiered storage can use a local directory instead of a bucket.
Hosted by usStreams in the region that serves your account, with limits in service quotas.
Get started
Set it up with the documentation.
What's next
Being built now.
These are in development and not part of the product today.
Endpoint hostnames in DNSPublishing each endpoint's hostname through Govern's own DNS service instead of a wildcard record.
Talk to our team.
Neural Inverse is available by request. We work with your engineering, security and procurement teams from the first conversation.