stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity

refactor(policy): merge policy gateway into policy-engine

Browse Source 
- Move 24 gateway source files (endpoints, services, contracts) into engine
  under Endpoints/Gateway/, Services/Gateway/, Contracts/Gateway/ namespaces
- Add gateway DI registrations and endpoint mappings to engine Program.cs
- Add missing project references (StellaOps.Policy.Scoring, DeltaVerdict, Localization)
- Remove HTTP proxy layer (PolicyEngineClient, DPoP, forwarding context not copied)
- Update gateway routes in router appsettings to point to policy-engine
- Comment out policy service in docker-compose, add backwards-compat network alias
- Update services-matrix (gateway build line commented out)
- Update all codebase references: AdvisoryAI, JobEngine, CLI, router tests, helm
- Update docs: OFFLINE_KIT, configuration-migration, gateway guide, port-registry
- Deprecate etc/policy-gateway.yaml.sample with notice
- Eliminates 1 container, 9 HTTP round-trips, DPoP token flow

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
master
2026-04-08 13:19:09 +03:00
parent 9eec100204
commit c1ecc75ace
40 changed files with 12135 additions and 116 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
docs/OFFLINE_KIT.md
View File

@@ -198,12 +198,15 @@ Outputs:
- `telemetry/telemetry-offline-bundle.tar.gz` + `.sha256` — packaged OTLP collector assets for environments without upstream access
- `plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Python/*.sig` (+ `.sha256`) — Cosign signatures for the Python analyzer DLL and manifest
### Policy Gateway configuration bundle
### Policy Engine configuration bundle (formerly Policy Gateway)
- Copy `etc/policy-gateway.yaml` (or the `*.sample` template if you expect operators to override values) into `config/policy-gateway/policy-gateway.yaml` within the staging tree.
- Include the gateway DPoP private key under `secrets/policy-gateway/policy-gateway-dpop.pem` and reference the location inside the manifest notes. Set the permissions explicitly (`chmod 600 secrets/policy-gateway/policy-gateway-dpop.pem`) so only the kit importer can read it; the importer will refuse keys that are broader.
- Document the gateway base URL and activation verification steps in `docs/modules/policy/guides/gateway.md` (bundled alongside the kit). Operators can use those curl snippets to smoke-test pack CRUD once the Offline Kit is imported.
- Ensure the Prometheus snapshot captured during packaging contains `policy_gateway_activation_requests_total` so auditors can reconcile activation attempts performed via the gateway during the validation window.
> **Note:** The Policy Gateway service has been merged into Policy Engine.
> All gateway endpoints (exceptions, deltas, gates, governance, tool-lattice)
> are now served directly by the policy-engine process.
- Copy `etc/policy-engine.yaml` (or the `*.sample` template if you expect operators to override values) into `config/policy-engine/policy-engine.yaml` within the staging tree.
- Document the policy-engine base URL and activation verification steps in `docs/modules/policy/guides/gateway.md` (bundled alongside the kit). Operators can use those curl snippets to smoke-test pack CRUD once the Offline Kit is imported.
- Ensure the Prometheus snapshot captured during packaging contains `policy_engine_activation_requests_total` so auditors can reconcile activation attempts performed during the validation window.
Provide `--cosign-key` / `--cosign-identity-token` (and optional `--cosign-password`) to generate Cosign signatures for both the tarball and manifest.

121
docs/modules/policy/guides/gateway.md
View File

@@ -1,17 +1,22 @@
# Policy Gateway
# Policy Gateway (Merged into Policy Engine)
> **Delivery scope:** `StellaOps.Policy.Gateway` minimal API service fronting Policy Engine pack CRUD + activation endpoints for UI/CLI clients. Sender-constrained with DPoP and tenant headers, suitable for online and Offline Kit deployments.
> **Status:** The `StellaOps.Policy.Gateway` service has been **merged into
> `StellaOps.Policy.Engine`**. All gateway endpoints (exceptions, deltas,
> gates, governance, tool-lattice, pack CRUD, activation) are now served
> directly by the policy-engine process. The separate gateway container,
> HTTP proxy layer, DPoP token flow, and `PolicyEngineClient` have been
> removed.
## 1 · Responsibilities
## 1 - Responsibilities (now handled by Policy Engine)
- Proxy policy pack CRUD and activation requests to Policy Engine while enforcing scope policies (`policy:read`, `policy:author`, `policy:review`, `policy:operate`, `policy:activate`).
- Normalise responses (DTO + `ProblemDetails`) so Console, CLI, and automation receive consistent payloads.
- Guard activation actions with structured logging and metrics so approvals are auditable.
- Support dual auth modes:
- Forwarded caller tokens (Console/CLI) with DPoP proofs + `X-Stella-Tenant` header.
- Gateway client credentials (DPoP) for service automation or Offline Kit flows when no caller token is present.
- Policy pack CRUD and activation endpoints enforcing scope policies (`policy:read`, `policy:author`, `policy:review`, `policy:operate`, `policy:activate`).
- Normalised responses (DTO + `ProblemDetails`) so Console, CLI, and automation receive consistent payloads.
- Structured logging and metrics for activation actions so approvals are auditable.
- Exception management, delta computation, gate evaluation, governance, and tool-lattice endpoints.
## 2 · Endpoints
## 2 - Endpoints
All endpoints below are served by the **policy-engine** service.
| Route | Method | Description | Required scope(s) |
|-------|--------|-------------|-------------------|
@@ -23,116 +28,76 @@
### Response shapes
- Successful responses return camel-case DTOs matching `PolicyPackDto`, `PolicyRevisionDto`, or `PolicyRevisionActivationDto` as described in the Policy Engine API doc (`/docs/api/policy.md`).
- Errors always return RFC 7807 `ProblemDetails` with deterministic fields (`title`, `detail`, `status`). Missing caller credentials now surface `401` with `"Upstream authorization missing"` detail.
- Errors always return RFC 7807 `ProblemDetails` with deterministic fields (`title`, `detail`, `status`).
### Dual-control activation
- **Config-driven.** Set `PolicyEngine.activation.forceTwoPersonApproval=true` when every activation must collect two distinct `policy:activate` approvals. When false, operators can opt into dual-control per revision (`requiresTwoPersonApproval: true`).
- **Defaults.** `PolicyEngine.activation.defaultRequiresTwoPersonApproval` feeds the default when callers omit the checkbox/flag.
- **Statuses.** First approval on a dual-control revision returns `202 pending_second_approval`; duplicate actors get `400 duplicate_approval`; the second distinct approver receives the usual `200 activated`.
- **Audit trail.** With `PolicyEngine.activation.emitAuditLogs` on, Policy Engine emits structured `policy.activation.*` scopes (pack id, revision, tenant, approver IDs, comments) so the gateway metrics/ELK dashboards can show who approved what.
- **Audit trail.** With `PolicyEngine.activation.emitAuditLogs` on, Policy Engine emits structured `policy.activation.*` scopes (pack id, revision, tenant, approver IDs, comments).
#### Activation configuration wiring
- **Helm ConfigMap.** `devops/helm/stellaops/values*.yaml` now include a `policy-engine-activation` ConfigMap. The chart automatically injects it via `envFrom` into both the Policy Engine and Policy Gateway pods, so overriding the ConfigMap data updates the services with no manifest edits.
- **Type safety.** Quote ConfigMap values (e.g., `"true"`, `"false"`) because Kubernetes ConfigMaps carry string data. This mirrors the defaults checked into the repo and keeps `helm template` deterministic.
- **File-based overrides (optional).** The Policy Engine host already probes `/config/policy-engine/activation.yaml`, `../etc/policy-engine.activation.yaml`, and ambient `policy-engine.activation.yaml` files beside the binary. Mounting the ConfigMap as a file at `/config/policy-engine/activation.yaml` works immediately if/when we add a volume.
- **Helm ConfigMap.** `devops/helm/stellaops/values*.yaml` now include a `policy-engine-activation` ConfigMap. The chart automatically injects it via `envFrom` into the Policy Engine pod, so overriding the ConfigMap data updates the service with no manifest edits.
- **Type safety.** Quote ConfigMap values (e.g., `"true"`, `"false"`) because Kubernetes ConfigMaps carry string data.
- **File-based overrides (optional).** The Policy Engine host probes `/config/policy-engine/activation.yaml`, `../etc/policy-engine.activation.yaml`, and ambient `policy-engine.activation.yaml` files beside the binary.
- **Offline/Compose.** Compose/offline bundles can continue exporting `STELLAOPS_POLICY_ENGINE__ACTIVATION__*` variables directly; the ConfigMap wiring simply mirrors those keys for Kubernetes clusters.
## 3 · Authentication & headers
## 3 - Authentication & headers
| Header | Source | Notes |
|--------|--------|-------|
| `Authorization` | Forwarded caller token *or* gateway client credentials. | Caller tokens must include tenant scope; gateway tokens default to `DPoP` scheme. |
| `DPoP` | Caller or gateway. | Required when Authority mandates proof-of-possession (default). Generated per request; gateway keeps ES256/ES384 key material under `etc/policy-gateway-dpop.pem`. |
| `X-Stella-Tenant` | Caller | Tenant isolation header. Forwarded unchanged; gateway automation omits it. |
| `Authorization` | Caller token. | Caller tokens must include tenant scope. |
| `X-Stella-Tenant` | Caller | Tenant isolation header. |
Gateway client credentials are configured in `policy-gateway.yaml`:
> **Note:** The previous DPoP proxy layer (gateway client credentials, `PolicyEngineClient`,
> `PolicyGatewayDpopHandler`) has been removed. Callers authenticate directly with Policy Engine
> using standard StellaOps resource server authentication.
```yaml
policyEngine:
baseAddress: "https://policy-engine.internal"
audience: "api://policy-engine"
clientCredentials:
enabled: true
clientId: "policy-gateway"
clientSecret: "<secret>"
scopes:
- policy:read
- policy:author
- policy:review
- policy:operate
- policy:activate
dpop:
enabled: true
keyPath: "../etc/policy-gateway-dpop.pem"
algorithm: "ES256"
```
> 🔐 **DPoP key** – store the private key alongside Offline Kit secrets; rotate it whenever the gateway identity or Authority configuration changes.
## 4 · Metrics & logging
## 4 - Metrics & logging
All activation calls emit:
- `policy_gateway_activation_requests_total{outcome,source}` – counter labelled with `outcome` (`activated`, `pending_second_approval`, `already_active`, `bad_request`, `not_found`, `unauthorized`, `forbidden`, `error`) and `source` (`caller`, `service`).
- `policy_gateway_activation_latency_ms{outcome,source}` – histogram measuring proxy latency.
- `policy_engine_activation_requests_total{outcome,source}` -- counter labelled with `outcome` (`activated`, `pending_second_approval`, `already_active`, `bad_request`, `not_found`, `unauthorized`, `forbidden`, `error`) and `source` (`caller`, `service`).
Structured logs (category `StellaOps.Policy.Gateway.Activation`) include `PackId`, `Version`, `Outcome`, `Source`, and upstream status code for audit trails.
Structured logs (category `StellaOps.Policy.Engine.Activation`) include `PackId`, `Version`, `Outcome`, `Source`, and status code for audit trails.
## 5 · Sample `curl` workflows
## 5 - Sample `curl` workflows
Assuming you already obtained a DPoP-bound access token (`$TOKEN`) for tenant `acme`:
Assuming you already obtained an access token (`$TOKEN`) for tenant `acme`:
```bash
# Generate a DPoP proof for GET via the CLI helper
DPoP_PROOF=$(stella auth dpop proof \
--htu https://gateway.example.com/api/policy/packs \
--htm GET \
--token "$TOKEN")
curl -sS https://gateway.example.com/api/policy/packs \
-H "Authorization: DPoP $TOKEN" \
-H "DPoP: $DPoP_PROOF" \
-H "Authorization: Bearer $TOKEN" \
-H "X-Stella-Tenant: acme"
# Draft a new revision
DPoP_PROOF=$(stella auth dpop proof \
--htu https://gateway.example.com/api/policy/packs/policy.core/revisions \
--htm POST \
--token "$TOKEN")
curl -sS https://gateway.example.com/api/policy/packs/policy.core/revisions \
-H "Authorization: DPoP $TOKEN" \
-H "DPoP: $DPoP_PROOF" \
-H "Authorization: Bearer $TOKEN" \
-H "X-Stella-Tenant: acme" \
-H "Content-Type: application/json" \
-d '{"version":5,"requiresTwoPersonApproval":true,"initialStatus":"Draft"}'
# Activate revision 5 (returns 202 when awaiting the second approver)
DPoP_PROOF=$(stella auth dpop proof \
--htu https://gateway.example.com/api/policy/packs/policy.core/revisions/5:activate \
--htm POST \
--token "$TOKEN")
curl -sS https://gateway.example.com/api/policy/packs/policy.core/revisions/5:activate \
-H "Authorization: DPoP $TOKEN" \
-H "DPoP: $DPoP_PROOF" \
-H "Authorization: Bearer $TOKEN" \
-H "X-Stella-Tenant: acme" \
-H "Content-Type: application/json" \
-d '{"comment":"Rollout baseline"}'
```
For air-gapped environments, bundle `policy-gateway.yaml` and the DPoP key in the Offline Kit (see `/docs/OFFLINE_KIT.md` §5.7).
## 6 - Offline Kit guidance
> **DPoP proof helper:** Use `stella auth dpop proof` to mint sender-constrained proofs locally. The command accepts `--htu`, `--htm`, and `--token` arguments and emits a ready-to-use header value. Teams maintaining alternate tooling (for example, `scripts/make-dpop.sh`) can substitute it as long as the inputs and output match the CLI behaviour.
- Include `policy-engine.yaml.sample` and the resolved runtime config in the Offline Kit's `config/` tree.
- When building verification scripts, use the policy-engine endpoints above. The Offline Kit validator expects `policy_engine_activation_requests_total` metrics in the Prometheus snapshot.
## 6 · Offline Kit guidance
## 7 - Backwards compatibility
- Include `policy-gateway.yaml.sample` and the resolved runtime config in the Offline Kit’s `config/` tree.
- Place the DPoP private key under `secrets/policy-gateway-dpop.pem` with restricted permissions; document rotation steps in the manifest.
- When building verification scripts, use the gateway endpoints above instead of hitting Policy Engine directly. The Offline Kit validator now expects `policy_gateway_activation_requests_total` metrics in the Prometheus snapshot.
- The `policy-gateway.stella-ops.local` hostname is preserved as a network alias on the policy-engine container in both docker-compose and hosts files. Existing configurations referencing `policy-gateway.stella-ops.local` will continue to work.
- The `STELLAOPS_POLICY_GATEWAY_URL` environment variable has been removed from service defaults. Services that previously used it should use `STELLAOPS_POLICY_ENGINE_URL` instead.
## 7 · Change log
## 8 - Change log
- **2025-10-27 – Sprint 18.5**: Initial gateway bootstrap + activation metrics + DPoP client credentials.
- **2026-04-08 -- Gateway merge**: Gateway merged into Policy Engine. Separate container, HTTP proxy layer, DPoP flow, and `PolicyEngineClient` removed. All endpoints served directly by policy-engine.
- **2025-10-27 -- Sprint 18.5**: Initial gateway bootstrap + activation metrics + DPoP client credentials.

7
docs/operations/configuration-migration.md
View File

@@ -163,10 +163,13 @@ authority:
- ../../etc/certificates/signing:/app/etc/signing:ro
```
### Policy Gateway
### Policy Engine (formerly Policy Gateway)
> **Note:** The `policy-gateway` service has been merged into `policy-engine`.
> Mount policy configuration on the `policy-engine` service instead.
```yaml
policy-gateway:
policy-engine:
volumes:
- ../../etc/policy:/app/etc/policy:ro
```