Add Authority Advisory AI and API Lifecycle Configuration
- Introduced AuthorityAdvisoryAiOptions and related classes for managing advisory AI configurations, including remote inference options and tenant-specific settings. - Added AuthorityApiLifecycleOptions to control API lifecycle settings, including legacy OAuth endpoint configurations. - Implemented validation and normalization methods for both advisory AI and API lifecycle options to ensure proper configuration. - Created AuthorityNotificationsOptions and its related classes for managing notification settings, including ack tokens, webhooks, and escalation options. - Developed IssuerDirectoryClient and related models for interacting with the issuer directory service, including caching mechanisms and HTTP client configurations. - Added support for dependency injection through ServiceCollectionExtensions for the Issuer Directory Client. - Updated project file to include necessary package references for the new Issuer Directory Client library.
This commit is contained in:
master
@@ -19,16 +19,22 @@ Attestor converts signed DSSE evidence from the Signer into transparency-log pro
|
||||
- `StellaOps.PolicyEvaluation@1`, `StellaOps.RiskProfileEvidence@1`
|
||||
All predicates capture subjects, issuer metadata, policy context, materials, optional witnesses, and versioned schemas. Unsupported predicates return `422 predicate_unsupported`.
|
||||
|
||||
## Trust & envelope model
|
||||
- DSSE envelopes are canonicalised, hashed, and stored alongside the Rekor UUID, index, and proof.
|
||||
- Signature modes span keyless (Fulcio), keyful (KMS/HSM), and hardware-backed (FIDO2). Multiple signatures are supported per envelope.
|
||||
- Proofs include Merkle inclusion path, checkpoint metadata, optional witness endorsements, and cached verification verdicts.
|
||||
- CAS/object storage retains envelopes + provenance for later replay; Rekor backends may be primary plus mirrors.
|
||||
|
||||
## UI, CLI, and SDK workflows
|
||||
- **Console:** Evidence browser, verification reports, chain-of-custody graph, issuer/key management, attestation workbench, and bulk verification flows.
|
||||
- **CLI / SDK:** `stella attest sign|verify|list|fetch|key` commands plus language SDKs to integrate build pipelines and offline verification scripts.
|
||||
- **Policy Studio:** Verification policies author required predicate types, issuers, witness requirements, and freshness windows; simulations show enforcement impact.
|
||||
## Trust & envelope model
|
||||
- DSSE envelopes are canonicalised, hashed, and stored alongside the Rekor UUID, index, and proof.
|
||||
- Signature modes span keyless (Fulcio), keyful (KMS/HSM), and hardware-backed (FIDO2). Multiple signatures are supported per envelope.
|
||||
- Proofs include Merkle inclusion path, checkpoint metadata, optional witness endorsements, and cached verification verdicts.
|
||||
- CAS/object storage retains envelopes + provenance for later replay; Rekor backends may be primary plus mirrors.
|
||||
|
||||
## Security hardening
|
||||
- `attestor.write`, `attestor.verify`, and `attestor.read` scopes are enforced per endpoint; verify/list flows accept read/verify scopes while submissions remain write-only.
|
||||
- JSON content-type is mandatory; malformed content returns `415 unsupported_media_type`.
|
||||
- DSSE payloads are capped at 2 MiB (configurable), certificate chains at six entries, and each envelope may carry up to six signatures to contain parsing abuse.
|
||||
- All verification/list APIs share the token-bucket rate limiter (`quotas.perCaller`) in addition to the existing submission limiter.
|
||||
|
||||
## UI, CLI, and SDK workflows
|
||||
- **Console:** Evidence browser, verification reports, chain-of-custody graph, issuer/key management, attestation workbench, and bulk verification flows.
|
||||
- **CLI / SDK:** `stella attest sign|verify|list|fetch|key` commands plus language SDKs to integrate build pipelines and offline verification scripts.
|
||||
- **Policy Studio:** Verification policies author required predicate types, issuers, witness requirements, and freshness windows; simulations show enforcement impact.
|
||||
|
||||
## Storage, offline & air-gap posture
|
||||
- MongoDB stores entry metadata, dedupe keys, and audit events; object storage optionally archives DSSE bundles.
|
||||
|
||||
@@ -45,17 +45,23 @@ Trust boundary: **Only the Signer** is allowed to call submission endpoints; enf
|
||||
- `StellaOps.BuildProvenance@1`
|
||||
- `StellaOps.SBOMAttestation@1`
|
||||
- `StellaOps.ScanResults@1`
|
||||
- `StellaOps.PolicyEvaluation@1`
|
||||
- `StellaOps.VEXAttestation@1`
|
||||
- `StellaOps.RiskProfileEvidence@1`
|
||||
- `StellaOps.PolicyEvaluation@1`
|
||||
- `StellaOps.VEXAttestation@1`
|
||||
- `StellaOps.RiskProfileEvidence@1`
|
||||
|
||||
Each predicate embeds subject digests, issuer metadata, policy context, materials, and optional transparency hints. Unsupported predicates return `422 predicate_unsupported`.
|
||||
|
||||
> **Golden fixtures:** Deterministic JSON statements for each predicate live in `src/Attestor/StellaOps.Attestor.Types/samples`. They are kept stable by the `StellaOps.Attestor.Types.Tests` project so downstream docs and contracts can rely on them without drifting.
|
||||
|
||||
Each predicate embeds subject digests, issuer metadata, policy context, materials, and optional transparency hints. Unsupported predicates return `422 predicate_unsupported`.
|
||||
|
||||
### Envelope & signature model
|
||||
- DSSE envelopes canonicalised (stable JSON ordering) prior to hashing.
|
||||
- Signature modes: keyless (Fulcio cert chain), keyful (KMS/HSM), hardware (FIDO2/WebAuthn). Multiple signatures allowed.
|
||||
- Rekor entry stores bundle hash, certificate chain, and optional witness endorsements.
|
||||
- Archive CAS retains original envelope plus metadata for offline verification.
|
||||
### Envelope & signature model
|
||||
- DSSE envelopes canonicalised (stable JSON ordering) prior to hashing.
|
||||
- Signature modes: keyless (Fulcio cert chain), keyful (KMS/HSM), hardware (FIDO2/WebAuthn). Multiple signatures allowed.
|
||||
- Rekor entry stores bundle hash, certificate chain, and optional witness endorsements.
|
||||
- Archive CAS retains original envelope plus metadata for offline verification.
|
||||
- Envelope serializer emits **compact** (canonical, minified) and **expanded** (annotated, indented) JSON variants off the same canonical byte stream so hashing stays deterministic while humans get context.
|
||||
- Payload handling supports **optional compression** (`gzip`, `brotli`) with compression metadata recorded in the expanded view and digesting always performed over the uncompressed bytes.
|
||||
- Expanded envelopes surface **detached payload references** (URI, digest, media type, size) so large artifacts can live in CAS/object storage while the canonical payload remains embedded for verification.
|
||||
- Payload previews auto-render JSON or UTF-8 text in the expanded output to simplify triage in air-gapped and offline review flows.
|
||||
|
||||
### Verification pipeline overview
|
||||
1. Fetch envelope (from request, cache, or storage) and validate DSSE structure.
|
||||
@@ -151,11 +157,53 @@ Indexes:
|
||||
|
||||
## 4) APIs
|
||||
|
||||
### 4.1 Submission
|
||||
|
||||
`POST /api/v1/rekor/entries` *(mTLS + OpTok required)*
|
||||
|
||||
* **Body**: as above.
|
||||
### 4.1 Signing
|
||||
|
||||
`POST /api/v1/attestations:sign` *(mTLS + OpTok required)*
|
||||
|
||||
* **Purpose**: Deterministically wrap Stella Ops payloads in DSSE envelopes before Rekor submission. Reuses the submission rate limiter and honours caller tenancy/audience scopes.
|
||||
* **Body**:
|
||||
|
||||
```json
|
||||
{
|
||||
"keyId": "signing-key-id",
|
||||
"payloadType": "application/vnd.in-toto+json",
|
||||
"payload": "<base64 payload>",
|
||||
"mode": "keyless|keyful|kms",
|
||||
"certificateChain": ["-----BEGIN CERTIFICATE-----..."],
|
||||
"artifact": {
|
||||
"sha256": "<subject sha256>",
|
||||
"kind": "sbom|report|vex-export",
|
||||
"imageDigest": "sha256:...",
|
||||
"subjectUri": "oci://..."
|
||||
},
|
||||
"logPreference": "primary|mirror|both",
|
||||
"archive": true
|
||||
}
|
||||
```
|
||||
|
||||
* **Behaviour**:
|
||||
* Resolve the signing key from `attestor.signing.keys[]` (includes algorithm, provider, and optional KMS version).
|
||||
* Compute DSSE pre‑authentication encoding, sign with the resolved provider (default EC, BouncyCastle Ed25519, or File‑KMS ES256), and add static + request certificate chains.
|
||||
* Canonicalise the resulting bundle, derive `bundleSha256`, and mirror the request meta shape used by `/api/v1/rekor/entries`.
|
||||
* Emit `attestor.sign_total{result,algorithm,provider}` and `attestor.sign_latency_seconds{algorithm,provider}` metrics and append an audit row (`action=sign`).
|
||||
* **Response 200**:
|
||||
|
||||
```json
|
||||
{
|
||||
"bundle": { "dsse": { "payloadType": "...", "payload": "...", "signatures": [{ "keyid": "signing-key-id", "sig": "..." }] }, "certificateChain": ["..."], "mode": "kms" },
|
||||
"meta": { "artifact": { "sha256": "...", "kind": "sbom" }, "bundleSha256": "...", "logPreference": "primary", "archive": true },
|
||||
"key": { "keyId": "signing-key-id", "algorithm": "ES256", "mode": "kms", "provider": "kms", "signedAt": "2025-11-01T12:34:56Z" }
|
||||
}
|
||||
```
|
||||
|
||||
* **Errors**: `400 key_not_found`, `400 payload_missing|payload_invalid_base64|artifact_sha_missing`, `400 mode_not_allowed`, `403 client_certificate_required`, `401 invalid_token`, `500 signing_failed`.
|
||||
|
||||
### 4.2 Submission
|
||||
|
||||
`POST /api/v1/rekor/entries` *(mTLS + OpTok required)*
|
||||
|
||||
* **Body**: as above.
|
||||
* **Behavior**:
|
||||
|
||||
* Verify caller (mTLS + OpTok).
|
||||
@@ -178,16 +226,16 @@ Indexes:
|
||||
"status": "included"
|
||||
}
|
||||
```
|
||||
* **Errors**: `401 invalid_token`, `403 not_signer|chain_untrusted`, `409 duplicate_bundle` (with existing `uuid`), `502 rekor_unavailable`, `504 proof_timeout`.
|
||||
|
||||
### 4.2 Proof retrieval
|
||||
|
||||
`GET /api/v1/rekor/entries/{uuid}`
|
||||
* **Errors**: `401 invalid_token`, `403 not_signer|chain_untrusted`, `409 duplicate_bundle` (with existing `uuid`), `502 rekor_unavailable`, `504 proof_timeout`.
|
||||
|
||||
### 4.3 Proof retrieval
|
||||
|
||||
`GET /api/v1/rekor/entries/{uuid}`
|
||||
|
||||
* Returns `entries` row (refreshes proof from Rekor if stale/missing).
|
||||
* Accepts `?refresh=true` to force backend query.
|
||||
|
||||
### 4.3 Verification (third‑party or internal)
|
||||
### 4.4 Verification (third‑party or internal)
|
||||
|
||||
`POST /api/v1/rekor/verify`
|
||||
|
||||
@@ -202,17 +250,28 @@ Indexes:
|
||||
1. **Bundle signature** → cert chain to Fulcio/KMS roots configured.
|
||||
2. **Inclusion proof** → recompute leaf hash; verify Merkle path against checkpoint root.
|
||||
3. Optionally verify **checkpoint** against local trust anchors (if Rekor signs checkpoints).
|
||||
4. Confirm **subject.digest** matches caller‑provided hash (when given).
|
||||
4. Confirm **subject.digest** matches caller‑provided hash (when given).
|
||||
5. Fetch **transparency witness** statement when enabled; cache results and downgrade status to WARN when endorsements are missing or mismatched.
|
||||
|
||||
* **Response**:
|
||||
|
||||
```json
|
||||
{ "ok": true, "uuid": "…", "index": 123, "logURL": "…", "checkedAt": "…" }
|
||||
```
|
||||
|
||||
### 4.4 Batch submission (optional)
|
||||
|
||||
`POST /api/v1/rekor/batch` accepts an array of submission objects; processes with per‑item results.
|
||||
* **Response**:
|
||||
|
||||
```json
|
||||
{ "ok": true, "uuid": "…", "index": 123, "logURL": "…", "checkedAt": "…" }
|
||||
```
|
||||
|
||||
### 4.5 Bulk verification
|
||||
|
||||
`POST /api/v1/rekor/verify:bulk` enqueues a verification job containing up to `quotas.bulk.maxItemsPerJob` items. Each item mirrors the single verification payload (uuid | artifactSha256 | subject+envelopeId, optional policyVersion/refreshProof). The handler persists a MongoDB job document (`bulk_jobs` collection) and returns `202 Accepted` with a job descriptor and polling URL.
|
||||
|
||||
`GET /api/v1/rekor/verify:bulk/{jobId}` returns progress and per-item results (subject/uuid, status, issues, cached verification report if available). Jobs are tenant- and subject-scoped; only the initiating principal can read their progress.
|
||||
|
||||
**Worker path:** `BulkVerificationWorker` claims queued jobs (`status=queued → running`), executes items sequentially through the cached verification service, updates progress counters, and records metrics:
|
||||
|
||||
- `attestor.bulk_jobs_total{status}` – completed/failed jobs
|
||||
- `attestor.bulk_job_duration_seconds{status}` – job runtime
|
||||
- `attestor.bulk_items_total{status}` – per-item outcomes (`succeeded`, `verification_failed`, `exception`)
|
||||
|
||||
The worker honours `bulkVerification.itemDelayMilliseconds` for throttling and reschedules persistence conflicts with optimistic version checks. Results hydrate the verification cache; failed items record the error reason without aborting the overall job.
|
||||
|
||||
---
|
||||
|
||||
@@ -244,8 +303,10 @@ Indexes:
|
||||
* `subject.digest.sha256` values must be present and well‑formed (hex).
|
||||
* **No public submission** path. **Never** accept bundles from untrusted clients.
|
||||
* **Client certificate allowlists**: optional `security.mtls.allowedSubjects` / `allowedThumbprints` tighten peer identity checks beyond CA pinning.
|
||||
* **Rate limits**: token-bucket per caller derived from `quotas.perCaller` (QPS/burst) returns `429` + `Retry-After` when exceeded.
|
||||
* **Redaction**: Attestor never logs secret material; DSSE payloads **should** be public by design (SBOMs/reports). If customers require redaction, enforce policy at Signer (predicate minimization) **before** Attestor.
|
||||
* **Rate limits**: token-bucket per caller derived from `quotas.perCaller` (QPS/burst) returns `429` + `Retry-After` when exceeded.
|
||||
* **Scope enforcement**: API separates `attestor.write`, `attestor.verify`, and `attestor.read` policies; verification/list endpoints accept read or verify scopes while submission endpoints remain write-only.
|
||||
* **Request hygiene**: JSON content-type is mandatory (415 returned otherwise); DSSE payloads are capped (default 2 MiB), certificate chains limited to six entries, and signatures to six per envelope to mitigate parsing abuse.
|
||||
* **Redaction**: Attestor never logs secret material; DSSE payloads **should** be public by design (SBOMs/reports). If customers require redaction, enforce policy at Signer (predicate minimization) **before** Attestor.
|
||||
|
||||
---
|
||||
|
||||
@@ -268,24 +329,32 @@ Indexes:
|
||||
|
||||
## 8) Observability & audit
|
||||
|
||||
**Metrics** (Prometheus):
|
||||
|
||||
* `attestor.submit_total{result,backend}`
|
||||
* `attestor.submit_latency_seconds{backend}`
|
||||
* `attestor.proof_fetch_total{result}`
|
||||
* `attestor.verify_total{result}`
|
||||
* `attestor.dedupe_hits_total`
|
||||
* `attestor.errors_total{type}`
|
||||
|
||||
**Correlation**:
|
||||
|
||||
* HTTP callers may supply `X-Correlation-Id`; Attestor will echo the header and push `CorrelationId` into the log scope for cross-service tracing.
|
||||
|
||||
**Tracing**:
|
||||
|
||||
* Spans: `validate`, `rekor.submit`, `rekor.poll`, `persist`, `archive`, `verify`.
|
||||
|
||||
**Audit**:
|
||||
**Metrics** (Prometheus):
|
||||
|
||||
* `attestor.sign_total{result,algorithm,provider}`
|
||||
* `attestor.sign_latency_seconds{algorithm,provider}`
|
||||
* `attestor.submit_total{result,backend}`
|
||||
* `attestor.submit_latency_seconds{backend}`
|
||||
* `attestor.proof_fetch_total{subject,issuer,policy,result,attestor.log.backend}`
|
||||
* `attestor.verify_total{subject,issuer,policy,result}`
|
||||
* `attestor.verify_latency_seconds{subject,issuer,policy,result}`
|
||||
* `attestor.dedupe_hits_total`
|
||||
* `attestor.errors_total{type}`
|
||||
|
||||
SLO guardrails:
|
||||
|
||||
* `attestor.verify_latency_seconds` P95 ≤ 2 s per policy.
|
||||
* `attestor.verify_total{result="failed"}` ≤ 1 % of `attestor.verify_total` over 30 min rolling windows.
|
||||
|
||||
**Correlation**:
|
||||
|
||||
* HTTP callers may supply `X-Correlation-Id`; Attestor will echo the header and push `CorrelationId` into the log scope for cross-service tracing.
|
||||
|
||||
**Tracing**:
|
||||
|
||||
* Spans: `attestor.sign`, `validate`, `rekor.submit`, `rekor.poll`, `persist`, `archive`, `attestor.verify`, `attestor.verify.refresh_proof`.
|
||||
|
||||
**Audit**:
|
||||
|
||||
* Immutable `audit` rows (ts, caller, action, hashes, uuid, index, backend, result, latency).
|
||||
|
||||
@@ -296,20 +365,45 @@ Indexes:
|
||||
```yaml
|
||||
attestor:
|
||||
listen: "https://0.0.0.0:8444"
|
||||
security:
|
||||
mtls:
|
||||
caBundle: /etc/ssl/signer-ca.pem
|
||||
requireClientCert: true
|
||||
authority:
|
||||
issuer: "https://authority.internal"
|
||||
jwksUrl: "https://authority.internal/jwks"
|
||||
requireSenderConstraint: "dpop" # or "mtls"
|
||||
signerIdentity:
|
||||
mode: ["keyless","kms"]
|
||||
fulcioRoots: ["/etc/fulcio/root.pem"]
|
||||
allowedSANs: ["urn:stellaops:signer"]
|
||||
kmsKeys: ["kms://cluster-kms/stellaops-signer"]
|
||||
rekor:
|
||||
security:
|
||||
mtls:
|
||||
caBundle: /etc/ssl/signer-ca.pem
|
||||
requireClientCert: true
|
||||
authority:
|
||||
issuer: "https://authority.internal"
|
||||
jwksUrl: "https://authority.internal/jwks"
|
||||
requireSenderConstraint: "dpop" # or "mtls"
|
||||
signerIdentity:
|
||||
mode: ["keyless","kms"]
|
||||
fulcioRoots: ["/etc/fulcio/root.pem"]
|
||||
allowedSANs: ["urn:stellaops:signer"]
|
||||
kmsKeys: ["kms://cluster-kms/stellaops-signer"]
|
||||
submissionLimits:
|
||||
maxPayloadBytes: 2097152
|
||||
maxCertificateChainEntries: 6
|
||||
maxSignatures: 6
|
||||
signing:
|
||||
preferredProviders: ["kms","bouncycastle.ed25519","default"]
|
||||
kms:
|
||||
enabled: true
|
||||
rootPath: "/var/lib/stellaops/kms"
|
||||
password: "${ATTESTOR_KMS_PASSWORD}"
|
||||
keys:
|
||||
- keyId: "kms-primary"
|
||||
algorithm: ES256
|
||||
mode: kms
|
||||
provider: "kms"
|
||||
providerKeyId: "kms-primary"
|
||||
kmsVersionId: "v1"
|
||||
- keyId: "ed25519-offline"
|
||||
algorithm: Ed25519
|
||||
mode: keyful
|
||||
provider: "bouncycastle.ed25519"
|
||||
materialFormat: base64
|
||||
materialPath: "/etc/stellaops/keys/ed25519.key"
|
||||
certificateChain:
|
||||
- "-----BEGIN CERTIFICATE-----...-----END CERTIFICATE-----"
|
||||
rekor:
|
||||
primary:
|
||||
url: "https://rekor-v2.internal"
|
||||
proofTimeoutMs: 15000
|
||||
@@ -328,13 +422,20 @@ attestor:
|
||||
objectLock: "governance"
|
||||
redis:
|
||||
url: "redis://redis:6379/2"
|
||||
quotas:
|
||||
perCaller:
|
||||
qps: 50
|
||||
burst: 100
|
||||
```
|
||||
|
||||
---
|
||||
quotas:
|
||||
perCaller:
|
||||
qps: 50
|
||||
burst: 100
|
||||
```
|
||||
|
||||
**Notes:**
|
||||
|
||||
* `signing.preferredProviders` defines the resolution order when multiple providers support the requested algorithm. Omit to fall back to registration order.
|
||||
* File-backed KMS (`signing.kms`) is required when at least one key uses `mode: kms`; the password should be injected via secret store or environment.
|
||||
* For keyful providers, supply inline `material` or `materialPath` plus `materialFormat` (`pem` (default), `base64`, or `hex`). KMS keys ignore these fields and require `kmsVersionId`.
|
||||
* `certificateChain` entries are appended to returned bundles so offline verifiers do not need to dereference external stores.
|
||||
|
||||
---
|
||||
|
||||
## 10) End‑to‑end sequences
|
||||
|
||||
|
||||
48
docs/modules/attestor/payloads.md
Normal file
48
docs/modules/attestor/payloads.md
Normal file
@@ -0,0 +1,48 @@
|
||||
# Attestor Payload Reference
|
||||
|
||||
StellaOps evidence predicates must remain reproducible, explainable, and portable across online and fully air-gapped deployments. This guide lists each predicate type, indicates where the canonical JSON Schema lives, highlights the producing service, and links to the matching golden samples.
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Type ID | Predicate URI | Schema file | Produced by | Primary consumers |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| StellaOps.BuildProvenance@1 | https://schemas.stella-ops.org/attestations/build-provenance@1 | src/Attestor/StellaOps.Attestor.Types/schemas/stellaops-build-provenance.v1.schema.json | Build pipelines, Scanner SBOM bake stage | Attestor, Export Center, Policy Engine |
|
||||
| StellaOps.SBOMAttestation@1 | https://schemas.stella-ops.org/attestations/sbom-attestation@1 | src/Attestor/StellaOps.Attestor.Types/schemas/stellaops-sbom-attestation.v1.schema.json | Scanner.Worker SBOM composer | Policy Engine, CLI, Export Center |
|
||||
| StellaOps.ScanResults@1 | https://schemas.stella-ops.org/attestations/scan-results@1 | src/Attestor/StellaOps.Attestor.Types/schemas/stellaops-scan-results.v1.schema.json | Scanner.Worker analyzers | Policy Engine, CLI, Orchestrator |
|
||||
| StellaOps.PolicyEvaluation@1 | https://schemas.stella-ops.org/attestations/policy-evaluation@1 | src/Attestor/StellaOps.Attestor.Types/schemas/stellaops-policy-evaluation.v1.schema.json | Policy Engine explain pipeline | CLI, Notify, Export Center |
|
||||
| StellaOps.VEXAttestation@1 | https://schemas.stella-ops.org/attestations/vex-attestation@1 | src/Attestor/StellaOps.Attestor.Types/schemas/stellaops-vex-attestation.v1.schema.json | Excititor consensus service | Policy Engine, CLI, Console |
|
||||
| StellaOps.RiskProfileEvidence@1 | https://schemas.stella-ops.org/attestations/risk-profile@1 | src/Attestor/StellaOps.Attestor.Types/schemas/stellaops-risk-profile.v1.schema.json | Policy Engine risk pipeline | Console, Notify, Export Center |
|
||||
| StellaOps.CustomEvidence@1 | https://schemas.stella-ops.org/attestations/custom-evidence@1 | src/Attestor/StellaOps.Attestor.Types/schemas/stellaops-custom-evidence.v1.schema.json | CLI custom evidence workflows and partner integrations | Policy Engine (policy hooks), Export Center |
|
||||
|
||||
Golden JSON fixtures that double as contract tests live under `src/Attestor/StellaOps.Attestor.Types/fixtures/v1/<predicate>.sample.json`. TypeScript and Go clients consume the generated sources in `src/Attestor/StellaOps.Attestor.Types/generated/ts` and `src/Attestor/StellaOps.Attestor.Types/generated/go`.
|
||||
|
||||
## Envelope Conventions
|
||||
|
||||
- DSSE envelopes are signed over canonical JSON (sorted keys, UTF-8, no insignificant whitespace).
|
||||
- The `subject` array must include at least one SHA-256 digest and may attach annotations such as `oci.reference` or `stellaops.asset`.
|
||||
- `predicateType` uses the URI shown in the table; `predicate.typeId` mirrors the short identifier.
|
||||
- `predicate.schemaVersion` follows semantic versioning. Consumers reject mismatched major versions.
|
||||
- Optional `metadata` and `materials` sections follow the in-toto Statement format to maximise provenance portability.
|
||||
|
||||
## Predicate Highlights
|
||||
|
||||
- **StellaOps.BuildProvenance@1** records builder identity, config source, materials, reproducibility flags, and the resulting artifact digests. Outputs must match the DSSE subject.
|
||||
- **StellaOps.SBOMAttestation@1** links an artifact digest to a CycloneDX 1.6 or SBOM 3.0.0 document, tracking inventory counts and the generator metadata. Component graph hashes reference CAS entries emitted by Scanner.Worker.
|
||||
- **StellaOps.ScanResults@1** captures deterministic findings from OS, language, and native analyzers. It reports summary counts, per-finding metadata (PURL, severity, exploitability), and the layer digests inspected.
|
||||
- **StellaOps.PolicyEvaluation@1** documents lattice-based policy outcomes, including decision traces and evidence digests consumed during evaluation.
|
||||
- **StellaOps.VEXAttestation@1** mirrors OpenVEX-aligned statements with justification, scope narrowing (package coordinates or component IDs), and issue timestamps.
|
||||
- **StellaOps.RiskProfileEvidence@1** summarises exploitability, ticketing load, runtime coverage, and maturity for downstream dashboards.
|
||||
- **StellaOps.CustomEvidence@1** allows regulated tenants to attach organisation-specific payloads referenced by a CAS-hosted schema while preserving provenance and retention controls.
|
||||
|
||||
## Validation and Tooling
|
||||
|
||||
- Run `npm install` once, then `npm run docs:attestor:validate` to validate JSON fixtures against their schemas, execute the generated TypeScript tests (`npm test`), and run `go test ./...` for the Go SDK. The command fails fast when any schema, fixture, or generated SDK drifts.
|
||||
- Regenerate schemas and SDKs after edits with `dotnet run --project src/Attestor/StellaOps.Attestor.Types/Tools/StellaOps.Attestor.Types.Generator`.
|
||||
- Offline Kit builds (`ops/devops/offline-kit/`) mirror schemas, fixtures, and SDK bundles so air-gapped operators can run the same validation stack.
|
||||
|
||||
## Related Material
|
||||
|
||||
- `docs/modules/attestor/architecture.md` — service topology, Rekor integration, caching model.
|
||||
- `docs/modules/platform/architecture-overview.md` — cross-module data flows and tenant boundaries.
|
||||
- `docs/ingestion/aggregation-only-contract.md` — guardrails for advisory feeds consumed by policy evaluation.
|
||||
- `src/Attestor/StellaOps.Attestor.Types/samples/README.md` — directory map for the golden evidence set referenced here.
|
||||
41
docs/modules/attestor/ttl-validation.md
Normal file
41
docs/modules/attestor/ttl-validation.md
Normal file
@@ -0,0 +1,41 @@
|
||||
# Attestor TTL Validation Runbook
|
||||
|
||||
> **Purpose:** confirm MongoDB TTL indexes and Redis expirations for the attestation dedupe store behave as expected on a production-like stack.
|
||||
|
||||
## Prerequisites
|
||||
- Docker Desktop or compatible daemon with the Compose plugin enabled.
|
||||
- Local ports `27017` and `6379` free.
|
||||
- `dotnet` SDK 10.0 preview (same as repo toolchain).
|
||||
- Network access to pull `mongo:7` and `redis:7` images.
|
||||
|
||||
## Quickstart
|
||||
1. From the repo root export any required proxy settings, then run
|
||||
```bash
|
||||
scripts/run-attestor-ttl-validation.sh
|
||||
```
|
||||
The helper script:
|
||||
- Spins up `mongo:7` and `redis:7` containers.
|
||||
- Sets `ATTESTOR_LIVE_MONGO_URI` / `ATTESTOR_LIVE_REDIS_URI`.
|
||||
- Executes the live TTL test suite (`Category=LiveTTL`) in `StellaOps.Attestor.Tests`.
|
||||
- Tears the stack down automatically.
|
||||
|
||||
2. Capture the test output (`ttl-validation-<timestamp>.log`) and attach it to the sprint evidence folder (`docs/modules/attestor/evidence/`).
|
||||
|
||||
## Result handling
|
||||
- **Success:** Tests complete in ~3–4 minutes with `Total tests: 2, Passed: 2`. Store the log and note the run in `SPRINT_100_identity_signing.md` under ATTESTOR-72-003.
|
||||
- **Failure:** Preserve:
|
||||
- `docker compose logs` for both services.
|
||||
- `mongosh` output of `db.dedupe.getIndexes()` and sample documents.
|
||||
- `redis-cli --raw ttl attestor:ttl:live:bundle:<id>`.
|
||||
File an incident in the Attestor Guild channel and link the captured artifacts.
|
||||
|
||||
## Manual verification (optional)
|
||||
If the helper script cannot be used:
|
||||
1. Start MongoDB and Redis manually with equivalent configuration.
|
||||
2. Set `ATTESTOR_LIVE_MONGO_URI` and `ATTESTOR_LIVE_REDIS_URI`.
|
||||
3. Run `dotnet test src/Attestor/StellaOps.Attestor.sln --no-build --filter "Category=LiveTTL"`.
|
||||
4. Follow the evidence handling steps above.
|
||||
|
||||
## Ownership
|
||||
- Primary: Attestor Service Guild.
|
||||
- Partner: QA Guild (observes TTL metrics, confirms evidence archiving).
|
||||
247
docs/modules/attestor/workflows.md
Normal file
247
docs/modules/attestor/workflows.md
Normal file
@@ -0,0 +1,247 @@
|
||||
# Attestor Verification Workflows
|
||||
|
||||
> How StellaOps turns DSSE bundles into verifiable evidence, how the verification API reports outcomes, and how explainability signals surface in UI/CLI flows.
|
||||
|
||||
> ⚠️ **2025-11-01 coordination note:** `StellaOps.Attestor.WebService` is failing to compile until downstream fixes land (`Contracts/AttestationBundleContracts.cs` null-coalescing update and scope/token variables restored in `Program.cs`). Verification flows ship in infrastructure/tests, but the WebService hand-off stays blocked — track via `ATTESTOR-73-002` (see Attestor task board).
|
||||
|
||||
## 1. Verification flow (API and service contract)
|
||||
|
||||
- **Entry point.** `POST /api/v1/rekor/verify` deserialises to `AttestorVerificationRequest`.
|
||||
- **Resolution order.** The service tries `uuid`, then canonicalised `bundle`, then `artifactSha256`. At least one selector must be present (`invalid_query` otherwise).
|
||||
- **Optional proof refresh.** `refreshProof=true` forces a Rekor lookup before returning. Proofs are cached in Mongo.
|
||||
- **Signature replay.** Supplying `bundle` lets the service recompute the canonical hash and re-run signature checks; omitting the bundle skips those steps but still validates Merkle proofs and cached policy decisions.
|
||||
- **Auth scopes.** Endpoints demand `attestor.verify` (write scope is also accepted); read-only detail/list APIs require `attestor.read` at minimum.
|
||||
|
||||
### 1.1 Request properties
|
||||
|
||||
| Field | Type | Required | Purpose |
|
||||
|-------|------|----------|---------|
|
||||
| `uuid` | string | optional | Rekor V2 UUID to verify and (optionally) refresh. |
|
||||
| `bundle` | object | optional | DSSE envelope (same shape as submission) for signature re-verification. |
|
||||
| `artifactSha256` | string | optional | Resolve the most recent entry for an attestable artefact digest. |
|
||||
| `subject` | string | optional | Logical subject identifier used for cache/telemetry tagging; defaults to the stored artifact digest. |
|
||||
| `envelopeId` | string | optional | Stable identifier for the DSSE bundle (typically the canonical hash); enables cache lookups. |
|
||||
| `policyVersion` | string | optional | Policy digest/version driving verification; feeds cache keys and observability dimensions. |
|
||||
| `refreshProof` | bool | optional (default `false`) | Pull the current inclusion proof and checkpoint from Rekor before evaluating. |
|
||||
|
||||
All selectors are mutually compatible; if more than one is set the service uses the first match (`uuid` → `bundle` → `artifactSha256`).
|
||||
|
||||
### 1.2 Response schema (`AttestorVerificationResult`)
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `ok` | bool | `true` when the entry status is `included` **and** no issues were recorded. |
|
||||
| `uuid` | string | Rekor UUID that satisfied the query. Useful for follow-up fetches. |
|
||||
| `index` | number (int64) | Rekor log index, when supplied by the backend. |
|
||||
| `logUrl` | string | Fully-qualified Rekor entry URL for operators and auditors. |
|
||||
| `status` | string | Transparency-log status seen in Mongo (`included`, `pending`, `failed`, …). |
|
||||
| `checkedAt` | string (ISO-8601 UTC) | Timestamp emitted when the response is created. |
|
||||
| `issues` | array[string] | Machine-readable explainability codes. Empty when `ok=true`. |
|
||||
|
||||
> **Note:** `checkedAt` is recomputed each call; cache hits do not recycle previous timestamps.
|
||||
|
||||
### 1.3 Success criteria
|
||||
|
||||
`ok=true` requires:
|
||||
|
||||
1. Entry exists and status equals `included`.
|
||||
2. Canonical DSSE hash matches the stored bundle hash.
|
||||
3. Signature re-verification (when a bundle is supplied) succeeds.
|
||||
4. Inclusion proof validates against the cached or refreshed checkpoint.
|
||||
|
||||
Any deviation records at least one issue and flips `ok` to `false`. Consumers **must** inspect `issues` rather than inferring from `status` alone.
|
||||
|
||||
## 2. Verification report schema
|
||||
|
||||
`AttestorVerificationResult` carries the flattened summary shown above. When callers request the detailed report (`GET /api/v1/rekor/entries/{uuid}?refresh=true` or via SDK) they receive a `VerificationReport` shaped as follows:
|
||||
|
||||
```json
|
||||
{
|
||||
"overallStatus": "pass",
|
||||
"succeeded": true,
|
||||
"policy": { ... },
|
||||
"issuer": { ... },
|
||||
"freshness": { ... },
|
||||
"signatures": { ... },
|
||||
"transparency": { ... },
|
||||
"issues": [ "bundle_hash_mismatch" ]
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `overallStatus` | string (`pass`, `warn`, `fail`, `skipped`) | Aggregated verdict derived from the individual section statuses. |
|
||||
| `succeeded` | bool | Convenience flag; `true` when `overallStatus ∈ {pass, warn}`. |
|
||||
| `policy` | object | Results from policy evaluation (see below). |
|
||||
| `issuer` | object | Identity/result of the signing entity. |
|
||||
| `freshness` | object | Age analysis relative to policy settings. |
|
||||
| `signatures` | object | Signature validation summary. |
|
||||
| `transparency` | object | Inclusion proof / checkpoint evaluation summary. |
|
||||
| `issues` | array[string] | De-duplicated set drawn from the sections; order is deterministic and stable. |
|
||||
|
||||
### 2.1 `policy`
|
||||
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| `status` | Section verdict (`pass`, `warn`, `fail`, `skipped`). |
|
||||
| `policyId` / `policyVersion` | DSL identifier and revision used for evaluation. |
|
||||
| `verdict` | Policy outcome (`allow`, `challenge`, `deny`, etc.). |
|
||||
| `issues` | Policy-specific explainability codes (e.g., `policy_rule_blocked`). |
|
||||
| `attributes` | Key/value map emitted by the policy for downstream observability (e.g., applicable rules, matched waivers). |
|
||||
|
||||
### 2.2 `issuer`
|
||||
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| `status` | Result of issuer validation. |
|
||||
| `mode` | Signing mode detected (`keyless`, `kms`, `unknown`). |
|
||||
| `issuer` | Distinguished name / issuer URI recorded during signing. |
|
||||
| `subjectAlternativeName` | SAN pulled from the Fulcio certificate (keyless) or recorded KMS identity. |
|
||||
| `keyId` | Logical key identifier associated with the signature. |
|
||||
| `issues` | Issuer-specific issues (e.g., `issuer_trust_root_mismatch`, `signer_mode_unsupported:kid`). |
|
||||
|
||||
### 2.3 `freshness`
|
||||
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| `status` | `fail` when the attestation exceeds `verification.freshnessMaxAgeMinutes`; `warn` when only the warning threshold is hit. |
|
||||
| `createdAt` | Timestamp embedded in the attestation metadata. |
|
||||
| `evaluatedAt` | Server-side timestamp used for age calculations. |
|
||||
| `age` | ISO8601 duration of `evaluatedAt - createdAt`. |
|
||||
| `maxAge` | Policy-driven ceiling (null when unchecked). |
|
||||
| `issues` | `freshness_max_age_exceeded`, `freshness_warning`, etc. |
|
||||
|
||||
### 2.4 `signatures`
|
||||
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| `status` | Signature validation verdict. |
|
||||
| `bundleProvided` | `true` when canonical DSSE bytes were supplied. |
|
||||
| `totalSignatures` | Count observed in the DSSE envelope. |
|
||||
| `verifiedSignatures` | Number of signatures that validated against trusted keys. |
|
||||
| `requiredSignatures` | Policy / configuration minimum enforced. |
|
||||
| `issues` | Signature codes such as `bundle_payload_invalid_base64`, `signature_invalid`, `signer_mode_unknown`. |
|
||||
|
||||
### 2.5 `transparency`
|
||||
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| `status` | Inclusion proof / checkpoint verdict. |
|
||||
| `proofPresent` | Whether a proof document was available. |
|
||||
| `checkpointPresent` | Indicates the Rekor checkpoint existed and parsed. |
|
||||
| `inclusionPathPresent` | `true` when the Merkle path array contained nodes. |
|
||||
| `issues` | Merkle/rekor codes (`proof_missing`, `proof_leafhash_mismatch`, `checkpoint_missing`, `proof_root_mismatch`). |
|
||||
|
||||
### 2.6 Issue catalogue (non-exhaustive)
|
||||
|
||||
| Code | Trigger | Notes |
|
||||
|------|---------|-------|
|
||||
| `bundle_hash_mismatch` | Canonical DSSE hash differs from stored value. | Often indicates tampering or inconsistent canonicalisation. |
|
||||
| `bundle_payload_invalid_base64` | DSSE payload cannot be base64-decoded. | Validate producer pipeline; the attestation is unusable. |
|
||||
| `signature_invalid` | At least one signature failed cryptographic verification. | Consider checking key rotation / revocation status. |
|
||||
| `signer_mode_unknown` / `signer_mode_unsupported:<mode>` | Signing mode not configured for this installation. | Update `attestorOptions.security.signerIdentity.mode`. |
|
||||
| `issuer_trust_root_mismatch` | Certificate chain does not terminate in configured Fulcio/KMS roots. | Check Fulcio bundle / KMS configuration. |
|
||||
| `freshness_max_age_exceeded` | Attestation older than permitted maximum. | Regenerate attestation or extend policy window. |
|
||||
| `proof_missing` | No inclusion proof stored or supplied. | When running offline, import bundles with proofs or allow warn-level policies. |
|
||||
| `proof_root_mismatch` | Rebuilt Merkle root differs from checkpoint. | Proof may be stale or log compromised; escalate. |
|
||||
| `checkpoint_missing` | No Rekor checkpoint available. | Configure `RequireCheckpoint=false` to downgrade severity. |
|
||||
|
||||
Downstream consumers (UI, CLI, policy studio) should render human-readable messages but must retain the exact issue codes for automation and audit replay.
|
||||
|
||||
## 3. Explainability signals
|
||||
|
||||
1. **Canonicalisation.** The service replays DSSE canonicalisation to derive `bundleSha256`. Failures surface as `bundle_hash_mismatch` or decoding errors.
|
||||
2. **Signature checks.** Mode-aware handling:
|
||||
- `kms` (HMAC) compares against configured shared secrets.
|
||||
- `keyless` rebuilds the certificate chain, enforces Fulcio roots, SAN allow-lists, and verifies with the leaf certificate.
|
||||
- Unknown modes emit `signer_mode_unknown` / `signer_mode_unsupported:<mode>`.
|
||||
3. **Proof acquisition.** When `refreshProof` is requested the Rekor backend may contribute a textual issue (`Proof refresh failed: …`) without stopping evaluation.
|
||||
4. **Merkle validation.** Structured helper ensures leaf hash, path orientation, and checkpoint root are consistent; each validation failure has a discrete issue code.
|
||||
5. **Observability.** The meter `attestor.verify_total` increments with `result=ok|failed`; structured logs and traces carry the same `issues` vector for UI/CLI drill-down.
|
||||
|
||||
All issues are appended in detection order to simplify chronological replay in the Console’s chain-of-custody view.
|
||||
|
||||
## 3. Issue catalogue
|
||||
|
||||
| Code | Trigger | Operator guidance |
|
||||
|------|---------|-------------------|
|
||||
| `bundle_hash_mismatch` | Canonicalised DSSE hash differs from stored bundle hash. | Re-download artefact; investigate tampering or submission races. |
|
||||
| `bundle_payload_invalid_base64` | Payload could not be base64-decoded. | Ensure bundle transport preserved payload; capture original DSSE for forensics. |
|
||||
| `signature_invalid_kms` | HMAC verification failed for `mode=kms`. | Confirm shared secret alignment with Signer; rotate keys if drift detected. |
|
||||
| `signer_mode_unknown` | Entry lacks signer mode metadata and bundle omitted it. | Re-ingest bundle or inspect submission pipeline metadata. |
|
||||
| `signer_mode_unsupported:<mode>` | Signer mode is unsupported by the verifier. | Add support or block unsupported issuers in policy. |
|
||||
| `kms_key_missing` | No configured KMS secrets to verify `mode=kms`. | Populate `security:signerIdentity:kmsKeys` in Attestor config before retry. |
|
||||
| `signature_invalid_base64` | One or more signatures were not valid base64. | Bundle corruption; capture raw payload and re-submit. |
|
||||
| `certificate_chain_missing` | `mode=keyless` bundle lacked any certificates. | Ensure Signer attaches Fulcio chain; review submission pipeline. |
|
||||
| `certificate_chain_invalid` | Certificates could not be parsed. | Fetch original DSSE bundle for repair; confirm certificate encoding. |
|
||||
| `certificate_chain_untrusted[:detail]` | Chain failed custom-root validation. | Import correct Fulcio roots or investigate potential impersonation. |
|
||||
| `certificate_san_untrusted` | Leaf SAN not in configured allow-list. | Update allow-list or revoke offending issuer. |
|
||||
| `signature_invalid` | No signature validated with supplied public keys. | Treat as tampering; trigger incident response. |
|
||||
| `proof_missing` | No Merkle proof stored for the entry. | Re-run with `refreshProof=true`; check Rekor availability. |
|
||||
| `bundle_hash_decode_failed` | Stored bundle hash could not be decoded. | Verify Mongo record integrity; re-enqueue submission if necessary. |
|
||||
| `proof_inclusion_missing` | Inclusion section absent from proof. | Retry proof refresh; inspect Rekor health. |
|
||||
| `proof_leafhash_decode_failed` | Leaf hash malformed. | Replay submission; inspect Rekor data corruption. |
|
||||
| `proof_leafhash_mismatch` | Leaf hash differs from canonical bundle hash. | Raises tamper alert; reconcile Rekor entry vs stored bundle. |
|
||||
| `proof_path_decode_failed` | Inclusion path entry malformed. | Same action as above; likely Rekor data corruption. |
|
||||
| `proof_path_orientation_missing` | Inclusion path lacks left/right marker. | File Rekor bug; fallback to mirror log if configured. |
|
||||
| `checkpoint_missing` | Proof lacks checkpoint metadata. | Retry refresh; ensure Rekor is configured to return checkpoints. |
|
||||
| `checkpoint_root_decode_failed` | Checkpoint root hash malformed. | Investigate Rekor/mirror integrity before trusting log. |
|
||||
| `proof_root_mismatch` | Computed root hash != checkpoint root. | Critical alert; assume inclusion proof compromised. |
|
||||
| `Proof refresh failed: …` | Rekor fetch threw an exception. | Message includes upstream error; surface alongside telemetry for debugging. |
|
||||
|
||||
Future explainability flags must follow the same pattern: short, lowercase codes with optional suffix payload (`code:detail`).
|
||||
|
||||
## 4. Worked examples
|
||||
|
||||
### 4.1 Successful verification
|
||||
|
||||
```json
|
||||
{
|
||||
"ok": true,
|
||||
"uuid": "0192fdb4-a82b-7f90-b894-6fd1dd918b85",
|
||||
"index": 73421,
|
||||
"logUrl": "https://rekor.stellaops.test/api/v2/log/entries/0192fdb4a82b7f90b8946fd1dd918b85",
|
||||
"status": "included",
|
||||
"checkedAt": "2025-11-01T17:06:52.182394Z",
|
||||
"issues": []
|
||||
}
|
||||
```
|
||||
|
||||
This mirrors the happy-path asserted in `AttestorVerificationServiceTests.VerifyAsync_ReturnsOk_ForExistingUuid`, which replays the entire submission→verification loop.
|
||||
|
||||
### 4.2 Tampered bundle
|
||||
|
||||
```json
|
||||
{
|
||||
"ok": false,
|
||||
"uuid": "0192fdb4-a82b-7f90-b894-6fd1dd918b85",
|
||||
"index": 73421,
|
||||
"logUrl": "https://rekor.stellaops.test/api/v2/log/entries/0192fdb4a82b7f90b8946fd1dd918b85",
|
||||
"status": "included",
|
||||
"checkedAt": "2025-11-01T17:09:05.443218Z",
|
||||
"issues": [
|
||||
"bundle_hash_mismatch",
|
||||
"signature_invalid"
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Derived from `AttestorVerificationServiceTests.VerifyAsync_FlagsTamperedBundle`, which flips the DSSE payload and expects both issues to surface. CLI and Console consumers should display these codes verbatim and provide remediation tips from the table above.
|
||||
|
||||
## 5. Validating the documentation
|
||||
|
||||
- Run `dotnet test src/Attestor/StellaOps.Attestor/StellaOps.Attestor.Tests` to exercise the scenarios behind the examples.
|
||||
- API integrators can `curl` the verify endpoint and compare responses with the JSON above.
|
||||
- UI/CLI teams should ensure explainability tooltips and runbooks reference the same issue catalogue.
|
||||
|
||||
Keeping the documentation aligned with the test suite guarantees explainability remains deterministic and audit-friendly.
|
||||
|
||||
## 6. Offline bundles & air-gapped verification
|
||||
|
||||
Stella Ops Attestor now supports packaging attestations for sealed environments and rehydrating them without calling Rekor:
|
||||
|
||||
- **Export bundles.** `POST /api/v1/attestations:export` accepts either a list of Rekor UUIDs or filter criteria (`subject`, `type`, `issuer`, `scope`, `createdAfter|Before`, `limit`, `continuationToken`) and returns an `attestor.bundle.v1` document. Each item contains the attestation entry, canonical DSSE payload (base64), optional proof payload, and metadata. Responses include a `continuationToken` so callers can page through large result sets (limits default to 100 and are capped at 200). JSON content is required and requests are gated by the `attestor.read` scope.
|
||||
- **Import bundles.** `POST /api/v1/attestations:import` ingests the bundle document, upserts attestation metadata, and restores the canonical DSSE/proof into the configured archive store. The S3 archive integration must be enabled; the response reports how many entries were imported versus updated, any skipped items, and issue codes (`bundle_payload_invalid_base64`, `bundle_hash_mismatch`, `archive_disabled`, …).
|
||||
- **Offline verification.** When replaying verification without log connectivity, submit the DSSE bundle and set `offline=true` on `POST /api/v1/rekor/verify`. The service reuses imported proofs when present and surfaces deterministic explainability codes (`proof_missing`, `proof_inclusion_missing`, …) instead of attempting Rekor fetches.
|
||||
|
||||
Tests `AttestorBundleServiceTests.ExportAsync_AppliesFiltersAndContinuation`, `AttestationBundleEndpointsTests`, `AttestorVerificationServiceTests.VerifyAsync_OfflineSkipsProofRefreshWhenMissing`, and `AttestorVerificationServiceTests.VerifyAsync_OfflineUsesImportedProof` exercise the exporter/importer, API contracts, and the offline verification path with and without witness data.
|
||||
@@ -1,445 +1,452 @@
|
||||
# component_architecture_authority.md — **Stella Ops Authority** (2025Q4)
|
||||
|
||||
> Consolidates identity and tenancy requirements documented across the AOC, Policy, and Platform guides, along with the dedicated Authority implementation plan.
|
||||
|
||||
> **Scope.** Implementation‑ready architecture for **Stella Ops Authority**: the on‑prem **OIDC/OAuth2** service that issues **short‑lived, sender‑constrained operational tokens (OpToks)** to first‑party services and tools. Covers protocols (DPoP & mTLS binding), token shapes, endpoints, storage, rotation, HA, RBAC, audit, and testing. This component is the trust anchor for *who* is calling inside a Stella Ops installation. (Entitlement is proven separately by **PoE** from the cloud Licensing Service; Authority does not issue PoE.)
|
||||
|
||||
---
|
||||
|
||||
## 0) Mission & boundaries
|
||||
|
||||
**Mission.** Provide **fast, local, verifiable** authentication for Stella Ops microservices and tools by minting **very short‑lived** OAuth2/OIDC tokens that are **sender‑constrained** (DPoP or mTLS‑bound). Support RBAC scopes, multi‑tenant claims, and deterministic validation for APIs (Scanner, Signer, Attestor, Excititor, Concelier, UI, CLI, Zastava).
|
||||
|
||||
**Boundaries.**
|
||||
|
||||
* Authority **does not** validate entitlements/licensing. That’s enforced by **Signer** using **PoE** with the cloud Licensing Service.
|
||||
* Authority tokens are **operational only** (2–5 min TTL) and must not be embedded in long‑lived artifacts or stored in SBOMs.
|
||||
* Authority is **stateless for validation** (JWT) and **optional introspection** for services that prefer online checks.
|
||||
|
||||
---
|
||||
|
||||
## 1) Protocols & cryptography
|
||||
|
||||
* **OIDC Discovery**: `/.well-known/openid-configuration`
|
||||
* **OAuth2** grant types:
|
||||
|
||||
* **Client Credentials** (service↔service, with mTLS or private_key_jwt)
|
||||
* **Device Code** (CLI login on headless agents; optional)
|
||||
* **Authorization Code + PKCE** (browser login for UI; optional)
|
||||
* **Sender constraint options** (choose per caller or per audience):
|
||||
|
||||
* **DPoP** (Demonstration of Proof‑of‑Possession): proof JWT on each HTTP request, bound to the access token via `cnf.jkt`.
|
||||
* **OAuth 2.0 mTLS** (certificate‑bound tokens): token bound to client certificate thumbprint via `cnf.x5t#S256`.
|
||||
* **Signing algorithms**: **EdDSA (Ed25519)** preferred; fallback **ES256 (P‑256)**. Rotation is supported via **kid** in JWKS.
|
||||
* **Token format**: **JWT** access tokens (compact), optionally opaque reference tokens for services that insist on introspection.
|
||||
* **Clock skew tolerance**: ±60 s; issue `nbf`, `iat`, `exp` accordingly.
|
||||
|
||||
---
|
||||
|
||||
## 2) Token model
|
||||
|
||||
### 2.1 Access token (OpTok) — short‑lived (120–300 s)
|
||||
|
||||
**Registered claims**
|
||||
|
||||
```
|
||||
iss = https://authority.<domain>
|
||||
sub = <client_id or user_id>
|
||||
aud = <service audience: signer|scanner|attestor|concelier|excititor|ui|zastava>
|
||||
exp = <unix ts> (<= 300 s from iat)
|
||||
iat = <unix ts>
|
||||
nbf = iat - 30
|
||||
jti = <uuid>
|
||||
scope = "scanner.scan scanner.export signer.sign ..."
|
||||
```
|
||||
|
||||
**Sender‑constraint (`cnf`)**
|
||||
|
||||
* **DPoP**:
|
||||
|
||||
```json
|
||||
"cnf": { "jkt": "<base64url(SHA-256(JWK))>" }
|
||||
```
|
||||
* **mTLS**:
|
||||
|
||||
```json
|
||||
"cnf": { "x5t#S256": "<base64url(SHA-256(client_cert_der))>" }
|
||||
```
|
||||
|
||||
**Install/tenant context (custom claims)**
|
||||
|
||||
```
|
||||
tid = <tenant id> // multi-tenant
|
||||
inst = <installation id> // unique installation
|
||||
roles = [ "svc.scanner", "svc.signer", "ui.admin", ... ]
|
||||
plan? = <plan name> // optional hint for UIs; not used for enforcement
|
||||
```
|
||||
|
||||
> **Note**: Do **not** copy PoE claims into OpTok; OpTok ≠ entitlement. Only **Signer** checks PoE.
|
||||
|
||||
### 2.2 Refresh tokens (optional)
|
||||
|
||||
* Default **disabled**. If enabled (for UI interactive logins), pair with **DPoP‑bound** refresh tokens or **mTLS** client sessions; short TTL (≤ 8 h), rotating on use (replay‑safe).
|
||||
|
||||
### 2.3 ID tokens (optional)
|
||||
|
||||
* Issued for UI/browser OIDC flows (Authorization Code + PKCE); not used for service auth.
|
||||
|
||||
---
|
||||
|
||||
## 3) Endpoints & flows
|
||||
|
||||
### 3.1 OIDC discovery & keys
|
||||
|
||||
* `GET /.well-known/openid-configuration` → endpoints, algs, jwks_uri
|
||||
* `GET /jwks` → JSON Web Key Set (rotating, at least 2 active keys during transition)
|
||||
|
||||
### 3.2 Token issuance
|
||||
|
||||
* `POST /oauth/token`
|
||||
|
||||
* **Client Credentials** (service→service):
|
||||
|
||||
|
||||
> **Scope.** Implementation‑ready architecture for **Stella Ops Authority**: the on‑prem **OIDC/OAuth2** service that issues **short‑lived, sender‑constrained operational tokens (OpToks)** to first‑party services and tools. Covers protocols (DPoP & mTLS binding), token shapes, endpoints, storage, rotation, HA, RBAC, audit, and testing. This component is the trust anchor for *who* is calling inside a Stella Ops installation. (Entitlement is proven separately by **PoE** from the cloud Licensing Service; Authority does not issue PoE.)
|
||||
|
||||
---
|
||||
|
||||
## 0) Mission & boundaries
|
||||
|
||||
**Mission.** Provide **fast, local, verifiable** authentication for Stella Ops microservices and tools by minting **very short‑lived** OAuth2/OIDC tokens that are **sender‑constrained** (DPoP or mTLS‑bound). Support RBAC scopes, multi‑tenant claims, and deterministic validation for APIs (Scanner, Signer, Attestor, Excititor, Concelier, UI, CLI, Zastava).
|
||||
|
||||
**Boundaries.**
|
||||
|
||||
* Authority **does not** validate entitlements/licensing. That’s enforced by **Signer** using **PoE** with the cloud Licensing Service.
|
||||
* Authority tokens are **operational only** (2–5 min TTL) and must not be embedded in long‑lived artifacts or stored in SBOMs.
|
||||
* Authority is **stateless for validation** (JWT) and **optional introspection** for services that prefer online checks.
|
||||
|
||||
---
|
||||
|
||||
## 1) Protocols & cryptography
|
||||
|
||||
* **OIDC Discovery**: `/.well-known/openid-configuration`
|
||||
* **OAuth2** grant types:
|
||||
|
||||
* **Client Credentials** (service↔service, with mTLS or private_key_jwt)
|
||||
* **Device Code** (CLI login on headless agents; optional)
|
||||
* **Authorization Code + PKCE** (browser login for UI; optional)
|
||||
* **Sender constraint options** (choose per caller or per audience):
|
||||
|
||||
* **DPoP** (Demonstration of Proof‑of‑Possession): proof JWT on each HTTP request, bound to the access token via `cnf.jkt`.
|
||||
* **OAuth 2.0 mTLS** (certificate‑bound tokens): token bound to client certificate thumbprint via `cnf.x5t#S256`.
|
||||
* **Signing algorithms**: **EdDSA (Ed25519)** preferred; fallback **ES256 (P‑256)**. Rotation is supported via **kid** in JWKS.
|
||||
* **Token format**: **JWT** access tokens (compact), optionally opaque reference tokens for services that insist on introspection.
|
||||
* **Clock skew tolerance**: ±60 s; issue `nbf`, `iat`, `exp` accordingly.
|
||||
|
||||
---
|
||||
|
||||
## 2) Token model
|
||||
|
||||
* **Incident mode tokens** require the `obs:incident` scope, a human-supplied `incident_reason`, and remain valid only while `auth_time` stays within a five-minute freshness window. Resource servers enforce the same window and persist `incident.reason`, `incident.auth_time`, and the fresh-auth verdict in `authority.resource.authorize` events. Authority exposes `/authority/audit/incident` so auditors can review recent activations.
|
||||
|
||||
|
||||
### 2.1 Access token (OpTok) — short‑lived (120–300 s)
|
||||
|
||||
**Registered claims**
|
||||
|
||||
```
|
||||
iss = https://authority.<domain>
|
||||
sub = <client_id or user_id>
|
||||
aud = <service audience: signer|scanner|attestor|concelier|excititor|ui|zastava>
|
||||
exp = <unix ts> (<= 300 s from iat)
|
||||
iat = <unix ts>
|
||||
nbf = iat - 30
|
||||
jti = <uuid>
|
||||
scope = "scanner.scan scanner.export signer.sign ..."
|
||||
```
|
||||
|
||||
**Sender‑constraint (`cnf`)**
|
||||
|
||||
* **DPoP**:
|
||||
|
||||
```json
|
||||
"cnf": { "jkt": "<base64url(SHA-256(JWK))>" }
|
||||
```
|
||||
* **mTLS**:
|
||||
|
||||
```json
|
||||
"cnf": { "x5t#S256": "<base64url(SHA-256(client_cert_der))>" }
|
||||
```
|
||||
|
||||
**Install/tenant context (custom claims)**
|
||||
|
||||
```
|
||||
tid = <tenant id> // multi-tenant
|
||||
inst = <installation id> // unique installation
|
||||
roles = [ "svc.scanner", "svc.signer", "ui.admin", ... ]
|
||||
plan? = <plan name> // optional hint for UIs; not used for enforcement
|
||||
```
|
||||
|
||||
> **Note**: Do **not** copy PoE claims into OpTok; OpTok ≠ entitlement. Only **Signer** checks PoE.
|
||||
|
||||
### 2.2 Refresh tokens (optional)
|
||||
|
||||
* Default **disabled**. If enabled (for UI interactive logins), pair with **DPoP‑bound** refresh tokens or **mTLS** client sessions; short TTL (≤ 8 h), rotating on use (replay‑safe).
|
||||
|
||||
### 2.3 ID tokens (optional)
|
||||
|
||||
* Issued for UI/browser OIDC flows (Authorization Code + PKCE); not used for service auth.
|
||||
|
||||
---
|
||||
|
||||
## 3) Endpoints & flows
|
||||
|
||||
### 3.1 OIDC discovery & keys
|
||||
|
||||
* `GET /.well-known/openid-configuration` → endpoints, algs, jwks_uri
|
||||
* `GET /jwks` → JSON Web Key Set (rotating, at least 2 active keys during transition)
|
||||
|
||||
### 3.2 Token issuance
|
||||
|
||||
* `POST /token`
|
||||
|
||||
> Legacy aliases under `/oauth/token` are deprecated as of 1 November 2025 and now emit `Deprecation/Sunset/Warning` headers. See [`docs/api/authority-legacy-auth-endpoints.md`](../../api/authority-legacy-auth-endpoints.md) for timelines and migration guidance.
|
||||
|
||||
* **Client Credentials** (service→service):
|
||||
|
||||
* **mTLS**: mutual TLS + `client_id` → bound token (`cnf.x5t#S256`)
|
||||
* `security.senderConstraints.mtls.enforceForAudiences` forces the mTLS path when requested `aud`/`resource` values intersect high-value audiences (defaults include `signer`). Authority rejects clients attempting to use DPoP/basic secrets for these audiences.
|
||||
* Stored `certificateBindings` are authoritative: thumbprint, subject, issuer, serial number, and SAN values are matched against the presented certificate, with rotation grace applied to activation windows. Failures surface deterministic error codes (e.g. `certificate_binding_subject_mismatch`).
|
||||
* **private_key_jwt**: JWT‑based client auth + **DPoP** header (preferred for tools and CLI)
|
||||
* **Device Code** (CLI): `POST /oauth/device/code` + `POST /oauth/token` poll
|
||||
* **Authorization Code + PKCE** (UI): standard
|
||||
|
||||
**DPoP handshake (example)**
|
||||
|
||||
1. Client prepares **JWK** (ephemeral keypair).
|
||||
2. Client sends **DPoP proof** header with fields:
|
||||
|
||||
```
|
||||
htm=POST
|
||||
htu=https://authority.../oauth/token
|
||||
iat=<now>
|
||||
jti=<uuid>
|
||||
```
|
||||
|
||||
signed with the DPoP private key; header carries JWK.
|
||||
3. Authority validates proof; issues access token with `cnf.jkt=<thumbprint(JWK)>`.
|
||||
4. Client uses the same DPoP key to sign **every subsequent API request** to services (Signer, Scanner, …).
|
||||
|
||||
**mTLS flow**
|
||||
|
||||
* Mutual TLS at the connection; Authority extracts client cert, validates chain; token carries `cnf.x5t#S256`.
|
||||
|
||||
### 3.3 Introspection & revocation (optional)
|
||||
|
||||
* `POST /oauth/introspect` → `{ active, sub, scope, aud, exp, cnf, ... }`
|
||||
* `POST /oauth/revoke` → revokes refresh tokens or opaque access tokens.
|
||||
* **Replay prevention**: maintain **DPoP `jti` cache** (TTL ≤ 10 min) to reject duplicate proofs when services supply DPoP nonces (Signer requires nonce for high‑value operations).
|
||||
|
||||
### 3.4 UserInfo (optional for UI)
|
||||
|
||||
* `GET /userinfo` (ID token context).
|
||||
|
||||
---
|
||||
|
||||
## 4) Audiences, scopes & RBAC
|
||||
|
||||
### 4.1 Audiences
|
||||
|
||||
* `signer` — only the **Signer** service should accept tokens with `aud=signer`.
|
||||
* `attestor`, `scanner`, `concelier`, `excititor`, `ui`, `zastava` similarly.
|
||||
|
||||
Services **must** verify `aud` and **sender constraint** (DPoP/mTLS) per their policy.
|
||||
|
||||
### 4.2 Core scopes
|
||||
|
||||
| Scope | Service | Operation |
|
||||
| ---------------------------------- | ------------------ | -------------------------- |
|
||||
| `signer.sign` | Signer | Request DSSE signing |
|
||||
| `attestor.write` | Attestor | Submit Rekor entries |
|
||||
| `scanner.scan` | Scanner.WebService | Submit scan jobs |
|
||||
| `scanner.export` | Scanner.WebService | Export SBOMs |
|
||||
| `scanner.read` | Scanner.WebService | Read catalog/SBOMs |
|
||||
| `vex.read` / `vex.admin` | Excititor | Query/operate |
|
||||
| `concelier.read` / `concelier.export` | Concelier | Query/exports |
|
||||
| `ui.read` / `ui.admin` | UI | View/admin |
|
||||
| `zastava.emit` / `zastava.enforce` | Scanner/Zastava | Runtime events / admission |
|
||||
|
||||
**Roles → scopes mapping** is configured centrally (Authority policy) and pushed during token issuance.
|
||||
|
||||
---
|
||||
|
||||
## 5) Storage & state
|
||||
|
||||
* **Configuration DB** (PostgreSQL/MySQL): clients, audiences, role→scope maps, tenant/installation registry, device code grants, persistent consents (if any).
|
||||
* **Cache** (Redis):
|
||||
|
||||
* DPoP **jti** replay cache (short TTL)
|
||||
* **Nonce** store (per resource server, if they demand nonce)
|
||||
* Device code pollers, rate limiting buckets
|
||||
* **JWKS**: key material in HSM/KMS or encrypted at rest; JWKS served from memory.
|
||||
|
||||
---
|
||||
|
||||
## 6) Key management & rotation
|
||||
|
||||
* Maintain **at least 2 signing keys** active during rotation; tokens carry `kid`.
|
||||
* Prefer **Ed25519** for compact tokens; maintain **ES256** fallback for FIPS contexts.
|
||||
* Rotation cadence: 30–90 days; emergency rotation supported.
|
||||
* Publish new JWKS **before** issuing tokens with the new `kid` to avoid cold‑start validation misses.
|
||||
* Keep **old keys** available **at least** for max token TTL + 5 minutes.
|
||||
|
||||
---
|
||||
|
||||
## 7) HA & performance
|
||||
|
||||
* **Stateless issuance** (except device codes/refresh) → scale horizontally behind a load‑balancer.
|
||||
* **DB** only for client metadata and optional flows; token checks are JWT‑local; introspection endpoints hit cache/DB minimally.
|
||||
* **Targets**:
|
||||
|
||||
* Token issuance P95 ≤ **20 ms** under warm cache.
|
||||
* DPoP proof validation ≤ **1 ms** extra per request at resource servers (Signer/Scanner).
|
||||
* 99.9% uptime; HPA on CPU/latency.
|
||||
|
||||
---
|
||||
|
||||
## 8) Security posture
|
||||
|
||||
* **Strict TLS** (1.3 preferred); HSTS; modern cipher suites.
|
||||
* **mTLS** enabled where required (Signer/Attestor paths).
|
||||
* **Replay protection**: DPoP `jti` cache, nonce support for **Signer** (add `DPoP-Nonce` header on 401; clients re‑sign).
|
||||
* **Rate limits** per client & per IP; exponential backoff on failures.
|
||||
* **Secrets**: clients use **private_key_jwt** or **mTLS**; never basic secrets over the wire.
|
||||
* **CSP/CSRF** hardening on UI flows; `SameSite=Lax` cookies; PKCE enforced.
|
||||
* **Logs** redact `Authorization` and DPoP proofs; store `sub`, `aud`, `scopes`, `inst`, `tid`, `cnf` thumbprints, not full keys.
|
||||
|
||||
---
|
||||
|
||||
## 9) Multi‑tenancy & installations
|
||||
|
||||
* **Tenant (`tid`)** and **Installation (`inst`)** registries define which audiences/scopes a client can request.
|
||||
* Cross‑tenant isolation enforced at issuance (disallow rogue `aud`), and resource servers **must** check that `tid` matches their configured tenant.
|
||||
|
||||
---
|
||||
|
||||
## 10) Admin & operations APIs
|
||||
|
||||
All under `/admin` (mTLS + `authority.admin` scope).
|
||||
|
||||
```
|
||||
POST /admin/clients # create/update client (confidential/public)
|
||||
POST /admin/audiences # register audience resource URIs
|
||||
POST /admin/roles # define role→scope mappings
|
||||
POST /admin/tenants # create tenant/install entries
|
||||
POST /admin/keys/rotate # rotate signing key (zero-downtime)
|
||||
GET /admin/metrics # Prometheus exposition (token issue rates, errors)
|
||||
GET /admin/healthz|readyz # health/readiness
|
||||
```
|
||||
|
||||
Declared client `audiences` flow through to the issued JWT `aud` claim and the token request's `resource` indicators. Authority relies on this metadata to enforce DPoP nonce challenges for `signer`, `attestor`, and other high-value services without requiring clients to repeat the audience parameter on every request.
|
||||
|
||||
---
|
||||
|
||||
## 11) Integration hard lines (what resource servers must enforce)
|
||||
|
||||
Every Stella Ops service that consumes Authority tokens **must**:
|
||||
|
||||
1. Verify JWT signature (`kid` in JWKS), `iss`, `aud`, `exp`, `nbf`.
|
||||
2. Enforce **sender‑constraint**:
|
||||
|
||||
* **DPoP**: validate DPoP proof (`htu`, `htm`, `iat`, `jti`) and match `cnf.jkt`; cache `jti` for replay defense; honor nonce challenges.
|
||||
* **mTLS**: match presented client cert thumbprint to token `cnf.x5t#S256`.
|
||||
3. Check **scopes**; optionally map to internal roles.
|
||||
4. Check **tenant** (`tid`) and **installation** (`inst`) as appropriate.
|
||||
5. For **Signer** only: require **both** OpTok and **PoE** in the request (enforced by Signer, not Authority).
|
||||
|
||||
---
|
||||
|
||||
## 12) Error surfaces & UX
|
||||
|
||||
* Token endpoint errors follow OAuth2 (`invalid_client`, `invalid_grant`, `invalid_scope`, `unauthorized_client`).
|
||||
* Resource servers use RFC 6750 style (`WWW-Authenticate: DPoP error="invalid_token", error_description="…", dpop_nonce="…" `).
|
||||
* For DPoP nonce challenges, clients retry with the server‑supplied nonce once.
|
||||
|
||||
---
|
||||
|
||||
## 13) Observability & audit
|
||||
|
||||
* **Metrics**:
|
||||
|
||||
* `authority.tokens_issued_total{grant,aud}`
|
||||
* `authority.dpop_validations_total{result}`
|
||||
* `authority.mtls_bindings_total{result}`
|
||||
* `authority.jwks_rotations_total`
|
||||
* `authority.errors_total{type}`
|
||||
* **Audit log** (immutable sink): token issuance (`sub`, `aud`, `scopes`, `tid`, `inst`, `cnf thumbprint`, `jti`), revocations, admin changes.
|
||||
* **Tracing**: token flows, DB reads, JWKS cache.
|
||||
|
||||
---
|
||||
|
||||
## 14) Configuration (YAML)
|
||||
|
||||
```yaml
|
||||
authority:
|
||||
issuer: "https://authority.internal"
|
||||
signing:
|
||||
enabled: true
|
||||
activeKeyId: "authority-signing-2025"
|
||||
keyPath: "../certificates/authority-signing-2025.pem"
|
||||
algorithm: "ES256"
|
||||
keySource: "file"
|
||||
security:
|
||||
rateLimiting:
|
||||
token:
|
||||
enabled: true
|
||||
permitLimit: 30
|
||||
window: "00:01:00"
|
||||
queueLimit: 0
|
||||
authorize:
|
||||
enabled: true
|
||||
permitLimit: 60
|
||||
window: "00:01:00"
|
||||
queueLimit: 10
|
||||
internal:
|
||||
enabled: false
|
||||
permitLimit: 5
|
||||
window: "00:01:00"
|
||||
queueLimit: 0
|
||||
senderConstraints:
|
||||
dpop:
|
||||
enabled: true
|
||||
allowedAlgorithms: [ "ES256", "ES384" ]
|
||||
proofLifetime: "00:02:00"
|
||||
allowedClockSkew: "00:00:30"
|
||||
replayWindow: "00:05:00"
|
||||
nonce:
|
||||
enabled: true
|
||||
ttl: "00:10:00"
|
||||
maxIssuancePerMinute: 120
|
||||
store: "redis"
|
||||
redisConnectionString: "redis://authority-redis:6379?ssl=false"
|
||||
requiredAudiences:
|
||||
- "signer"
|
||||
- "attestor"
|
||||
mtls:
|
||||
enabled: true
|
||||
requireChainValidation: true
|
||||
rotationGrace: "00:15:00"
|
||||
enforceForAudiences:
|
||||
- "signer"
|
||||
allowedSanTypes:
|
||||
- "dns"
|
||||
- "uri"
|
||||
allowedCertificateAuthorities:
|
||||
- "/etc/ssl/mtls/clients-ca.pem"
|
||||
clients:
|
||||
- clientId: scanner-web
|
||||
grantTypes: [ "client_credentials" ]
|
||||
audiences: [ "scanner" ]
|
||||
auth: { type: "private_key_jwt", jwkFile: "/secrets/scanner-web.jwk" }
|
||||
senderConstraint: "dpop"
|
||||
scopes: [ "scanner.scan", "scanner.export", "scanner.read" ]
|
||||
- clientId: signer
|
||||
grantTypes: [ "client_credentials" ]
|
||||
audiences: [ "signer" ]
|
||||
auth: { type: "mtls" }
|
||||
senderConstraint: "mtls"
|
||||
scopes: [ "signer.sign" ]
|
||||
- clientId: notify-web-dev
|
||||
grantTypes: [ "client_credentials" ]
|
||||
audiences: [ "notify.dev" ]
|
||||
auth: { type: "client_secret", secretFile: "/secrets/notify-web-dev.secret" }
|
||||
senderConstraint: "dpop"
|
||||
scopes: [ "notify.read", "notify.admin" ]
|
||||
- clientId: notify-web
|
||||
grantTypes: [ "client_credentials" ]
|
||||
audiences: [ "notify" ]
|
||||
auth: { type: "client_secret", secretFile: "/secrets/notify-web.secret" }
|
||||
senderConstraint: "dpop"
|
||||
scopes: [ "notify.read", "notify.admin" ]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 15) Testing matrix
|
||||
|
||||
* **JWT validation**: wrong `aud`, expired `exp`, skewed `nbf`, stale `kid`.
|
||||
* **DPoP**: invalid `htu`/`htm`, replayed `jti`, stale `iat`, wrong `jkt`, nonce dance.
|
||||
* **mTLS**: wrong client cert, wrong CA, thumbprint mismatch.
|
||||
* **RBAC**: scope enforcement per audience; over‑privileged client denied.
|
||||
* **Rotation**: JWKS rotation while load‑testing; zero‑downtime verification.
|
||||
* **HA**: kill one Authority instance; verify issuance continues; JWKS served by peers.
|
||||
* **Performance**: 1k token issuance/sec on 2 cores with Redis enabled for jti caching.
|
||||
|
||||
---
|
||||
|
||||
## 16) Threat model & mitigations (summary)
|
||||
|
||||
| Threat | Vector | Mitigation |
|
||||
| ------------------- | ---------------- | ------------------------------------------------------------------------------------------ |
|
||||
| Token theft | Copy of JWT | **Short TTL**, **sender‑constraint** (DPoP/mTLS); replay blocked by `jti` cache and nonces |
|
||||
| Replay across hosts | Reuse DPoP proof | Enforce `htu`/`htm`, `iat` freshness, `jti` uniqueness; services may require **nonce** |
|
||||
| Impersonation | Fake client | mTLS or `private_key_jwt` with pinned JWK; client registration & rotation |
|
||||
| Key compromise | Signing key leak | HSM/KMS storage, key rotation, audit; emergency key revoke path; narrow token TTL |
|
||||
| Cross‑tenant abuse | Scope elevation | Enforce `aud`, `tid`, `inst` at issuance and resource servers |
|
||||
| Downgrade to bearer | Strip DPoP | Resource servers require DPoP/mTLS based on `aud`; reject bearer without `cnf` |
|
||||
|
||||
---
|
||||
|
||||
## 17) Deployment & HA
|
||||
|
||||
* **Stateless** microservice, containerized; run ≥ 2 replicas behind LB.
|
||||
* **DB**: HA Postgres (or MySQL) for clients/roles; **Redis** for device codes, DPoP nonces/jtis.
|
||||
* **Secrets**: mount client JWKs via K8s Secrets/HashiCorp Vault; signing keys via KMS.
|
||||
* **Backups**: DB daily; Redis not critical (ephemeral).
|
||||
* **Disaster recovery**: export/import of client registry; JWKS rehydrate from KMS.
|
||||
* **Compliance**: TLS audit; penetration testing for OIDC flows.
|
||||
|
||||
---
|
||||
|
||||
## 18) Implementation notes
|
||||
|
||||
* Reference stack: **.NET 10** + **OpenIddict 6** (or IdentityServer if licensed) with custom DPoP validator and mTLS binding middleware.
|
||||
* Keep the DPoP/JTI cache pluggable; allow Redis/Memcached.
|
||||
* Provide **client SDKs** for C# and Go: DPoP key mgmt, proof generation, nonce handling, token refresh helper.
|
||||
|
||||
---
|
||||
|
||||
## 19) Quick reference — wire examples
|
||||
|
||||
**Access token (payload excerpt)**
|
||||
|
||||
```json
|
||||
{
|
||||
"iss": "https://authority.internal",
|
||||
"sub": "scanner-web",
|
||||
"aud": "signer",
|
||||
"exp": 1760668800,
|
||||
"iat": 1760668620,
|
||||
"nbf": 1760668620,
|
||||
"jti": "9d9c3f01-6e1a-49f1-8f77-9b7e6f7e3c50",
|
||||
"scope": "signer.sign",
|
||||
"tid": "tenant-01",
|
||||
"inst": "install-7A2B",
|
||||
"cnf": { "jkt": "KcVb2V...base64url..." }
|
||||
}
|
||||
```
|
||||
|
||||
**DPoP proof header fields (for POST /sign/dsse)**
|
||||
|
||||
```json
|
||||
{
|
||||
"htu": "https://signer.internal/sign/dsse",
|
||||
"htm": "POST",
|
||||
"iat": 1760668620,
|
||||
"jti": "4b1c9b3c-8a95-4c58-8a92-9c6cfb4a6a0b"
|
||||
}
|
||||
```
|
||||
|
||||
Signer validates that `hash(JWK)` in the proof matches `cnf.jkt` in the token.
|
||||
|
||||
---
|
||||
|
||||
## 20) Rollout plan
|
||||
|
||||
1. **MVP**: Client Credentials (private_key_jwt + DPoP), JWKS, short OpToks, per‑audience scopes.
|
||||
2. **Add**: mTLS‑bound tokens for Signer/Attestor; device code for CLI; optional introspection.
|
||||
3. **Hardening**: DPoP nonce support; full audit pipeline; HA tuning.
|
||||
4. **UX**: Tenant/installation admin UI; role→scope editors; client bootstrap wizards.
|
||||
* **Device Code** (CLI): `POST /oauth/device/code` + `POST /oauth/token` poll
|
||||
* **Authorization Code + PKCE** (UI): standard
|
||||
|
||||
**DPoP handshake (example)**
|
||||
|
||||
1. Client prepares **JWK** (ephemeral keypair).
|
||||
2. Client sends **DPoP proof** header with fields:
|
||||
|
||||
```
|
||||
htm=POST
|
||||
htu=https://authority.../token
|
||||
iat=<now>
|
||||
jti=<uuid>
|
||||
```
|
||||
|
||||
signed with the DPoP private key; header carries JWK.
|
||||
3. Authority validates proof; issues access token with `cnf.jkt=<thumbprint(JWK)>`.
|
||||
4. Client uses the same DPoP key to sign **every subsequent API request** to services (Signer, Scanner, …).
|
||||
|
||||
**mTLS flow**
|
||||
|
||||
* Mutual TLS at the connection; Authority extracts client cert, validates chain; token carries `cnf.x5t#S256`.
|
||||
|
||||
### 3.3 Introspection & revocation (optional)
|
||||
|
||||
* `POST /introspect` → `{ active, sub, scope, aud, exp, cnf, ... }`
|
||||
* `POST /revoke` → revokes refresh tokens or opaque access tokens.
|
||||
|
||||
> Requests targeting the legacy `/oauth/{introspect|revoke}` paths receive deprecation headers and are scheduled for removal after 1 May 2026.
|
||||
* **Replay prevention**: maintain **DPoP `jti` cache** (TTL ≤ 10 min) to reject duplicate proofs when services supply DPoP nonces (Signer requires nonce for high‑value operations).
|
||||
|
||||
### 3.4 UserInfo (optional for UI)
|
||||
|
||||
* `GET /userinfo` (ID token context).
|
||||
|
||||
---
|
||||
|
||||
## 4) Audiences, scopes & RBAC
|
||||
|
||||
### 4.1 Audiences
|
||||
|
||||
* `signer` — only the **Signer** service should accept tokens with `aud=signer`.
|
||||
* `attestor`, `scanner`, `concelier`, `excititor`, `ui`, `zastava` similarly.
|
||||
|
||||
Services **must** verify `aud` and **sender constraint** (DPoP/mTLS) per their policy.
|
||||
|
||||
### 4.2 Core scopes
|
||||
|
||||
| Scope | Service | Operation |
|
||||
| ---------------------------------- | ------------------ | -------------------------- |
|
||||
| `signer.sign` | Signer | Request DSSE signing |
|
||||
| `attestor.write` | Attestor | Submit Rekor entries |
|
||||
| `scanner.scan` | Scanner.WebService | Submit scan jobs |
|
||||
| `scanner.export` | Scanner.WebService | Export SBOMs |
|
||||
| `scanner.read` | Scanner.WebService | Read catalog/SBOMs |
|
||||
| `vex.read` / `vex.admin` | Excititor | Query/operate |
|
||||
| `concelier.read` / `concelier.export` | Concelier | Query/exports |
|
||||
| `ui.read` / `ui.admin` | UI | View/admin |
|
||||
| `zastava.emit` / `zastava.enforce` | Scanner/Zastava | Runtime events / admission |
|
||||
|
||||
**Roles → scopes mapping** is configured centrally (Authority policy) and pushed during token issuance.
|
||||
|
||||
---
|
||||
|
||||
## 5) Storage & state
|
||||
|
||||
* **Configuration DB** (PostgreSQL/MySQL): clients, audiences, role→scope maps, tenant/installation registry, device code grants, persistent consents (if any).
|
||||
* **Cache** (Redis):
|
||||
|
||||
* DPoP **jti** replay cache (short TTL)
|
||||
* **Nonce** store (per resource server, if they demand nonce)
|
||||
* Device code pollers, rate limiting buckets
|
||||
* **JWKS**: key material in HSM/KMS or encrypted at rest; JWKS served from memory.
|
||||
|
||||
---
|
||||
|
||||
## 6) Key management & rotation
|
||||
|
||||
* Maintain **at least 2 signing keys** active during rotation; tokens carry `kid`.
|
||||
* Prefer **Ed25519** for compact tokens; maintain **ES256** fallback for FIPS contexts.
|
||||
* Rotation cadence: 30–90 days; emergency rotation supported.
|
||||
* Publish new JWKS **before** issuing tokens with the new `kid` to avoid cold‑start validation misses.
|
||||
* Keep **old keys** available **at least** for max token TTL + 5 minutes.
|
||||
|
||||
---
|
||||
|
||||
## 7) HA & performance
|
||||
|
||||
* **Stateless issuance** (except device codes/refresh) → scale horizontally behind a load‑balancer.
|
||||
* **DB** only for client metadata and optional flows; token checks are JWT‑local; introspection endpoints hit cache/DB minimally.
|
||||
* **Targets**:
|
||||
|
||||
* Token issuance P95 ≤ **20 ms** under warm cache.
|
||||
* DPoP proof validation ≤ **1 ms** extra per request at resource servers (Signer/Scanner).
|
||||
* 99.9% uptime; HPA on CPU/latency.
|
||||
|
||||
---
|
||||
|
||||
## 8) Security posture
|
||||
|
||||
* **Strict TLS** (1.3 preferred); HSTS; modern cipher suites.
|
||||
* **mTLS** enabled where required (Signer/Attestor paths).
|
||||
* **Replay protection**: DPoP `jti` cache, nonce support for **Signer** (add `DPoP-Nonce` header on 401; clients re‑sign).
|
||||
* **Rate limits** per client & per IP; exponential backoff on failures.
|
||||
* **Secrets**: clients use **private_key_jwt** or **mTLS**; never basic secrets over the wire.
|
||||
* **CSP/CSRF** hardening on UI flows; `SameSite=Lax` cookies; PKCE enforced.
|
||||
* **Logs** redact `Authorization` and DPoP proofs; store `sub`, `aud`, `scopes`, `inst`, `tid`, `cnf` thumbprints, not full keys.
|
||||
|
||||
---
|
||||
|
||||
## 9) Multi‑tenancy & installations
|
||||
|
||||
* **Tenant (`tid`)** and **Installation (`inst`)** registries define which audiences/scopes a client can request.
|
||||
* Cross‑tenant isolation enforced at issuance (disallow rogue `aud`), and resource servers **must** check that `tid` matches their configured tenant.
|
||||
|
||||
---
|
||||
|
||||
## 10) Admin & operations APIs
|
||||
|
||||
All under `/admin` (mTLS + `authority.admin` scope).
|
||||
|
||||
```
|
||||
POST /admin/clients # create/update client (confidential/public)
|
||||
POST /admin/audiences # register audience resource URIs
|
||||
POST /admin/roles # define role→scope mappings
|
||||
POST /admin/tenants # create tenant/install entries
|
||||
POST /admin/keys/rotate # rotate signing key (zero-downtime)
|
||||
GET /admin/metrics # Prometheus exposition (token issue rates, errors)
|
||||
GET /admin/healthz|readyz # health/readiness
|
||||
```
|
||||
|
||||
Declared client `audiences` flow through to the issued JWT `aud` claim and the token request's `resource` indicators. Authority relies on this metadata to enforce DPoP nonce challenges for `signer`, `attestor`, and other high-value services without requiring clients to repeat the audience parameter on every request.
|
||||
|
||||
---
|
||||
|
||||
## 11) Integration hard lines (what resource servers must enforce)
|
||||
|
||||
Every Stella Ops service that consumes Authority tokens **must**:
|
||||
|
||||
1. Verify JWT signature (`kid` in JWKS), `iss`, `aud`, `exp`, `nbf`.
|
||||
2. Enforce **sender‑constraint**:
|
||||
|
||||
* **DPoP**: validate DPoP proof (`htu`, `htm`, `iat`, `jti`) and match `cnf.jkt`; cache `jti` for replay defense; honor nonce challenges.
|
||||
* **mTLS**: match presented client cert thumbprint to token `cnf.x5t#S256`.
|
||||
3. Check **scopes**; optionally map to internal roles.
|
||||
4. Check **tenant** (`tid`) and **installation** (`inst`) as appropriate.
|
||||
5. For **Signer** only: require **both** OpTok and **PoE** in the request (enforced by Signer, not Authority).
|
||||
|
||||
---
|
||||
|
||||
## 12) Error surfaces & UX
|
||||
|
||||
* Token endpoint errors follow OAuth2 (`invalid_client`, `invalid_grant`, `invalid_scope`, `unauthorized_client`).
|
||||
* Resource servers use RFC 6750 style (`WWW-Authenticate: DPoP error="invalid_token", error_description="…", dpop_nonce="…" `).
|
||||
* For DPoP nonce challenges, clients retry with the server‑supplied nonce once.
|
||||
|
||||
---
|
||||
|
||||
## 13) Observability & audit
|
||||
|
||||
* **Metrics**:
|
||||
|
||||
* `authority.tokens_issued_total{grant,aud}`
|
||||
* `authority.dpop_validations_total{result}`
|
||||
* `authority.mtls_bindings_total{result}`
|
||||
* `authority.jwks_rotations_total`
|
||||
* `authority.errors_total{type}`
|
||||
* **Audit log** (immutable sink): token issuance (`sub`, `aud`, `scopes`, `tid`, `inst`, `cnf thumbprint`, `jti`), revocations, admin changes.
|
||||
* **Tracing**: token flows, DB reads, JWKS cache.
|
||||
|
||||
---
|
||||
|
||||
## 14) Configuration (YAML)
|
||||
|
||||
```yaml
|
||||
authority:
|
||||
issuer: "https://authority.internal"
|
||||
signing:
|
||||
enabled: true
|
||||
activeKeyId: "authority-signing-2025"
|
||||
keyPath: "../certificates/authority-signing-2025.pem"
|
||||
algorithm: "ES256"
|
||||
keySource: "file"
|
||||
security:
|
||||
rateLimiting:
|
||||
token:
|
||||
enabled: true
|
||||
permitLimit: 30
|
||||
window: "00:01:00"
|
||||
queueLimit: 0
|
||||
authorize:
|
||||
enabled: true
|
||||
permitLimit: 60
|
||||
window: "00:01:00"
|
||||
queueLimit: 10
|
||||
internal:
|
||||
enabled: false
|
||||
permitLimit: 5
|
||||
window: "00:01:00"
|
||||
queueLimit: 0
|
||||
senderConstraints:
|
||||
dpop:
|
||||
enabled: true
|
||||
allowedAlgorithms: [ "ES256", "ES384" ]
|
||||
proofLifetime: "00:02:00"
|
||||
allowedClockSkew: "00:00:30"
|
||||
replayWindow: "00:05:00"
|
||||
nonce:
|
||||
enabled: true
|
||||
ttl: "00:10:00"
|
||||
maxIssuancePerMinute: 120
|
||||
store: "redis"
|
||||
redisConnectionString: "redis://authority-redis:6379?ssl=false"
|
||||
requiredAudiences:
|
||||
- "signer"
|
||||
- "attestor"
|
||||
mtls:
|
||||
enabled: true
|
||||
requireChainValidation: true
|
||||
rotationGrace: "00:15:00"
|
||||
enforceForAudiences:
|
||||
- "signer"
|
||||
allowedSanTypes:
|
||||
- "dns"
|
||||
- "uri"
|
||||
allowedCertificateAuthorities:
|
||||
- "/etc/ssl/mtls/clients-ca.pem"
|
||||
clients:
|
||||
- clientId: scanner-web
|
||||
grantTypes: [ "client_credentials" ]
|
||||
audiences: [ "scanner" ]
|
||||
auth: { type: "private_key_jwt", jwkFile: "/secrets/scanner-web.jwk" }
|
||||
senderConstraint: "dpop"
|
||||
scopes: [ "scanner.scan", "scanner.export", "scanner.read" ]
|
||||
- clientId: signer
|
||||
grantTypes: [ "client_credentials" ]
|
||||
audiences: [ "signer" ]
|
||||
auth: { type: "mtls" }
|
||||
senderConstraint: "mtls"
|
||||
scopes: [ "signer.sign" ]
|
||||
- clientId: notify-web-dev
|
||||
grantTypes: [ "client_credentials" ]
|
||||
audiences: [ "notify.dev" ]
|
||||
auth: { type: "client_secret", secretFile: "/secrets/notify-web-dev.secret" }
|
||||
senderConstraint: "dpop"
|
||||
scopes: [ "notify.viewer", "notify.operator", "notify.admin" ]
|
||||
- clientId: notify-web
|
||||
grantTypes: [ "client_credentials" ]
|
||||
audiences: [ "notify" ]
|
||||
auth: { type: "client_secret", secretFile: "/secrets/notify-web.secret" }
|
||||
senderConstraint: "dpop"
|
||||
scopes: [ "notify.viewer", "notify.operator" ]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 15) Testing matrix
|
||||
|
||||
* **JWT validation**: wrong `aud`, expired `exp`, skewed `nbf`, stale `kid`.
|
||||
* **DPoP**: invalid `htu`/`htm`, replayed `jti`, stale `iat`, wrong `jkt`, nonce dance.
|
||||
* **mTLS**: wrong client cert, wrong CA, thumbprint mismatch.
|
||||
* **RBAC**: scope enforcement per audience; over‑privileged client denied.
|
||||
* **Rotation**: JWKS rotation while load‑testing; zero‑downtime verification.
|
||||
* **HA**: kill one Authority instance; verify issuance continues; JWKS served by peers.
|
||||
* **Performance**: 1k token issuance/sec on 2 cores with Redis enabled for jti caching.
|
||||
|
||||
---
|
||||
|
||||
## 16) Threat model & mitigations (summary)
|
||||
|
||||
| Threat | Vector | Mitigation |
|
||||
| ------------------- | ---------------- | ------------------------------------------------------------------------------------------ |
|
||||
| Token theft | Copy of JWT | **Short TTL**, **sender‑constraint** (DPoP/mTLS); replay blocked by `jti` cache and nonces |
|
||||
| Replay across hosts | Reuse DPoP proof | Enforce `htu`/`htm`, `iat` freshness, `jti` uniqueness; services may require **nonce** |
|
||||
| Impersonation | Fake client | mTLS or `private_key_jwt` with pinned JWK; client registration & rotation |
|
||||
| Key compromise | Signing key leak | HSM/KMS storage, key rotation, audit; emergency key revoke path; narrow token TTL |
|
||||
| Cross‑tenant abuse | Scope elevation | Enforce `aud`, `tid`, `inst` at issuance and resource servers |
|
||||
| Downgrade to bearer | Strip DPoP | Resource servers require DPoP/mTLS based on `aud`; reject bearer without `cnf` |
|
||||
|
||||
---
|
||||
|
||||
## 17) Deployment & HA
|
||||
|
||||
* **Stateless** microservice, containerized; run ≥ 2 replicas behind LB.
|
||||
* **DB**: HA Postgres (or MySQL) for clients/roles; **Redis** for device codes, DPoP nonces/jtis.
|
||||
* **Secrets**: mount client JWKs via K8s Secrets/HashiCorp Vault; signing keys via KMS.
|
||||
* **Backups**: DB daily; Redis not critical (ephemeral).
|
||||
* **Disaster recovery**: export/import of client registry; JWKS rehydrate from KMS.
|
||||
* **Compliance**: TLS audit; penetration testing for OIDC flows.
|
||||
|
||||
---
|
||||
|
||||
## 18) Implementation notes
|
||||
|
||||
* Reference stack: **.NET 10** + **OpenIddict 6** (or IdentityServer if licensed) with custom DPoP validator and mTLS binding middleware.
|
||||
* Keep the DPoP/JTI cache pluggable; allow Redis/Memcached.
|
||||
* Provide **client SDKs** for C# and Go: DPoP key mgmt, proof generation, nonce handling, token refresh helper.
|
||||
|
||||
---
|
||||
|
||||
## 19) Quick reference — wire examples
|
||||
|
||||
**Access token (payload excerpt)**
|
||||
|
||||
```json
|
||||
{
|
||||
"iss": "https://authority.internal",
|
||||
"sub": "scanner-web",
|
||||
"aud": "signer",
|
||||
"exp": 1760668800,
|
||||
"iat": 1760668620,
|
||||
"nbf": 1760668620,
|
||||
"jti": "9d9c3f01-6e1a-49f1-8f77-9b7e6f7e3c50",
|
||||
"scope": "signer.sign",
|
||||
"tid": "tenant-01",
|
||||
"inst": "install-7A2B",
|
||||
"cnf": { "jkt": "KcVb2V...base64url..." }
|
||||
}
|
||||
```
|
||||
|
||||
**DPoP proof header fields (for POST /sign/dsse)**
|
||||
|
||||
```json
|
||||
{
|
||||
"htu": "https://signer.internal/sign/dsse",
|
||||
"htm": "POST",
|
||||
"iat": 1760668620,
|
||||
"jti": "4b1c9b3c-8a95-4c58-8a92-9c6cfb4a6a0b"
|
||||
}
|
||||
```
|
||||
|
||||
Signer validates that `hash(JWK)` in the proof matches `cnf.jkt` in the token.
|
||||
|
||||
---
|
||||
|
||||
## 20) Rollout plan
|
||||
|
||||
1. **MVP**: Client Credentials (private_key_jwt + DPoP), JWKS, short OpToks, per‑audience scopes.
|
||||
2. **Add**: mTLS‑bound tokens for Signer/Attestor; device code for CLI; optional introspection.
|
||||
3. **Hardening**: DPoP nonce support; full audit pipeline; HA tuning.
|
||||
4. **UX**: Tenant/installation admin UI; role→scope editors; client bootstrap wizards.
|
||||
|
||||
@@ -145,14 +145,14 @@ Both subcommands honour offline-first expectations (no network access) and norma
|
||||
|
||||
### 3.3 Multi‑audience & scopes
|
||||
|
||||
* CLI requests **audiences** as needed per verb:
|
||||
|
||||
* `scanner` for scan/export/report/diff
|
||||
* `signer` (indirect; usually backend calls Signer)
|
||||
* `attestor` for verify
|
||||
* `concelier`/`excititor` for admin verbs
|
||||
|
||||
CLI rejects verbs if required scopes are missing.
|
||||
* CLI requests **audiences** as needed per verb:
|
||||
|
||||
* `scanner` for scan/export/report/diff
|
||||
* `signer` (indirect; usually backend calls Signer)
|
||||
* `attestor` for verify (requires `attestor.verify` scope; read-only verbs fall back to `attestor.read`)
|
||||
* `concelier`/`excititor` for admin verbs
|
||||
|
||||
CLI rejects verbs if required scopes are missing.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -1,316 +1,318 @@
|
||||
# CLI AOC Commands Reference
|
||||
|
||||
> **Audience:** DevEx engineers, operators, and CI authors integrating the `stella` CLI with Aggregation-Only Contract (AOC) workflows.
|
||||
> **Scope:** Command synopsis, options, exit codes, and offline considerations for `stella sources ingest --dry-run` and `stella aoc verify` as introduced in Sprint 19.
|
||||
|
||||
# CLI AOC Commands Reference
|
||||
|
||||
> **Audience:** DevEx engineers, operators, and CI authors integrating the `stella` CLI with Aggregation-Only Contract (AOC) workflows.
|
||||
> **Scope:** Command synopsis, options, exit codes, and offline considerations for `stella sources ingest --dry-run` and `stella aoc verify` as introduced in Sprint 19.
|
||||
|
||||
Both commands are designed to enforce the AOC guardrails documented in the [aggregation-only reference](../../../ingestion/aggregation-only-contract.md) and the [architecture overview](../architecture.md). They consume Authority-issued tokens with tenant scopes and never mutate ingestion stores.
|
||||
|
||||
---
|
||||
|
||||
## 1 · Prerequisites
|
||||
|
||||
- CLI version: `stella` ≥ 0.19.0 (AOC feature gate enabled).
|
||||
- Required scopes (DPoP-bound):
|
||||
- `advisory:read` for Concelier sources.
|
||||
- `vex:read` for Excititor sources (optional but required for VEX checks).
|
||||
- `aoc:verify` to invoke guard verification endpoints.
|
||||
- `tenant:select` if your deployment uses tenant switching.
|
||||
- Connectivity: direct access to Concelier/Excititor APIs or Offline Kit snapshot (see § 4).
|
||||
- Environment: set `STELLA_AUTHORITY_URL`, `STELLA_TENANT`, and export a valid OpTok via `stella auth login` or existing token cache.
|
||||
|
||||
---
|
||||
|
||||
## 2 · `stella sources ingest --dry-run`
|
||||
|
||||
### 2.1 Synopsis
|
||||
|
||||
```bash
|
||||
stella sources ingest --dry-run \
|
||||
--source <source-key> \
|
||||
--input <path-or-uri> \
|
||||
[--tenant <tenant-id>] \
|
||||
[--format json|table] \
|
||||
[--no-color] \
|
||||
[--output <file>]
|
||||
```
|
||||
|
||||
### 2.2 Description
|
||||
|
||||
Previews an ingestion write without touching MongoDB. The command loads an upstream advisory or VEX document, computes the would-write payload, runs it through the `AOCWriteGuard`, and reports any forbidden fields, provenance gaps, or idempotency issues. Use it during connector development, CI validation, or while triaging incidents.
|
||||
|
||||
### 2.3 Options
|
||||
|
||||
| Option | Description |
|
||||
|--------|-------------|
|
||||
| `--source <source-key>` | Logical source name (`redhat`, `ubuntu`, `osv`, etc.). Mirrors connector configuration. |
|
||||
| `--input <path-or-uri>` | Path to local CSAF/OSV/VEX file or HTTPS URI. CLI normalises transport (gzip/base64) before guard evaluation. |
|
||||
| `--tenant <tenant-id>` | Overrides default tenant for multi-tenant deployments. Mandatory when `STELLA_TENANT` is not set. |
|
||||
| `--format json|table` | Output format. `table` (default) prints summary with highlighted violations; `json` emits machine-readable report (see below). |
|
||||
| `--no-color` | Disables ANSI colour output for CI logs. |
|
||||
| `--output <file>` | Writes the JSON report to file while still printing human-readable summary to stdout. |
|
||||
|
||||
### 2.4 Output schema (JSON)
|
||||
|
||||
```json
|
||||
{
|
||||
"source": "redhat",
|
||||
"tenant": "default",
|
||||
"guardVersion": "1.0.0",
|
||||
"status": "ok",
|
||||
"document": {
|
||||
"contentHash": "sha256:…",
|
||||
"supersedes": null,
|
||||
"provenance": {
|
||||
"signature": { "format": "pgp", "present": true }
|
||||
}
|
||||
},
|
||||
"violations": []
|
||||
}
|
||||
```
|
||||
|
||||
When violations exist, `status` becomes `error` and `violations` contains entries with `code` (`ERR_AOC_00x`), a short `message`, and JSON Pointer `path` values indicating offending fields.
|
||||
|
||||
### 2.5 Exit codes
|
||||
|
||||
| Exit code | Meaning |
|
||||
|-----------|---------|
|
||||
| `0` | Guard passed; would-write payload is AOC compliant. |
|
||||
| `11` | `ERR_AOC_001` – Forbidden field (`severity`, `cvss`, etc.) detected. |
|
||||
| `12` | `ERR_AOC_002` – Merge attempt (multiple upstream sources fused). |
|
||||
| `13` | `ERR_AOC_003` – Idempotency violation (duplicate without supersedes). |
|
||||
| `14` | `ERR_AOC_004` – Missing provenance fields. |
|
||||
| `15` | `ERR_AOC_005` – Signature/checksum mismatch. |
|
||||
| `16` | `ERR_AOC_006` – Effective findings present (Policy-only data). |
|
||||
| `17` | `ERR_AOC_007` – Unknown top-level fields / schema violation. |
|
||||
| `70` | Transport error (network, auth, malformed input). |
|
||||
|
||||
> Exit codes map directly to the `ERR_AOC_00x` table for scripting consistency. Multiple violations yield the highest-priority code (e.g., 11 takes precedence over 14).
|
||||
|
||||
### 2.6 Examples
|
||||
|
||||
Dry-run a local CSAF file:
|
||||
|
||||
```bash
|
||||
stella sources ingest --dry-run \
|
||||
--source redhat \
|
||||
--input ./fixtures/redhat/RHSA-2025-1234.json
|
||||
```
|
||||
|
||||
Stream from HTTPS and emit JSON for CI:
|
||||
|
||||
```bash
|
||||
stella sources ingest --dry-run \
|
||||
--source osv \
|
||||
--input https://osv.dev/vulnerability/GHSA-aaaa-bbbb \
|
||||
--format json \
|
||||
--output artifacts/osv-dry-run.json
|
||||
|
||||
cat artifacts/osv-dry-run.json | jq '.violations'
|
||||
```
|
||||
|
||||
### 2.7 Offline notes
|
||||
|
||||
When operating in sealed/offline mode:
|
||||
|
||||
- Use `--input` paths pointing to Offline Kit snapshots (`offline-kit/advisories/*.json`).
|
||||
- Provide `--tenant` explicitly if the offline bundle contains multiple tenants.
|
||||
- The command does not attempt network access when given a file path.
|
||||
- Store reports with `--output` to include in transfer packages for policy review.
|
||||
|
||||
---
|
||||
|
||||
## 3 · `stella aoc verify`
|
||||
|
||||
### 3.1 Synopsis
|
||||
|
||||
```bash
|
||||
stella aoc verify \
|
||||
[--since <iso8601|duration>] \
|
||||
[--limit <count>] \
|
||||
[--sources <list>] \
|
||||
[--codes <ERR_AOC_00x,...>] \
|
||||
[--format table|json] \
|
||||
[--export <file>] \
|
||||
[--tenant <tenant-id>] \
|
||||
[--no-color]
|
||||
```
|
||||
|
||||
### 3.2 Description
|
||||
|
||||
Replays the AOC guard against stored raw documents. By default it checks all advisories and VEX statements ingested in the last 24 hours for the active tenant, reporting totals, top violation codes, and sample documents. Use it in CI pipelines, scheduled verifications, or during incident response.
|
||||
|
||||
### 3.3 Options
|
||||
|
||||
| Option | Description |
|
||||
|--------|-------------|
|
||||
| `--since <value>` | Verification window. Accepts ISO 8601 timestamp (`2025-10-25T12:00:00Z`) or duration (`48h`, `7d`). Defaults to `24h`. |
|
||||
| `--limit <count>` | Maximum number of violations to display (per code). `0` means show all. Defaults to `20`. |
|
||||
| `--sources <list>` | Comma-separated list of sources (`redhat,ubuntu,osv`). Filters both advisories and VEX entries. |
|
||||
| `--codes <list>` | Restricts output to specific `ERR_AOC_00x` codes. Useful for regression tracking. |
|
||||
| `--format table|json` | `table` (default) prints summary plus top violations; `json` outputs machine-readable report identical to the `/aoc/verify` API. |
|
||||
| `--export <file>` | Writes the JSON report to disk (useful for audits/offline uploads). |
|
||||
| `--tenant <tenant-id>` | Overrides tenant context. Required for cross-tenant verifications when run by platform operators. |
|
||||
| `--no-color` | Disables ANSI colours. |
|
||||
|
||||
`table` mode prints a summary showing the active tenant, evaluated window, counts of checked advisories/VEX statements, the active limit, total writes/violations, and whether the page was truncated. Status is colour-coded as `ok`, `violations`, or `truncated`. When violations exist the detail table lists the code, total occurrences, first sample document (`source` + `documentId` + `contentHash`), and JSON pointer path.
|
||||
|
||||
### 3.4 Report structure (JSON)
|
||||
|
||||
```json
|
||||
{
|
||||
"tenant": "default",
|
||||
"window": {
|
||||
"from": "2025-10-25T12:00:00Z",
|
||||
"to": "2025-10-26T12:00:00Z"
|
||||
},
|
||||
"checked": {
|
||||
"advisories": 482,
|
||||
"vex": 75
|
||||
},
|
||||
"violations": [
|
||||
{
|
||||
"code": "ERR_AOC_001",
|
||||
"count": 2,
|
||||
"examples": [
|
||||
{
|
||||
"source": "redhat",
|
||||
"documentId": "advisory_raw:redhat:RHSA-2025:1",
|
||||
"contentHash": "sha256:…",
|
||||
"path": "/content/raw/cvss"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"metrics": {
|
||||
"ingestion_write_total": 557,
|
||||
"aoc_violation_total": 2
|
||||
},
|
||||
"truncated": false
|
||||
}
|
||||
```
|
||||
|
||||
### 3.5 Exit codes
|
||||
|
||||
| Exit code | Meaning |
|
||||
|-----------|---------|
|
||||
| `0` | Verification succeeded with zero violations. |
|
||||
| `11…17` | Same mapping as § 2.5 when violations are detected. Highest-priority code returned. |
|
||||
| `18` | Verification ran but results truncated (limit reached) – treat as warning; rerun with higher `--limit`. |
|
||||
| `70` | Transport/authentication error. |
|
||||
| `71` | CLI misconfiguration (missing tenant, invalid `--since`, etc.). |
|
||||
|
||||
### 3.6 Examples
|
||||
|
||||
Daily verification across all sources:
|
||||
|
||||
```bash
|
||||
stella aoc verify --since 24h --format table
|
||||
```
|
||||
|
||||
CI pipeline focusing on errant sources and exporting evidence:
|
||||
|
||||
```bash
|
||||
stella aoc verify \
|
||||
--sources redhat,ubuntu \
|
||||
--codes ERR_AOC_001,ERR_AOC_004 \
|
||||
--format json \
|
||||
--limit 100 \
|
||||
--export artifacts/aoc-verify.json
|
||||
|
||||
jq '.violations[] | {code, count}' artifacts/aoc-verify.json
|
||||
```
|
||||
|
||||
Air-gapped verification using Offline Kit snapshot (example script):
|
||||
|
||||
```bash
|
||||
stella aoc verify \
|
||||
--since 7d \
|
||||
--format json \
|
||||
--export /mnt/offline/aoc-verify-$(date +%F).json
|
||||
|
||||
sha256sum /mnt/offline/aoc-verify-*.json > /mnt/offline/checksums.txt
|
||||
```
|
||||
|
||||
### 3.7 Automation tips
|
||||
|
||||
- Schedule with `cron` or platform scheduler and fail the job when exit code ≥ 11.
|
||||
- Pair with `stella sources ingest --dry-run` for pre-flight validation before re-enabling a paused source.
|
||||
- Push JSON exports to observability pipelines for historical tracking of violation counts.
|
||||
|
||||
### 3.8 Offline notes
|
||||
|
||||
- Works against Offline Kit Mongo snapshots when CLI is pointed at the local API gateway included in the bundle.
|
||||
- When fully disconnected, run against exported `aoc verify` reports generated on production and replay them using `--format json --export` (automation recipe above).
|
||||
- Include verification output in compliance packages alongside Offline Kit manifests.
|
||||
|
||||
---
|
||||
|
||||
## 4 · Global exit-code reference
|
||||
|
||||
| Code | Summary |
|
||||
|------|---------|
|
||||
| `0` | Success / no violations. |
|
||||
| `11` | `ERR_AOC_001` – Forbidden field present. |
|
||||
| `12` | `ERR_AOC_002` – Merge attempt detected. |
|
||||
| `13` | `ERR_AOC_003` – Idempotency violation. |
|
||||
| `14` | `ERR_AOC_004` – Missing provenance/signature metadata. |
|
||||
| `15` | `ERR_AOC_005` – Signature/checksum mismatch. |
|
||||
| `16` | `ERR_AOC_006` – Effective findings in ingestion payload. |
|
||||
| `17` | `ERR_AOC_007` – Schema violation / unknown fields. |
|
||||
| `18` | Partial verification (limit reached). |
|
||||
| `70` | Transport or HTTP failure. |
|
||||
| `71` | CLI usage error (invalid arguments, missing tenant). |
|
||||
|
||||
Use these codes in CI to map outcomes to build statuses or alert severities.
|
||||
|
||||
---
|
||||
|
||||
## 4 · `stella vuln observations` (Overlay paging)
|
||||
|
||||
`stella vuln observations` lists raw advisory observations for downstream overlays (Graph Explorer, Policy simulations, Console). Large tenants can now page through results deterministically.
|
||||
|
||||
| Option | Description |
|
||||
|--------|-------------|
|
||||
| `--limit <count>` | Caps the number of observations returned in a single call. Defaults to `200`; values above `500` are clamped server-side. |
|
||||
| `--cursor <token>` | Opaque continuation token produced by the previous page (`nextCursor` in JSON output). Pass it back to resume iteration. |
|
||||
|
||||
Additional notes:
|
||||
|
||||
- Table mode prints a hint when `hasMore` is `true`:
|
||||
`[yellow]More observations available. Continue with --cursor <token>[/]`.
|
||||
- JSON mode returns `nextCursor` and `hasMore` alongside the observation list so automation can loop until `hasMore` is `false`.
|
||||
- Supplying a non-positive limit falls back to the default (`200`). Invalid/expired cursors yield `400 Bad Request`; restart without `--cursor` to begin a fresh iteration.
|
||||
|
||||
---
|
||||
|
||||
## 5 · Related references
|
||||
|
||||
|
||||
---
|
||||
|
||||
## 1 · Prerequisites
|
||||
|
||||
- CLI version: `stella` ≥ 0.19.0 (AOC feature gate enabled).
|
||||
- Required scopes (DPoP-bound):
|
||||
- `advisory:read` for Concelier sources.
|
||||
- `vex:read` for Excititor sources (optional but required for VEX checks).
|
||||
- `aoc:verify` to invoke guard verification endpoints.
|
||||
- `tenant:select` if your deployment uses tenant switching.
|
||||
- Connectivity: direct access to Concelier/Excititor APIs or Offline Kit snapshot (see § 4).
|
||||
- Environment: set `STELLA_AUTHORITY_URL`, `STELLA_TENANT`, and export a valid OpTok via `stella auth login` or existing token cache.
|
||||
|
||||
---
|
||||
|
||||
## 2 · `stella sources ingest --dry-run`
|
||||
|
||||
### 2.1 Synopsis
|
||||
|
||||
```bash
|
||||
stella sources ingest --dry-run \
|
||||
--source <source-key> \
|
||||
--input <path-or-uri> \
|
||||
[--tenant <tenant-id>] \
|
||||
[--format json|table] \
|
||||
[--no-color] \
|
||||
[--output <file>]
|
||||
```
|
||||
|
||||
### 2.2 Description
|
||||
|
||||
Previews an ingestion write without touching MongoDB. The command loads an upstream advisory or VEX document, computes the would-write payload, runs it through the `AOCWriteGuard`, and reports any forbidden fields, provenance gaps, or idempotency issues. Use it during connector development, CI validation, or while triaging incidents.
|
||||
|
||||
### 2.3 Options
|
||||
|
||||
| Option | Description |
|
||||
|--------|-------------|
|
||||
| `--source <source-key>` | Logical source name (`redhat`, `ubuntu`, `osv`, etc.). Mirrors connector configuration. |
|
||||
| `--input <path-or-uri>` | Path to local CSAF/OSV/VEX file or HTTPS URI. CLI normalises transport (gzip/base64) before guard evaluation. |
|
||||
| `--tenant <tenant-id>` | Overrides default tenant for multi-tenant deployments. Mandatory when `STELLA_TENANT` is not set. |
|
||||
| `--format json|table` | Output format. `table` (default) prints summary with highlighted violations; `json` emits machine-readable report (see below). |
|
||||
| `--no-color` | Disables ANSI colour output for CI logs. |
|
||||
| `--output <file>` | Writes the JSON report to file while still printing human-readable summary to stdout. |
|
||||
|
||||
### 2.4 Output schema (JSON)
|
||||
|
||||
```json
|
||||
{
|
||||
"source": "redhat",
|
||||
"tenant": "default",
|
||||
"guardVersion": "1.0.0",
|
||||
"status": "ok",
|
||||
"document": {
|
||||
"contentHash": "sha256:…",
|
||||
"supersedes": null,
|
||||
"provenance": {
|
||||
"signature": { "format": "pgp", "present": true }
|
||||
}
|
||||
},
|
||||
"violations": []
|
||||
}
|
||||
```
|
||||
|
||||
When violations exist, `status` becomes `error` and `violations` contains entries with `code` (`ERR_AOC_00x`), a short `message`, and JSON Pointer `path` values indicating offending fields.
|
||||
|
||||
### 2.5 Exit codes
|
||||
|
||||
| Exit code | Meaning |
|
||||
|-----------|---------|
|
||||
| `0` | Guard passed; would-write payload is AOC compliant. |
|
||||
| `11` | `ERR_AOC_001` – Forbidden field (`severity`, `cvss`, etc.) detected. |
|
||||
| `12` | `ERR_AOC_002` – Merge attempt (multiple upstream sources fused). |
|
||||
| `13` | `ERR_AOC_003` – Idempotency violation (duplicate without supersedes). |
|
||||
| `14` | `ERR_AOC_004` – Missing provenance fields. |
|
||||
| `15` | `ERR_AOC_005` – Signature/checksum mismatch. |
|
||||
| `16` | `ERR_AOC_006` – Effective findings present (Policy-only data). |
|
||||
| `17` | `ERR_AOC_007` – Unknown top-level fields / schema violation. |
|
||||
| `70` | Transport error (network, auth, malformed input). |
|
||||
|
||||
> Exit codes map directly to the `ERR_AOC_00x` table for scripting consistency. Multiple violations yield the highest-priority code (e.g., 11 takes precedence over 14).
|
||||
|
||||
### 2.6 Examples
|
||||
|
||||
Dry-run a local CSAF file:
|
||||
|
||||
```bash
|
||||
stella sources ingest --dry-run \
|
||||
--source redhat \
|
||||
--input ./fixtures/redhat/RHSA-2025-1234.json
|
||||
```
|
||||
|
||||
Stream from HTTPS and emit JSON for CI:
|
||||
|
||||
```bash
|
||||
stella sources ingest --dry-run \
|
||||
--source osv \
|
||||
--input https://osv.dev/vulnerability/GHSA-aaaa-bbbb \
|
||||
--format json \
|
||||
--output artifacts/osv-dry-run.json
|
||||
|
||||
cat artifacts/osv-dry-run.json | jq '.violations'
|
||||
```
|
||||
|
||||
### 2.7 Offline notes
|
||||
|
||||
When operating in sealed/offline mode:
|
||||
|
||||
- Use `--input` paths pointing to Offline Kit snapshots (`offline-kit/advisories/*.json`).
|
||||
- Provide `--tenant` explicitly if the offline bundle contains multiple tenants.
|
||||
- The command does not attempt network access when given a file path.
|
||||
- Store reports with `--output` to include in transfer packages for policy review.
|
||||
|
||||
---
|
||||
|
||||
## 3 · `stella aoc verify`
|
||||
|
||||
### 3.1 Synopsis
|
||||
|
||||
```bash
|
||||
stella aoc verify \
|
||||
[--since <iso8601|duration>] \
|
||||
[--limit <count>] \
|
||||
[--sources <list>] \
|
||||
[--codes <ERR_AOC_00x,...>] \
|
||||
[--format table|json] \
|
||||
[--export <file>] \
|
||||
[--tenant <tenant-id>] \
|
||||
[--no-color]
|
||||
```
|
||||
|
||||
### 3.2 Description
|
||||
|
||||
Replays the AOC guard against stored raw documents. By default it checks all advisories and VEX statements ingested in the last 24 hours for the active tenant, reporting totals, top violation codes, and sample documents. Use it in CI pipelines, scheduled verifications, or during incident response.
|
||||
|
||||
### 3.3 Options
|
||||
|
||||
| Option | Description |
|
||||
|--------|-------------|
|
||||
| `--since <value>` | Verification window. Accepts ISO 8601 timestamp (`2025-10-25T12:00:00Z`) or duration (`48h`, `7d`). Defaults to `24h`. |
|
||||
| `--limit <count>` | Maximum number of violations to display (per code). `0` means show all. Defaults to `20`. |
|
||||
| `--sources <list>` | Comma-separated list of sources (`redhat,ubuntu,osv`). Filters both advisories and VEX entries. |
|
||||
| `--codes <list>` | Restricts output to specific `ERR_AOC_00x` codes. Useful for regression tracking. |
|
||||
| `--format table|json` | `table` (default) prints summary plus top violations; `json` outputs machine-readable report identical to the `/aoc/verify` API. |
|
||||
| `--export <file>` | Writes the JSON report to disk (useful for audits/offline uploads). |
|
||||
| `--tenant <tenant-id>` | Overrides tenant context. Required for cross-tenant verifications when run by platform operators. |
|
||||
| `--no-color` | Disables ANSI colours. |
|
||||
|
||||
`table` mode prints a summary showing the active tenant, evaluated window, counts of checked advisories/VEX statements, the active limit, total writes/violations, and whether the page was truncated. Status is colour-coded as `ok`, `violations`, or `truncated`. When violations exist the detail table lists the code, total occurrences, first sample document (`source` + `documentId` + `contentHash`), and JSON pointer path.
|
||||
|
||||
### 3.4 Report structure (JSON)
|
||||
|
||||
```json
|
||||
{
|
||||
"tenant": "default",
|
||||
"window": {
|
||||
"from": "2025-10-25T12:00:00Z",
|
||||
"to": "2025-10-26T12:00:00Z"
|
||||
},
|
||||
"checked": {
|
||||
"advisories": 482,
|
||||
"vex": 75
|
||||
},
|
||||
"violations": [
|
||||
{
|
||||
"code": "ERR_AOC_001",
|
||||
"count": 2,
|
||||
"examples": [
|
||||
{
|
||||
"source": "redhat",
|
||||
"documentId": "advisory_raw:redhat:RHSA-2025:1",
|
||||
"contentHash": "sha256:…",
|
||||
"path": "/content/raw/cvss"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"metrics": {
|
||||
"ingestion_write_total": 557,
|
||||
"aoc_violation_total": 2
|
||||
},
|
||||
"truncated": false
|
||||
}
|
||||
```
|
||||
|
||||
### 3.5 Exit codes
|
||||
|
||||
| Exit code | Meaning |
|
||||
|-----------|---------|
|
||||
| `0` | Verification succeeded with zero violations. |
|
||||
| `11…17` | Same mapping as § 2.5 when violations are detected. Highest-priority code returned. |
|
||||
| `18` | Verification ran but results truncated (limit reached) – treat as warning; rerun with higher `--limit`. |
|
||||
| `70` | Transport/authentication error. |
|
||||
| `71` | CLI misconfiguration (missing tenant, invalid `--since`, etc.). |
|
||||
|
||||
### 3.6 Examples
|
||||
|
||||
Daily verification across all sources:
|
||||
|
||||
```bash
|
||||
stella aoc verify --since 24h --format table
|
||||
```
|
||||
|
||||
CI pipeline focusing on errant sources and exporting evidence:
|
||||
|
||||
```bash
|
||||
stella aoc verify \
|
||||
--sources redhat,ubuntu \
|
||||
--codes ERR_AOC_001,ERR_AOC_004 \
|
||||
--format json \
|
||||
--limit 100 \
|
||||
--export artifacts/aoc-verify.json
|
||||
|
||||
jq '.violations[] | {code, count}' artifacts/aoc-verify.json
|
||||
```
|
||||
|
||||
Air-gapped verification using Offline Kit snapshot (example script):
|
||||
|
||||
```bash
|
||||
stella aoc verify \
|
||||
--since 7d \
|
||||
--format json \
|
||||
--export /mnt/offline/aoc-verify-$(date +%F).json
|
||||
|
||||
sha256sum /mnt/offline/aoc-verify-*.json > /mnt/offline/checksums.txt
|
||||
```
|
||||
|
||||
### 3.7 Automation tips
|
||||
|
||||
- Schedule with `cron` or platform scheduler and fail the job when exit code ≥ 11.
|
||||
- Pair with `stella sources ingest --dry-run` for pre-flight validation before re-enabling a paused source.
|
||||
- Push JSON exports to observability pipelines for historical tracking of violation counts.
|
||||
|
||||
### 3.8 Offline notes
|
||||
|
||||
- Works against Offline Kit Mongo snapshots when CLI is pointed at the local API gateway included in the bundle.
|
||||
- When fully disconnected, run against exported `aoc verify` reports generated on production and replay them using `--format json --export` (automation recipe above).
|
||||
- Include verification output in compliance packages alongside Offline Kit manifests.
|
||||
|
||||
---
|
||||
|
||||
## 4 · Global exit-code reference
|
||||
|
||||
| Code | Summary |
|
||||
|------|---------|
|
||||
| `0` | Success / no violations. |
|
||||
| `11` | `ERR_AOC_001` – Forbidden field present. |
|
||||
| `12` | `ERR_AOC_002` – Merge attempt detected. |
|
||||
| `13` | `ERR_AOC_003` – Idempotency violation. |
|
||||
| `14` | `ERR_AOC_004` – Missing provenance/signature metadata. |
|
||||
| `15` | `ERR_AOC_005` – Signature/checksum mismatch. |
|
||||
| `16` | `ERR_AOC_006` – Effective findings in ingestion payload. |
|
||||
| `17` | `ERR_AOC_007` – Schema violation / unknown fields. |
|
||||
| `18` | Partial verification (limit reached). |
|
||||
| `70` | Transport or HTTP failure. |
|
||||
| `71` | CLI usage error (invalid arguments, missing tenant). |
|
||||
|
||||
Use these codes in CI to map outcomes to build statuses or alert severities.
|
||||
|
||||
---
|
||||
|
||||
## 4 · `stella vuln observations` (Overlay paging)
|
||||
|
||||
`stella vuln observations` lists raw advisory observations for downstream overlays (Graph Explorer, Policy simulations, Console). Large tenants can now page through results deterministically.
|
||||
|
||||
| Option | Description |
|
||||
|--------|-------------|
|
||||
| `--limit <count>` | Caps the number of observations returned in a single call. Defaults to `200`; values above `500` are clamped server-side. |
|
||||
| `--cursor <token>` | Opaque continuation token produced by the previous page (`nextCursor` in JSON output). Pass it back to resume iteration. |
|
||||
|
||||
Additional notes:
|
||||
|
||||
- Table mode prints a hint when `hasMore` is `true`:
|
||||
`[yellow]More observations available. Continue with --cursor <token>[/]`.
|
||||
- JSON mode returns `nextCursor` and `hasMore` alongside the observation list so automation can loop until `hasMore` is `false`.
|
||||
- Supplying a non-positive limit falls back to the default (`200`). Invalid/expired cursors yield `400 Bad Request`; restart without `--cursor` to begin a fresh iteration.
|
||||
|
||||
---
|
||||
|
||||
## 5 · Related references
|
||||
|
||||
- [Aggregation-Only Contract reference](../../../ingestion/aggregation-only-contract.md)
|
||||
- [Architecture overview](../../platform/architecture-overview.md)
|
||||
- [Architecture overview](../../platform/architecture-overview.md)
|
||||
- [Console AOC dashboard](../../../ui/console.md)
|
||||
- [Authority scopes](../../authority/architecture.md)
|
||||
|
||||
---
|
||||
|
||||
## 6 · Compliance checklist
|
||||
|
||||
- [ ] Usage documented for both table and JSON formats.
|
||||
- [ ] Exit-code mapping matches `ERR_AOC_00x` definitions and automation guidance.
|
||||
- [ ] Offline/air-gap workflow captured for both commands.
|
||||
- [ ] References to AOC architecture and console docs included.
|
||||
- [ ] Examples validated against current CLI syntax (update post-implementation).
|
||||
- [ ] Docs guild screenshot/narrative placeholder logged for release notes (pending CLI team capture).
|
||||
|
||||
---
|
||||
|
||||
*Last updated: 2025-10-29 (Sprint 24).*
|
||||
|
||||
## 13. Authority configuration quick reference
|
||||
|
||||
| Setting | Purpose | How to set |
|
||||
|---------|---------|------------|
|
||||
| `StellaOps:Authority:OperatorReason` | Incident/change description recorded with `orch:operate` tokens. | CLI flag `--Authority:OperatorReason=...` or env `STELLAOPS_ORCH_REASON`. |
|
||||
| `StellaOps:Authority:OperatorTicket` | Change/incident ticket reference paired with orchestrator control actions. | CLI flag `--Authority:OperatorTicket=...` or env `STELLAOPS_ORCH_TICKET`. |
|
||||
|
||||
> Tokens requesting `orch:operate` will fail with `invalid_request` unless both values are present. Choose concise strings (≤256 chars for reason, ≤128 chars for ticket) and avoid sensitive data.
|
||||
|
||||
- [Authority scopes](../../authority/architecture.md)
|
||||
|
||||
---
|
||||
|
||||
## 6 · Compliance checklist
|
||||
|
||||
- [ ] Usage documented for both table and JSON formats.
|
||||
- [ ] Exit-code mapping matches `ERR_AOC_00x` definitions and automation guidance.
|
||||
- [ ] Offline/air-gap workflow captured for both commands.
|
||||
- [ ] References to AOC architecture and console docs included.
|
||||
- [ ] Examples validated against current CLI syntax (update post-implementation).
|
||||
- [ ] Docs guild screenshot/narrative placeholder logged for release notes (pending CLI team capture).
|
||||
|
||||
---
|
||||
|
||||
*Last updated: 2025-10-29 (Sprint 24).*
|
||||
|
||||
## 13. Authority configuration quick reference
|
||||
|
||||
| Setting | Purpose | How to set |
|
||||
|---------|---------|------------|
|
||||
| `StellaOps:Authority:OperatorReason` | Incident/change description recorded with `orch:operate` tokens. | CLI flag `--Authority:OperatorReason=...` or env `STELLAOPS_ORCH_REASON`. |
|
||||
| `StellaOps:Authority:OperatorTicket` | Change/incident ticket reference paired with orchestrator control actions. | CLI flag `--Authority:OperatorTicket=...` or env `STELLAOPS_ORCH_TICKET`. |
|
||||
| `StellaOps:Authority:QuotaReason` | Required justification recorded with `orch:quota` tokens. | CLI flag `--Authority:QuotaReason=...` or env `STELLAOPS_ORCH_QUOTA_REASON`. |
|
||||
| `StellaOps:Authority:QuotaTicket` | Optional change ticket/reference accompanying quota adjustments. | CLI flag `--Authority:QuotaTicket=...` or env `STELLAOPS_ORCH_QUOTA_TICKET`. |
|
||||
|
||||
> Tokens requesting `orch:operate` fail with `invalid_request` unless both operator values are present. `orch:quota` tokens require `quota_reason` (≤256 chars) and accept an optional `quota_ticket` (≤128 chars). Avoid embedding secrets in either field.
|
||||
|
||||
|
||||
@@ -107,7 +107,9 @@ Excititor derives `vex_normalized` tuples (without making decisions) for downstr
|
||||
}
|
||||
```
|
||||
|
||||
These tuples allow VEX Lens to compute deterministic consensus without re-parsing heavy upstream documents.
|
||||
These tuples allow VEX Lens to compute deterministic consensus without re-parsing heavy upstream documents.
|
||||
|
||||
Excititor workers now hydrate signature metadata with issuer trust data retrieved from the Issuer Directory service. The worker-side IssuerDirectoryClient performs tenant-aware lookups (including global fallbacks) and caches responses offline so attestation verification exposes an effective trust weight alongside the cryptographic details captured on ingest.
|
||||
|
||||
### 1.4 AI-ready citations
|
||||
|
||||
|
||||
95
docs/modules/issuer-directory/architecture.md
Normal file
95
docs/modules/issuer-directory/architecture.md
Normal file
@@ -0,0 +1,95 @@
|
||||
# Issuer Directory Architecture
|
||||
|
||||
> **Status:** Initial service scaffold (Sprint 100 – Identity & Signing)
|
||||
|
||||
## 1. Purpose
|
||||
|
||||
Issuer Directory centralises trusted VEX/CSAF publisher metadata so downstream services (VEX Lens, Excititor, Policy Engine) can resolve issuer identity, active keys, and trust weights. The initial milestone delivers tenant-scoped CRUD APIs with audit logging plus bootstrap import for CSAF publishers.
|
||||
|
||||
## 2. Runtime Topology
|
||||
|
||||
- **Service name:** `stellaops/issuer-directory`
|
||||
- **Framework:** ASP.NET Core minimal APIs (`net10.0`)
|
||||
- **Persistence:** MongoDB (`issuer-directory.issuers`, `issuer-directory.issuer_keys`, `issuer-directory.issuer_audit`)
|
||||
- **AuthZ:** StellaOps resource server scopes (`issuer-directory:read`, `issuer-directory:write`, `issuer-directory:admin`)
|
||||
- **Audit:** Every create/update/delete emits an audit record with actor, reason, and context.
|
||||
- **Bootstrap:** On startup, the service imports `data/csaf-publishers.json` into the global tenant (`@global`) and records a `seeded` audit the first time each publisher is added.
|
||||
- **Key lifecycle:** API validates Ed25519 public keys, X.509 certificates, and DSSE public keys, enforces future expiries, deduplicates fingerprints, and records audit entries for create/rotate/revoke actions.
|
||||
|
||||
```
|
||||
Clients ──> Authority (DPoP/JWT) ──> IssuerDirectory WebService ──> MongoDB
|
||||
│
|
||||
└─> Audit sink (Mongo)
|
||||
```
|
||||
|
||||
## 3. Configuration
|
||||
|
||||
Configuration is resolved via `IssuerDirectoryWebServiceOptions` (section name `IssuerDirectory`). The default YAML sample lives at `etc/issuer-directory.yaml.sample` and exposes:
|
||||
|
||||
```yaml
|
||||
IssuerDirectory:
|
||||
telemetry:
|
||||
minimumLogLevel: Information
|
||||
authority:
|
||||
enabled: true
|
||||
issuer: https://authority.example.com/realms/stellaops
|
||||
requireHttpsMetadata: true
|
||||
audiences:
|
||||
- stellaops-platform
|
||||
readScope: issuer-directory:read
|
||||
writeScope: issuer-directory:write
|
||||
adminScope: issuer-directory:admin
|
||||
tenantHeader: X-StellaOps-Tenant
|
||||
seedCsafPublishers: true
|
||||
csafSeedPath: data/csaf-publishers.json
|
||||
Mongo:
|
||||
connectionString: mongodb://localhost:27017
|
||||
database: issuer-directory
|
||||
issuersCollection: issuers
|
||||
issuerKeysCollection: issuer_keys
|
||||
auditCollection: issuer_audit
|
||||
```
|
||||
|
||||
## 4. API Surface (v0)
|
||||
|
||||
| Method | Route | Scope | Description |
|
||||
|--------|-------|-------|-------------|
|
||||
| `GET` | `/issuer-directory/issuers` | `issuer-directory:read` | List tenant issuers (optionally include global seeds). |
|
||||
| `GET` | `/issuer-directory/issuers/{id}` | `issuer-directory:read` | Fetch a single issuer by identifier. |
|
||||
| `POST` | `/issuer-directory/issuers` | `issuer-directory:write` | Create a tenant issuer. Requires `X-StellaOps-Tenant` header and optional `X-StellaOps-Reason`. |
|
||||
| `PUT` | `/issuer-directory/issuers/{id}` | `issuer-directory:write` | Update issuer metadata/endpoints/tags. |
|
||||
| `DELETE` | `/issuer-directory/issuers/{id}` | `issuer-directory:admin` | Delete issuer (records audit). |
|
||||
| `GET` | `/issuer-directory/issuers/{id}/keys` | `issuer-directory:read` | List issuer keys (tenant + optional `@global` seeds). |
|
||||
| `POST` | `/issuer-directory/issuers/{id}/keys` | `issuer-directory:write` | Add a signing key (validates format, deduplicates fingerprint, audits). |
|
||||
| `POST` | `/issuer-directory/issuers/{id}/keys/{keyId}/rotate` | `issuer-directory:write` | Retire an active key and create a replacement atomically. |
|
||||
| `DELETE` | `/issuer-directory/issuers/{id}/keys/{keyId}` | `issuer-directory:admin` | Revoke a key (status → revoked, audit logged). |
|
||||
|
||||
Payloads follow the contract in `Contracts/IssuerDtos.cs` and align with domain types (`IssuerRecord`, `IssuerMetadata`, `IssuerEndpoint`).
|
||||
|
||||
## 5. Dependencies & Reuse
|
||||
|
||||
- `StellaOps.IssuerDirectory.Core` — domain model (`IssuerRecord`, `IssuerKeyRecord`) + application services.
|
||||
- `StellaOps.IssuerDirectory.Infrastructure` — MongoDB persistence, audit sink, seed loader.
|
||||
- `StellaOps.IssuerDirectory.WebService` — minimal API host, authentication wiring.
|
||||
- Shared libraries: `StellaOps.Configuration`, `StellaOps.Auth.ServerIntegration`.
|
||||
|
||||
## 6. Testing
|
||||
|
||||
- Unit coverage for issuer CRUD (`IssuerDirectoryServiceTests`) and key lifecycle (`IssuerKeyServiceTests`) in `StellaOps.IssuerDirectory.Core.Tests`.
|
||||
- Test infrastructure leverages `FakeTimeProvider` for deterministic timestamps and in-memory fakes for repository + audit sink.
|
||||
|
||||
## 7. Observability
|
||||
|
||||
- **Metrics.** `issuer_directory_changes_total` (labels: `tenant`, `issuer`, `action`) tracks issuer create/update/delete events; `issuer_directory_key_operations_total` (labels: `tenant`, `issuer`, `operation`, `key_type`) covers key create/rotate/revoke flows; `issuer_directory_key_validation_failures_total` (labels: `tenant`, `issuer`, `reason`) captures validation/verification failures. The WebService exports these via OpenTelemetry (`StellaOps.IssuerDirectory` meter).
|
||||
- **Logs.** Service-level `ILogger` instrumentation records structured entries for issuer CRUD, key lifecycle operations, and validation failures; audit logs remain the authoritative trail.
|
||||
|
||||
## 8. Roadmap (next milestones)
|
||||
|
||||
1. **Key management APIs (ISSUER-30-002)** — manage signing keys, enforce expiry, integrate with KMS.
|
||||
2. **Trust weight overrides (ISSUER-30-003)** — expose policy-friendly trust weighting with audit trails.
|
||||
3. **SDK integration (ISSUER-30-004)** — supply cached issuer metadata to VEX Lens and Excititor clients.
|
||||
4. **Observability & Ops (ISSUER-30-005/006)** — metrics, dashboards, deployment automation, offline kit.
|
||||
|
||||
---
|
||||
|
||||
*Document owner: Issuer Directory Guild*
|
||||
@@ -313,17 +313,28 @@ Internal tooling can hit `/internal/notify/<entity>/normalize` to upgrade legacy
|
||||
* `GET /deliveries/{id}` → detail (redacted body + metadata)
|
||||
* `POST /deliveries/{id}/retry` → force retry (admin, future sprint)
|
||||
|
||||
* **Admin**
|
||||
|
||||
* `GET /stats` (per tenant counts, last hour/day)
|
||||
* `GET /healthz|readyz` (liveness)
|
||||
* `POST /locks/acquire` | `POST /locks/release` – worker coordination primitives (short TTL).
|
||||
* `POST /digests` | `GET /digests/{actionKey}` | `DELETE /digests/{actionKey}` – manage open digest windows.
|
||||
* `POST /audit` | `GET /audit?since=&limit=` – append/query structured audit trail entries.
|
||||
|
||||
**Ingestion**: workers do **not** expose public ingestion; they **subscribe** to the internal bus. (Optional `/events/test` for integration testing, admin‑only.)
|
||||
|
||||
---
|
||||
* **Admin**
|
||||
|
||||
* `GET /stats` (per tenant counts, last hour/day)
|
||||
* `GET /healthz|readyz` (liveness)
|
||||
* `POST /locks/acquire` | `POST /locks/release` – worker coordination primitives (short TTL).
|
||||
* `POST /digests` | `GET /digests/{actionKey}` | `DELETE /digests/{actionKey}` – manage open digest windows.
|
||||
* `POST /audit` | `GET /audit?since=&limit=` – append/query structured audit trail entries.
|
||||
|
||||
### 8.1 Ack tokens & escalation workflows
|
||||
|
||||
To support one-click acknowledgements from chat/email, the Notify WebService mints **DSSE ack tokens** via Authority:
|
||||
|
||||
* `POST /notify/ack-tokens/issue` → returns a DSSE envelope (payload type `application/vnd.stellaops.notify-ack-token+json`) describing the tenant, notification/delivery ids, channel, webhook URL, nonce, permitted actions, and TTL. Requires `notify.operator`; requesting escalation requires the caller to hold `notify.escalate` (and `notify.admin` when configured). Issuance enforces the Authority-side webhook allowlist (`notifications.webhooks.allowedHosts`) before minting tokens.
|
||||
* `POST /notify/ack-tokens/verify` → verifies the DSSE signature, enforces expiry/tenant/action constraints, and emits audit events (`notify.ack.verified`, `notify.ack.escalated`). Scope: `notify.operator` (+`notify.escalate` for escalation).
|
||||
* `POST /notify/ack-tokens/rotate` → rotates the signing key used for ack tokens, requires `notify.admin`, and emits `notify.ack.key_rotated`/`notify.ack.key_rotation_failed` audit events. Operators must supply the new key material (file/KMS/etc. depending on `notifications.ackTokens.keySource`); Authority updates JWKS entries with `use: "notify-ack"` and retires the previous key.
|
||||
* `POST /internal/notifications/ack-tokens/rotate` → legacy bootstrap path (API-key protected) retained for air-gapped initial provisioning; it forwards to the same rotation pipeline as the public endpoint.
|
||||
|
||||
Authority signs ack tokens using keys configured under `notifications.ackTokens`. Public JWKS responses expose these keys with `use: "notify-ack"` and `status: active|retired`, enabling offline verification by the worker/UI/CLI.
|
||||
|
||||
**Ingestion**: workers do **not** expose public ingestion; they **subscribe** to the internal bus. (Optional `/events/test` for integration testing, admin-only.)
|
||||
|
||||
---
|
||||
|
||||
## 9) Delivery pipeline (worker)
|
||||
|
||||
|
||||
@@ -19,9 +19,10 @@
|
||||
|
||||
## 3) Rate-limit & quota governance
|
||||
|
||||
- Quotas defined per tenant/profile (`maxActive`, `maxPerHour`, `burst`). Stored in `quotas` and enforced before leasing.
|
||||
- Dynamic throttles allow ops to pause specific sources (`pauseSource`, `resumeSource`) or reduce concurrency.
|
||||
- Circuit breakers automatically pause job types when failure rate > configured threshold; incidents generated via Notify and Observability stack.
|
||||
- Quotas defined per tenant/profile (`maxActive`, `maxPerHour`, `burst`). Stored in `quotas` and enforced before leasing.
|
||||
- Dynamic throttles allow ops to pause specific sources (`pauseSource`, `resumeSource`) or reduce concurrency.
|
||||
- Circuit breakers automatically pause job types when failure rate > configured threshold; incidents generated via Notify and Observability stack.
|
||||
- Control plane quota updates require Authority scope `orch:quota` (issued via `Orch.Admin` role). Token requests include `quota_reason` (mandatory) and optional `quota_ticket`; Authority persists both values for audit replay.
|
||||
|
||||
## 4) APIs
|
||||
|
||||
|
||||
@@ -73,7 +73,8 @@ Surface.FS library for .NET hosts provides:
|
||||
- `ISurfaceManifestWriter` / `ISurfaceManifestReader` interfaces.
|
||||
- Content-addressed path builder (`SurfacePathBuilder`).
|
||||
- Tenant namespace isolation and bucket configuration (via Surface.Env).
|
||||
- Local cache management (using `SCANNER_SURFACE_CACHE_ROOT` and quota).
|
||||
- Local cache abstraction `ISurfaceCache` with default `FileSurfaceCache` implementation (uses `Surface:Cache:Root` / `SCANNER_SURFACE_CACHE_ROOT`, enforces quotas, serialises writes with per-key semaphores).
|
||||
- `SurfaceCacheKey` helper that normalises cache entries as `{namespace}/{tenant}/{sha256}`. EntryTrace graphs use the `entrytrace.graph` namespace so Worker/WebService/CLI can share cached results deterministically.
|
||||
- Metrics: `surface_manifest_put_seconds`, `surface_manifest_cache_hit_total`, etc.
|
||||
|
||||
## 5. Retention & Eviction
|
||||
@@ -97,6 +98,10 @@ offline/surface/
|
||||
|
||||
Import script calls `PutManifest` for each manifest, verifying digests. This enables Zastava and Scheduler running offline to consume cached data without re-scanning.
|
||||
|
||||
### 6.1 EntryTrace Cache Usage
|
||||
|
||||
Scanner.Worker serialises EntryTrace graphs into Surface.FS using `SurfaceCacheKey(namespace: "entrytrace.graph", tenant, sha256(options|env|entrypoint))`. At runtime the worker checks the cache before invoking analyzers; cache hits bypass parsing and feed the result store/attestor pipeline directly. The same namespace is consumed by WebService and CLI to retrieve cached graphs for reporting.
|
||||
|
||||
## 7. Security & Tenancy
|
||||
|
||||
- Tenant ID is mandatory; Surface.Validation enforces match with Authority token.
|
||||
|
||||
@@ -52,7 +52,18 @@ public sealed record SurfaceSecretRequest
|
||||
|
||||
### 3.2 Secret Handle
|
||||
|
||||
`SurfaceSecretHandle` exposes typed accessors (`AsCredentials()`, `AsTlsCertificate()`) and ensures sensitive data is cleared when disposed.
|
||||
`SurfaceSecretHandle` exposes typed accessors (`AsBytes()`, `AsCredentials()`, `AsTlsCertificate()`) and ensures sensitive data is cleared when disposed. Consumers that expect string material attempt UTF-8 decoding first and, if decoding fails, fall back to returning a base64 representation rather than dropping binary content.
|
||||
|
||||
### 3.3 Environment & Config References
|
||||
|
||||
Runtime configuration can reference secrets using the URI scheme `secret://{secretType}/{name?}`. Example:
|
||||
|
||||
```
|
||||
SCANNER_ENTRYTRACE_ENV__0=API_TOKEN=secret://registry/primary
|
||||
SCANNER_ENTRYTRACE_ENV__1=TLS_CERT=secret://tls/edge-gateway
|
||||
```
|
||||
|
||||
During scan execution, Scanner.Worker resolves each placeholder via `ISurfaceSecretProvider` before invoking analyzers, replacing the environment variable with the resolved value (base64 when non-text). Missing secrets raise `SurfaceSecretNotFoundException` and are surfaced as warnings without hard-failing the scan.
|
||||
|
||||
## 4. Configuration
|
||||
|
||||
|
||||
@@ -13,32 +13,36 @@ Surface.Validation provides a shared validator framework to ensure all surface c
|
||||
```csharp
|
||||
public interface ISurfaceValidator
|
||||
{
|
||||
ValueTask<ValidationResult> ValidateAsync(SurfaceValidationContext context, CancellationToken ct = default);
|
||||
ValueTask<SurfaceValidationResult> ValidateAsync(SurfaceValidationContext context, CancellationToken ct = default);
|
||||
}
|
||||
|
||||
public sealed record SurfaceValidationContext
|
||||
(
|
||||
SurfaceEnvironmentSettings Environment,
|
||||
public sealed record SurfaceValidationContext(
|
||||
IServiceProvider Services,
|
||||
string ComponentName
|
||||
);
|
||||
string ComponentName,
|
||||
SurfaceEnvironmentSettings Environment,
|
||||
IReadOnlyDictionary<string, object?> Properties)
|
||||
{
|
||||
public static SurfaceValidationContext Create(
|
||||
IServiceProvider services,
|
||||
string componentName,
|
||||
SurfaceEnvironmentSettings environment,
|
||||
IReadOnlyDictionary<string, object?>? properties = null);
|
||||
}
|
||||
|
||||
public sealed record ValidationResult
|
||||
(
|
||||
bool IsSuccess,
|
||||
IReadOnlyCollection<SurfaceValidationIssue> Issues
|
||||
);
|
||||
public interface ISurfaceValidatorRunner
|
||||
{
|
||||
ValueTask<SurfaceValidationResult> RunAllAsync(SurfaceValidationContext context, CancellationToken ct = default);
|
||||
ValueTask EnsureAsync(SurfaceValidationContext context, CancellationToken ct = default);
|
||||
}
|
||||
|
||||
public sealed record SurfaceValidationIssue
|
||||
(
|
||||
public sealed record SurfaceValidationIssue(
|
||||
string Code,
|
||||
string Message,
|
||||
SurfaceValidationSeverity Severity,
|
||||
string? Hint = null
|
||||
);
|
||||
string? Hint = null);
|
||||
```
|
||||
|
||||
Validators register with DI (`services.AddSurfaceValidation()`). Hosts call `ISurfaceValidatorRunner.RunAllAsync()` during startup and periodically (optional) to re-check configuration.
|
||||
`Properties` carries optional context-specific metadata (e.g., `jobId`, `imageDigest`, cache paths) so validators can tailor diagnostics without pulling additional services. Validators register with DI (`services.AddSurfaceValidation()`). Hosts call `ISurfaceValidatorRunner.RunAllAsync()` during startup and before workload execution to capture misconfiguration early; `EnsureAsync()` rethrows when `Surface:Validation:ThrowOnFailure=true`.
|
||||
|
||||
## 3. Built-in Validators
|
||||
|
||||
@@ -76,6 +80,7 @@ Validators can access DI services (e.g., HttpClient, Authority token provider) t
|
||||
## 6. Integration Guidelines
|
||||
|
||||
- **Scanner Worker/WebService**: fail startup if any error-level issue occurs; log warnings but continue running.
|
||||
- **Scanner EntryTrace**: execute `RunAllAsync` for each scan job with properties `{imageDigest, jobId, configPath, rootPath}`. If the result contains errors, skip analysis and log the issue summary instead of failing the entire scan.
|
||||
- **Zastava Webhook**: treat validation errors as fatal (webhook should not enforce policies when surface preconditions fail). Display validation error summary in `/readyz` response to aid debugging.
|
||||
- **Analysers**: call `SurfaceValidation.Ensure()` before executing heavy work to catch misconfiguration during integration tests.
|
||||
|
||||
|
||||
@@ -1,11 +1,89 @@
|
||||
# Entry-Point Static Analysis
|
||||
|
||||
This guide captures the static half of Stella Ops’ entry-point detection pipeline: how we turn image metadata and filesystem contents into a resolved binary, an execution chain, and a confidence score.
|
||||
|
||||
## 1) Loading OCI images
|
||||
|
||||
### 1.1 Supported inputs
|
||||
- Registry references (`repo:tag@sha256:digest`) using the existing content store.
|
||||
# Entry-Point Static Analysis
|
||||
|
||||
This guide captures the static half of Stella Ops’ entry-point detection pipeline: how we turn image metadata and filesystem contents into a resolved binary, an execution chain, and a confidence score.
|
||||
|
||||
## 0) Implementation snapshot — Sprint 130.A (2025-11-02)
|
||||
|
||||
The `StellaOps.Scanner.EntryTrace` stack (analyzer + worker + surfaces) currently provides:
|
||||
|
||||
- **OCI config + layered FS context**: `EntryTraceImageContextFactory` normalises environment (`PATH` fallback), user, and working directory while `LayeredRootFileSystem` handles whiteouts, symlinks, and bounded byte reads (`TryReadBytes`) so ELF/PE probing stays offline friendly.
|
||||
- **Wrapper-aware exec expansion**: the analyzer unwraps init/user-switch/environment/supervisor wrappers (`tini`, `dumb-init`, `gosu`, `su-exec`, `chpst`, `env`, `supervisord`, `s6-supervise`, `runsv*`) and records guard metadata plus environment/user deltas on nodes and edges.
|
||||
- **Script + interpreter resolution**: POSIX shell parsing (AST-driven) covers `source`, `run-parts`, `exec`, and supervisor service directories, with Windows `cmd /c` support. Python `-m`, Node script, and Java `-jar` lookups add evidence when targets are located.
|
||||
- **Terminal classification & scoring**: `ClassifyTerminal` fingerprints ELF (`PT_INTERP`, Go build ID, Rust notes), PE/CLR, and JAR manifests, pairs them with shebang/runtime heuristics (`python`, `node`, `java`, `.NET`, `php-fpm`, `nginx`, `ruby`), and emits `EntryTracePlan/EntryTraceTerminal` records capped at 95-point confidence.
|
||||
- **NDJSON + capability stream**: `EntryTraceNdjsonWriter` produces deterministic `entrytrace.entry/node/edge/target/warning/capability` lines consumed by AOC, CLI, and policy surfaces.
|
||||
- **Runtime reconciliation**: `ProcFileSystemSnapshot` + `ProcGraphBuilder` replay `/proc`, `EntryTraceRuntimeReconciler` merges runtime terminals with static predictions, and diagnostics note matches/mismatches.
|
||||
- **Surface integration**: Scanner Worker caches graphs (`SurfaceCache`), persists `EntryTraceResult` via the shared store, exposes NDJSON + graph through `ScanAnalysisKeys`, and the WebService/CLI (`scan entrytrace`) return the stored result.
|
||||
|
||||
Open follow-ups tracked for this wave:
|
||||
|
||||
- **SCANNER-ENTRYTRACE-18-507** – fallback candidate discovery (Docker history, `/etc/services/**`, `/usr/local/bin/*-entrypoint`) when ENTRYPOINT/CMD are empty.
|
||||
- **SCANNER-ENTRYTRACE-18-508** – broaden wrapper catalogue (package/tool runners such as `bundle exec`, `npm`, `yarn node`, `docker-php-entrypoint`, `pipenv`, `poetry run`).
|
||||
- **ENTRYTRACE-SURFACE-01** (DOING) / **ENTRYTRACE-SURFACE-02** (TODO) – finish wiring Surface.Validation/FS/Secrets to gate prerequisites and remove direct env/secret reads.
|
||||
|
||||
_Sections §4–§7 below capture the long-term reduction design; features not yet implemented are explicitly noted in the task board._
|
||||
|
||||
### Probing the analyzer today
|
||||
|
||||
1. **Load the image config**
|
||||
```csharp
|
||||
using var stream = File.OpenRead("config.json");
|
||||
var config = OciImageConfigLoader.Load(stream);
|
||||
```
|
||||
2. **Create a layered filesystem** from extracted layer directories or tar archives:
|
||||
```csharp
|
||||
var fs = LayeredRootFileSystem.FromArchives(layers);
|
||||
```
|
||||
3. **Build the image context** (normalises env, PATH, user, working dir):
|
||||
```csharp
|
||||
var imageCtx = EntryTraceImageContextFactory.Create(
|
||||
config, fs, new EntryTraceAnalyzerOptions(), imageDigest, scanId);
|
||||
```
|
||||
4. **Resolve the entry trace**:
|
||||
```csharp
|
||||
var analyzer = serviceProvider.GetRequiredService<IEntryTraceAnalyzer>();
|
||||
var graph = await analyzer.ResolveAsync(imageCtx.Entrypoint, imageCtx.Context, cancellationToken);
|
||||
```
|
||||
5. **Inspect results** – `graph.Terminals` lists classified candidates (path, runtime, confidence, evidence), `graph.Nodes/Edges` capture the explainable chain, and `graph.Diagnostics` highlight unresolved steps. Emit metrics/telemetry via `EntryTraceMetrics`.
|
||||
6. **Serialize if needed** – pass the graph through `EntryTraceNdjsonWriter.Serialize` to obtain deterministic NDJSON lines; the helper already computes capability summaries.
|
||||
|
||||
For ad-hoc investigation, snapshotting `EntryTraceResult` keeps graph and NDJSON aligned. Avoid ad-hoc JSON writers to maintain ordering guarantees.
|
||||
|
||||
#### Probing through Scanner.Worker
|
||||
|
||||
EntryTrace runs automatically inside the worker when these metadata keys exist on the lease:
|
||||
|
||||
| Key | Purpose |
|
||||
| --- | --- |
|
||||
| `ScanMetadataKeys.ImageConfigPath` (default `scanner.analyzers.entrytrace.configMetadataKey`) | Absolute path to the OCI `config.json`. |
|
||||
| `ScanMetadataKeys.LayerDirectories` or `ScanMetadataKeys.LayerArchives` | Semicolon-delimited list of extracted layer folders or tar archives. |
|
||||
| `ScanMetadataKeys.RuntimeProcRoot` *(optional)* | Path to a captured `/proc` tree for runtime reconciliation (air-gapped runs can mount a snapshot). |
|
||||
|
||||
Worker output lands in `context.Analysis` (`EntryTraceGraph`, `EntryTraceNdjson`) and is persisted via `IEntryTraceResultStore`. Ensure Surface Validation prerequisites pass before dispatching the analyzer.
|
||||
|
||||
#### Probing via WebService & CLI
|
||||
|
||||
- **REST**: `GET /api/scans/{scanId}/entrytrace` returns `EntryTraceResponse` (`graph + ndjson + metadata`). Requires scan ownership/authz.
|
||||
- **CLI**: `stella scan entrytrace <scan-id> [--ndjson] [--verbose]` renders a confidence-sorted terminal table, diagnostics, and optionally the NDJSON payload.
|
||||
|
||||
Both surfaces consume the persisted result; rerunning the worker updates the stored document atomically.
|
||||
|
||||
### NDJSON reference
|
||||
|
||||
`EntryTraceNdjsonWriter.Serialize` emits newline-delimited JSON in the following order so AOC consumers can stream without buffering:
|
||||
|
||||
- `entrytrace.entry` — scan metadata (scan id, image digest, outcome, counts).
|
||||
- `entrytrace.node` — every node in the graph with arguments, interpreter, evidence, and metadata.
|
||||
- `entrytrace.edge` — directed relationships between nodes with optional wrapper metadata.
|
||||
- `entrytrace.target` — resolved terminal programmes (`EntryTracePlan`), including runtime, confidence, arguments, environment, and evidence.
|
||||
- `entrytrace.warning` — diagnostics (severity, reason, span, related path).
|
||||
- `entrytrace.capability` — aggregated wrapper capabilities discovered during traversal.
|
||||
|
||||
Every line ends with a newline and is emitted in deterministic order (IDs ascending, keys lexicographically sorted) so downstream tooling can hash or diff outputs reproducibly.
|
||||
|
||||
## 1) Loading OCI images
|
||||
|
||||
### 1.1 Supported inputs
|
||||
- Registry references (`repo:tag@sha256:digest`) using the existing content store.
|
||||
- Local OCI/Docker v2 archives (`docker save` tarball, OCI layout directory with `index.json` + `blobs/sha256/*`).
|
||||
|
||||
### 1.2 Normalised model
|
||||
@@ -53,14 +131,18 @@ Compose the runtime argv as `Entrypoint ++ Cmd`, honouring shell-form vs exec-fo
|
||||
- For non-ELF/PE files: read first line; interpret `#!interpreter args`.
|
||||
- Replace `argv[0]` with the interpreter, prepend shebang args, append script path per kernel semantics.
|
||||
|
||||
### 3.3 Binary probes
|
||||
- Identify ELF via magic `\x7FELF`, parse `.interp`, `.dynamic`, linked libs, `.note.go.buildid`, DWARF producer.
|
||||
- Identify PE (Windows) and detect .NET single-file bundles via CLI header.
|
||||
- Record features for runtime scoring (Go vs Rust vs glibc vs musl).
|
||||
### 3.3 Binary probes
|
||||
- Identify ELF via magic `\x7FELF`, parse `.interp`, `.dynamic`, linked libs, `.note.go.buildid`, DWARF producer, `.rustc` notes, and musl/glibc fingerprints.
|
||||
- Identify PE (Windows) and detect .NET single-file bundles via CLI header / metadata tables; capture ready-to-run vs IL-only markers.
|
||||
- Inspect archives (JAR/WAR/EAR) for `META-INF/MANIFEST.MF` `Main-Class`/`Main-Module` and signed entries.
|
||||
- Detect PHP-FPM / nginx launchers (`php-fpm`, `apache2-foreground`, `nginx -g 'daemon off;'`) via binary names + nearby config (php.ini, nginx.conf).
|
||||
- Record evidence tuples for runtime scoring (interpreter, build ID, runtime note) so downstream components can explain the classification.
|
||||
|
||||
## 4) Wrapper catalogue
|
||||
|
||||
Collapse known wrappers before analysing the target command:
|
||||
## 4) Wrapper catalogue
|
||||
|
||||
> _Roadmap note_: extended package/tool runners land with **SCANNER-ENTRYTRACE-18-508**; today the catalogue covers init/user-switch/environment/supervisor wrappers listed above.
|
||||
|
||||
Collapse known wrappers before analysing the target command:
|
||||
|
||||
- Init shims: `tini`, `dumb-init`, `s6-svscan`, `runit`, `supervisord`.
|
||||
- Privilege droppers: `gosu`, `su-exec`, `chpst`.
|
||||
@@ -111,12 +193,31 @@ Use a simple logistic model with feature contributions captured for the evidence
|
||||
|
||||
Persist per-feature evidence strings so UI/CLI users can see **why** the scanner picked a given entry point.
|
||||
|
||||
## 8) Outputs
|
||||
|
||||
Return a populated `EntryTraceResult`:
|
||||
|
||||
- `Terminals` contains the best candidate(s) and their runtime classification.
|
||||
- `Evidence` aggregates feature messages, ShellFlow reasoning, wrapper reductions, and runtime detector hints.
|
||||
- `Chain` shows the explainable path from initial Docker argv to the final binary.
|
||||
|
||||
Static and dynamic reducers share this shape, enabling downstream modules to remain agnostic of the detection mode.
|
||||
## 8) Outputs
|
||||
|
||||
Return a populated `EntryTraceResult`:
|
||||
|
||||
- `Terminals` contains the best candidate(s) and their runtime classification.
|
||||
- `Evidence` aggregates feature messages, ShellFlow reasoning, wrapper reductions, and runtime detector hints.
|
||||
- `Chain` shows the explainable path from initial Docker argv to the final binary.
|
||||
|
||||
Static and dynamic reducers share this shape, enabling downstream modules to remain agnostic of the detection mode.
|
||||
|
||||
## 9) ProcGraph replay (runtime parity)
|
||||
|
||||
Static resolution must be reconciled with live observations when a workload is running under the Stella Ops runtime agent:
|
||||
|
||||
1. Read `/proc/1/{cmdline,exe}` and walk descendants via `/proc/*/stat` to construct the initial exec chain (ascending PID order).
|
||||
2. Collapse known wrappers (`tini`, `dumb-init`, `gosu`, `su-exec`, `s6-supervise`, `runsv`, `supervisord`) and privilege switches, mirroring the static wrapper catalogue.
|
||||
3. Materialise a `ProcGraph` object that records each transition and the resolved executable path (via `/proc/<pid>/exe` symlinks).
|
||||
4. Compare `ProcGraph.Terminal` with `EntryTraceResult.Terminals[0]`, emitting `confidence=high` when they match and downgrade when divergence occurs.
|
||||
5. Persist the merged view so the CLI/UI can highlight static vs runtime discrepancies and feed drift detection in Zastava.
|
||||
|
||||
This replay is optional offline, but required when runtime evidence is available so policy decisions can lean on High-confidence matches.
|
||||
|
||||
## 10) Service & CLI surfaces
|
||||
|
||||
- **Scanner.WebService** must expose `/scans/{scanId}/entrytrace` returning chain, terminal classification, evidence, and runtime agreement markers.
|
||||
- **CLI** gains `stella scan entrypoint <scanId>` (and JSON streaming) for air-gapped review.
|
||||
- **Policy / Export** payloads include `entrytrace_terminal`, `entrytrace_confidence`, and evidence arrays so downstream consumers retain provenance.
|
||||
- All outputs reuse the same `EntryTraceResult` schema and NDJSON stream defined in §7, keeping the Offline Kit and DSSE attestations deterministic.
|
||||
|
||||
Reference in New Issue
Block a user