stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity

Add Authority Advisory AI and API Lifecycle Configuration

Browse Source 
- Introduced AuthorityAdvisoryAiOptions and related classes for managing advisory AI configurations, including remote inference options and tenant-specific settings.
- Added AuthorityApiLifecycleOptions to control API lifecycle settings, including legacy OAuth endpoint configurations.
- Implemented validation and normalization methods for both advisory AI and API lifecycle options to ensure proper configuration.
- Created AuthorityNotificationsOptions and its related classes for managing notification settings, including ack tokens, webhooks, and escalation options.
- Developed IssuerDirectoryClient and related models for interacting with the issuer directory service, including caching mechanisms and HTTP client configurations.
- Added support for dependency injection through ServiceCollectionExtensions for the Issuer Directory Client.
- Updated project file to include necessary package references for the new Issuer Directory Client library.
This commit is contained in:
master
2025-11-02 13:40:38 +02:00
parent 66cb6c4b8a
commit f98cea3bcf
516 changed files with 68157 additions and 24754 deletions
Download Patch File Download Diff File Expand all files Collapse all files

26
docs/modules/attestor/README.md
View File

@@ -19,16 +19,22 @@ Attestor converts signed DSSE evidence from the Signer into transparency-log pro
- `StellaOps.PolicyEvaluation@1`, `StellaOps.RiskProfileEvidence@1`
All predicates capture subjects, issuer metadata, policy context, materials, optional witnesses, and versioned schemas. Unsupported predicates return `422 predicate_unsupported`.
## Trust & envelope model
- DSSE envelopes are canonicalised, hashed, and stored alongside the Rekor UUID, index, and proof.
- Signature modes span keyless (Fulcio), keyful (KMS/HSM), and hardware-backed (FIDO2). Multiple signatures are supported per envelope.
- Proofs include Merkle inclusion path, checkpoint metadata, optional witness endorsements, and cached verification verdicts.
- CAS/object storage retains envelopes + provenance for later replay; Rekor backends may be primary plus mirrors.
## UI, CLI, and SDK workflows
- **Console:** Evidence browser, verification reports, chain-of-custody graph, issuer/key management, attestation workbench, and bulk verification flows.
- **CLI / SDK:** `stella attest sign|verify|list|fetch|key` commands plus language SDKs to integrate build pipelines and offline verification scripts.
- **Policy Studio:** Verification policies author required predicate types, issuers, witness requirements, and freshness windows; simulations show enforcement impact.
## Trust & envelope model
- DSSE envelopes are canonicalised, hashed, and stored alongside the Rekor UUID, index, and proof.
- Signature modes span keyless (Fulcio), keyful (KMS/HSM), and hardware-backed (FIDO2). Multiple signatures are supported per envelope.
- Proofs include Merkle inclusion path, checkpoint metadata, optional witness endorsements, and cached verification verdicts.
- CAS/object storage retains envelopes + provenance for later replay; Rekor backends may be primary plus mirrors.
## Security hardening
- `attestor.write`, `attestor.verify`, and `attestor.read` scopes are enforced per endpoint; verify/list flows accept read/verify scopes while submissions remain write-only.
- JSON content-type is mandatory; malformed content returns `415 unsupported_media_type`.
- DSSE payloads are capped at 2 MiB (configurable), certificate chains at six entries, and each envelope may carry up to six signatures to contain parsing abuse.
- All verification/list APIs share the token-bucket rate limiter (`quotas.perCaller`) in addition to the existing submission limiter.
## UI, CLI, and SDK workflows
- **Console:** Evidence browser, verification reports, chain-of-custody graph, issuer/key management, attestation workbench, and bulk verification flows.
- **CLI / SDK:** `stella attest sign|verify|list|fetch|key` commands plus language SDKs to integrate build pipelines and offline verification scripts.
- **Policy Studio:** Verification policies author required predicate types, issuers, witness requirements, and freshness windows; simulations show enforcement impact.
## Storage, offline & air-gap posture
- MongoDB stores entry metadata, dedupe keys, and audit events; object storage optionally archives DSSE bundles.

245
docs/modules/attestor/architecture.md
View File

@@ -45,17 +45,23 @@ Trust boundary: **Only the Signer** is allowed to call submission endpoints; enf
- `StellaOps.BuildProvenance@1`
- `StellaOps.SBOMAttestation@1`
- `StellaOps.ScanResults@1`
- `StellaOps.PolicyEvaluation@1`
- `StellaOps.VEXAttestation@1`
- `StellaOps.RiskProfileEvidence@1`
- `StellaOps.PolicyEvaluation@1`
- `StellaOps.VEXAttestation@1`
- `StellaOps.RiskProfileEvidence@1`
Each predicate embeds subject digests, issuer metadata, policy context, materials, and optional transparency hints. Unsupported predicates return `422 predicate_unsupported`.
> **Golden fixtures:** Deterministic JSON statements for each predicate live in `src/Attestor/StellaOps.Attestor.Types/samples`. They are kept stable by the `StellaOps.Attestor.Types.Tests` project so downstream docs and contracts can rely on them without drifting.
Each predicate embeds subject digests, issuer metadata, policy context, materials, and optional transparency hints. Unsupported predicates return `422 predicate_unsupported`.
### Envelope & signature model
- DSSE envelopes canonicalised (stable JSON ordering) prior to hashing.
- Signature modes: keyless (Fulcio cert chain), keyful (KMS/HSM), hardware (FIDO2/WebAuthn). Multiple signatures allowed.
- Rekor entry stores bundle hash, certificate chain, and optional witness endorsements.
- Archive CAS retains original envelope plus metadata for offline verification.
### Envelope & signature model
- DSSE envelopes canonicalised (stable JSON ordering) prior to hashing.
- Signature modes: keyless (Fulcio cert chain), keyful (KMS/HSM), hardware (FIDO2/WebAuthn). Multiple signatures allowed.
- Rekor entry stores bundle hash, certificate chain, and optional witness endorsements.
- Archive CAS retains original envelope plus metadata for offline verification.
- Envelope serializer emits **compact** (canonical, minified) and **expanded** (annotated, indented) JSON variants off the same canonical byte stream so hashing stays deterministic while humans get context.
- Payload handling supports **optional compression** (`gzip`, `brotli`) with compression metadata recorded in the expanded view and digesting always performed over the uncompressed bytes.
- Expanded envelopes surface **detached payload references** (URI, digest, media type, size) so large artifacts can live in CAS/object storage while the canonical payload remains embedded for verification.
- Payload previews auto-render JSON or UTF-8 text in the expanded output to simplify triage in air-gapped and offline review flows.
### Verification pipeline overview
1. Fetch envelope (from request, cache, or storage) and validate DSSE structure.
@@ -151,11 +157,53 @@ Indexes:
## 4) APIs
### 4.1 Submission
`POST /api/v1/rekor/entries` *(mTLS + OpTok required)*
* **Body**: as above.
### 4.1 Signing
`POST /api/v1/attestations:sign` *(mTLS + OpTok required)*
* **Purpose**: Deterministically wrap Stella Ops payloads in DSSE envelopes before Rekor submission. Reuses the submission rate limiter and honours caller tenancy/audience scopes.
* **Body**:
```json
{
"keyId": "signing-key-id",
"payloadType": "application/vnd.in-toto+json",
"payload": "<base64 payload>",
"mode": "keyless|keyful|kms",
"certificateChain": ["-----BEGIN CERTIFICATE-----..."],
"artifact": {
"sha256": "<subject sha256>",
"kind": "sbom|report|vex-export",
"imageDigest": "sha256:...",
"subjectUri": "oci://..."
},
"logPreference": "primary|mirror|both",
"archive": true
}
```
* **Behaviour**:
* Resolve the signing key from `attestor.signing.keys[]` (includes algorithm, provider, and optional KMS version).
* Compute DSSE pre‑authentication encoding, sign with the resolved provider (default EC, BouncyCastle Ed25519, or File‑KMS ES256), and add static + request certificate chains.
* Canonicalise the resulting bundle, derive `bundleSha256`, and mirror the request meta shape used by `/api/v1/rekor/entries`.
* Emit `attestor.sign_total{result,algorithm,provider}` and `attestor.sign_latency_seconds{algorithm,provider}` metrics and append an audit row (`action=sign`).
* **Response 200**:
```json
{
"bundle": { "dsse": { "payloadType": "...", "payload": "...", "signatures": [{ "keyid": "signing-key-id", "sig": "..." }] }, "certificateChain": ["..."], "mode": "kms" },
"meta": { "artifact": { "sha256": "...", "kind": "sbom" }, "bundleSha256": "...", "logPreference": "primary", "archive": true },
"key": { "keyId": "signing-key-id", "algorithm": "ES256", "mode": "kms", "provider": "kms", "signedAt": "2025-11-01T12:34:56Z" }
}
```
* **Errors**: `400 key_not_found`, `400 payload_missing|payload_invalid_base64|artifact_sha_missing`, `400 mode_not_allowed`, `403 client_certificate_required`, `401 invalid_token`, `500 signing_failed`.
### 4.2 Submission
`POST /api/v1/rekor/entries` *(mTLS + OpTok required)*
* **Body**: as above.
* **Behavior**:
* Verify caller (mTLS + OpTok).
@@ -178,16 +226,16 @@ Indexes:
"status": "included"
}
```
* **Errors**: `401 invalid_token`, `403 not_signer|chain_untrusted`, `409 duplicate_bundle` (with existing `uuid`), `502 rekor_unavailable`, `504 proof_timeout`.
### 4.2 Proof retrieval
`GET /api/v1/rekor/entries/{uuid}`
* **Errors**: `401 invalid_token`, `403 not_signer|chain_untrusted`, `409 duplicate_bundle` (with existing `uuid`), `502 rekor_unavailable`, `504 proof_timeout`.
### 4.3 Proof retrieval
`GET /api/v1/rekor/entries/{uuid}`
* Returns `entries` row (refreshes proof from Rekor if stale/missing).
* Accepts `?refresh=true` to force backend query.
### 4.3 Verification (third‑party or internal)
### 4.4 Verification (third‑party or internal)
`POST /api/v1/rekor/verify`
@@ -202,17 +250,28 @@ Indexes:
1. **Bundle signature** → cert chain to Fulcio/KMS roots configured.
2. **Inclusion proof** → recompute leaf hash; verify Merkle path against checkpoint root.
3. Optionally verify **checkpoint** against local trust anchors (if Rekor signs checkpoints).
4. Confirm **subject.digest** matches caller‑provided hash (when given).
4. Confirm **subject.digest** matches caller‑provided hash (when given).
5. Fetch **transparency witness** statement when enabled; cache results and downgrade status to WARN when endorsements are missing or mismatched.
* **Response**:
```json
{ "ok": true, "uuid": "…", "index": 123, "logURL": "…", "checkedAt": "…" }
```
### 4.4 Batch submission (optional)
`POST /api/v1/rekor/batch` accepts an array of submission objects; processes with per‑item results.
* **Response**:
```json
{ "ok": true, "uuid": "…", "index": 123, "logURL": "…", "checkedAt": "…" }
```
### 4.5 Bulk verification
`POST /api/v1/rekor/verify:bulk` enqueues a verification job containing up to `quotas.bulk.maxItemsPerJob` items. Each item mirrors the single verification payload (uuid | artifactSha256 | subject+envelopeId, optional policyVersion/refreshProof). The handler persists a MongoDB job document (`bulk_jobs` collection) and returns `202 Accepted` with a job descriptor and polling URL.
`GET /api/v1/rekor/verify:bulk/{jobId}` returns progress and per-item results (subject/uuid, status, issues, cached verification report if available). Jobs are tenant- and subject-scoped; only the initiating principal can read their progress.
**Worker path:** `BulkVerificationWorker` claims queued jobs (`status=queued → running`), executes items sequentially through the cached verification service, updates progress counters, and records metrics:
- `attestor.bulk_jobs_total{status}` – completed/failed jobs
- `attestor.bulk_job_duration_seconds{status}` – job runtime
- `attestor.bulk_items_total{status}` – per-item outcomes (`succeeded`, `verification_failed`, `exception`)
The worker honours `bulkVerification.itemDelayMilliseconds` for throttling and reschedules persistence conflicts with optimistic version checks. Results hydrate the verification cache; failed items record the error reason without aborting the overall job.
---
@@ -244,8 +303,10 @@ Indexes:
* `subject.digest.sha256` values must be present and well‑formed (hex).
* **No public submission** path. **Never** accept bundles from untrusted clients.
* **Client certificate allowlists**: optional `security.mtls.allowedSubjects` / `allowedThumbprints` tighten peer identity checks beyond CA pinning.
* **Rate limits**: token-bucket per caller derived from `quotas.perCaller` (QPS/burst) returns `429` + `Retry-After` when exceeded.
* **Redaction**: Attestor never logs secret material; DSSE payloads **should** be public by design (SBOMs/reports). If customers require redaction, enforce policy at Signer (predicate minimization) **before** Attestor.
* **Rate limits**: token-bucket per caller derived from `quotas.perCaller` (QPS/burst) returns `429` + `Retry-After` when exceeded.
* **Scope enforcement**: API separates `attestor.write`, `attestor.verify`, and `attestor.read` policies; verification/list endpoints accept read or verify scopes while submission endpoints remain write-only.
* **Request hygiene**: JSON content-type is mandatory (415 returned otherwise); DSSE payloads are capped (default 2 MiB), certificate chains limited to six entries, and signatures to six per envelope to mitigate parsing abuse.
* **Redaction**: Attestor never logs secret material; DSSE payloads **should** be public by design (SBOMs/reports). If customers require redaction, enforce policy at Signer (predicate minimization) **before** Attestor.
---
@@ -268,24 +329,32 @@ Indexes:
## 8) Observability & audit
**Metrics** (Prometheus):
* `attestor.submit_total{result,backend}`
* `attestor.submit_latency_seconds{backend}`
* `attestor.proof_fetch_total{result}`
* `attestor.verify_total{result}`
* `attestor.dedupe_hits_total`
* `attestor.errors_total{type}`
**Correlation**:
* HTTP callers may supply `X-Correlation-Id`; Attestor will echo the header and push `CorrelationId` into the log scope for cross-service tracing.
**Tracing**:
* Spans: `validate`, `rekor.submit`, `rekor.poll`, `persist`, `archive`, `verify`.
**Audit**:
**Metrics** (Prometheus):
* `attestor.sign_total{result,algorithm,provider}`
* `attestor.sign_latency_seconds{algorithm,provider}`
* `attestor.submit_total{result,backend}`
* `attestor.submit_latency_seconds{backend}`
* `attestor.proof_fetch_total{subject,issuer,policy,result,attestor.log.backend}`
* `attestor.verify_total{subject,issuer,policy,result}`
* `attestor.verify_latency_seconds{subject,issuer,policy,result}`
* `attestor.dedupe_hits_total`
* `attestor.errors_total{type}`
SLO guardrails:
* `attestor.verify_latency_seconds` P95 ≤ 2 s per policy.
* `attestor.verify_total{result="failed"}` ≤ 1 % of `attestor.verify_total` over 30 min rolling windows.
**Correlation**:
* HTTP callers may supply `X-Correlation-Id`; Attestor will echo the header and push `CorrelationId` into the log scope for cross-service tracing.
**Tracing**:
* Spans: `attestor.sign`, `validate`, `rekor.submit`, `rekor.poll`, `persist`, `archive`, `attestor.verify`, `attestor.verify.refresh_proof`.
**Audit**:
* Immutable `audit` rows (ts, caller, action, hashes, uuid, index, backend, result, latency).
@@ -296,20 +365,45 @@ Indexes:
```yaml
attestor:
listen: "https://0.0.0.0:8444"
security:
mtls:
caBundle: /etc/ssl/signer-ca.pem
requireClientCert: true
authority:
issuer: "https://authority.internal"
jwksUrl: "https://authority.internal/jwks"
requireSenderConstraint: "dpop" # or "mtls"
signerIdentity:
mode: ["keyless","kms"]
fulcioRoots: ["/etc/fulcio/root.pem"]
allowedSANs: ["urn:stellaops:signer"]
kmsKeys: ["kms://cluster-kms/stellaops-signer"]
rekor:
security:
mtls:
caBundle: /etc/ssl/signer-ca.pem
requireClientCert: true
authority:
issuer: "https://authority.internal"
jwksUrl: "https://authority.internal/jwks"
requireSenderConstraint: "dpop" # or "mtls"
signerIdentity:
mode: ["keyless","kms"]
fulcioRoots: ["/etc/fulcio/root.pem"]
allowedSANs: ["urn:stellaops:signer"]
kmsKeys: ["kms://cluster-kms/stellaops-signer"]
submissionLimits:
maxPayloadBytes: 2097152
maxCertificateChainEntries: 6
maxSignatures: 6
signing:
preferredProviders: ["kms","bouncycastle.ed25519","default"]
kms:
enabled: true
rootPath: "/var/lib/stellaops/kms"
password: "${ATTESTOR_KMS_PASSWORD}"
keys:
- keyId: "kms-primary"
algorithm: ES256
mode: kms
provider: "kms"
providerKeyId: "kms-primary"
kmsVersionId: "v1"
- keyId: "ed25519-offline"
algorithm: Ed25519
mode: keyful
provider: "bouncycastle.ed25519"
materialFormat: base64
materialPath: "/etc/stellaops/keys/ed25519.key"
certificateChain:
- "-----BEGIN CERTIFICATE-----...-----END CERTIFICATE-----"
rekor:
primary:
url: "https://rekor-v2.internal"
proofTimeoutMs: 15000
@@ -328,13 +422,20 @@ attestor:
objectLock: "governance"
redis:
url: "redis://redis:6379/2"
quotas:
perCaller:
qps: 50
burst: 100
```
---
quotas:
perCaller:
qps: 50
burst: 100
```
**Notes:**
* `signing.preferredProviders` defines the resolution order when multiple providers support the requested algorithm. Omit to fall back to registration order.
* File-backed KMS (`signing.kms`) is required when at least one key uses `mode: kms`; the password should be injected via secret store or environment.
* For keyful providers, supply inline `material` or `materialPath` plus `materialFormat` (`pem` (default), `base64`, or `hex`). KMS keys ignore these fields and require `kmsVersionId`.
* `certificateChain` entries are appended to returned bundles so offline verifiers do not need to dereference external stores.
---
## 10) End‑to‑end sequences

48
docs/modules/attestor/payloads.md Normal file
View File

@@ -0,0 +1,48 @@
# Attestor Payload Reference
StellaOps evidence predicates must remain reproducible, explainable, and portable across online and fully air-gapped deployments. This guide lists each predicate type, indicates where the canonical JSON Schema lives, highlights the producing service, and links to the matching golden samples.
## Quick Reference
| Type ID | Predicate URI | Schema file | Produced by | Primary consumers |
| --- | --- | --- | --- | --- |
| StellaOps.BuildProvenance@1 | https://schemas.stella-ops.org/attestations/build-provenance@1 | src/Attestor/StellaOps.Attestor.Types/schemas/stellaops-build-provenance.v1.schema.json | Build pipelines, Scanner SBOM bake stage | Attestor, Export Center, Policy Engine |
| StellaOps.SBOMAttestation@1 | https://schemas.stella-ops.org/attestations/sbom-attestation@1 | src/Attestor/StellaOps.Attestor.Types/schemas/stellaops-sbom-attestation.v1.schema.json | Scanner.Worker SBOM composer | Policy Engine, CLI, Export Center |
| StellaOps.ScanResults@1 | https://schemas.stella-ops.org/attestations/scan-results@1 | src/Attestor/StellaOps.Attestor.Types/schemas/stellaops-scan-results.v1.schema.json | Scanner.Worker analyzers | Policy Engine, CLI, Orchestrator |
| StellaOps.PolicyEvaluation@1 | https://schemas.stella-ops.org/attestations/policy-evaluation@1 | src/Attestor/StellaOps.Attestor.Types/schemas/stellaops-policy-evaluation.v1.schema.json | Policy Engine explain pipeline | CLI, Notify, Export Center |
| StellaOps.VEXAttestation@1 | https://schemas.stella-ops.org/attestations/vex-attestation@1 | src/Attestor/StellaOps.Attestor.Types/schemas/stellaops-vex-attestation.v1.schema.json | Excititor consensus service | Policy Engine, CLI, Console |
| StellaOps.RiskProfileEvidence@1 | https://schemas.stella-ops.org/attestations/risk-profile@1 | src/Attestor/StellaOps.Attestor.Types/schemas/stellaops-risk-profile.v1.schema.json | Policy Engine risk pipeline | Console, Notify, Export Center |
| StellaOps.CustomEvidence@1 | https://schemas.stella-ops.org/attestations/custom-evidence@1 | src/Attestor/StellaOps.Attestor.Types/schemas/stellaops-custom-evidence.v1.schema.json | CLI custom evidence workflows and partner integrations | Policy Engine (policy hooks), Export Center |
Golden JSON fixtures that double as contract tests live under `src/Attestor/StellaOps.Attestor.Types/fixtures/v1/<predicate>.sample.json`. TypeScript and Go clients consume the generated sources in `src/Attestor/StellaOps.Attestor.Types/generated/ts` and `src/Attestor/StellaOps.Attestor.Types/generated/go`.
## Envelope Conventions
- DSSE envelopes are signed over canonical JSON (sorted keys, UTF-8, no insignificant whitespace).
- The `subject` array must include at least one SHA-256 digest and may attach annotations such as `oci.reference` or `stellaops.asset`.
- `predicateType` uses the URI shown in the table; `predicate.typeId` mirrors the short identifier.
- `predicate.schemaVersion` follows semantic versioning. Consumers reject mismatched major versions.
- Optional `metadata` and `materials` sections follow the in-toto Statement format to maximise provenance portability.
## Predicate Highlights
- **StellaOps.BuildProvenance@1** records builder identity, config source, materials, reproducibility flags, and the resulting artifact digests. Outputs must match the DSSE subject.
- **StellaOps.SBOMAttestation@1** links an artifact digest to a CycloneDX 1.6 or SBOM 3.0.0 document, tracking inventory counts and the generator metadata. Component graph hashes reference CAS entries emitted by Scanner.Worker.
- **StellaOps.ScanResults@1** captures deterministic findings from OS, language, and native analyzers. It reports summary counts, per-finding metadata (PURL, severity, exploitability), and the layer digests inspected.
- **StellaOps.PolicyEvaluation@1** documents lattice-based policy outcomes, including decision traces and evidence digests consumed during evaluation.
- **StellaOps.VEXAttestation@1** mirrors OpenVEX-aligned statements with justification, scope narrowing (package coordinates or component IDs), and issue timestamps.
- **StellaOps.RiskProfileEvidence@1** summarises exploitability, ticketing load, runtime coverage, and maturity for downstream dashboards.
- **StellaOps.CustomEvidence@1** allows regulated tenants to attach organisation-specific payloads referenced by a CAS-hosted schema while preserving provenance and retention controls.
## Validation and Tooling
- Run `npm install` once, then `npm run docs:attestor:validate` to validate JSON fixtures against their schemas, execute the generated TypeScript tests (`npm test`), and run `go test ./...` for the Go SDK. The command fails fast when any schema, fixture, or generated SDK drifts.
- Regenerate schemas and SDKs after edits with `dotnet run --project src/Attestor/StellaOps.Attestor.Types/Tools/StellaOps.Attestor.Types.Generator`.
- Offline Kit builds (`ops/devops/offline-kit/`) mirror schemas, fixtures, and SDK bundles so air-gapped operators can run the same validation stack.
## Related Material
- `docs/modules/attestor/architecture.md` — service topology, Rekor integration, caching model.
- `docs/modules/platform/architecture-overview.md` — cross-module data flows and tenant boundaries.
- `docs/ingestion/aggregation-only-contract.md` — guardrails for advisory feeds consumed by policy evaluation.
- `src/Attestor/StellaOps.Attestor.Types/samples/README.md` — directory map for the golden evidence set referenced here.

41
docs/modules/attestor/ttl-validation.md Normal file
View File

@@ -0,0 +1,41 @@
# Attestor TTL Validation Runbook
> **Purpose:** confirm MongoDB TTL indexes and Redis expirations for the attestation dedupe store behave as expected on a production-like stack.
## Prerequisites
- Docker Desktop or compatible daemon with the Compose plugin enabled.
- Local ports `27017` and `6379` free.
- `dotnet` SDK 10.0 preview (same as repo toolchain).
- Network access to pull `mongo:7` and `redis:7` images.
## Quickstart
1. From the repo root export any required proxy settings, then run
```bash
scripts/run-attestor-ttl-validation.sh
```
The helper script:
- Spins up `mongo:7` and `redis:7` containers.
- Sets `ATTESTOR_LIVE_MONGO_URI` / `ATTESTOR_LIVE_REDIS_URI`.
- Executes the live TTL test suite (`Category=LiveTTL`) in `StellaOps.Attestor.Tests`.
- Tears the stack down automatically.
2. Capture the test output (`ttl-validation-<timestamp>.log`) and attach it to the sprint evidence folder (`docs/modules/attestor/evidence/`).
## Result handling
- **Success:** Tests complete in ~3–4 minutes with `Total tests: 2, Passed: 2`. Store the log and note the run in `SPRINT_100_identity_signing.md` under ATTESTOR-72-003.
- **Failure:** Preserve:
- `docker compose logs` for both services.
- `mongosh` output of `db.dedupe.getIndexes()` and sample documents.
- `redis-cli --raw ttl attestor:ttl:live:bundle:<id>`.
File an incident in the Attestor Guild channel and link the captured artifacts.
## Manual verification (optional)
If the helper script cannot be used:
1. Start MongoDB and Redis manually with equivalent configuration.
2. Set `ATTESTOR_LIVE_MONGO_URI` and `ATTESTOR_LIVE_REDIS_URI`.
3. Run `dotnet test src/Attestor/StellaOps.Attestor.sln --no-build --filter "Category=LiveTTL"`.
4. Follow the evidence handling steps above.
## Ownership
- Primary: Attestor Service Guild.
- Partner: QA Guild (observes TTL metrics, confirms evidence archiving).

247
docs/modules/attestor/workflows.md Normal file
View File

@@ -0,0 +1,247 @@
# Attestor Verification Workflows
> How StellaOps turns DSSE bundles into verifiable evidence, how the verification API reports outcomes, and how explainability signals surface in UI/CLI flows.
> ⚠️ **2025-11-01 coordination note:** `StellaOps.Attestor.WebService` is failing to compile until downstream fixes land (`Contracts/AttestationBundleContracts.cs` null-coalescing update and scope/token variables restored in `Program.cs`). Verification flows ship in infrastructure/tests, but the WebService hand-off stays blocked — track via `ATTESTOR-73-002` (see Attestor task board).
## 1. Verification flow (API and service contract)
- **Entry point.** `POST /api/v1/rekor/verify` deserialises to `AttestorVerificationRequest`.
- **Resolution order.** The service tries `uuid`, then canonicalised `bundle`, then `artifactSha256`. At least one selector must be present (`invalid_query` otherwise).
- **Optional proof refresh.** `refreshProof=true` forces a Rekor lookup before returning. Proofs are cached in Mongo.
- **Signature replay.** Supplying `bundle` lets the service recompute the canonical hash and re-run signature checks; omitting the bundle skips those steps but still validates Merkle proofs and cached policy decisions.
- **Auth scopes.** Endpoints demand `attestor.verify` (write scope is also accepted); read-only detail/list APIs require `attestor.read` at minimum.
### 1.1 Request properties
| Field | Type | Required | Purpose |
|-------|------|----------|---------|
| `uuid` | string | optional | Rekor V2 UUID to verify and (optionally) refresh. |
| `bundle` | object | optional | DSSE envelope (same shape as submission) for signature re-verification. |
| `artifactSha256` | string | optional | Resolve the most recent entry for an attestable artefact digest. |
| `subject` | string | optional | Logical subject identifier used for cache/telemetry tagging; defaults to the stored artifact digest. |
| `envelopeId` | string | optional | Stable identifier for the DSSE bundle (typically the canonical hash); enables cache lookups. |
| `policyVersion` | string | optional | Policy digest/version driving verification; feeds cache keys and observability dimensions. |
| `refreshProof` | bool | optional (default `false`) | Pull the current inclusion proof and checkpoint from Rekor before evaluating. |
All selectors are mutually compatible; if more than one is set the service uses the first match (`uuid` → `bundle` → `artifactSha256`).
### 1.2 Response schema (`AttestorVerificationResult`)
| Field | Type | Description |
|-------|------|-------------|
| `ok` | bool | `true` when the entry status is `included` **and** no issues were recorded. |
| `uuid` | string | Rekor UUID that satisfied the query. Useful for follow-up fetches. |
| `index` | number (int64) | Rekor log index, when supplied by the backend. |
| `logUrl` | string | Fully-qualified Rekor entry URL for operators and auditors. |
| `status` | string | Transparency-log status seen in Mongo (`included`, `pending`, `failed`, …). |
| `checkedAt` | string (ISO-8601 UTC) | Timestamp emitted when the response is created. |
| `issues` | array[string] | Machine-readable explainability codes. Empty when `ok=true`. |
> **Note:** `checkedAt` is recomputed each call; cache hits do not recycle previous timestamps.
### 1.3 Success criteria
`ok=true` requires:
1. Entry exists and status equals `included`.
2. Canonical DSSE hash matches the stored bundle hash.
3. Signature re-verification (when a bundle is supplied) succeeds.
4. Inclusion proof validates against the cached or refreshed checkpoint.
Any deviation records at least one issue and flips `ok` to `false`. Consumers **must** inspect `issues` rather than inferring from `status` alone.
## 2. Verification report schema
`AttestorVerificationResult` carries the flattened summary shown above. When callers request the detailed report (`GET /api/v1/rekor/entries/{uuid}?refresh=true` or via SDK) they receive a `VerificationReport` shaped as follows:
```json
{
"overallStatus": "pass",
"succeeded": true,
"policy": { ... },
"issuer": { ... },
"freshness": { ... },
"signatures": { ... },
"transparency": { ... },
"issues": [ "bundle_hash_mismatch" ]
}
```
| Field | Type | Description |
|-------|------|-------------|
| `overallStatus` | string (`pass`, `warn`, `fail`, `skipped`) | Aggregated verdict derived from the individual section statuses. |
| `succeeded` | bool | Convenience flag; `true` when `overallStatus ∈ {pass, warn}`. |
| `policy` | object | Results from policy evaluation (see below). |
| `issuer` | object | Identity/result of the signing entity. |
| `freshness` | object | Age analysis relative to policy settings. |
| `signatures` | object | Signature validation summary. |
| `transparency` | object | Inclusion proof / checkpoint evaluation summary. |
| `issues` | array[string] | De-duplicated set drawn from the sections; order is deterministic and stable. |
### 2.1 `policy`
| Field | Description |
|-------|-------------|
| `status` | Section verdict (`pass`, `warn`, `fail`, `skipped`). |
| `policyId` / `policyVersion` | DSL identifier and revision used for evaluation. |
| `verdict` | Policy outcome (`allow`, `challenge`, `deny`, etc.). |
| `issues` | Policy-specific explainability codes (e.g., `policy_rule_blocked`). |
| `attributes` | Key/value map emitted by the policy for downstream observability (e.g., applicable rules, matched waivers). |
### 2.2 `issuer`
| Field | Description |
|-------|-------------|
| `status` | Result of issuer validation. |
| `mode` | Signing mode detected (`keyless`, `kms`, `unknown`). |
| `issuer` | Distinguished name / issuer URI recorded during signing. |
| `subjectAlternativeName` | SAN pulled from the Fulcio certificate (keyless) or recorded KMS identity. |
| `keyId` | Logical key identifier associated with the signature. |
| `issues` | Issuer-specific issues (e.g., `issuer_trust_root_mismatch`, `signer_mode_unsupported:kid`). |
### 2.3 `freshness`
| Field | Description |
|-------|-------------|
| `status` | `fail` when the attestation exceeds `verification.freshnessMaxAgeMinutes`; `warn` when only the warning threshold is hit. |
| `createdAt` | Timestamp embedded in the attestation metadata. |
| `evaluatedAt` | Server-side timestamp used for age calculations. |
| `age` | ISO8601 duration of `evaluatedAt - createdAt`. |
| `maxAge` | Policy-driven ceiling (null when unchecked). |
| `issues` | `freshness_max_age_exceeded`, `freshness_warning`, etc. |
### 2.4 `signatures`
| Field | Description |
|-------|-------------|
| `status` | Signature validation verdict. |
| `bundleProvided` | `true` when canonical DSSE bytes were supplied. |
| `totalSignatures` | Count observed in the DSSE envelope. |
| `verifiedSignatures` | Number of signatures that validated against trusted keys. |
| `requiredSignatures` | Policy / configuration minimum enforced. |
| `issues` | Signature codes such as `bundle_payload_invalid_base64`, `signature_invalid`, `signer_mode_unknown`. |
### 2.5 `transparency`
| Field | Description |
|-------|-------------|
| `status` | Inclusion proof / checkpoint verdict. |
| `proofPresent` | Whether a proof document was available. |
| `checkpointPresent` | Indicates the Rekor checkpoint existed and parsed. |
| `inclusionPathPresent` | `true` when the Merkle path array contained nodes. |
| `issues` | Merkle/rekor codes (`proof_missing`, `proof_leafhash_mismatch`, `checkpoint_missing`, `proof_root_mismatch`). |
### 2.6 Issue catalogue (non-exhaustive)
| Code | Trigger | Notes |
|------|---------|-------|
| `bundle_hash_mismatch` | Canonical DSSE hash differs from stored value. | Often indicates tampering or inconsistent canonicalisation. |
| `bundle_payload_invalid_base64` | DSSE payload cannot be base64-decoded. | Validate producer pipeline; the attestation is unusable. |
| `signature_invalid` | At least one signature failed cryptographic verification. | Consider checking key rotation / revocation status. |
| `signer_mode_unknown` / `signer_mode_unsupported:<mode>` | Signing mode not configured for this installation. | Update `attestorOptions.security.signerIdentity.mode`. |
| `issuer_trust_root_mismatch` | Certificate chain does not terminate in configured Fulcio/KMS roots. | Check Fulcio bundle / KMS configuration. |
| `freshness_max_age_exceeded` | Attestation older than permitted maximum. | Regenerate attestation or extend policy window. |
| `proof_missing` | No inclusion proof stored or supplied. | When running offline, import bundles with proofs or allow warn-level policies. |
| `proof_root_mismatch` | Rebuilt Merkle root differs from checkpoint. | Proof may be stale or log compromised; escalate. |
| `checkpoint_missing` | No Rekor checkpoint available. | Configure `RequireCheckpoint=false` to downgrade severity. |
Downstream consumers (UI, CLI, policy studio) should render human-readable messages but must retain the exact issue codes for automation and audit replay.
## 3. Explainability signals
1. **Canonicalisation.** The service replays DSSE canonicalisation to derive `bundleSha256`. Failures surface as `bundle_hash_mismatch` or decoding errors.
2. **Signature checks.** Mode-aware handling:
- `kms` (HMAC) compares against configured shared secrets.
- `keyless` rebuilds the certificate chain, enforces Fulcio roots, SAN allow-lists, and verifies with the leaf certificate.
- Unknown modes emit `signer_mode_unknown` / `signer_mode_unsupported:<mode>`.
3. **Proof acquisition.** When `refreshProof` is requested the Rekor backend may contribute a textual issue (`Proof refresh failed: …`) without stopping evaluation.
4. **Merkle validation.** Structured helper ensures leaf hash, path orientation, and checkpoint root are consistent; each validation failure has a discrete issue code.
5. **Observability.** The meter `attestor.verify_total` increments with `result=ok|failed`; structured logs and traces carry the same `issues` vector for UI/CLI drill-down.
All issues are appended in detection order to simplify chronological replay in the Console’s chain-of-custody view.
## 3. Issue catalogue
| Code | Trigger | Operator guidance |
|------|---------|-------------------|
| `bundle_hash_mismatch` | Canonicalised DSSE hash differs from stored bundle hash. | Re-download artefact; investigate tampering or submission races. |
| `bundle_payload_invalid_base64` | Payload could not be base64-decoded. | Ensure bundle transport preserved payload; capture original DSSE for forensics. |
| `signature_invalid_kms` | HMAC verification failed for `mode=kms`. | Confirm shared secret alignment with Signer; rotate keys if drift detected. |
| `signer_mode_unknown` | Entry lacks signer mode metadata and bundle omitted it. | Re-ingest bundle or inspect submission pipeline metadata. |
| `signer_mode_unsupported:<mode>` | Signer mode is unsupported by the verifier. | Add support or block unsupported issuers in policy. |
| `kms_key_missing` | No configured KMS secrets to verify `mode=kms`. | Populate `security:signerIdentity:kmsKeys` in Attestor config before retry. |
| `signature_invalid_base64` | One or more signatures were not valid base64. | Bundle corruption; capture raw payload and re-submit. |
| `certificate_chain_missing` | `mode=keyless` bundle lacked any certificates. | Ensure Signer attaches Fulcio chain; review submission pipeline. |
| `certificate_chain_invalid` | Certificates could not be parsed. | Fetch original DSSE bundle for repair; confirm certificate encoding. |
| `certificate_chain_untrusted[:detail]` | Chain failed custom-root validation. | Import correct Fulcio roots or investigate potential impersonation. |
| `certificate_san_untrusted` | Leaf SAN not in configured allow-list. | Update allow-list or revoke offending issuer. |
| `signature_invalid` | No signature validated with supplied public keys. | Treat as tampering; trigger incident response. |
| `proof_missing` | No Merkle proof stored for the entry. | Re-run with `refreshProof=true`; check Rekor availability. |
| `bundle_hash_decode_failed` | Stored bundle hash could not be decoded. | Verify Mongo record integrity; re-enqueue submission if necessary. |
| `proof_inclusion_missing` | Inclusion section absent from proof. | Retry proof refresh; inspect Rekor health. |
| `proof_leafhash_decode_failed` | Leaf hash malformed. | Replay submission; inspect Rekor data corruption. |
| `proof_leafhash_mismatch` | Leaf hash differs from canonical bundle hash. | Raises tamper alert; reconcile Rekor entry vs stored bundle. |
| `proof_path_decode_failed` | Inclusion path entry malformed. | Same action as above; likely Rekor data corruption. |
| `proof_path_orientation_missing` | Inclusion path lacks left/right marker. | File Rekor bug; fallback to mirror log if configured. |
| `checkpoint_missing` | Proof lacks checkpoint metadata. | Retry refresh; ensure Rekor is configured to return checkpoints. |
| `checkpoint_root_decode_failed` | Checkpoint root hash malformed. | Investigate Rekor/mirror integrity before trusting log. |
| `proof_root_mismatch` | Computed root hash != checkpoint root. | Critical alert; assume inclusion proof compromised. |
| `Proof refresh failed: …` | Rekor fetch threw an exception. | Message includes upstream error; surface alongside telemetry for debugging. |
Future explainability flags must follow the same pattern: short, lowercase codes with optional suffix payload (`code:detail`).
## 4. Worked examples
### 4.1 Successful verification
```json
{
"ok": true,
"uuid": "0192fdb4-a82b-7f90-b894-6fd1dd918b85",
"index": 73421,
"logUrl": "https://rekor.stellaops.test/api/v2/log/entries/0192fdb4a82b7f90b8946fd1dd918b85",
"status": "included",
"checkedAt": "2025-11-01T17:06:52.182394Z",
"issues": []
}
```
This mirrors the happy-path asserted in `AttestorVerificationServiceTests.VerifyAsync_ReturnsOk_ForExistingUuid`, which replays the entire submission→verification loop.
### 4.2 Tampered bundle
```json
{
"ok": false,
"uuid": "0192fdb4-a82b-7f90-b894-6fd1dd918b85",
"index": 73421,
"logUrl": "https://rekor.stellaops.test/api/v2/log/entries/0192fdb4a82b7f90b8946fd1dd918b85",
"status": "included",
"checkedAt": "2025-11-01T17:09:05.443218Z",
"issues": [
"bundle_hash_mismatch",
"signature_invalid"
]
}
```
Derived from `AttestorVerificationServiceTests.VerifyAsync_FlagsTamperedBundle`, which flips the DSSE payload and expects both issues to surface. CLI and Console consumers should display these codes verbatim and provide remediation tips from the table above.
## 5. Validating the documentation
- Run `dotnet test src/Attestor/StellaOps.Attestor/StellaOps.Attestor.Tests` to exercise the scenarios behind the examples.
- API integrators can `curl` the verify endpoint and compare responses with the JSON above.
- UI/CLI teams should ensure explainability tooltips and runbooks reference the same issue catalogue.
Keeping the documentation aligned with the test suite guarantees explainability remains deterministic and audit-friendly.
## 6. Offline bundles & air-gapped verification
Stella Ops Attestor now supports packaging attestations for sealed environments and rehydrating them without calling Rekor:
- **Export bundles.** `POST /api/v1/attestations:export` accepts either a list of Rekor UUIDs or filter criteria (`subject`, `type`, `issuer`, `scope`, `createdAfter|Before`, `limit`, `continuationToken`) and returns an `attestor.bundle.v1` document. Each item contains the attestation entry, canonical DSSE payload (base64), optional proof payload, and metadata. Responses include a `continuationToken` so callers can page through large result sets (limits default to 100 and are capped at 200). JSON content is required and requests are gated by the `attestor.read` scope.
- **Import bundles.** `POST /api/v1/attestations:import` ingests the bundle document, upserts attestation metadata, and restores the canonical DSSE/proof into the configured archive store. The S3 archive integration must be enabled; the response reports how many entries were imported versus updated, any skipped items, and issue codes (`bundle_payload_invalid_base64`, `bundle_hash_mismatch`, `archive_disabled`, …).
- **Offline verification.** When replaying verification without log connectivity, submit the DSSE bundle and set `offline=true` on `POST /api/v1/rekor/verify`. The service reuses imported proofs when present and surfaces deterministic explainability codes (`proof_missing`, `proof_inclusion_missing`, …) instead of attempting Rekor fetches.
Tests `AttestorBundleServiceTests.ExportAsync_AppliesFiltersAndContinuation`, `AttestationBundleEndpointsTests`, `AttestorVerificationServiceTests.VerifyAsync_OfflineSkipsProofRefreshWhenMissing`, and `AttestorVerificationServiceTests.VerifyAsync_OfflineUsesImportedProof` exercise the exporter/importer, API contracts, and the offline verification path with and without witness data.