stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 1 Releases Wiki Activity

feat(docs): Add comprehensive documentation for Vexer, Vulnerability Explorer, and Zastava modules

Browse Source 
- Introduced AGENTS.md, README.md, TASKS.md, and implementation_plan.md for Vexer, detailing mission, responsibilities, key components, and operational notes.
- Established similar documentation structure for Vulnerability Explorer and Zastava modules, including their respective workflows, integrations, and observability notes.
- Created risk scoring profiles documentation outlining the core workflow, factor model, governance, and deliverables.
- Ensured all modules adhere to the Aggregation-Only Contract and maintain determinism and provenance in outputs.
This commit is contained in:
master
2025-10-30 00:09:39 +02:00
parent 3154c67978
commit 7b5bdcf4d3
503 changed files with 16136 additions and 54638 deletions
Download Patch File Download Diff File Expand all files Collapse all files

432
docs/modules/attestor/architecture.md Normal file
View File

@@ -0,0 +1,432 @@
# component_architecture_attestor.md — **Stella Ops Attestor** (2025Q4)
> Derived from Epic 19 – Attestor Console with provenance hooks aligned to the Export Center bundle workflows scoped in Epic 10.
> **Scope.** Implementation‑ready architecture for the **Attestor**: the service that **submits** DSSE envelopes to **Rekor v2**, retrieves/validates inclusion proofs, caches results, and exposes verification APIs. It accepts DSSE **only** from the **Signer** over mTLS, enforces chain‑of‑trust to Stella Ops roots, and returns `{uuid, index, proof, logURL}` to calling services (Scanner.WebService for SBOMs; backend for final reports; Excititor exports when configured).
---
## 0) Mission & boundaries
**Mission.** Turn a signed DSSE envelope from the Signer into a **transparency‑logged, verifiable fact** with a durable, replayable proof (Merkle inclusion + (optional) checkpoint anchoring). Provide **fast verification** for downstream consumers and a stable retrieval interface for UI/CLI.
**Boundaries.**
* Attestor **does not sign**; it **must not** accept unsigned or third‑party‑signed bundles.
* Attestor **does not decide PASS/FAIL**; it logs attestations for SBOMs, reports, and export artifacts.
* Rekor v2 backends may be **local** (self‑hosted) or **remote**; Attestor handles both with retries, backoff, and idempotency.
---
## 1) Topology & dependencies
**Process shape:** single stateless service `stellaops/attestor` behind mTLS.
**Dependencies:**
* **Signer** (caller) — authenticated via **mTLS** and **Authority** OpToks.
* **Rekor v2** — tile‑backed transparency log endpoint(s).
* **MinIO (S3)** — optional archive store for DSSE envelopes & verification bundles.
* **MongoDB** — local cache of `{uuid, index, proof, artifactSha256, bundleSha256}`; job state; audit.
* **Redis** — dedupe/idempotency keys and short‑lived rate‑limit buckets.
* **Licensing Service (optional)** — “endorse” call for cross‑log publishing when customer opts‑in.
Trust boundary: **Only the Signer** is allowed to call submission endpoints; enforced by **mTLS peer cert allowlist** + `aud=attestor` OpTok.
---
### Roles, identities & scopes
- **Subjects** — immutable digests for artifacts (container images, SBOMs, reports) referenced in DSSE envelopes.
- **Issuers** — authenticated builders/scanners/policy engines signing evidence; tracked with mode (`keyless`, `kms`, `hsm`, `fido2`) and tenant scope.
- **Consumers** — Scanner, Export Center, CLI, Console, Policy Engine that verify proofs using Attestor APIs.
- **Authority scopes** — `attestor.write`, `attestor.verify`, `attestor.read`, and administrative scopes for key management; all calls mTLS/DPoP-bound.
### Supported predicate types
- `StellaOps.BuildProvenance@1`
- `StellaOps.SBOMAttestation@1`
- `StellaOps.ScanResults@1`
- `StellaOps.PolicyEvaluation@1`
- `StellaOps.VEXAttestation@1`
- `StellaOps.RiskProfileEvidence@1`
Each predicate embeds subject digests, issuer metadata, policy context, materials, and optional transparency hints. Unsupported predicates return `422 predicate_unsupported`.
### Envelope & signature model
- DSSE envelopes canonicalised (stable JSON ordering) prior to hashing.
- Signature modes: keyless (Fulcio cert chain), keyful (KMS/HSM), hardware (FIDO2/WebAuthn). Multiple signatures allowed.
- Rekor entry stores bundle hash, certificate chain, and optional witness endorsements.
- Archive CAS retains original envelope plus metadata for offline verification.
### Verification pipeline overview
1. Fetch envelope (from request, cache, or storage) and validate DSSE structure.
2. Verify signature(s) against configured trust roots; evaluate issuer policy.
3. Retrieve or acquire inclusion proof from Rekor (primary + optional mirror).
4. Validate Merkle proof against checkpoint; optionally verify witness endorsement.
5. Return cached verification bundle including policy verdict and timestamps.
### UI & CLI touchpoints
- Console: Evidence browser, verification report, chain-of-custody graph, issuer/key management, attestation workbench, bulk verification views.
- CLI: `stella attest sign|verify|list|fetch|key` with offline verification and export bundle support.
- SDKs expose sign/verify primitives for build pipelines.
### Performance & observability targets
- Throughput goal: ≥1 000 envelopes/minute per worker with cached verification.
- Metrics: `attestor_submission_total`, `attestor_verify_seconds`, `attestor_rekor_latency_seconds`, `attestor_cache_hit_ratio`.
- Logs include `tenant`, `issuer`, `subjectDigest`, `rekorUuid`, `proofStatus`; traces cover submission → Rekor → cache → response path.
---
## 2) Data model (Mongo)
Database: `attestor`
**Collections & schemas**
* `entries`
```
{ _id: "<rekor-uuid>",
artifact: { sha256: "<sha256>", kind: "sbom|report|vex-export", imageDigest?, subjectUri? },
bundleSha256: "<sha256>", // canonicalized DSSE
index: <int>, // log index/sequence if provided by backend
proof: { // inclusion proof
checkpoint: { origin, size, rootHash, timestamp },
inclusion: { leafHash, path[] } // Merkle path (tiles)
},
log: { url, logId? },
createdAt, status: "included|pending|failed",
signerIdentity: { mode: "keyless|kms", issuer, san?, kid? }
}
```
* `dedupe`
```
{ key: "bundle:<sha256>", rekorUuid, createdAt, ttlAt } // idempotency key
```
* `audit`
```
{ _id, ts, caller: { cn, mTLSThumbprint, sub, aud }, // from mTLS + OpTok
action: "submit|verify|fetch",
artifactSha256, bundleSha256, rekorUuid?, index?, result, latencyMs, backend }
```
Indexes:
* `entries` on `artifact.sha256`, `bundleSha256`, `createdAt`, and `{status:1, createdAt:-1}`.
* `dedupe.key` unique (TTL 24–48h).
* `audit.ts` for time‑range queries.
---
## 3) Input contract (from Signer)
**Attestor accepts only** DSSE envelopes that satisfy all of:
1. **mTLS** peer certificate maps to `signer` service (CA‑pinned).
2. **Authority** OpTok with `aud=attestor`, `scope=attestor.write`, DPoP or mTLS bound.
3. DSSE envelope is **signed by the Signer’s key** (or includes a **Fulcio‑issued** cert chain) and **chains to configured roots** (Fulcio/KMS).
4. **Predicate type** is one of Stella Ops types (sbom/report/vex‑export) with valid schema.
5. `subject[*].digest.sha256` is present and canonicalized.
**Wire shape (JSON):**
```json
{
"bundle": { "dsse": { "payloadType": "application/vnd.in-toto+json", "payload": "<b64>", "signatures": [ ... ] },
"certificateChain": [ "-----BEGIN CERTIFICATE-----..." ],
"mode": "keyless" },
"meta": {
"artifact": { "sha256": "<subject sha256>", "kind": "sbom|report|vex-export", "imageDigest": "sha256:..." },
"bundleSha256": "<sha256 of canonical dsse>",
"logPreference": "primary", // "primary" | "mirror" | "both"
"archive": true // whether Attestor should archive bundle to S3
}
}
```
---
## 4) APIs
### 4.1 Submission
`POST /api/v1/rekor/entries` *(mTLS + OpTok required)*
* **Body**: as above.
* **Behavior**:
* Verify caller (mTLS + OpTok).
* Validate DSSE bundle (signature, cert chain to Fulcio/KMS; DSSE structure; payloadType allowed).
* Idempotency: compute `bundleSha256`; check `dedupe`. If present, return existing `rekorUuid`.
* Submit canonicalized bundle to Rekor v2 (primary or mirror according to `logPreference`).
* Retrieve **inclusion proof** (blocking until inclusion or up to `proofTimeoutMs`); if backend returns promise only, return `status=pending` and retry asynchronously.
* Persist `entries` record; archive DSSE to S3 if `archive=true`.
* **Response 200**:
```json
{
"uuid": "…",
"index": 123456,
"proof": {
"checkpoint": { "origin": "rekor@site", "size": 987654, "rootHash": "…", "timestamp": "…" },
"inclusion": { "leafHash": "…", "path": ["…","…"] }
},
"logURL": "https://rekor…/api/v2/log/…/entries/…",
"status": "included"
}
```
* **Errors**: `401 invalid_token`, `403 not_signer|chain_untrusted`, `409 duplicate_bundle` (with existing `uuid`), `502 rekor_unavailable`, `504 proof_timeout`.
### 4.2 Proof retrieval
`GET /api/v1/rekor/entries/{uuid}`
* Returns `entries` row (refreshes proof from Rekor if stale/missing).
* Accepts `?refresh=true` to force backend query.
### 4.3 Verification (third‑party or internal)
`POST /api/v1/rekor/verify`
* **Body** (one of):
* `{ "uuid": "…" }`
* `{ "bundle": { …DSSE… } }`
* `{ "artifactSha256": "…" }` *(looks up most recent entry)*
* **Checks**:
1. **Bundle signature** → cert chain to Fulcio/KMS roots configured.
2. **Inclusion proof** → recompute leaf hash; verify Merkle path against checkpoint root.
3. Optionally verify **checkpoint** against local trust anchors (if Rekor signs checkpoints).
4. Confirm **subject.digest** matches caller‑provided hash (when given).
* **Response**:
```json
{ "ok": true, "uuid": "…", "index": 123, "logURL": "…", "checkedAt": "…" }
```
### 4.4 Batch submission (optional)
`POST /api/v1/rekor/batch` accepts an array of submission objects; processes with per‑item results.
---
## 5) Rekor v2 driver (backend)
* **Canonicalization**: DSSE envelopes are **normalized** (stable JSON ordering, no insignificant whitespace) before hashing and submission.
* **Transport**: HTTP/2 with retries (exponential backoff, jitter), budgeted timeouts.
* **Idempotency**: if backend returns “already exists,” map to existing `uuid`.
* **Proof acquisition**:
* In synchronous mode, poll the log for inclusion up to `proofTimeoutMs`.
* In asynchronous mode, return `pending` and schedule a **proof fetcher** job (Mongo job doc + backoff).
* **Mirrors/dual logs**:
* When `logPreference="both"`, submit to primary and mirror; store **both** UUIDs (primary canonical).
* Optional **cloud endorsement**: POST to the Stella Ops cloud `/attest/endorse` with `{uuid, artifactSha256}`; store returned endorsement id.
---
## 6) Security model
* **mTLS required** for submission from **Signer** (CA‑pinned).
* **Authority token** with `aud=attestor` and DPoP/mTLS binding must be presented; Attestor verifies both.
* **Bundle acceptance policy**:
* DSSE signature must chain to the configured **Fulcio** (keyless) or **KMS/HSM** roots.
* SAN (Subject Alternative Name) must match **Signer identity** policy (e.g., `urn:stellaops:signer` or pinned OIDC issuer).
* Predicate `predicateType` must be on allowlist (sbom/report/vex-export).
* `subject.digest.sha256` values must be present and well‑formed (hex).
* **No public submission** path. **Never** accept bundles from untrusted clients.
* **Client certificate allowlists**: optional `security.mtls.allowedSubjects` / `allowedThumbprints` tighten peer identity checks beyond CA pinning.
* **Rate limits**: token-bucket per caller derived from `quotas.perCaller` (QPS/burst) returns `429` + `Retry-After` when exceeded.
* **Redaction**: Attestor never logs secret material; DSSE payloads **should** be public by design (SBOMs/reports). If customers require redaction, enforce policy at Signer (predicate minimization) **before** Attestor.
---
## 7) Storage & archival
* **Entries** in Mongo provide a local ledger keyed by `rekorUuid` and **artifact sha256** for quick reverse lookups.
* **S3 archival** (if enabled):
```
s3://stellaops/attest/
dsse/<bundleSha256>.json
proof/<rekorUuid>.json
bundle/<artifactSha256>.zip # optional verification bundle
```
* **Verification bundles** (zip):
* DSSE (`*.dsse.json`), proof (`*.proof.json`), `chain.pem` (certs), `README.txt` with verification steps & hashes.
---
## 8) Observability & audit
**Metrics** (Prometheus):
* `attestor.submit_total{result,backend}`
* `attestor.submit_latency_seconds{backend}`
* `attestor.proof_fetch_total{result}`
* `attestor.verify_total{result}`
* `attestor.dedupe_hits_total`
* `attestor.errors_total{type}`
**Correlation**:
* HTTP callers may supply `X-Correlation-Id`; Attestor will echo the header and push `CorrelationId` into the log scope for cross-service tracing.
**Tracing**:
* Spans: `validate`, `rekor.submit`, `rekor.poll`, `persist`, `archive`, `verify`.
**Audit**:
* Immutable `audit` rows (ts, caller, action, hashes, uuid, index, backend, result, latency).
---
## 9) Configuration (YAML)
```yaml
attestor:
listen: "https://0.0.0.0:8444"
security:
mtls:
caBundle: /etc/ssl/signer-ca.pem
requireClientCert: true
authority:
issuer: "https://authority.internal"
jwksUrl: "https://authority.internal/jwks"
requireSenderConstraint: "dpop" # or "mtls"
signerIdentity:
mode: ["keyless","kms"]
fulcioRoots: ["/etc/fulcio/root.pem"]
allowedSANs: ["urn:stellaops:signer"]
kmsKeys: ["kms://cluster-kms/stellaops-signer"]
rekor:
primary:
url: "https://rekor-v2.internal"
proofTimeoutMs: 15000
pollIntervalMs: 250
maxAttempts: 60
mirror:
enabled: false
url: "https://rekor-v2.mirror"
mongo:
uri: "mongodb://mongo/attestor"
s3:
enabled: true
endpoint: "http://minio:9000"
bucket: "stellaops"
prefix: "attest/"
objectLock: "governance"
redis:
url: "redis://redis:6379/2"
quotas:
perCaller:
qps: 50
burst: 100
```
---
## 10) End‑to‑end sequences
**A) Submit & include (happy path)**
```mermaid
sequenceDiagram
autonumber
participant SW as Scanner.WebService
participant SG as Signer
participant AT as Attestor
participant RK as Rekor v2
SW->>SG: POST /sign/dsse (OpTok+PoE)
SG-->>SW: DSSE bundle (+certs)
SW->>AT: POST /rekor/entries (mTLS + OpTok)
AT->>AT: Validate DSSE (chain to Fulcio/KMS; signer identity)
AT->>RK: submit(bundle)
RK-->>AT: {uuid, index?}
AT->>RK: poll inclusion until proof or timeout
RK-->>AT: inclusion proof (checkpoint + path)
AT-->>SW: {uuid, index, proof, logURL}
```
**B) Verify by artifact digest (CLI)**
```mermaid
sequenceDiagram
autonumber
participant CLI as stellaops verify
participant SW as Scanner.WebService
participant AT as Attestor
CLI->>SW: GET /catalog/artifacts/{id}
SW-->>CLI: {artifactSha256, rekor: {uuid}}
CLI->>AT: POST /rekor/verify { uuid }
AT-->>CLI: { ok: true, index, logURL }
```
---
## 11) Failure modes & responses
| Condition | Return | Details | | |
| ------------------------------------- | ----------------------- | --------------------------------------------------------- | -------- | ------------ |
| mTLS/OpTok invalid | `401 invalid_token` | Include `WWW-Authenticate` DPoP challenge when applicable | | |
| Bundle not signed by trusted identity | `403 chain_untrusted` | DSSE accepted only from Signer identities | | |
| Duplicate bundle | `409 duplicate_bundle` | Return existing `uuid` (idempotent) | | |
| Rekor unreachable/timeout | `502 rekor_unavailable` | Retry with backoff; surface `Retry-After` | | |
| Inclusion proof timeout | `202 accepted` | `status=pending`, background job continues to fetch proof | | |
| Archive failure | `207 multi-status` | Entry recorded; archive will retry asynchronously | | |
| Verification mismatch | `400 verify_failed` | Include reason: chain | leafHash | rootMismatch |
---
## 12) Performance & scale
* Stateless; scale horizontally.
* **Targets**:
* Submit+proof P95 ≤ **300 ms** (warm log; local Rekor).
* Verify P95 ≤ **30 ms** from cache; ≤ **120 ms** with live proof fetch.
* 1k submissions/minute per replica sustained.
* **Hot caches**: `dedupe` (bundle hash → uuid), recent `entries` by artifact sha256.
---
## 13) Testing matrix
* **Happy path**: valid DSSE, inclusion within timeout.
* **Idempotency**: resubmit same `bundleSha256` → same `uuid`.
* **Security**: reject non‑Signer mTLS, wrong `aud`, DPoP replay, untrusted cert chain, forbidden predicateType.
* **Rekor variants**: promise‑then‑proof, proof delayed, mirror dual‑submit, mirror failure.
* **Verification**: corrupt leaf path, wrong root, tampered bundle.
* **Throughput**: soak test with 10k submissions; latency SLOs, zero drops.
---
## 14) Implementation notes
* Language: **.NET 10** minimal API; `HttpClient` with **sockets handler** tuned for HTTP/2.
* JSON: **canonical writer** for DSSE payload hashing.
* Crypto: use **BouncyCastle**/**System.Security.Cryptography**; PEM parsing for cert chains.
* Rekor client: pluggable driver; treat backend errors as retryable/non‑retryable with granular mapping.
* Safety: size caps on bundles; decompress bombs guarded; strict UTF‑8.
* CLI integration: `stellaops verify attestation <uuid|bundle|artifact>` calls `/rekor/verify`.
---
## 15) Optional features
* **Dual‑log** write (primary + mirror) and **cross‑log proof** packaging.
* **Cloud endorsement**: send `{uuid, artifactSha256}` to Stella Ops cloud; store returned endorsement id for marketing/chain‑of‑custody.
* **Checkpoint pinning**: periodically pin latest Rekor checkpoints to an external audit store for independent monitoring.