feat(zastava): add evidence locker plan and schema examples

- Introduced README.md for Zastava Evidence Locker Plan detailing artifacts to sign and post-signing steps.
- Added example JSON schemas for observer events and webhook admissions.
- Updated implementor guidelines with checklist for CI linting, determinism, secrets management, and schema control.
- Created alert rules for Vuln Explorer to monitor API latency and projection errors.
- Developed analytics ingestion plan for Vuln Explorer, focusing on telemetry and PII guardrails.
- Implemented Grafana dashboard configuration for Vuln Explorer metrics visualization.
- Added expected projection SHA256 for vulnerability events.
- Created k6 load testing script for Vuln Explorer API.
- Added sample projection and replay event data for testing.
- Implemented ReplayInputsLock for deterministic replay inputs management.
- Developed tests for ReplayInputsLock to ensure stable hash computation.
- Created SurfaceManifestDeterminismVerifier to validate manifest determinism and integrity.
- Added unit tests for SurfaceManifestDeterminismVerifier to ensure correct functionality.
- Implemented Angular tests for VulnerabilityHttpClient and VulnerabilityDetailComponent to verify API interactions and UI rendering.
This commit is contained in:
StellaOps Bot
2025-12-02 09:27:31 +02:00
parent 885ce86af4
commit 2d08f52715
74 changed files with 1690 additions and 131 deletions

View File

@@ -38,6 +38,20 @@ The endpoint reuses `EvidenceBundlePackagingService` and caches the packaged obj
## Verification guidance
Upcoming EB1EB10 remediation (Sprint 0161; advisory `docs/product-advisories/28-Nov-2025 - Evidence Bundle and Replay Contracts.md`):
- Publish `bundle.manifest.schema.json` and `checksums.schema.json` with canonical JSON rules and signatures.
- Document the Merkle hash recipe and DSSE predicate/log policy.
- Ship an offline verifier script and golden bundles/replay fixtures to prove determinism.
- Add incident-mode activation/exit records and redaction/tenant isolation guidance for portable bundles.
### Merkle recipe (example)
```bash
cd bundle
find . -type f ! -name checksums.txt -print0 | sort -z | xargs -0 sha256sum > checksums.txt
sha256sum checksums.txt | awk '{print $1}' > merkle-root.txt
```
Use the resulting root as the DSSE subject and store `checksums.txt` inside the bundle.
1. Download `bundle.tgz` and read `instructions.txt`; the first section lists bundle id, root hash, and creation/timestamp information.
2. Verify `checksums.txt` against the transferred archive to detect transit corruption.
3. Use the StellaOps CLI (`stella evidence verify bundle.tgz`) or the provenance verifier library to validate `signature.json`.

View File

@@ -4,7 +4,7 @@
The Export Center is the dedicated service layer that packages StellaOps evidence and policy overlays into reproducible bundles. It runs as a multi-surface API backed by asynchronous workers and format adapters, enforcing Aggregation-Only Contract (AOC) guardrails while providing deterministic manifests, signing, and distribution paths.
## Runtime topology
## Runtime topology
- **Export Center API (`StellaOps.ExportCenter.WebService`).** Receives profile CRUD, export run requests, status queries, and download streams through the unified Web API gateway. Enforces tenant scopes, RBAC, quotas, and concurrency guards.
- **Export Center Worker (`StellaOps.ExportCenter.Worker`).** Dequeues export jobs from the Orchestrator, resolves selectors, invokes adapters, and writes manifests and bundle artefacts. Stateless; scales horizontally.
- **Backing stores.**
@@ -16,7 +16,16 @@ The Export Center is the dedicated service layer that packages StellaOps evidenc
- **Policy Engine** for deterministic policy snapshots and evaluated findings.
- **Orchestrator** for job scheduling, quotas, and telemetry fan-out.
- **Authority** for tenant-aware access tokens and KMS key references.
- **Console & CLI** as presentation surfaces consuming the API.
- **Console & CLI** as presentation surfaces consuming the API.
## Gap remediation (EC1EC10)
- Schemas: publish signed `ExportProfile` + manifest schemas with selector validation; keep in repo alongside OpenAPI docs.
- Determinism: per-adapter ordering/compression rules with rerun-hash CI; pin Trivy DB schema versions.
- Provenance: DSSE/SLSA attestations with log metadata for every export run; include tenant IDs in predicates.
- Integrity: require checksum/signature headers and OCI annotations; mirror delta/tombstone rules documented for adapters.
- Security: cross-tenant exports denied by default; enforce approval tokens and encryption recipient validation.
- Offline parity: provide export-kit packaging + verify script for air-gap consumers; include fixtures under `src/ExportCenter/__fixtures`.
- Advisory link: see `docs/product-advisories/28-Nov-2025 - Export Center and Reporting Strategy.md` (EC1EC10) for original requirements and keep it alongside sprint tasks for implementers.
## Job lifecycle
1. **Profile selection.** Operator or automation picks a profile (`json:raw`, `json:policy`, `trivy:db`, `trivy:java-db`, `mirror:full`, `mirror:delta`) and submits scope selectors (tenant, time window, products, SBOM subjects, ecosystems). See `docs/modules/export-center/profiles.md` for profile definitions and configuration fields.

View File

@@ -0,0 +1,34 @@
# Export Center Determinism & Rerun Hash Guide
Advisory: `docs/product-advisories/28-Nov-2025 - Export Center and Reporting Strategy.md` (EC1EC10).
## Adapter settings (runnable example)
- JSON adapters: `--compression zstd --compression-level 19 --deterministic-order`
- Mirror adapter: sort descriptors by digest, emit annotations in lexicographic order, disable mtime in tar (`--mtime 0`).
- Delta adapter: include `baseManifestHash` and sorted `added`/`removed` lists; tombstones must be explicit.
## Rerun-hash check
```bash
set -euo pipefail
run_id=$(uuidgen)
stella export run --profile demo --run-id "$run_id" --out /tmp/export1
sha256sum /tmp/export1/manifest.json > /tmp/export1/manifest.sha256
# second run
run_id2=$(uuidgen)
stella export run --profile demo --run-id "$run_id2" --out /tmp/export2
sha256sum /tmp/export2/manifest.json > /tmp/export2/manifest.sha256
diff -u /tmp/export1/manifest.sha256 /tmp/export2/manifest.sha256
```
## Integrity headers (HTTP example)
- `Digest: sha-256=<base64>`
- `X-Stella-Signature: dsse-b64=<payload>`
- `X-Stella-Immutability: true`
## Offline kit packaging
- Tar flags: `tar --sort=name --mtime=@0 --owner=0 --group=0 --numeric-owner`
- Include `export-kit/manifest.json` + `manifest.dsse`; add `verify-export-kit.sh` to check hashes and signatures.
## Where to place fixtures
- `src/ExportCenter/__fixtures/` for deterministic manifests/outputs used by tests.
- Add rerun-hash CI to compare fixture hash against regenerated outputs.

View File

@@ -18,6 +18,7 @@ The service operates strictly downstream of the **Aggregation-Only Contract (AOC
- Compile and evaluate `stella-dsl@1` policy packs into deterministic verdicts.
- Join SBOM inventory, Concelier advisories, and Excititor VEX evidence via canonical linksets and equivalence tables.
- Materialise effective findings (`effective_finding_{policyId}`) with append-only history and produce explain traces.
- Emit CVSS v4.0 receipts with canonical hashing and policy replay/backfill rules; store tenant-scoped receipts with RBAC; export receipts deterministically (UTC/fonts/order) and flag v3.1→v4.0 conversions (see Sprint 0190 CVSS-GAPS-190-014 / `docs/modules/policy/cvss-v4.md`).
- Emit per-finding OpenVEX decisions anchored to reachability evidence, forward them to Signer/Attestor for DSSE/Rekor, and publish the resulting artifacts for bench/verification consumers.
- Consume reachability lattice decisions (`ReachDecision`, `docs/reachability/lattice.md`) to drive confidence-based VEX gates (not_affected / under_investigation / affected) and record the policy hash used for each decision.
- Honor **hybrid reachability attestations**: graph-level DSSE is required input; when edge-bundle DSSEs exist, prefer their per-edge provenance for quarantine, dispute, and high-risk decisions. Quarantined edges (revoked in bundles or listed in Unknowns registry) must be excluded before VEX emission.

View File

@@ -0,0 +1,49 @@
# CVSS v4.0 Receipts Hardening Guide
Source advisory: `docs/product-advisories/25-Nov-2025 - Add CVSSv4.0 Score Receipts for Transparency.md` (CV1CV10). This guide turns the gaps into implementable rules for Sprint 0190.
## Canonical hashing (CV2)
- Serializer: JSON Canonicalization Scheme (JCS).
- Ordering: lexicographic keys; arrays keep order; drop nulls.
- Numbers: fixed 4-decimal precision; invariant culture; no exponent.
- Time: UTC ISO-8601 `Z`; strip milliseconds unless non-zero.
- Hash: SHA-256 of canonical JSON; store as `inputsHash` and DSSE subject.
- Test vectors: `tests/Policy/StellaOps.Policy.Scoring.Tests/Fixtures/hashing/`.
## Policy replay & backfill (CV1)
- Policies immutable; bump version for any change.
- On change, emit new receipts with `supersedesReceiptId` and retain old ones.
- Backfill job: re-score under new policy, append history, re-sign DSSE.
## Tenant segregation & RBAC (CV4, CV9)
- Storage keys include `tenantId`; hashes/DSSE annotate tenant.
- Roles: Security Engineer (Base), SOC Analyst (Threat), Customer Admin (Env), Viewer (read-only).
- Enforce at API/repo layer and in canonical hash.
## Deterministic exports (CV8)
- JSON export: JCS ordering, UTF-8, UTC timestamps, stable severity palette.
- PDF export: embed fonts (Source Sans 3 + Roboto Mono), A4, fixed margins; hash PDF bytes and persist `exportHash`.
## v3.1 → v4.0 conversion (CV5)
- Deterministic mapping; tag `source: "converted-v3.1"`, set `conversionMethod` + `confidence`; retain vendor vector.
## Evidence provenance (CV6)
- Evidence items use CAS URIs + DSSE refs, include `retentionClass`, `redactionStatus`, `verifiedAt`, `hashMismatch`.
## Immutability & monitoring (CV7, CV10)
- Receipts append-only; amendments create new IDs + DSSE.
- Alerts: DSSE verify failures, policy hash drift, hash mismatch, engine version skew. Prometheus counters: `cvss_receipt_dsse_failures_total`, `cvss_policy_drift_total`, `cvss_hash_mismatch_total`.
## Golden fixtures & locations
- Hashing vectors: `src/Policy/__Tests/StellaOps.Policy.Scoring.Tests/Fixtures/hashing/example-receipt-input.json` with expected hash `example-receipt-input.sha256`.
- Receipts/exports under `tests/Policy/StellaOps.Policy.Scoring.Tests/Fixtures/` (expand as features land).
- Sample PDFs in `Fixtures/exports/` once generated.
## Implementation checklist
- Wire `ReceiptCanonicalizer` to JCS rules above.
- Add backfill job + history persistence.
- Enforce tenant/RBAC and annotate hashes/DSSE.
- Implement deterministic PDF export and record `exportHash`.
- Store conversion metadata for v3.1 sources.
- Verify evidence CAS/DSSE on ingest; fail closed.
- Expose metrics/alerts listed above.

View File

@@ -45,6 +45,14 @@ Manifests describe the artefact metadata and storage pointers. They are stored i
"format": "json",
"sizeBytes": 524288,
"view": "runtime",
"attestations": [
{
"kind": "dsse",
"mediaType": "application/vnd.dsse+json",
"digest": "sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
"uri": "cas://surface-cache/attestations/entrytrace.graph.dsse/e3b0c442....json"
}
],
"storage": {
"bucket": "surface-cache",
"objectKey": "payloads/acme/entrytrace/sha256/ab/cd/abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789.ndjson.zst",
@@ -64,10 +72,16 @@ Manifest URIs follow the deterministic pattern:
```
cas://{bucket}/{prefix}/{tenant}/{digest[0..1]}/{digest[2..3]}/{digest}.json
```
```
The hex portion of the manifest digest is split into two directory levels to avoid hot directories. The same layout is mirrored on disk by the default `FileSurfaceManifestStore`, which keeps offline bundle sync trivial (copy the `manifests/` tree verbatim).
Deterministic composition adds:
- Artifact kind `composition.recipe` (media type `application/vnd.stellaops.composition.recipe+json`) describing the merge recipe and Merkle root.
- `attestations[]` per artefact (currently DSSE envelopes) so offline kits can verify payloads without re-signing.
- `determinismRoot` and `determinism` metadata on the manifest that capture the Merkle root plus the composition recipe digest/URI.
### 2.3 Payload Storage
Large payloads (SBOM fragments, entry traces, runtime events) live in the same object store as manifests (RustFS/S3). Manifests record relative paths so offline bundles can copy both manifest and payload without modification.
@@ -151,6 +165,7 @@ Scanner.Worker serialises EntryTrace graphs into Surface.FS using `SurfaceCacheK
## 9. Testing Strategy
- Unit tests for path builder, manifest serializer, and local cache eviction.
- Determinism verifier tests assert that `composition.recipe` + DSSE payloads match the Merkle root and surface artefact digests.
- Integration tests using embedded RustFS or MinIO container to validate API interactions.
- Offline kit tests verifying export/import cycle round-trips manifests and payloads.

View File

@@ -2,7 +2,7 @@
## Dashboards (offline-friendly)
- Grafana JSON: `docs/modules/vuln-explorer/runbooks/dashboards/vuln-explorer-observability.json` (import locally; no external data sources assumed).
- Panels: projection lag, open findings by severity/tenant, accepted-risk ageing, API 5xx rate, export duration p95, ledger replay backlog.
- Ops dashboards: `ops/devops/vuln/dashboards/vuln-explorer.json` (CI/staging) adds API latency p95, projection lag, error rate, query budget enforcement.
## Key metrics
- `vuln_projection_lag_seconds{tenant}` seconds between latest ledger event and projector head.
@@ -11,9 +11,12 @@
- `vuln_projection_backlog_total` queued events awaiting projection.
- `vuln_triage_actions_total{type}` immutable triage actions (assign, comment, risk_accept, remediation_note).
- `vuln_api_request_duration_seconds_bucket{route}` API latency for `GET /v1/findings*` and `POST /v1/reports`.
- `vuln_query_hashes_total{tenant,query_hash}` hashed query shapes (no PII) to observe cache effectiveness.
- `vuln_api_payload_bytes_bucket{direction}` request/response size histograms to spot oversized payloads.
## Logs & traces
- Correlate by `correlationId` and `findingId`. Structured fields: `tenant`, `advisoryKey`, `policyVersion`, `projectId`, `route`.
- Query PII guardrail: request filters are hashed (SHA-256 with deployment salt); raw filters are not logged. Strings longer than 128 chars are truncated; known PII fields (`email`, `userId`) are dropped before logging.
- Trace exemplar anchors: `traceparent` headers are copied into logs; exporters stay disabled by default for air-gap. Enable by setting `Telemetry:ExportEnabled=true` and pointing to on-prem Tempo/Jaeger.
## Health/diagnostics

View File

@@ -3,3 +3,5 @@ f466bf2b399f065558867eaf3c961cff8803f4a1506bae5539c9ce62e9ab005d schemas/webhoo
40fabd4d7bc75c35ae063b2e931e79838c79b447528440456f5f4846951ff59d thresholds.yaml
652fce7d7b622ae762c8fb65a1e592bec14b124c3273312f93a63d2c29a2b989 kit/verify.sh
f3f84fbe780115608268a91a5203d2d3ada50b4317e7641d88430a692e61e1f4 kit/README.md
2411a16a68c98c8fdd402e19b9c29400b469c0054d0b6067541ee343988b85e0 schemas/examples/observer_event.example.json
4ab47977b0717c8bdb39c52f52880742785cbcf0b5ba73d9ecc835155d445dc1 schemas/examples/webhook_admission.example.json

View File

@@ -7,6 +7,7 @@
| ZASTAVA-OPS-0001 | DONE (2025-11-30) | Ops Guild | Observability runbook stub + Grafana JSON placeholder added under `operations/`. |
| ZASTAVA-SCHEMAS-0001 | TODO | Zastava Guild | Publish signed observer/admission schemas + test vectors under `docs/modules/zastava/schemas/`; DSSE + SHA256 required. |
| ZASTAVA-KIT-0001 | TODO | Zastava Guild | Build signed `zastava-kit` bundle with thresholds.yaml, schemas, observations/admissions export, SHA256SUMS, and verify.sh; ensure offline parity. |
| ZASTAVA-THRESHOLDS-0001 | TODO | Zastava Guild | DSSE-sign `thresholds.yaml` and align with kit; publish Evidence Locker URI and update sprint 0144 checkpoints. |
| ZASTAVA-GAPS-144-007 | DONE (2025-12-02) | Zastava Guild | Remediation plan for ZR1ZR10 published at `docs/modules/zastava/gaps/2025-12-02-zr-gaps.md`; follow-on schemas/kit/thresholds to be produced and signed. |
> Keep this table in lockstep with the sprint Delivery Tracker (TODO/DOING/DONE/BLOCKED updates go to both places).

View File

@@ -0,0 +1,29 @@
# Zastava Evidence Locker Plan (schemas/kit)
Artifacts to sign (target 2025-12-06):
- `schemas/observer_event.schema.json` — predicate `stella.ops/zastavaSchema@v1`
- `schemas/webhook_admission.schema.json` — predicate `stella.ops/zastavaSchema@v1`
- `thresholds.yaml` — predicate `stella.ops/zastavaThresholds@v1`
- `zastava-kit.tzst` + `SHA256SUMS` — predicate `stella.ops/zastavaKit@v1`
Planned Evidence Locker paths (fill after signing):
- `evidence-locker/zastava/2025-12-06/observer_event.schema.dsse`
- `evidence-locker/zastava/2025-12-06/webhook_admission.schema.dsse`
- `evidence-locker/zastava/2025-12-06/thresholds.dsse`
- `evidence-locker/zastava/2025-12-06/zastava-kit.tzst`
- `evidence-locker/zastava/2025-12-06/SHA256SUMS`
Signing template (replace KEY and file):
```bash
cosign sign-blob \
--key cosign.key \
--predicate-type stella.ops/zastavaSchema@v1 \
--output-signature schemas/observer_event.schema.dsse \
schemas/observer_event.schema.json
```
Post-sign steps:
1) Verify DSSEs with `cosign verify-blob` using `cosign.pub`.
2) Upload DSSEs + SHA256SUMS to Evidence Locker paths above.
3) Update `docs/implplan/SPRINT_0144_0001_0001_zastava_runtime_signals.md` Decisions & Risks and Next Checkpoints with final URIs.
4) Mark tasks ZASTAVA-SCHEMAS-0001 / ZASTAVA-THRESHOLDS-0001 / ZASTAVA-KIT-0001 to DONE in both sprint and TASKS tables.

View File

@@ -10,3 +10,8 @@ Contents to include when built:
Deterministic packaging: `tar --mtime @0 --owner 0 --group 0 --numeric-owner -cf - kit | zstd -19 --long=27 --no-progress > zastava-kit.tzst`.
Pending: fill with signed artefacts and Evidence Locker URIs after DSSE signing.
Planned Evidence Locker paths (post-signing):
- `evidence-locker/zastava/2025-12-06/observer_event.schema.dsse`
- `evidence-locker/zastava/2025-12-06/webhook_admission.schema.dsse`
- `evidence-locker/zastava/2025-12-06/thresholds.dsse`
- `evidence-locker/zastava/2025-12-06/zastava-kit.tzst` + `SHA256SUMS`

View File

@@ -8,7 +8,17 @@ if ! command -v sha256sum >/dev/null; then
fi
sha256sum --check SHA256SUMS
# TODO: add DSSE verification once signatures are available; placeholder below
# cosign verify-blob --key cosign.pub --signature observer_event.schema.json.sig observer_event.schema.json
if command -v cosign >/dev/null && [ -f cosign.pub ]; then
echo "cosign present; DSSE verification placeholders (update paths when signed):"
echo "- observer_event.schema.dsse"
echo "- webhook_admission.schema.dsse"
echo "- thresholds.dsse"
# Example commands (uncomment once DSSE files exist):
# cosign verify-blob --key cosign.pub --signature observer_event.schema.dsse schemas/observer_event.schema.json
# cosign verify-blob --key cosign.pub --signature webhook_admission.schema.dsse schemas/webhook_admission.schema.json
# cosign verify-blob --key cosign.pub --signature thresholds.dsse thresholds.yaml
else
echo "cosign not found or cosign.pub missing; skipped DSSE verification"
fi
echo "OK: hashes verified (DSSE verification pending)"

View File

@@ -0,0 +1,19 @@
{
"tenant_id": "tenant-a",
"project_id": "proj-123",
"sensor_id": "observer-01",
"firmware_version": "1.2.3",
"policy_hash": "sha256:deadbeef",
"graph_revision_id": "graph-r1",
"ledger_id": "ledger-789",
"replay_manifest": "manifest-r1",
"event_type": "runtime_fact",
"observed_at": "2025-12-02T00:00:00Z",
"monotonic_nanos": 123456789,
"payload": {
"process": "nginx",
"pid": 4242
},
"payload_hash": "sha256:payloadhash",
"signature": "dsse://observer-event"
}

View File

@@ -0,0 +1,21 @@
{
"tenant_id": "tenant-a",
"project_id": "proj-123",
"request_uid": "abcd-1234",
"resource_kind": "Deployment",
"namespace": "prod",
"workload_name": "api",
"policy_hash": "sha256:deadbeef",
"graph_revision_id": "graph-r1",
"ledger_id": "ledger-789",
"replay_manifest": "manifest-r1",
"manifest_pointer": "surfacefs://cache/sha256:abc",
"decision": "allow",
"decision_reason": "surface cache fresh",
"decision_at": "2025-12-02T00:00:00Z",
"monotonic_nanos": 2233445566,
"side_effect": "none",
"bypass_waiver_id": null,
"payload_hash": "sha256:payloadhash",
"signature": "dsse://webhook-admission"
}