stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 1 Releases Wiki Activity

Add unit tests for SBOM ingestion and transformation
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Details

Browse Source 
- Implement `SbomIngestServiceCollectionExtensionsTests` to verify the SBOM ingestion pipeline exports snapshots correctly.
- Create `SbomIngestTransformerTests` to ensure the transformation produces expected nodes and edges, including deduplication of license nodes and normalization of timestamps.
- Add `SbomSnapshotExporterTests` to test the export functionality for manifest, adjacency, nodes, and edges.
- Introduce `VexOverlayTransformerTests` to validate the transformation of VEX nodes and edges.
- Set up project file for the test project with necessary dependencies and configurations.
- Include JSON fixture files for testing purposes.
This commit is contained in:
master
2025-11-04 07:49:39 +02:00
parent f72c5c513a
commit 2eb6852d34
491 changed files with 39445 additions and 3917 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
docs/modules/advisory-ai/architecture.md
View File

@@ -56,6 +56,24 @@
- Dependency paths (runtime vs build/test, deduped by coordinate chain).
- Tenant environment flags (prod/stage toggles) with optional blast radius summary.
- Service-side clamps: max 500 timeline entries, 200 dependency paths, with client-provided toggles for env/blast data.
- `AddSbomContextHttpClient(...)` registers the typed HTTP client that calls `/v1/sbom/context`, while `NullSbomContextClient` remains the safe default for environments that have not yet exposed the SBOM service.
**Sample configuration** (wire real SBOM base URL + API key):
```csharp
services.AddSbomContextHttpClient(options =>
{
options.BaseAddress = new Uri("https://sbom-service.internal");
options.Endpoint = "/v1/sbom/context";
options.ApiKey = configuration["SBOM_SERVICE_API_KEY"];
options.UserAgent = "stellaops-advisoryai/1.0";
options.Tenant = configuration["TENANT_ID"];
});
services.AddAdvisoryPipeline();
```
After configuration, issue a smoke request (e.g., `ISbomContextRetriever.RetrieveAsync`) during deployment validation to confirm end-to-end connectivity and credentials before enabling Advisory AI endpoints.
Retriever requests and results are trimmed/normalized before hashing; metadata (counts, provenance keys) is returned for downstream guardrails. Unit coverage ensures deterministic ordering and flag handling.
@@ -69,6 +87,7 @@ All context references include `content_hash` and `source_id` enabling verifiabl
- Citations follow `[n]` indexing referencing actual sources.
- Remediation suggestions only cite policy-approved sources (fixed versions, vendor hotfixes).
- Moderation/PII filters prevent leaking secrets; responses failing validation are rejected and logged.
- Pre-flight guardrails redact secrets (AWS keys, generic API tokens, PEM blobs), block "ignore previous instructions"-style prompt injection attempts, enforce citation presence, and cap prompt payload length (default 16 kB). Guardrail outcomes and redaction counts surface via `advisory_guardrail_blocks` / `advisory_outputs_stored` metrics.
## 5) Deterministic tooling
@@ -95,10 +114,8 @@ All context references include `content_hash` and `source_id` enabling verifiabl
## 8) APIs
- `POST /v1/advisory-ai/summaries` — generate (or retrieve cached) summary for `{advisoryKey, artifactId, policyVersion}`.
- `POST /v1/advisory-ai/conflicts` — explain conflicting VEX statements with trust ranking.
- `POST /v1/advisory-ai/remediation` — fetch remediation plan with target fix versions, prerequisites, verification steps.
- `GET /v1/advisory-ai/outputs/{hash}` — retrieve cached artefact (used by CLI/Console/Export Center).
- `POST /api/v1/advisory/{task}` — executes Summary/Conflict/Remediation pipeline (`task` ∈ `summary|conflict|remediation`). Requests accept `{advisoryKey, artifactId?, policyVersion?, profile, preferredSections?, forceRefresh}` and return sanitized prompt payloads, citations, guardrail metadata, provenance hash, and cache hints.
- `GET /api/v1/advisory/outputs/{cacheKey}?taskType=SUMMARY&profile=default` — retrieves cached artefacts for downstream consumers (Console, CLI, Export Center). Guardrail state and provenance hash accompany results.
All endpoints accept `profile` parameter (default `fips-local`) and return `output_hash`, `input_digest`, and `citations` for verification.

8
docs/modules/advisory-ai/orchestration-pipeline.md
View File

@@ -50,6 +50,7 @@ Wire the deterministic pipeline (Summary / Conflict / Remediation flows) into th
- **Scope:** Implement prompt assembler, connect to guardrails, persist cache entries w/ DSSE metadata.
- **Dependencies:** Prompt templates, cache storage decision, guardrail interface.
- **Exit:** Deterministic outputs stored; force-refresh honoured; tests cover prompt assembly + caching.
> 2025-11-03: Prompt assembler now emits deterministic JSON payloads, guardrail pipeline wiring is stubbed for upcoming security hardening, and outputs persist with DSSE-ready provenance metadata plus golden test coverage.
### AIAI-31-004C (CLI integration & docs)
@@ -57,6 +58,13 @@ Wire the deterministic pipeline (Summary / Conflict / Remediation flows) into th
- **Dependencies:** Service endpoints stable, caching semantics documented.
- **Exit:** CLI command produces deterministic output, docs updated, smoke tests recorded.
### AIAI-31-006 (Service API surface)
- **Scope:** Expose REST endpoints for summary/conflict/remediation execution plus cached output retrieval (`POST /api/v1/advisory/{task}`, `GET /api/v1/advisory/outputs/{cacheKey}`). Include guardrail execution, provenance hashing, metrics, and stubs for RBAC/rate limits.
- **Dependencies:** Guardrail enforcement (AIAI-31-005), Authority scope wiring (`advisory-ai:view` / `advisory-ai:operate`), Offline kit docs.
- **Exit:** Endpoints return sanitized prompts with citations, guardrail metadata, DSSE hash, and plan cache indicators; OpenAPI description updated; rate-limit hooks ready for Authority integration.
> 2025-11-03: Initial REST surface shipped – direct execution runs through guardrail pipeline, outputs persist with DSSE-ready provenance, metrics `advisory_outputs_stored`/`advisory_guardrail_blocks` emit, and cache retrieval endpoint exposes stored artefacts (RBAC/header enforcement pending scope delivery).
### Supporting tasks (other guilds)
- **AUTH-AIAI-31-004** – Update scopes and DSSE policy (Authority guild).

55
docs/modules/attestor/evidence/2025-11-03-mongo-ttl-validation.txt Normal file
View File

@@ -0,0 +1,55 @@
Using database: attestor_ttl_validation
true
{ ok: 1 }
Created collection and resetting indexes...
dedupe_key_unique
dedupe_ttl
{
acknowledged: true,
insertedId: ObjectId("6909225d9ddb8e8caf11da28")
}
Inserted document scheduled to expire at 2025-11-03T21:45:21.054Z
Current indexes:
[
{
v: 2,
key: {
_id: 1
},
name: '_id_'
},
{
v: 2,
key: {
key: 1
},
name: 'dedupe_key_unique',
unique: true
},
{
v: 2,
key: {
ttlAt: 1
},
name: 'dedupe_ttl',
expireAfterSeconds: 0
}
]
Still present at 2025-11-03T21:45:06.275Z (waiting for TTL monitor)...
Still present at 2025-11-03T21:45:09.473Z (waiting for TTL monitor)...
Still present at 2025-11-03T21:45:14.492Z (waiting for TTL monitor)...
Still present at 2025-11-03T21:45:19.510Z (waiting for TTL monitor)...
Still present at 2025-11-03T21:45:24.532Z (waiting for TTL monitor)...
Still present at 2025-11-03T21:45:29.565Z (waiting for TTL monitor)...
Still present at 2025-11-03T21:45:34.590Z (waiting for TTL monitor)...
Still present at 2025-11-03T21:45:37.779Z (waiting for TTL monitor)...
Still present at 2025-11-03T21:45:42.799Z (waiting for TTL monitor)...
Still present at 2025-11-03T21:45:47.818Z (waiting for TTL monitor)...
Still present at 2025-11-03T21:45:52.838Z (waiting for TTL monitor)...
Still present at 2025-11-03T21:45:57.855Z (waiting for TTL monitor)...
Still present at 2025-11-03T21:46:02.875Z (waiting for TTL monitor)...
Still present at 2025-11-03T21:46:07.885Z (waiting for TTL monitor)...
Still present at 2025-11-03T21:46:11.070Z (waiting for TTL monitor)...
Still present at 2025-11-03T21:46:16.075Z (waiting for TTL monitor)...
Document expired and removed after 80.0 seconds.
Remaining documents: 0

13
docs/modules/attestor/evidence/2025-11-03-redis-ttl-validation.txt Normal file
View File

@@ -0,0 +1,13 @@
redis> OK
2025-11-03T21:49:38+00:00 ttl=45
2025-11-03T21:49:41+00:00 ttl=42
2025-11-03T21:49:46+00:00 ttl=37
2025-11-03T21:49:51+00:00 ttl=32
2025-11-03T21:49:56+00:00 ttl=27
2025-11-03T21:50:01+00:00 ttl=22
2025-11-03T21:50:06+00:00 ttl=17
2025-11-03T21:50:11+00:00 ttl=12
2025-11-03T21:50:14+00:00 ttl=8
2025-11-03T21:50:20+00:00 ttl=3
2025-11-03T21:50:25+00:00 ttl=-2
Key expired after 47s

6
docs/modules/attestor/ttl-validation.md
View File

@@ -39,3 +39,9 @@ If the helper script cannot be used:
## Ownership
- Primary: Attestor Service Guild.
- Partner: QA Guild (observes TTL metrics, confirms evidence archiving).
## 2025-11-03 validation summary
- **Stack:** `mongod` 7.0.5 (tarball) + `mongosh` 2.0.2, `redis-server` 7.2.4 (source build) running on localhost without Docker.
- **Mongo results:** `dedupe` TTL index (`ttlAt`, `expireAfterSeconds: 0`) confirmed; document inserted with 20 s TTL expired automatically after ~80 s (expected allocator sweep). Evidence: `docs/modules/attestor/evidence/2025-11-03-mongo-ttl-validation.txt`.
- **Redis results:** Key `attestor:ttl:live:bundle:validation` set with 45 s TTL reached `TTL=-2` after ~47 s confirming expiry propagation. Evidence: `docs/modules/attestor/evidence/2025-11-03-redis-ttl-validation.txt`.
- **Notes:** Local binaries built/run to accommodate sandbox without Docker; services shut down after validation.

2
docs/modules/authority/architecture.md
View File

@@ -97,6 +97,8 @@ plan? = <plan name> // optional hint for UIs; not used for e
* `GET /.well-known/openid-configuration` → endpoints, algs, jwks_uri
* `GET /jwks` → JSON Web Key Set (rotating, at least 2 active keys during transition)
> **KMS-backed keys.** When the signing provider is `kms`, Authority fetches only the public coordinates (`Qx`, `Qy`) and version identifiers from the backing KMS. Private scalars never leave the provider; JWKS entries are produced by re-exporting the public material via the `kms.version` metadata attached to each key. Retired keys keep the same `kms.version` metadata so audits can trace which cloud KMS version produced a token.
### 3.2 Token issuance
* `POST /token`

12
docs/modules/cli/architecture.md
View File

@@ -128,10 +128,14 @@ src/
* Imports a previously exported bundle into the local KMS root (`kms/` by default), promotes the imported version to `Active`, and preserves existing versions by marking them `PendingRotation`. Prompts for the passphrase when not provided to keep automation password-safe.
Both subcommands honour offline-first expectations (no network access) and normalise relative roots via `--root` when operators mirror the credential store.
---
## 3) AuthN: Authority + DPoP
### 2.11 Air-gap guard
- CLI outbound HTTP flows (Authority auth, backend APIs, advisory downloads) route through `StellaOps.AirGap.Policy`. When sealed mode is active the CLI refuses commands that would require external egress and surfaces the shared `AIRGAP_EGRESS_BLOCKED` remediation guidance instead of attempting the request.
---
## 3) AuthN: Authority + DPoP
### 3.1 Token acquisition

15
docs/modules/cli/guides/cli-reference.md
View File

@@ -304,7 +304,20 @@ Additional notes:
---
*Last updated: 2025-11-02 (Sprint 100).*
## 7 · Policy lifecycle CLI quick start
- `stella policy submit --policy <id> --version <n> --attach <simulation.json> --reviewers group`
- `stella policy review --policy <id> --version <n> --request-changes|--approve`
- `stella policy approve --policy <id> --version <n> --note "<justification>"`
- `stella policy publish --policy <id> --version <n> --reason "<why>" --ticket SEC-2048 --sign`
- `stella policy promote --policy <id> --version <n> --environment stage --note "<context>"`
- `stella policy activate --policy <id> --version <n> --note "<deployment reason>"`
All publish/promote operations require interactive identities with `policy:publish`/`policy:promote` and inject attestation metadata headers (`policy_reason`, `policy_ticket`, `policy_digest`). See [Policy Lifecycle & Approvals](../../../policy/lifecycle.md) § 3–4 for the full workflow and compliance checklist.
---
*Last updated: 2025-11-03 (Sprint 100).*
## 13. Authority configuration quick reference

75
docs/modules/evidence-locker/bundle-packaging.md Normal file
View File

@@ -0,0 +1,75 @@
# Evidence Locker Bundle Packaging
> Sprint 160 / Task EVID-OBS-54-002 — deterministic tarball packaging for download/export.
The Evidence Locker emits a **single `bundle.tgz` artifact** for every sealed bundle. The artifact is deterministic so that operators can re-run packaging and obtain identical bytes when the manifest and signature are unchanged.
## Layout
The tar stream is written with **POSIX/PAX entries** and wrapped in a gzip layer:
```
bundle.tgz
├── manifest.json # Re-emitted DSSE payload (pretty JSON, canonical ordering)
├── signature.json # DSSE signature + key metadata + RFC3161 timestamp (if present)
├── bundle.json # Locker metadata (ids, status, root hash, storage key, timestamps)
├── checksums.txt # SHA-256 root hash + per-entry hashes from the manifest
└── instructions.txt # Offline verification steps and retention guidance
```
### Determinism traits
- **Gzip header timestamp** is pinned to `2025-01-01T00:00:00Z` so CI fixtures remain stable.
- All tar entries use the same fixed mtime/atime/ctime, `0644` permissions, and UTF-8 encoding.
- JSON files are serialized with `JsonSerializerDefaults.Web` + indentation to stabilise ordering.
- `checksums.txt` sorts manifest entries by `canonicalPath` and prefixes the Merkle root (`root <hash>`).
- `instructions.txt` conditionally adds timestamp verification steps when an RFC3161 token exists.
## Download endpoint
`GET /evidence/{bundleId}/download`
- Requires scopes: `evidence:read`.
- Streams `application/gzip` content with `Content-Disposition: attachment; filename="bundle.tgz"`.
- Emits quota headers (`X-Stella-Quota-*`) and audit events mirroring snapshot fetches.
- Returns `404` when the bundle is not sealed or the package has not been materialised.
The endpoint reuses `EvidenceBundlePackagingService` and caches the packaged object in the configured object store (`tenants/{tenant}/bundles/{bundle}/bundle.tgz`). If the underlying storage key changes (for example, during migration from filesystem to S3), the repository is updated atomically.
## Verification guidance
1. Download `bundle.tgz` and read `instructions.txt`; the first section lists bundle id, root hash, and creation/timestamp information.
2. Verify `checksums.txt` against the transferred archive to detect transit corruption.
3. Use the StellaOps CLI (`stella evidence verify bundle.tgz`) or the provenance verifier library to validate `signature.json`.
4. When present, validate the RFC3161 timestamp token with the configured TSA endpoint.
These steps match the offline procedure described in `docs/forensics/evidence-locker.md` (Portable Evidence section). Update that guide whenever packaging fields change.
## Portable bundle (`portable-bundle-v1.tgz`)
When sealed or air-gapped environments need a redacted evidence artifact, request:
`GET /evidence/{bundleId}/portable`
The portable archive is deterministic and contains only non-sensitive metadata:
```
portable-bundle-v1.tgz
├── manifest.json # Canonical manifest (identical to sealed bundle)
├── signature.json # DSSE signature + optional RFC3161 token
├── bundle.json # Redacted metadata (no tenant/storage identifiers)
├── checksums.txt # SHA-256 root + entry checksums
├── instructions-portable.txt # Sealed-mode transfer + verification guidance
└── verify-offline.sh # Offline verification helper script (POSIX shell)
```
Portable packaging traits:
- `bundle.json` excludes tenant identifiers, storage keys, and free-form descriptions. It adds `portableGeneratedAt` along with entry counts and totals for audit purposes.
- `incidentMetadata` is preserved only when incident mode injects `incident.*` keys into the manifest metadata.
- `verify-offline.sh` extracts the bundle, validates checksums (using `sha256sum`/`shasum`), surfaces the Merkle root hash, and reminds operators to run `stella evidence verify --bundle <archive>`.
- `instructions-portable.txt` mirrors the sealed documentation but calls out the offline script and redaction constraints.
Portable bundles reuse the same DSSE payload and timestamp, so downstream verifiers can validate signatures without additional configuration. The Evidence Locker tracks the portable storage key separately to honour write-once semantics for both sealed and portable artifacts.
For step-by-step sealed-mode guidance see `docs/airgap/portable-evidence.md`.

13
docs/modules/evidence-locker/compliance-checklist.md Normal file
View File

@@ -0,0 +1,13 @@
# Evidence Locker Compliance Checklist (Sprint 160)
- [x] Postgres schema created via deterministic SQL migrations (`evidence_locker.*` tables, schema version tracking).
- [x] Row-level security enforced per tenant via `app.current_tenant` guard function.
- [x] Evidence bundle storage keys are content-addressed (sha256) and unique per tenant/bundle.
- [x] Object-store abstraction provides local filesystem and Amazon S3 drivers with optional WORM enforcement.
- [x] Startup migrations wired via hosted service with opt-out flag (`ApplyMigrationsAtStartup`).
- [x] Integration tests cover schema bootstrap, RLS behaviour, and storage drivers (filesystem, S3 fake client).
- [x] Temporary artifacts cleaned deterministically; filesystem targets validated in tests.
- [x] Timeline publisher emits bundle sealed and hold events with DSSE metadata when enabled; offline deployments fall back to null publisher.
- [x] Bundle packaging outputs deterministic `bundle.tgz` (fixed gzip mtime, sorted checksums, timestamp-aware instructions) and `/evidence/{id}/download` streams the cached object with audit logging.
- [x] Incident mode extends bundle retention, captures incident request snapshots, and emits activation/deactivation events to Timeline Indexer and Notifier stubs with unit + web integration coverage.
- [x] Portable bundle flow (`/evidence/{id}/portable`) emits `portable-bundle-v1.tgz` with redacted metadata, deterministic offline script, and write-once storage tracking.

24
docs/modules/evidence-locker/incident-mode.md Normal file
View File

@@ -0,0 +1,24 @@
# Evidence Locker Incident Mode
> Sprint 55 / Task EVID-OBS-55-001 – retention & debug hooks
Incident mode is a service-wide switch that increases forensic fidelity when StellaOps enters a suspected compromise or SLO breach. The Evidence Locker reacts to the flag in four ways:
1. **Extended retention.** Every newly sealed bundle receives an `ExpiresAt` timestamp of `CreatedAt + Incident.RetentionExtensionDays` so downstream TTL jobs keep artefacts long enough for investigation.
2. **Debug artefacts.** Snapshot requests emit an `incident/request-*.json` payload into the object store. The payload captures the normalized request metadata/materials plus the incident stamp so offline replay tooling has everything it needs. The manifest surfaces the artefact under the `incident/` section and packaging streams it alongside the canonical bundle files.
3. **Manifest metadata.** Bundles carry `incident.mode`, `incident.changedAt`, and `incident.retentionExtensionDays` metadata so verifiers and auditors can see exactly when the mode toggled and how long retention was extended.
4. **Operational signals.** Activation/deactivation events are published to the Timeline Indexer (and, via the notifier stub, to the future Notify integration). The `IEvidenceTimelinePublisher` now emits `evidence.incident.mode` with `state` and retention attributes, giving Ops a canonical audit trail.
Configuration lives under `EvidenceLocker:Incident`:
```jsonc
"EvidenceLocker": {
"Incident": {
"Enabled": true,
"RetentionExtensionDays": 60,
"CaptureRequestSnapshot": true
}
}
```
`IncidentModeManager` watches the options and raises events whenever the state flips. Tests cover retention math, timeline/notifier fan-out, and the new debug artefact path.

15
docs/modules/export-center/architecture.md
View File

@@ -78,12 +78,15 @@ All endpoints require Authority-issued JWT + DPoP tokens with scopes `export:run
- Maps StellaOps advisory schema to Trivy DB format, handling namespace collisions and ecosystem-specific ranges.
- Validates compatibility against supported Trivy schema versions; run fails fast if mismatch.
- Emits optional manifest summarising package counts and severity distribution.
- **Mirror (`mirror:full`, `mirror:delta`).**
- Builds self-contained filesystem layout (`/manifests`, `/data/raw`, `/data/policy`, `/indexes`).
- Delta variant compares against base manifest (`base_export_id`) to write only changed artefacts; records `removed` entries for cleanup.
- Supports optional encryption of `/data` subtree (age/AES-GCM) with key wrapping stored in `provenance.json`.
Adapters expose structured telemetry events (`adapter.start`, `adapter.chunk`, `adapter.complete`) with record counts and byte totals per chunk. Failures emit `adapter.error` with reason codes.
- **Mirror (`mirror:full`, `mirror:delta`).**
- Builds self-contained filesystem layout (`/manifests`, `/data/raw`, `/data/policy`, `/indexes`).
- Delta variant compares against base manifest (`base_export_id`) to write only changed artefacts; records `removed` entries for cleanup.
- Supports optional encryption of `/data` subtree (age/AES-GCM) with key wrapping stored in `provenance.json`.
- **DevPortal (`devportal:offline`).**
- Packages developer portal static assets, OpenAPI specs, SDK releases, and changelog content into a reproducible archive with manifest/checksum pairs.
- Emits `manifest.json`, `checksums.txt`, and helper scripts described in [DevPortal Offline Bundle Specification](devportal-offline.md); signing/DSSE wiring follows the shared Export Center signing service.
Adapters expose structured telemetry events (`adapter.start`, `adapter.chunk`, `adapter.complete`) with record counts and byte totals per chunk. Failures emit `adapter.error` with reason codes.
## Signing and provenance
- **Manifest schema.** `export.json` contains run metadata, profile descriptor, selector summary, counts, SHA-256 digests, compression hints, and distribution list. Deterministic field ordering and normalized timestamps.

115
docs/modules/export-center/devportal-offline.md Normal file
View File

@@ -0,0 +1,115 @@
# DevPortal Offline Bundle Specification
> Sprint 160 · Task DVOFF-64-001
> Owners: DevPortal Offline Guild · Exporter Service Guild
The DevPortal offline bundle packages developer portal assets, OpenAPI specifications, SDK binaries, and changelog content into a deterministic archive for air-gapped distribution. This document captures the first iteration of the profile produced by the new `devportal --offline` export job.
## 1. Archive layout
The bundle ships as a gzip-compressed tar archive (`devportal-offline-bundle.tgz`) with the following structure:
```
manifest.json # Bundle manifest (schema v1)
checksums.txt # SHA-256 root + per-entry checksums
instructions-portable.txt # Human-readable verification guidance
verify-offline.sh # POSIX helper that extracts + validates checksums
portal/** # Static site assets (HTML, CSS, JS, etc.)
specs/** # OpenAPI / additional specs
sdks/<name>/** # SDK artifacts grouped by logical name (dotnet, python, ...)
changelog/** # Changelog and release notes
```
Every file entry is written with fixed permissions (`0644`, `0755` for scripts) and a pinned timestamp (`2025-01-01T00:00:00Z`) so the archive is byte-for-byte reproducible.
## 2. Manifest (`manifest.json`)
The manifest is emitted with camel-cased JSON (`JsonSerializerDefaults.Web`) and the following schema:
```jsonc
{
"version": "devportal-offline/v1",
"bundleId": "14b094c9-f0b4-4f9e-b221-b7a77c3f3445",
"generatedAt": "2025-11-04T12:30:00Z",
"metadata": {
"releaseVersion": "2025.11.0"
},
"sources": {
"portalIncluded": true,
"specsIncluded": true,
"sdkNames": ["dotnet", "python"],
"changelogIncluded": true
},
"totals": {
"entryCount": 6,
"totalSizeBytes": 123456
},
"entries": [
{
"category": "portal",
"path": "portal/index.html",
"sha256": "850db3...",
"sizeBytes": 5120,
"contentType": "text/html"
},
{
"category": "sdk",
"path": "sdks/dotnet/stellaops.sdk.nupkg",
"sha256": "0e1f23...",
"sizeBytes": 20480,
"contentType": "application/zip"
}
]
}
```
- `metadata` is a free-form dictionary (release version, build tag, etc.).
- `sdkNames` is a sorted list of logical SDK identifiers (sanitised to lowercase alphanumeric / `-_.`).
- `entries` are ordered lexicographically by `path` and include per-file SHA-256 digests, size, and inferred media type.
## 3. Checksums and root hash
`checksums.txt` follows the evidence locker format:
```
# DevPortal offline bundle checksums (sha256)
root <sha256(manifest.json)>
<sha256> portal/index.html
<sha256> specs/openapi.yaml
...
```
The `root` value is the SHA-256 hash of the serialized manifest and is exposed separately in the result object for downstream signing.
## 4. Verification script
`verify-offline.sh` is a POSIX-compatible helper that:
1. Extracts the archive into a temporary directory.
2. Validates `checksums.txt` via `sha256sum` (or `shasum -a 256` fallback).
3. Prints the manifest root hash and reminds operators to run `stella devportal verify --bundle <archive>` once DSSE signing is wired.
Operators can override the archive name via the first argument (`./verify-offline.sh mybundle.tgz`).
## 5. Content categories
| Category | Target prefix | Notes |
|-----------|---------------|-------|
| `portal` | `portal/` | Static site assets (HTML, CSS, JS, images). |
| `specs` | `specs/` | OpenAPI/JSON/YAML specifications. |
| `sdk` | `sdks/<name>/`| Each SDK source defines `<name>`; files are copied recursively. |
| `changelog` | `changelog/`| Markdown, text, or PDF release notes. |
Paths are normalised to forward slashes and guarded against directory traversal.
## 6. Determinism and hashing rules
- Files are enumerated and emitted in ordinal path order.
- SHA-256 digests use lowercase hex encoding.
- Optional directories (specs, SDKs, changelog) are skipped when absent; at least one category must contain files or the builder fails fast.
## 7. Next steps
- Attach DSSE signing + timestamping (`signature.json`) once Export Center signing infrastructure is ready.
- Integrate the builder into the Export Center worker profile (`devportal --offline`) and plumb orchestration/persistence.
- Produce CLI validation tooling (`stella devportal verify`) per DVOFF-64-002 and document operator workflows under `docs/airgap/devportal-offline.md`.

274
docs/modules/findings-ledger/schema.md Normal file
View File

@@ -0,0 +1,274 @@
# Findings Ledger Schema (Sprint 120)
> **Owners:** Findings Ledger Guild • Vuln Explorer Guild
> **Status:** Draft schema delivered 2025-11-03 for LEDGER-29-001
## 1. Storage profile
| Concern | Decision | Notes |
|---------|----------|-------|
| Engine | PostgreSQL 14+ with UTF-8, `jsonb`, and partitioning support | Aligns with shared data plane; deterministic ordering enforced via primary keys. |
| Tenancy | Range/list partition on `tenant_id` for ledger + projection tables | Simplifies retention and cross-tenant anchoring. |
| Time zone | All timestamps stored as `timestamptz` UTC | Canonical JSON uses ISO-8601 (`yyyy-MM-ddTHH:mm:ss.fffZ`). |
| Hashing | SHA-256 (lower-case hex) over canonical JSON | Implemented client-side and verified by DB constraint. |
| Migrations | SQL files under `src/Findings/StellaOps.Findings.Ledger/migrations` | Applied via DatabaseMigrator (part of platform toolchain). |
## 2. Ledger event model
Events are immutable append-only records representing every workflow change. Records capture the original event payload, cryptographic hashes, and actor metadata.
### 2.1 `ledger_events`
| Column | Type | Description |
|--------|------|-------------|
| `tenant_id` | `text` | Tenant partition key. |
| `chain_id` | `uuid` | Logical chain grouping (per tenant/policy combination). |
| `sequence_no` | `bigint` | Monotonic sequence within a chain (gapless). |
| `event_id` | `uuid` | Globally unique event identifier. |
| `event_type` | `ledger_event_type` | Enumerated type (see §2.2). |
| `policy_version` | `text` | Policy digest (e.g., SHA-256). |
| `finding_id` | `text` | Stable finding identity `(artifactId + vulnId + policyVersion)`. |
| `artifact_id` | `text` | Asset identifier (image digest, SBOM id, etc.). |
| `source_run_id` | `uuid` | Policy run that produced the event (nullable). |
| `actor_id` | `text` | Operator/service initiating the mutation. |
| `actor_type` | `text` | `system`, `operator`, `integration`. |
| `occurred_at` | `timestamptz` | Domain timestamp supplied by source. |
| `recorded_at` | `timestamptz` | Ingestion timestamp (defaults to `now()`). |
| `event_body` | `jsonb` | Canonical payload (see §2.3). |
| `event_hash` | `char(64)` | SHA-256 over canonical payload envelope. |
| `previous_hash` | `char(64)` | Hash of prior event in chain (all zeroes for first). |
| `merkle_leaf_hash` | `char(64)` | Leaf hash used for Merkle anchoring (hash over `event_hash || sequence_no`). |
**Constraints & indexes**
```
PRIMARY KEY (tenant_id, chain_id, sequence_no);
UNIQUE (tenant_id, event_id);
UNIQUE (tenant_id, chain_id, event_hash);
CHECK (event_hash ~ '^[0-9a-f]{64}$');
CHECK (previous_hash ~ '^[0-9a-f]{64}$');
CREATE INDEX ix_ledger_events_finding ON ledger_events (tenant_id, finding_id, policy_version);
CREATE INDEX ix_ledger_events_type ON ledger_events (tenant_id, event_type, recorded_at DESC);
```
Partitions: top-level partitioned by `tenant_id` (list) with a default partition. Optional sub-partition by month on `recorded_at` for large tenants. PostgreSQL requires the partition key in unique constraints; global uniqueness for `event_id` is enforced as `(tenant_id, event_id)` with application-level guards maintaining cross-tenant uniqueness.
### 2.2 Event types
```
CREATE TYPE ledger_event_type AS ENUM (
'finding.created',
'finding.status_changed',
'finding.severity_changed',
'finding.tag_updated',
'finding.comment_added',
'finding.assignment_changed',
'finding.accepted_risk',
'finding.remediation_plan_added',
'finding.attachment_added',
'finding.closed'
);
```
Additional types can be appended via migrations; canonical JSON must include `event_type` key.
### 2.3 Canonical ledger JSON
Canonical payload envelope (before hashing):
```json
{
"event": {
"id": "3ac1f4ef-3c26-4b0d-91d4-6a6d3a5bde10",
"type": "finding.status_changed",
"tenant": "tenant-a",
"chainId": "5fa2b970-9da2-4ef4-9a63-463c5d98d3cc",
"sequence": 42,
"policyVersion": "sha256:5f38...",
"finding": {
"id": "artifact:sha256:abc|pkg:cpe:/o:vendor:product",
"artifactId": "sha256:abc",
"vulnId": "CVE-2025-1234"
},
"actor": {
"id": "user:alice@tenant",
"type": "operator"
},
"occurredAt": "2025-11-03T15:12:05.123Z",
"payload": {
"previousStatus": "affected",
"status": "triaged",
"justification": "Ticket SEC-1234 created",
"ticket": {
"id": "SEC-1234",
"url": "https://tracker/sec-1234"
}
}
}
}
```
Canonicalisation rules:
1. Serialize using UTF-8, no BOM.
2. Sort object keys lexicographically at every level.
3. Represent enums/flags as lower-case strings.
4. Timestamps formatted as `yyyy-MM-ddTHH:mm:ss.fffZ` (millisecond precision, UTC).
5. Numbers use decimal notation; omit trailing zeros.
6. Arrays maintain supplied order.
Hash pipeline:
```
canonical_json = CanonicalJsonSerializer.Serialize(envelope)
sha256_bytes = SHA256(canonical_json)
event_hash = HexLower(sha256_bytes)
```
`merkle_leaf_hash = HexLower(SHA256(event_hash || '-' || sequence_no)).`
## 3. Merkle anchoring
Anchoring batches events per tenant across fixed windows (default: 1,000 events or 15 minutes). Anchors are stored in `ledger_merkle_roots`.
| Column | Type | Description |
|--------|------|-------------|
| `tenant_id` | `text` | Tenant key. |
| `anchor_id` | `uuid` | Anchor identifier. |
| `window_start` | `timestamptz` | Inclusive start of batch. |
| `window_end` | `timestamptz` | Exclusive end. |
| `sequence_start` | `bigint` | First sequence included. |
| `sequence_end` | `bigint` | Last sequence included. |
| `root_hash` | `char(64)` | Merkle root (SHA-256). |
| `leaf_count` | `integer` | Number of events aggregated. |
| `anchored_at` | `timestamptz` | Timestamp root stored/signed. |
| `anchor_reference` | `text` | Optional reference to external ledger (e.g., Rekor UUID). |
Indexes: `PRIMARY KEY (tenant_id, anchor_id)`, `UNIQUE (tenant_id, root_hash)`, `INDEX ix_merkle_sequences ON ledger_merkle_roots (tenant_id, sequence_end DESC)`.
## 4. Projection tables
### 4.1 `findings_projection`
Stores the latest verdict/state per finding.
| Column | Type | Description |
|--------|------|-------------|
| `tenant_id` | `text` | Partition key. |
| `finding_id` | `text` | Matches ledger payload. |
| `policy_version` | `text` | Active policy digest. |
| `status` | `text` | e.g., `affected`, `triaged`, `accepted_risk`, `resolved`. |
| `severity` | `numeric(6,3)` | Normalised severity score (0-10). |
| `labels` | `jsonb` | Key-value metadata (tags, KEV flag, runtime signals). |
| `current_event_id` | `uuid` | Ledger event that produced this state. |
| `explain_ref` | `text` | Reference to explain bundle or object storage key. |
| `policy_rationale` | `jsonb` | Array of policy rationale references (explain bundle IDs, remediation notes). |
| `updated_at` | `timestamptz` | Last projection update. |
| `cycle_hash` | `char(64)` | Deterministic hash of projection record (used in export bundles). |
Primary key: `(tenant_id, finding_id, policy_version)`.
Indexes:
- `ix_projection_status` on `(tenant_id, status, severity DESC)`.
- `ix_projection_labels_gin` using `labels` GIN for KEV/runtime filters.
### 4.2 `finding_history`
Delta view derived from ledger events for quick UI queries.
| Column | Type | Description |
|--------|------|-------------|
| `tenant_id` | `text` | Partition key. |
| `finding_id` | `text` | Finding identity. |
| `policy_version` | `text` | Policy digest. |
| `event_id` | `uuid` | Ledger event ID. |
| `status` | `text` | Status after event. |
| `severity` | `numeric(6,3)` | Severity after event (nullable). |
| `actor_id` | `text` | Actor performing change. |
| `comment` | `text` | Optional summary/message. |
| `occurred_at` | `timestamptz` | Domain event timestamp. |
Materialized view or table updated by projector. Indexed by `(tenant_id, finding_id, occurred_at DESC)`.
### 4.3 `triage_actions`
Audit table for operator actions needing tailored queries.
| Column | Type | Description |
|--------|------|-------------|
| `tenant_id` | `text` | Partition key. |
| `action_id` | `uuid` | Primary key. |
| `event_id` | `uuid` | Source ledger event. |
| `finding_id` | `text` | Finding identity. |
| `action_type` | `ledger_action_type` | e.g., `assign`, `comment`, `attach_evidence`, `link_ticket`. |
| `payload` | `jsonb` | Structured action body (canonical stored separately). |
| `created_at` | `timestamptz` | Timestamp stored. |
| `created_by` | `text` | Actor ID. |
`ledger_action_type` enum mirrors CLI/UX operations.
```
CREATE TYPE ledger_action_type AS ENUM (
'assign',
'comment',
'attach_evidence',
'link_ticket',
'remediation_plan',
'status_change',
'accept_risk',
'reopen',
'close'
);
### 4.4 `ledger_projection_offsets`
Checkpoint store for the projection background worker. Ensures idempotent replays across restarts.
| Column | Type | Description |
|--------|------|-------------|
| `worker_id` | `text` | Logical worker identifier (defaults to `default`). |
| `last_recorded_at` | `timestamptz` | Timestamp of the last projected ledger event. |
| `last_event_id` | `uuid` | Event identifier paired with `last_recorded_at` for deterministic ordering. |
| `updated_at` | `timestamptz` | Last time the checkpoint was persisted. |
Seed row inserted on migration ensures catch-up from epoch (`1970-01-01T00:00:00Z` with empty GUID).
## 5. Hashing & verification
1. Canonical serialize the envelope (§2.3).
2. Compute `event_hash` and store along with `previous_hash`.
3. Build Merkle tree per anchoring window using leaf hash `SHA256(event_hash || '-' || sequence_no)`.
4. Persist root in `ledger_merkle_roots` and, when configured, submit to external transparency log (Rekor v2). Store receipt/UUID in `anchor_reference`.
5. Projection rows compute `cycle_hash = SHA256(canonical_projection_json)` where canonical projection includes fields `{tenant_id, finding_id, policy_version, status, severity, labels, current_event_id}` with sorted keys.
Verification flow for auditors:
- Fetch event, recompute canonical hash, validate `previous_hash` chain.
- Reconstruct Merkle path from stored leaf hash; verify matches recorded root.
- Cross-check projection `cycle_hash` matches ledger state derived from last event.
## 6. Fixtures & migrations
- Initial migration script: `src/Findings/StellaOps.Findings.Ledger/migrations/001_initial.sql`.
- Sample canonical event: `seed-data/findings-ledger/fixtures/ledger-event.sample.json` (includes pre-computed `eventHash`, `previousHash`, and `merkleLeafHash` values).
- Sample projection row: `seed-data/findings-ledger/fixtures/finding-projection.sample.json` (includes canonical `cycleHash` for replay validation).
Fixtures follow canonical key ordering and include precomputed hashes to validate tooling.
## 7. Projection worker
- `LedgerProjectionWorker` consumes ledger events via `PostgresLedgerEventStream`, applying deterministic reductions with `LedgerProjectionReducer`.
- Checkpoint state is stored in `ledger_projection_offsets`, allowing replay from any point in time.
- Batch processing is configurable via `findings:ledger:projection` (`batchSize`, `idleDelay`).
- Each event writes:
- `findings_projection` (upserted current state with `cycle_hash`).
- `finding_history` (timeline entry keyed by event ID).
- `triage_actions` when applicable (status change, comment, assignment, remediation, attachment, accept-risk, close).
## 8. Next steps
- Integrate Policy Engine batch evaluation with the projector (`LEDGER-29-004`).
- Align Vulnerability Explorer queries with the new projection state and timeline endpoints.
- Externalise Merkle anchor publishing to transparency log once anchoring cadence is finalised.
| | | Array of policy rationale references (explain bundle IDs, remediation notes). |

11
docs/modules/graph/README.md
View File

@@ -2,11 +2,12 @@
Graph module (upcoming) will power graph-indexed queries for SBOM relationships, lineage, and blast-radius analysis.
## Responsibilities
- Model SBOM and advisory entities as a navigable graph.
- Provide APIs for dependency impact, provenance chains, and reachability analysis.
- Integrate with Scheduler/Policy for graph-driven re-evaluation.
- Expose tooling for offline explorers.
## Responsibilities
- Model SBOM and advisory entities as a navigable graph.
- Provide APIs for dependency impact, provenance chains, and reachability analysis.
- Integrate with Scheduler/Policy for graph-driven re-evaluation.
- Expose tooling for offline explorers.
- Maintain [Graph Index Canonical Schema](schema.md) with deterministic identities, fixtures, and attribute dictionary.
### Domain highlights (Epic 5)
- **Nodes:** artifacts/images, SBOM components, packages/versions, files/paths, licences, advisories, VEX statements, provenance attestations, policy versions.

19
docs/modules/graph/architecture.md
View File

@@ -38,8 +38,9 @@
## 5) Offline & export
- Each snapshot packages `nodes.jsonl`, `edges.jsonl`, `overlays/` plus manifest with hash, counts, and provenance. Export Center consumes these artefacts for graph-specific bundles.
- Saved queries and overlays include deterministic IDs so Offline Kit consumers can import and replay results.
- Each snapshot packages `nodes.jsonl`, `edges.jsonl`, `overlays/` plus manifest with hash, counts, and provenance. Export Center consumes these artefacts for graph-specific bundles.
- Saved queries and overlays include deterministic IDs so Offline Kit consumers can import and replay results.
- Runtime hosts register the SBOM ingest pipeline via `services.AddSbomIngestPipeline(...)`. Snapshot exports default to `./artifacts/graph-snapshots` but can be redirected with `STELLAOPS_GRAPH_SNAPSHOT_DIR` or the `SbomIngestOptions.SnapshotRootDirectory` callback.
## 6) Observability
@@ -47,10 +48,14 @@
- Logs: structured events for ETL stages and query execution (with trace IDs).
- Traces: ETL pipeline spans, query engine spans.
## 7) Rollout notes
- Phase 1: ingest SBOM + advisories, deliver impact queries.
- Phase 2: add VEX overlays, policy overlays, diff tooling.
- Phase 3: expose runtime/Zastava edges and AI-assisted recommendations (future).
## 7) Rollout notes
- Phase 1: ingest SBOM + advisories, deliver impact queries.
- Phase 2: add VEX overlays, policy overlays, diff tooling.
- Phase 3: expose runtime/Zastava edges and AI-assisted recommendations (future).
### Local testing note
Set `STELLAOPS_TEST_MONGO_URI` to a reachable MongoDB instance before running `tests/Graph/StellaOps.Graph.Indexer.Tests`. The test harness falls back to `mongodb://127.0.0.1:27017`, then Mongo2Go, but the CI workflow requires the environment variable to be present to ensure upsert coverage runs against a managed database. Use `STELLAOPS_GRAPH_SNAPSHOT_DIR` (or the `AddSbomIngestPipeline` options callback) to control where graph snapshot artefacts land during local runs.
Refer to the module README and implementation plan for immediate context, and update this document once component boundaries and data flows are finalised.

98
docs/modules/graph/schema.md Normal file
View File

@@ -0,0 +1,98 @@
# Graph Index Canonical Schema
> Ownership: Graph Indexer Guild • Version 2025-11-03 (Sprint 140)\
> Scope: Canonical node and edge schemas, attribute dictionary, identity rules, and fixture references for the Graph Indexer foundations (GRAPH-INDEX-28-001).
## 1. Purpose
- Provide a deterministic schema contract for graph indexing pipelines.
- Document the attribute dictionary consumed by SBOM, Advisory, VEX, Policy, and Runtime signal feeds.
- Define the identity rules that guarantee stable node and edge identifiers across rebuilds.
- Point implementers and QA to the seed fixtures used in unit/integration tests.
## 2. Node taxonomy
| Node kind | Identity tuple (ordered) | Required attributes | Primary sources |
|-----------|--------------------------|---------------------|-----------------|
| `artifact` | `tenant`, `artifact_digest`, `sbom_digest` | `display_name`, `artifact_digest`, `sbom_digest`, `environment`, `labels[]`, `origin_registry`, `supply_chain_stage` | Scanner WebService, SBOM Service |
| `component` | `tenant`, `purl`, `source_type` | `purl`, `version`, `ecosystem`, `scope`, `license_spdx`, `usage` | SBOM Service analyzers |
| `file` | `tenant`, `artifact_digest`, `normalized_path`, `content_sha256` | `normalized_path`, `content_sha256`, `language_hint`, `size_bytes`, `scope` | SBOM layer analyzers |
| `license` | `tenant`, `license_spdx`, `source_digest` | `license_spdx`, `name`, `classification`, `notice_uri` | SBOM Service, Concelier |
| `advisory` | `tenant`, `advisory_source`, `advisory_id`, `content_hash` | `advisory_source`, `advisory_id`, `severity`, `published_at`, `content_hash`, `linkset_digest` | Concelier |
| `vex_statement` | `tenant`, `vex_source`, `statement_id`, `content_hash` | `status`, `statement_id`, `justification`, `issued_at`, `expires_at`, `content_hash` | Excititor |
| `policy_version` | `tenant`, `policy_pack_digest`, `effective_from` | `policy_pack_digest`, `policy_name`, `effective_from`, `expires_at`, `explain_hash` | Policy Engine |
| `runtime_context` | `tenant`, `runtime_fingerprint`, `collector`, `observed_at` | `runtime_fingerprint`, `collector`, `observed_at`, `cluster`, `namespace`, `workload_kind`, `runtime_state` | Signals, Zastava |
## 3. Edge taxonomy
| Edge kind | Source → Target | Identity tuple (ordered) | Required attributes | Default validity |
|-----------|-----------------|--------------------------|---------------------|------------------|
| `CONTAINS` | `artifact` → `component` | `tenant`, `artifact_node_id`, `component_node_id`, `sbom_digest` | `detected_by`, `layer_digest`, `scope`, `evidence_digest` | `valid_from = sbom_collected_at`, `valid_to = null` |
| `DEPENDS_ON` | `component` → `component` | `tenant`, `component_node_id`, `dependency_purl`, `sbom_digest` | `dependency_purl`, `dependency_version`, `relationship`, `evidence_digest` | Derived from SBOM dependency graph |
| `DECLARED_IN` | `component` → `file` | `tenant`, `component_node_id`, `file_node_id`, `sbom_digest` | `detected_by`, `scope`, `evidence_digest` | Mirrors SBOM declaration |
| `BUILT_FROM` | `artifact` → `artifact` | `tenant`, `parent_artifact_node_id`, `child_artifact_digest` | `build_type`, `builder_id`, `attestation_digest` | Derived from provenance attestations |
| `AFFECTED_BY` | `component` → `advisory` | `tenant`, `component_node_id`, `advisory_node_id`, `linkset_digest` | `evidence_digest`, `matched_versions`, `cvss`, `confidence` | Concelier overlays |
| `VEX_EXEMPTS` | `component` → `vex_statement` | `tenant`, `component_node_id`, `vex_node_id`, `statement_hash` | `status`, `justification`, `impact_statement`, `evidence_digest` | Excititor overlays |
| `GOVERNS_WITH` | `policy_version` → `component` | `tenant`, `policy_node_id`, `component_node_id`, `finding_explain_hash` | `verdict`, `explain_hash`, `policy_rule_id`, `evaluation_timestamp` | Policy Engine overlays |
| `OBSERVED_RUNTIME` | `runtime_context` → `component` | `tenant`, `runtime_node_id`, `component_node_id`, `runtime_fingerprint` | `process_name`, `entrypoint_kind`, `runtime_evidence_digest`, `confidence` | Signals/Zastava ingestion |
## 4. Attribute dictionary
| Attribute | Type | Applies to | Description |
|-----------|------|------------|-------------|
| `tenant` | `string` | nodes, edges | Tenant identifier (enforced on storage and query). |
| `kind` | `string` | nodes, edges | One of the values listed in the taxonomy tables. |
| `canonical_key` | `object` | nodes | Ordered tuple persisted as a JSON object matching the identity tuple components. |
| `id` | `string` | nodes, edges | Deterministic identifier (`gn:` or `ge:` prefix + Base32-encoded SHA-256). |
| `hash` | `string` | nodes, edges | SHA-256 of the canonical JSON representation (normalized by sorted keys). |
| `attributes` | `object` | nodes, edges | Domain-specific attributes (all dictionary keys kebab-case). |
| `provenance` | `object` | nodes, edges | Includes `source`, `collected_at`, `sbom_digest`, `attestation_digest`, `event_offset`. |
| `valid_from` | `string (ISO-8601)` | nodes, edges | Inclusive timestamp describing when the record became effective. |
| `valid_to` | `string (ISO-8601 or null)` | nodes, edges | Exclusive timestamp; `null` means open-ended. |
| `scope` | `string` | nodes, edges | Scope label (e.g., `runtime`, `build`, `dev-dependency`). |
| `labels` | `array[string]` | nodes | Free-form but deterministic ordering (ASCII sort). |
| `confidence` | `number` | edges | 0-1 numeric confidence score for overlay-derived edges. |
| `evidence_digest` | `string` | edges | SHA-256 digest referencing the immutable evidence payload. |
| `linkset_digest` | `string` | nodes, edges | SHA-256 digest to Concelier linkset documents. |
| `explain_hash` | `string` | nodes, edges | Hash of Policy Engine explain trace payload. |
| `runtime_state` | `string` | `runtime_context` nodes | Aggregated runtime state (e.g., `Running`, `Terminated`). |
## 5. Identity rules
1. **Node IDs (`gn:` prefix).**
`id = "gn:" + tenant + ":" + kind + ":" + base32(sha256(identity_tuple))`\
`identity_tuple` concatenates tuple components with `|` (no escaping) and lower-cases both keys and values unless the component is a hash or digest.
2. **Edge IDs (`ge:` prefix).**
`id = "ge:" + tenant + ":" + kind + ":" + base32(sha256(identity_tuple))`\
Edge tuples must include the resolved node IDs rather than only the canonical keys to ensure immutability under re-key events.
3. **Hashes.**
`hash` is computed by serializing the canonical document with:
- UTF-8 JSON
- Object keys sorted lexicographically
- Arrays sorted where semantics allow (e.g., `labels`, `matched_versions`)
- Timestamps normalized to UTC ISO-8601 (`YYYY-MM-DDTHH:MM:SSZ`)
4. **Deterministic provenance.**
`provenance.source` is a dotted string (`scanner.sbom.v1`, `concelier.linkset.v1`) and `provenance.event_offset` is a monotonic integer for replay.
## 6. Validity window semantics
- `valid_from` equals the upstream event timestamp at ingestion time (SBOM collected timestamp, advisory published timestamp, policy evaluation timestamp, runtime observation timestamp).
- `valid_to` stays `null` until a newer version supersedes the record. Superseding records carry a `supersedes` reference in `attributes`.
- Snapshots freeze the set of nodes/edges with `valid_from <= snapshot_at < coalesce(valid_to, +∞)`.
## 7. Fixtures & verification
- Seed fixtures live under `tests/Graph/StellaOps.Graph.Indexer.Tests/Fixtures/v1/`.
- Fixture files:
- `nodes.json` — canonical node samples (per node kind).
- `edges.json` — canonical edge samples including overlay references.
- `schema-matrix.json` — lists attribute coverage per node/edge kind for regression tests.
- Unit tests assert:
- Identifier determinism (`GraphIdentityTests.NodeIds_are_stable`).
- Hash determinism under property ordering variations.
- Attribute coverage against `schema-matrix.json`.
- Fixtures follow the attribute dictionary above; new attributes require dictionary updates and fixture refresh.
## 8. Change control
- Increment schema version in fixture folder (`v1`, `v2`, …) when making breaking changes.
- Update this document and the JSON fixtures together; do not ship mismatched versions.
- Notify SBOM Service, Concelier, Excititor, Policy, Signals, and Zastava owners before promoting changes to DOING/DONE state.
## 9. References
- `docs/modules/graph/architecture.md` — high-level architecture.
- `docs/modules/platform/architecture-overview.md` — platform context.
- `src/Graph/StellaOps.Graph.Indexer/TASKS.md` — task tracking.
- `seed-data/` — additional sample payloads for offline kit packaging (future work).

5
docs/modules/issuer-directory/architecture.md
View File

@@ -63,6 +63,11 @@ IssuerDirectory:
| `POST` | `/issuer-directory/issuers/{id}/keys` | `issuer-directory:write` | Add a signing key (validates format, deduplicates fingerprint, audits). |
| `POST` | `/issuer-directory/issuers/{id}/keys/{keyId}/rotate` | `issuer-directory:write` | Retire an active key and create a replacement atomically. |
| `DELETE` | `/issuer-directory/issuers/{id}/keys/{keyId}` | `issuer-directory:admin` | Revoke a key (status → revoked, audit logged). |
| `GET` | `/issuer-directory/issuers/{id}/trust` | `issuer-directory:read` | Retrieve tenant/global trust overrides with effective weight. |
| `PUT` | `/issuer-directory/issuers/{id}/trust` | `issuer-directory:write` | Set or update a tenant trust override; reason may be supplied in body/header. |
| `DELETE` | `/issuer-directory/issuers/{id}/trust` | `issuer-directory:admin` | Remove a tenant trust override (falls back to global/default weight). |
All write/delete operations accept an optional audit reason header (`X-StellaOps-Reason`) which is persisted alongside trust override changes.
Payloads follow the contract in `Contracts/IssuerDtos.cs` and align with domain types (`IssuerRecord`, `IssuerMetadata`, `IssuerEndpoint`).

24
docs/modules/notify/architecture.md
View File

@@ -514,11 +514,31 @@ sequenceDiagram
* **Templates**: compile and cache per rule+channel+locale; version with rule `updatedAt` to invalidate.
* **Rules**: store raw YAML + parsed AST; validate with schema + static checks (e.g., nonsensical combos).
* **Secrets**: pluggable secret resolver (Authority Secret proxy, K8s, Vault).
* **Rate limiting**: `System.Threading.RateLimiting` + per‑connector adapters.
* **Rate limiting**: `System.Threading.RateLimiting` + per-connector adapters.
---
## 19) Roadmap (post‑v1)
## 19) Air-gapped bootstrap configuration
Air-gapped deployments ship a deterministic Notifier profile inside the
Bootstrap Pack. The artefacts live under `bootstrap/notify/` after running the
Offline Kit builder and include:
- `notify.yaml` — configuration derived from `etc/notify.airgap.yaml`, pointing
to the sealed MongoDB/Authority endpoints and loading connectors from the
local plug-in directory.
- `notify-web.secret.example` — template for the Authority client secret,
intended to be renamed to `notify-web.secret` before deployment.
- `README.md` — operator guide (`docs/modules/notify/bootstrap-pack.md`).
These files are copied automatically by `ops/offline-kit/build_offline_kit.py`
via `copy_bootstrap_configs`. Operators mount the configuration and secret into
the `StellaOps.Notifier.WebService` container (Compose or Kubernetes) to keep
sealed-mode roll-outs reproducible.
---
## 20) Roadmap (post-v1)
* **PagerDuty/Opsgenie** connectors; **Jira** ticket creation.
* **User inbox** (in‑app notifications) + mobile push via webhook relay.

59
docs/modules/notify/bootstrap-pack.md Normal file
View File

@@ -0,0 +1,59 @@
# Notifier Bootstrap Pack Guide
The Bootstrap Pack gives operators a deterministic set of configuration files
to stage the Notifier service in sealed or fully air-gapped environments. The
assets ship alongside the Offline Kit under `bootstrap/notify/` and can be
copied directly onto the hosts that run `StellaOps.Notifier.WebService`.
## Contents
| File | Purpose |
| ---- | ------- |
| `notify.yaml` | Sealed-mode configuration derived from `etc/notify.airgap.yaml`. It disables external resolution by pointing to in-cluster services and honours the shared `EgressPolicy`. |
| `notify-web.secret.example` | Deterministic template for the Authority client secret. Replace the value before running the service. |
| `rules/airgap-ops.rule.json` | Bootstrap rule subscribing to air-gap drift, bundle import, and portable export completion events. Update channel identifiers before import. |
| `templates/airgap-ops-email.template.json` | Email template used by the bootstrap rule with remediation guidance, checksum context, and download locations. |
| `README.md` | This guide, also embedded in the pack for quick operator reference. |
## Usage
1. **Populate secrets** – copy `notify-web.secret.example` to
`notify-web.secret`, change `NOTIFY_WEB_CLIENT_SECRET` to the value issued by
Authority, and store it with restrictive permissions (for example
`chmod 600`).
2. **Drop configuration** – place `notify.yaml` in the location expected by
the runtime (`/app/etc/notify.yaml` for the containers we ship). The file
assumes MongoDB is reachable at `mongodb://stellaops:airgap-password@mongo:27017`
and Authority at `https://authority.airgap.local` – adjust if your
deployment uses different hostnames.
3. **Import rule/template** – with the Notify CLI or REST API, import
`templates/airgap-ops-email.template.json` first, then
`rules/airgap-ops.rule.json`. Update the `channel` identifiers inside the
rule so they match your sealed SMTP relay (for example `email:airgap-ops`).
The rule now also delivers portable export completion notices; ensure your
downstream process watches for checksum and location details in the payload.
4. **Mount secrets/config** – for Docker Compose use:
```yaml
volumes:
- ./bootstrap/notify/notify.yaml:/app/etc/notify.yaml:ro
env_file:
- ./bootstrap/notify/notify-web.secret
```
In Kubernetes, create a Secret from the two files and mount them into the
Notifier pod.
5. **Verify sealed mode** – with the configuration in place the Notifier
resolves channels that point to local relays (SMTP, syslog, file sink). Any
attempt to contact an external webhook is denied by `StellaOps.AirGap.Policy`
with remediation guidance.
## How it is packaged
`ops/offline-kit/build_offline_kit.py` automatically copies the configuration
and secret template into `bootstrap/notify/` during Offline Kit creation. The
same staging directory is what we sign and publish as the Bootstrap Pack, so
the artefacts stay deterministic across releases.
Refer to `etc/notify.airgap.yaml` if you need to regenerate the pack or build a
site-specific overlay from source control.

29
docs/modules/notify/resources/samples/airgap-bundle-import@1.sample.json Normal file
View File

@@ -0,0 +1,29 @@
{
"eventId": "99b40f9c-4a34-4a5f-9e1a-0c1e5a97c0bb",
"kind": "airgap.bundle.import",
"version": "1",
"tenant": "tenant-01",
"ts": "2025-10-30T22:05:18+00:00",
"actor": "airgap-importer",
"payload": {
"bundleId": "mirror-2025-10-30",
"importedAt": "2025-10-30T22:02:03Z",
"links": {
"audit": "https://authority.airgap.local/authority/audit/airgap/entries/mirror-2025-10-30",
"docs": "https://docs.stella-ops.org/airgap/airgap-mode.html#import"
},
"remediation": "Review bundle warnings and plan the next mirror export to keep advisories within the allowed freshness budget.",
"severity": "medium",
"status": "completed",
"warnings": [
{
"code": "feeds.outdated",
"message": "OSV feed is 2 days old; schedule next mirror export sooner."
}
]
},
"attributes": {
"importer": "ops:olivia",
"category": "airgap"
}
}

40
docs/modules/notify/resources/samples/airgap-portable-export-completed@1.sample.json Normal file
View File

@@ -0,0 +1,40 @@
{
"eventId": "6a9f4e37-5053-4a40-9761-5c0a0a857ee2",
"kind": "airgap.portable.export.completed",
"version": "1",
"tenant": "tenant-01",
"ts": "2025-11-02T21:45:12+00:00",
"actor": "export-center",
"payload": {
"bundleId": "portable-2025-11-02",
"checksum": {
"sha256": "f4f56c7d9a68ee4d4c324e7d782f9e2a3c1c4b2fa83f39c7c4b5f9f047d3af11",
"sha512": "c2d1b5ec4784f8cfe317aa7c501c7fd736b119f1d0d1eaa99bf3d6f4f70a85b7e699f356e2f3d3d8c536a8a6f2a506d1c2f458f8829b6f8d9c4abe0ac7edb5a1"
},
"exportedAt": "2025-11-02T21:43:05Z",
"links": {
"docs": "https://docs.stella-ops.org/airgap/portable-evidence.html",
"manifest": "https://authority.airgap.local/export/bundles/portable-2025-11-02/export.json"
},
"locations": [
{
"availableUntil": "2025-11-09T00:00:00Z",
"path": "/data/offline-kit/bootstrap/exports/portable/portable-2025-11-02.tar.zst",
"type": "file"
},
{
"reference": "registry.airgap.local/offline/portable@sha256:f4f56c7d9a68ee4d4c324e7d782f9e2a3c1c4b2fa83f39c7c4b5f9f047d3af11",
"type": "oci"
}
],
"profile": "portable-evidence:mirror-full",
"remediation": "Distribute the portable evidence bundle to downstream enclaves before the freshness budget expires.",
"severity": "medium",
"sizeBytes": 73400320,
"status": "completed"
},
"attributes": {
"profile": "portable-evidence",
"category": "airgap"
}
}

26
docs/modules/notify/resources/samples/airgap-time-drift@1.sample.json Normal file
View File

@@ -0,0 +1,26 @@
{
"eventId": "5cf2d3b2-9b90-4f7d-9a9a-54b8b5a5e3aa",
"kind": "airgap.time.drift",
"version": "1",
"tenant": "tenant-01",
"ts": "2025-10-31T06:15:00+00:00",
"actor": "airgap-time",
"payload": {
"anchorId": "anchor-2025-10-28",
"anchorIssuedAt": "2025-10-28T05:00:00Z",
"budgetSeconds": 259200,
"driftSeconds": 216000,
"links": {
"docs": "https://docs.stella-ops.org/airgap/staleness-and-time.html"
},
"nextAnchorDueBy": "2025-11-01T05:00:00Z",
"remainingSeconds": 43200,
"remediation": "Import the latest mirror bundle to refresh the time anchor or extend the budget via policy override before sealed mode blocks scheduled jobs.",
"severity": "high",
"status": "critical"
},
"attributes": {
"category": "airgap",
"anchorBudgetHours": "72"
}
}

352
docs/modules/platform/architecture-overview.md
View File

@@ -1,168 +1,186 @@
# StellaOps Architecture Overview (Sprint 19)
> **Ownership:** Architecture Guild • Docs Guild
> **Audience:** Service owners, platform engineers, solution architects
> **Related:** [High-Level Architecture](../../07_HIGH_LEVEL_ARCHITECTURE.md), [Concelier Architecture](../concelier/architecture.md), [Policy Engine Architecture](../policy/architecture.md), [Aggregation-Only Contract](../../ingestion/aggregation-only-contract.md)
This dossier summarises the end-to-end runtime topology after the Aggregation-Only Contract (AOC) rollout. It highlights where raw facts live, how ingest services enforce guardrails, and how downstream components consume those facts to derive policy decisions and user-facing experiences.
---
## 1 · System landscape
```mermaid
graph TD
subgraph Edge["Clients & Automation"]
CLI[stella CLI]
UI[Console SPA]
APIClients[CI / API Clients]
end
Gateway[API Gateway<br/>(JWT + DPoP scopes)]
subgraph Scanner["Fact Collection"]
ScannerWeb[Scanner.WebService]
ScannerWorkers[Scanner.Workers]
Agent[Agent Runtime]
end
subgraph Ingestion["Aggregation-Only Ingestion (AOC)"]
Concelier[Concelier.WebService]
Excititor[Excititor.WebService]
RawStore[(MongoDB<br/>advisory_raw / vex_raw)]
end
subgraph Derivation["Policy & Overlay"]
Policy[Policy Engine]
Scheduler[Scheduler Services]
Notify[Notifier]
end
subgraph Experience["UX & Export"]
UIService[Console Backend]
Exporters[Export / Offline Kit]
end
Observability[Telemetry Stack]
CLI --> Gateway
UI --> Gateway
APIClients --> Gateway
Gateway --> ScannerWeb
ScannerWeb --> ScannerWorkers
ScannerWorkers --> Concelier
ScannerWorkers --> Excititor
Concelier --> RawStore
Excititor --> RawStore
RawStore --> Policy
Policy --> Scheduler
Policy --> Notify
Policy --> UIService
Scheduler --> UIService
UIService --> Exporters
Exporters --> CLI
Exporters --> Offline[Offline Kit]
Observability -.-> ScannerWeb
Observability -.-> Concelier
Observability -.-> Excititor
Observability -.-> Policy
Observability -.-> Scheduler
Observability -.-> Notify
```
Key boundaries:
- **AOC border.** Everything inside the Ingestion subgraph writes only immutable raw facts plus link hints. Derived severity, consensus, and risk remain outside the border.
- **Policy-only derivation.** Policy Engine materialises `effective_finding_*` collections and emits overlays; other services consume but never mutate them.
- **Tenant enforcement.** Authority-issued DPoP scopes flow through Gateway to every service; raw stores and overlays include `tenant` strictly.
---
## 2 · Aggregation-Only Contract focus
### 2.1 Responsibilities at the boundary
| Area | Services | Responsibilities under AOC | Forbidden under AOC |
|------|----------|-----------------------------|---------------------|
| **Ingestion (Concelier / Excititor)** | `StellaOps.Concelier.WebService`, `StellaOps.Excititor.WebService` | Fetch upstream advisories/VEX, verify signatures, compute linksets, append immutable documents to `advisory_raw` / `vex_raw`, emit observability signals, expose raw read APIs. | Computing severity, consensus, suppressions, or policy hints; merging upstream sources into a single derived record; mutating existing documents. |
| **Policy & Overlay** | `StellaOps.Policy.Engine`, Scheduler | Join SBOM inventory with raw advisories/VEX, evaluate policies, issue `effective_finding_*` overlays, drive remediation workflows. | Writing to raw collections; bypassing guard scopes; running without recorded provenance. |
| **Experience layers** | Console, CLI, Exporters | Surface raw facts + policy overlays; run `stella aoc verify`; render AOC dashboards and reports. | Accepting ingestion payloads that lack provenance or violate guard results. |
### 2.2 Raw stores
| Collection | Purpose | Key fields | Notes |
|------------|---------|------------|-------|
| `advisory_raw` | Immutable vendor/ecosystem advisory documents. | `_id`, `tenant`, `source.*`, `upstream.*`, `content.raw`, `linkset`, `supersedes`. | Idempotent by `(source.vendor, upstream.upstream_id, upstream.content_hash)`. |
| `vex_raw` | Immutable vendor VEX statements. | Mirrors `advisory_raw`; `identifiers.statements` summarises affected components. | Maintains supersedes chain identical to advisory flow. |
| Change streams (`advisory_raw_stream`, `vex_raw_stream`) | Feed Policy Engine and Scheduler. | `operationType`, `documentKey`, `fullDocument`, `tenant`, `traceId`. | Scope filtered per tenant before delivery. |
### 2.3 Guarded ingestion sequence
```mermaid
sequenceDiagram
participant Upstream as Upstream Source
participant Connector as Concelier/Excititor Connector
participant Guard as AOCWriteGuard
participant Mongo as MongoDB (advisory_raw / vex_raw)
participant Stream as Change Stream
participant Policy as Policy Engine
Upstream-->>Connector: CSAF / OSV / VEX document
Connector->>Connector: Normalize transport, compute content_hash
Connector->>Guard: Candidate raw doc (source + upstream + content + linkset)
Guard-->>Connector: ERR_AOC_00x on violation
Guard->>Mongo: Append immutable document (with tenant & supersedes)
Mongo-->>Stream: Change event (tenant scoped)
Stream->>Policy: Raw delta payload
Policy->>Policy: Evaluate policies, compute effective findings
```
---
### 2.4 Authority scopes & tenancy
| Scope | Holder | Purpose | Notes |
|-------|--------|---------|-------|
| `advisory:ingest` / `vex:ingest` | Concelier / Excititor collectors | Append raw documents through ingestion endpoints. | Paired with tenant claims; requests without tenant are rejected. |
| `advisory:read` / `vex:read` | DevOps verify identity, CLI | Run `stella aoc verify` or call `/aoc/verify`. | Read-only; cannot mutate raw docs. |
| `effective:write` | Policy Engine | Materialise `effective_finding_*` overlays. | Only Policy Engine identity may hold; ingestion contexts receive `ERR_AOC_006` if they attempt. |
| `findings:read` | Console, CLI, exports | Consume derived findings. | Enforced by Gateway and downstream services. |
---
## 3 · Data & control flow highlights
1. **Ingestion:** Concelier / Excititor connectors fetch upstream documents, compute linksets, and hand payloads to `AOCWriteGuard`. Guards validate schema, provenance, forbidden fields, supersedes pointers, and append-only rules before writing to Mongo.
2. **Verification:** `stella aoc verify` (CLI/CI) and `/aoc/verify` endpoints replay guard checks against stored documents, mapping `ERR_AOC_00x` codes to exit codes for automation.
3. **Policy evaluation:** Mongo change streams deliver tenant-scoped raw deltas. Policy Engine joins SBOM inventory (via BOM Index), executes deterministic policies, writes overlays, and emits events to Scheduler/Notify.
4. **Experience surfaces:** Console renders an AOC dashboard showing ingestion latency, guard violations, and supersedes depth. CLI exposes raw-document fetch helpers for auditing. Offline Kit bundles raw collections alongside guard configs to keep air-gapped installs verifiable.
5. **Observability:** All services emit `ingestion_write_total`, `aoc_violation_total{code}`, `ingestion_latency_seconds`, and trace spans `ingest.fetch`, `ingest.transform`, `ingest.write`, `aoc.guard`. Logs correlate via `traceId`, `tenant`, `source.vendor`, and `content_hash`.
---
## 4 · Offline & disaster readiness
- **Offline Kit:** Packages raw Mongo snapshots (`advisory_raw`, `vex_raw`) plus guard configuration and CLI verifier binaries so air-gapped sites can re-run AOC checks before promotion.
- **Recovery:** Supersedes chains allow rollback to prior revisions without mutating documents. Disaster exercises must rehearse restoring from snapshot, replaying change streams into Policy Engine, and re-validating guard compliance.
- **Migration:** Legacy normalised fields are moved to temporary views during cutover; ingestion runtime removes writes once guard-enforced path is live (see [Migration playbook](../../ingestion/aggregation-only-contract.md#8-migration-playbook)).
---
## 5 · References
- [Aggregation-Only Contract reference](../../ingestion/aggregation-only-contract.md)
- [Concelier architecture](../concelier/architecture.md)
- [Excititor architecture](../excititor/architecture.md)
# StellaOps Architecture Overview (Sprint 19)
> **Ownership:** Architecture Guild • Docs Guild
> **Audience:** Service owners, platform engineers, solution architects
> **Related:** [High-Level Architecture](../../07_HIGH_LEVEL_ARCHITECTURE.md), [Concelier Architecture](../concelier/architecture.md), [Policy Engine Architecture](../policy/architecture.md), [Aggregation-Only Contract](../../ingestion/aggregation-only-contract.md)
This dossier summarises the end-to-end runtime topology after the Aggregation-Only Contract (AOC) rollout. It highlights where raw facts live, how ingest services enforce guardrails, and how downstream components consume those facts to derive policy decisions and user-facing experiences.
---
## 1 · System landscape
```mermaid
graph TD
subgraph Edge["Clients & Automation"]
CLI[stella CLI]
UI[Console SPA]
APIClients[CI / API Clients]
end
Gateway[API Gateway<br/>(JWT + DPoP scopes)]
subgraph Scanner["Fact Collection"]
ScannerWeb[Scanner.WebService]
ScannerWorkers[Scanner.Workers]
Agent[Agent Runtime]
end
subgraph Ingestion["Aggregation-Only Ingestion (AOC)"]
Concelier[Concelier.WebService]
Excititor[Excititor.WebService]
RawStore[(MongoDB<br/>advisory_raw / vex_raw)]
end
subgraph Derivation["Policy & Overlay"]
Policy[Policy Engine]
Scheduler[Scheduler Services]
Notify[Notifier]
end
subgraph Experience["UX & Export"]
UIService[Console Backend]
Exporters[Export / Offline Kit]
end
Observability[Telemetry Stack]
CLI --> Gateway
UI --> Gateway
APIClients --> Gateway
Gateway --> ScannerWeb
ScannerWeb --> ScannerWorkers
ScannerWorkers --> Concelier
ScannerWorkers --> Excititor
Concelier --> RawStore
Excititor --> RawStore
RawStore --> Policy
Policy --> Scheduler
Policy --> Notify
Policy --> UIService
Scheduler --> UIService
UIService --> Exporters
Exporters --> CLI
Exporters --> Offline[Offline Kit]
Observability -.-> ScannerWeb
Observability -.-> Concelier
Observability -.-> Excititor
Observability -.-> Policy
Observability -.-> Scheduler
Observability -.-> Notify
```
Key boundaries:
- **AOC border.** Everything inside the Ingestion subgraph writes only immutable raw facts plus link hints. Derived severity, consensus, and risk remain outside the border.
- **Policy-only derivation.** Policy Engine materialises `effective_finding_*` collections and emits overlays; other services consume but never mutate them.
- **Tenant enforcement.** Authority-issued DPoP scopes flow through Gateway to every service; raw stores and overlays include `tenant` strictly.
---
## 2 · Aggregation-Only Contract focus
### 2.1 Responsibilities at the boundary
| Area | Services | Responsibilities under AOC | Forbidden under AOC |
|------|----------|-----------------------------|---------------------|
| **Ingestion (Concelier / Excititor)** | `StellaOps.Concelier.WebService`, `StellaOps.Excititor.WebService` | Fetch upstream advisories/VEX, verify signatures, compute linksets, append immutable documents to `advisory_raw` / `vex_raw`, emit observability signals, expose raw read APIs. | Computing severity, consensus, suppressions, or policy hints; merging upstream sources into a single derived record; mutating existing documents. |
| **Policy & Overlay** | `StellaOps.Policy.Engine`, Scheduler | Join SBOM inventory with raw advisories/VEX, evaluate policies, issue `effective_finding_*` overlays, drive remediation workflows. | Writing to raw collections; bypassing guard scopes; running without recorded provenance. |
| **Experience layers** | Console, CLI, Exporters | Surface raw facts + policy overlays; run `stella aoc verify`; render AOC dashboards and reports. | Accepting ingestion payloads that lack provenance or violate guard results. |
### 2.2 Raw stores
| Collection | Purpose | Key fields | Notes |
|------------|---------|------------|-------|
| `advisory_raw` | Immutable vendor/ecosystem advisory documents. | `_id`, `tenant`, `source.*`, `upstream.*`, `content.raw`, `linkset`, `supersedes`. | Idempotent by `(source.vendor, upstream.upstream_id, upstream.content_hash)`. |
| `vex_raw` | Immutable vendor VEX statements. | Mirrors `advisory_raw`; `identifiers.statements` summarises affected components. | Maintains supersedes chain identical to advisory flow. |
| Change streams (`advisory_raw_stream`, `vex_raw_stream`) | Feed Policy Engine and Scheduler. | `operationType`, `documentKey`, `fullDocument`, `tenant`, `traceId`. | Scope filtered per tenant before delivery. |
### 2.3 Guarded ingestion sequence
```mermaid
sequenceDiagram
participant Upstream as Upstream Source
participant Connector as Concelier/Excititor Connector
participant Guard as AOCWriteGuard
participant Mongo as MongoDB (advisory_raw / vex_raw)
participant Stream as Change Stream
participant Policy as Policy Engine
Upstream-->>Connector: CSAF / OSV / VEX document
Connector->>Connector: Normalize transport, compute content_hash
Connector->>Guard: Candidate raw doc (source + upstream + content + linkset)
Guard-->>Connector: ERR_AOC_00x on violation
Guard->>Mongo: Append immutable document (with tenant & supersedes)
Mongo-->>Stream: Change event (tenant scoped)
Stream->>Policy: Raw delta payload
Policy->>Policy: Evaluate policies, compute effective findings
```
---
### 2.4 Authority scopes & tenancy
| Scope | Holder | Purpose | Notes |
|-------|--------|---------|-------|
| `advisory:ingest` / `vex:ingest` | Concelier / Excititor collectors | Append raw documents through ingestion endpoints. | Paired with tenant claims; requests without tenant are rejected. |
| `advisory:read` / `vex:read` | DevOps verify identity, CLI | Run `stella aoc verify` or call `/aoc/verify`. | Read-only; cannot mutate raw docs. |
| `effective:write` | Policy Engine | Materialise `effective_finding_*` overlays. | Only Policy Engine identity may hold; ingestion contexts receive `ERR_AOC_006` if they attempt. |
| `findings:read` | Console, CLI, exports | Consume derived findings. | Enforced by Gateway and downstream services. |
---
## 3 · Data & control flow highlights
1. **Ingestion:** Concelier / Excititor connectors fetch upstream documents, compute linksets, and hand payloads to `AOCWriteGuard`. Guards validate schema, provenance, forbidden fields, supersedes pointers, and append-only rules before writing to Mongo.
2. **Verification:** `stella aoc verify` (CLI/CI) and `/aoc/verify` endpoints replay guard checks against stored documents, mapping `ERR_AOC_00x` codes to exit codes for automation.
3. **Policy evaluation:** Mongo change streams deliver tenant-scoped raw deltas. Policy Engine joins SBOM inventory (via BOM Index), executes deterministic policies, writes overlays, and emits events to Scheduler/Notify.
4. **Experience surfaces:** Console renders an AOC dashboard showing ingestion latency, guard violations, and supersedes depth. CLI exposes raw-document fetch helpers for auditing. Offline Kit bundles raw collections alongside guard configs to keep air-gapped installs verifiable.
5. **Observability:** All services emit `ingestion_write_total`, `aoc_violation_total{code}`, `ingestion_latency_seconds`, and trace spans `ingest.fetch`, `ingest.transform`, `ingest.write`, `aoc.guard`. Logs correlate via `traceId`, `tenant`, `source.vendor`, and `content_hash`.
---
## 4 · Offline & disaster readiness
- **Offline Kit:** Packages raw Mongo snapshots (`advisory_raw`, `vex_raw`) plus guard configuration and CLI verifier binaries so air-gapped sites can re-run AOC checks before promotion.
- **Recovery:** Supersedes chains allow rollback to prior revisions without mutating documents. Disaster exercises must rehearse restoring from snapshot, replaying change streams into Policy Engine, and re-validating guard compliance.
- **Migration:** Legacy normalised fields are moved to temporary views during cutover; ingestion runtime removes writes once guard-enforced path is live (see [Migration playbook](../../ingestion/aggregation-only-contract.md#8-migration-playbook)).
---
## 5 · Replay CAS & deterministic bundles
- **Replay CAS:** Content-addressed storage lives under `cas://replay/<sha256-prefix>/<digest>.tar.zst`. Writers must use [StellaOps.Replay.Core](../../src/__Libraries/StellaOps.Replay.Core/AGENTS.md) helpers to ensure lexicographic file ordering, POSIX mode normalisation (0644/0755), LF newlines, and zstd level 19 compression. Bundle metadata (size, hash, created) feeds the platform-wide `replay_bundles` collection defined in `docs/data/replay_schema.md`.
- **Artifacts:** Each recorded scan stores three bundles:
1. `manifest.json` (canonical JSON, hashed and signed via DSSE).
2. `inputbundle.tar.zst` (feeds, policies, tools, environment snapshot).
3. `outputbundle.tar.zst` (SBOM, findings, VEX, logs, Merkle proofs).
Every artifact is signed with multi-profile keys (FIPS, GOST, SM, etc.) managed by Authority. See `docs/replay/DETERMINISTIC_REPLAY.md` §2–§5 for the full schema.
- **Storage tiers:** Primary storage is Mongo (`replay_runs`, `replay_subjects`) plus the CAS bucket. Evidence Locker mirrors bundles for long-term retention and legal hold workflows (`docs/modules/evidence-locker/architecture.md`). Offline kits package bundles under `offline/replay/<scan-id>` with detached DSSE envelopes for air-gapped verification.
- **APIs & ownership:** Scanner WebService produces the bundles via `record` mode, Scanner Worker emits Merkle metadata, Signer/Authority provide DSSE signatures, Attestor anchors manifests to Rekor, CLI/Evidence Locker handle retrieval, and Docs Guild maintains runbooks. Responsibilities are tracked in `docs/implplan/SPRINT_185_replay_core.md` through `SPRINT_187_evidence_cli_replay.md`.
- **Operational policies:** Retention defaults to 180 days for hot CAS storage and 2 years for cold Evidence Locker copies. Rotation and pruning follow the checklist in `docs/runbooks/replay_ops.md`.
---
## 6 · References
- [Aggregation-Only Contract reference](../../ingestion/aggregation-only-contract.md)
- [Concelier architecture](../concelier/architecture.md)
- [Excititor architecture](../excititor/architecture.md)
- [Policy Engine architecture](../policy/architecture.md)
- [Authority service](../authority/architecture.md)
- [Observability standards (upcoming)](../../observability/policy.md) – interim reference for telemetry naming.
---
## 6 · Compliance checklist
- [ ] AOC guard enabled for all Concelier and Excititor write paths in production.
- [ ] Mongo schema validators deployed for `advisory_raw` and `vex_raw`; change streams scoped per tenant.
- [ ] Authority scopes (`advisory:*`, `vex:*`, `effective:*`) configured in Gateway and validated via integration tests.
- [ ] `stella aoc verify` wired into CI/CD pipelines with seeded violation fixtures.
- [ ] Console AOC dashboard and CLI documentation reference the new ingestion contract.
- [ ] Offline Kit bundles include guard configs, verifier tooling, and documentation updates.
- [ ] Observability dashboards include violation, latency, and supersedes depth metrics with alert thresholds.
---
*Last updated: 2025-10-26 (Sprint 19).*
- [Authority service](../authority/architecture.md)
- [Replay specification](../../replay/DETERMINISTIC_REPLAY.md)
- [Replay developer guide](../../replay/DEVS_GUIDE_REPLAY.md)
- [Replay schema](../../data/replay_schema.md) *(pending)*
- [Replay test strategy](../../replay/TEST_STRATEGY.md) *(draft)*
- [Observability standards (upcoming)](../../observability/policy.md) – interim reference for telemetry naming.
---
## 7 · Compliance checklist
- [ ] AOC guard enabled for all Concelier and Excititor write paths in production.
- [ ] Mongo schema validators deployed for `advisory_raw` and `vex_raw`; change streams scoped per tenant.
- [ ] Authority scopes (`advisory:*`, `vex:*`, `effective:*`) configured in Gateway and validated via integration tests.
- [ ] `stella aoc verify` wired into CI/CD pipelines with seeded violation fixtures.
- [ ] Console AOC dashboard and CLI documentation reference the new ingestion contract.
- [ ] Offline Kit bundles include guard configs, verifier tooling, and documentation updates.
- [ ] Observability dashboards include violation, latency, and supersedes depth metrics with alert thresholds.
---
*Last updated: 2025-11-03 (Replay planning refresh).*

8
docs/modules/scheduler/architecture.md
View File

@@ -269,6 +269,8 @@ At cron tick:
* `scheduler.run_latency_seconds{quantile}` // event → first verdict
* `scheduler.delta_images_total{severity}`
* `scheduler.rate_limited_total{reason}`
* `policy_simulation_queue_depth{status}` (WebService gauge)
* `policy_simulation_latency_seconds` (WebService histogram)
**Targets**
@@ -278,6 +280,12 @@ At cron tick:
**Tracing** (OTEL): spans `plan`, `resolve`, `enqueue`, `report_call`, `persist`, `emit`.
**Webhooks**
* Policy simulation webhooks fire on terminal states (`succeeded`, `failed`, `cancelled`).
* Configure under `Scheduler:Worker:Policy:Webhook` (see `SCHED-WEB-27-002-POLICY-SIMULATION-WEBHOOKS.md`).
* Requests include headers `X-StellaOps-Tenant` and `X-StellaOps-Run-Id` for idempotency.
---
## 10) Configuration (YAML)

36
docs/modules/signer/architecture.md
View File

@@ -125,15 +125,33 @@ Response:
{ "trusted": true, "signatures": [ { "type": "cosign", "digest": "sha256:...", "signedBy": "StellaOps Release 2027 Q2" } ] }
```
> **Note:** This endpoint is also used internally by Signer before issuing signatures.
---
## 4) Validation pipeline (hot path)
```mermaid
sequenceDiagram
autonumber
> **Note:** This endpoint is also used internally by Signer before issuing signatures.
---
### KMS drivers (keyful mode)
Signer now ships five deterministic KMS adapters alongside the default keyless flow:
- `services.AddFileKms(...)` – stores encrypted ECDSA material on disk for air-gapped or lab installs.
- `services.AddAwsKms(options => { options.Region = "us-east-1"; /* optional: options.Endpoint, UseFipsEndpoint */ });` – delegates signing to AWS KMS, caches metadata/public keys offline, and never exports the private scalar. Rotation/revocation still run through AWS tooling (this library intentionally throws for those APIs so we do not paper over operator approvals).
- `services.AddGcpKms(options => { options.Endpoint = "kms.googleapis.com"; });` – integrates with Google Cloud KMS asymmetric keys, auto-resolves the primary key version when callers omit a version, and verifies signatures locally with exported PEM material.
- `services.AddPkcs11Kms(options => { options.LibraryPath = "/opt/hsm/libpkcs11.so"; options.PrivateKeyLabel = "stella-attestor"; });` – loads a PKCS#11 module, opens read-only sessions, signs digests via HSM mechanisms, and never hoists the private scalar into process memory.
- `services.AddFido2Kms(options => { options.CredentialId = "<base64url>"; options.PublicKeyPem = "-----BEGIN PUBLIC KEY-----..."; options.AuthenticatorFactory = sp => new WebAuthnAuthenticator(); });` – routes signing to a WebAuthn/FIDO2 authenticator for dual-control or air-gap scenarios. The authenticator must supply the CTAP/WebAuthn plumbing; the library handles digesting, key material caching, and verification.
Cloud & hardware-backed drivers share a few invariants:
1. Hash payloads server-side (SHA-256) before invoking provider APIs – signatures remain reproducible and digest inputs are observable in structured audit logs.
2. Cache metadata for the configurable window (default 5 min) and subject-public-key-info blobs for 10 min; tune these per sovereignty policy when running in sealed/offline environments.
3. Only expose public coordinates (`Qx`, `Qy`) to the host ― `KmsKeyMaterial.D` is blank for non-exportable keys so downstream code cannot accidentally persist secrets.
> **Security review checkpoint:** rotate/destroy remains an administrative action in the provider. Document those runbooks per tenant, and gate AWS/GCP traffic in sealed-mode via the existing egress allowlist. PKCS#11 loads native code, so keep library paths on the allowlist and validate HSM policies separately. FIDO2 authenticators expect an operator in the loop; plan for session timeouts and explicit audit fields when enabling interactive signing.
## 4) Validation pipeline (hot path)
```mermaid
sequenceDiagram
autonumber
participant Client as Scanner.WebService
participant Auth as Authority (OIDC)
participant Sign as Signer

5
docs/modules/telemetry/architecture.md
View File

@@ -18,8 +18,9 @@
## 3) Pipelines & Guardrails
- **Redaction.** Attribute processors strip PII/secrets based on policy-managed allowed keys. Redaction profiles mirrored in Offline Kit.
- **Sampling.** Tail sampling by service/error; incident mode (triggered by Orchestrator) promotes services to 100 % sampling, extends retention, and toggles Notify alerts.
- **Alerting.** Prometheus rules/Dashboards packaged with Export Center: service SLOs, queue depth, policy run latency, ingestion AOC violations.
- **Sampling.** Tail sampling by service/error; incident mode (triggered by Orchestrator) promotes services to 100 % sampling, extends retention, and toggles Notify alerts.
- **Alerting.** Prometheus rules/Dashboards packaged with Export Center: service SLOs, queue depth, policy run latency, ingestion AOC violations.
- **Sealed-mode guard.** `StellaOps.Telemetry.Core` enforces `IEgressPolicy` on OTLP exporters; when air-gap mode is sealed any non-loopback collector endpoints are automatically disabled and a structured warning with remediation is emitted.
## 4) APIs & integration

112
docs/modules/vuln-explorer/architecture.md
View File

@@ -1,52 +1,54 @@
# Vulnerability Explorer architecture
> Based on Epic 6 – Vulnerability Explorer; this specification summarises the ledger model, triage workflows, APIs, and export requirements.
## 1) Ledger data model
- **Collections / tables**
- `finding_records` – canonical, policy-derived findings enriched with advisory, VEX, SBOM, runtime context. Includes `policyVersion`, `advisoryRawIds`, `vexRawIds`, `sbomComponentId`, and `explainBundleRef`.
- `finding_history` – append-only state transitions (`new`, `triaged`, `accepted_risk`, `remediated`, `false_positive`, etc.) with timestamps, actor, and justification.
- `triage_actions` – discrete operator actions (comment, assignment, remediation note, ticket link) with immutable provenance.
- `remediation_plans` – structured remediation steps (affected assets, deadlines, recommended fixes, auto-generated from SRM/AI hints).
- `reports` – saved report definitions, export manifests, and signatures.
- **Immutability & provenance** – All updates are append-only; previous state is recoverable. Records capture `tenant`, `artifactId`, `findingKey`, `policyVersion`, `sourceRunId`, `sr mDigest`.
## 2) Triage workflow
1. **Ingest effective findings** from Policy Engine (stream `policy.finding.delta`). Each delta updates `finding_records`, generates history entries, and triggers notification rules.
2. **Prioritisation** uses contextual heuristics: runtime exposure, VEX status, policy severity, AI hints. Stored as `priorityScore` with provenance from Zastava/AI modules.
3. **Assignment & collaboration** – Operators claim findings, add comments, attach evidence, and link tickets. Assignment uses Authority identities and RBAC.
4. **Remediation tracking** – Link remediation plans, record progress, and integrate with Scheduler for follow-up scans once fixes deploy.
5. **Closure** – When Policy or rescans mark finding resolved, system logs closure with explain trace and updates audit ledger.
State machine summary:
```
new -> (triage) triaged -> (remediate) in_progress -> (verify) awaiting_verification -> (scan) remediated
new -> (false_positive) closed_false_positive
new -> (risk_accept) accepted_risk
```
All transitions require justification; certain transitions (accepted risk) require multi-approver workflow defined by Policy Studio.
## 3) APIs
- `GET /v1/findings` — filtered listing with pagination, search (artifact, advisory, priority, status, assignee).
- `GET /v1/findings/{id}` — detail view (policy context, explain trace, evidence timeline).
- `POST /v1/findings/{id}/actions` — create triage action (assign, comment, status change, remediation, ticket link) with DSSE signature support.
- `POST /v1/reports` — generate report artifact (JSON, CSV, PDF) defined by saved templates; records manifest + signature.
- `GET /v1/exports/offline` — stream deterministic bundle for Offline Kit (findings JSON, history, attachments, manifest).
CLI mirrors these endpoints (`stella findings list|view|update|export`). Console UI consumes the same APIs via typed clients.
## 4) AI/automation integration
- Advisory AI contributes remediation hints and conflict explanations stored alongside findings (`aiInsights`).
- Scheduler integration triggers follow-up scans or policy re-evaluation when remediation plan reaches checkpoint.
- Zastava (Differential SBOM) feeds runtime exposure signals to reprioritise findings automatically.
# Vulnerability Explorer architecture
> Based on Epic 6 – Vulnerability Explorer; this specification summarises the ledger model, triage workflows, APIs, and export requirements.
## 1) Ledger data model
- See [`../findings-ledger/schema.md`](../findings-ledger/schema.md) for the canonical SQL schema, hashing strategy, and fixtures underpinning these collections.
- **Collections / tables**
- `finding_records` – canonical, policy-derived findings enriched with advisory, VEX, SBOM, runtime context. Includes `policyVersion`, `advisoryRawIds`, `vexRawIds`, `sbomComponentId`, and `explainBundleRef`.
- `finding_history` – append-only state transitions (`new`, `triaged`, `accepted_risk`, `remediated`, `false_positive`, etc.) with timestamps, actor, and justification.
- `triage_actions` – discrete operator actions (comment, assignment, remediation note, ticket link) with immutable provenance.
- `remediation_plans` – structured remediation steps (affected assets, deadlines, recommended fixes, auto-generated from SRM/AI hints).
- `reports` – saved report definitions, export manifests, and signatures.
- **Immutability & provenance** – All updates are append-only; previous state is recoverable. Records capture `tenant`, `artifactId`, `findingKey`, `policyVersion`, `sourceRunId`, `sr mDigest`.
## 2) Triage workflow
1. **Ingest effective findings** from Policy Engine (stream `policy.finding.delta`). Each delta updates `finding_records`, generates history entries, and triggers notification rules.
2. **Prioritisation** uses contextual heuristics: runtime exposure, VEX status, policy severity, AI hints. Stored as `priorityScore` with provenance from Zastava/AI modules.
3. **Assignment & collaboration** – Operators claim findings, add comments, attach evidence, and link tickets. Assignment uses Authority identities and RBAC.
4. **Remediation tracking** – Link remediation plans, record progress, and integrate with Scheduler for follow-up scans once fixes deploy.
5. **Closure** – When Policy or rescans mark finding resolved, system logs closure with explain trace and updates audit ledger.
State machine summary:
```
new -> (triage) triaged -> (remediate) in_progress -> (verify) awaiting_verification -> (scan) remediated
new -> (false_positive) closed_false_positive
new -> (risk_accept) accepted_risk
```
All transitions require justification; certain transitions (accepted risk) require multi-approver workflow defined by Policy Studio.
## 3) APIs
- `GET /v1/findings` — filtered listing with pagination, search (artifact, advisory, priority, status, assignee).
- `GET /v1/findings/{id}` — detail view (policy context, explain trace, evidence timeline).
- `POST /v1/findings/{id}/actions` — create triage action (assign, comment, status change, remediation, ticket link) with DSSE signature support.
- `POST /v1/reports` — generate report artifact (JSON, CSV, PDF) defined by saved templates; records manifest + signature.
- `GET /v1/exports/offline` — stream deterministic bundle for Offline Kit (findings JSON, history, attachments, manifest).
CLI mirrors these endpoints (`stella findings list|view|update|export`). Console UI consumes the same APIs via typed clients.
## 4) AI/automation integration
- Advisory AI contributes remediation hints and conflict explanations stored alongside findings (`aiInsights`).
- Scheduler integration triggers follow-up scans or policy re-evaluation when remediation plan reaches checkpoint.
- Zastava (Differential SBOM) feeds runtime exposure signals to reprioritise findings automatically.
## 5) Observability & compliance
- Metrics: `findings_open_total{severity,tenant}`, `findings_mttr_seconds`, `triage_actions_total{type}`, `report_generation_seconds`.
@@ -60,14 +62,16 @@ CLI mirrors these endpoints (`stella findings list|view|update|export`). Console
- **Attribute filters (ABAC)** – Authority enforces per-service-account filters via the client-credential parameters `vuln_env`, `vuln_owner`, and `vuln_business_tier`. Service accounts define the allowed values in `authority.yaml` (`attributes` block). Tokens include the resolved filters as claims (`stellaops:vuln_env`, `stellaops:vuln_owner`, `stellaops:vuln_business_tier`), and tokens persisted to Mongo retain the same values for audit and revocation.
- **Audit trail** – Every token issuance emits `authority.vuln_attr.*` audit properties that mirror the resolved filter set, along with `delegation.service_account` and ordered `delegation.actor[n]` entries so Vuln Explorer can correlate access decisions.
- **Permalinks** – Signed permalinks inherit the caller’s ABAC filters; consuming services must enforce the embedded claims in addition to scope checks when resolving permalinks.
- **Attachment tokens** – Authority mints short-lived tokens (`POST /vuln/attachments/tokens/issue`) to gate evidence downloads. Verification (`POST /vuln/attachments/tokens/verify`) enforces tenant, scope, and ABAC metadata, and emits `vuln.attachment.token.*` audit records.
- **Ledger verification** – Offline workflows validate attachments by comparing the Authority-issued token, the Vuln Explorer ledger hash, and the downloaded artefact hash before distribution.
## 7) Offline bundle requirements
- Bundle structure:
- `manifest.json` (hashes, counts, policy version, generation timestamp).
- `findings.jsonl` (current open findings).
- `history.jsonl` (state changes).
- `actions.jsonl` (comments, assignments, tickets).
- `reports/` (generated PDFs/CSVs).
- `signatures/` (DSSE envelopes).
- Bundles produced deterministically; Export Center consumes them for mirror profiles.
- `history.jsonl` (state changes).
- `actions.jsonl` (comments, assignments, tickets).
- `reports/` (generated PDFs/CSVs).
- `signatures/` (DSSE envelopes).
- Bundles produced deterministically; Export Center consumes them for mirror profiles.