up
Some checks failed
api-governance / spectral-lint (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled
oas-ci / oas-validate (push) Has been cancelled
SDK Publish & Sign / sdk-publish (push) Has been cancelled
Policy Lint & Smoke / policy-lint (push) Has been cancelled
Policy Simulation / policy-simulate (push) Has been cancelled
devportal-offline / build-offline (push) Has been cancelled

This commit is contained in:
StellaOps Bot
2025-11-26 20:23:28 +02:00
parent 4831c7fcb0
commit d63af51f84
139 changed files with 8010 additions and 2795 deletions

View File

@@ -0,0 +1,74 @@
# DevPortal Offline Bundle Manifest (draft v0.1)
Applies to sprint: SPRINT_0206_0001_0001_devportal · Action #2 (DEVPORT-64-001/64-002 interlock with Export Center)
## Purpose
- Define a deterministic, air-gap-friendly manifest for Developer Portal offline bundles.
- Ensure SDK archives, OpenAPI specs, and static site assets can be verified and consumed by Export Center and SDK Release pipelines.
## Bundle layout
```
devportal-offline/
manifest.json # see schema below
site/ # static HTML/CSS/JS (Astro/Starlight build)
specs/
stella-aggregate.yaml # merged OpenAPI used by portal
*.yaml # per-service OpenAPI (authority, scanner, policy, graph, etc.)
sdks/
node-sdk.tar.gz
python-sdk.tar.gz
java-sdk.zip # optional, language-dependent
assets/
fonts/* # self-hosted; no external CDNs
icons/* # SVG/PNG used by site
```
## Manifest schema (manifest.json)
```json
{
"version": "0.1",
"generatedAt": "2025-11-26T00:00:00Z",
"site": {
"path": "site",
"sha256": "<hex>",
"bytes": 0
},
"specs": [
{ "name": "stella-aggregate.yaml", "path": "specs/stella-aggregate.yaml", "sha256": "<hex>", "bytes": 0 },
{ "name": "authority.yaml", "path": "specs/authority.yaml", "sha256": "<hex>", "bytes": 0 }
],
"sdks": [
{ "name": "node-sdk", "path": "sdks/node-sdk.tar.gz", "sha256": "<hex>", "bytes": 0 },
{ "name": "python-sdk", "path": "sdks/python-sdk.tar.gz", "sha256": "<hex>", "bytes": 0 }
],
"checks": {
"integrity": "sha256",
"policy": "no-external-assets"
}
}
```
### Rules
- `version` is additive; bump on breaking shape change.
- `sha256` is hex lowercase of the file contents.
- `bytes` is the exact byte length for deterministic validation.
- `checks.policy` documents applied constraints; default `no-external-assets` (verify no `http(s)://` fonts/scripts).
## Production contract
- Export Center expects `manifest.json` at bundle root; validates sha256/bytes before publishing.
- Offline bundle must build via `npm run build:offline` without network calls after initial `npm ci` + `npm run sync:spec`.
- Specs and SDK archives are treated as opaque; manifest carries their hashes for downstream verification.
## Open items
- Add per-language SDK metadata (version, commit SHA) once SDKREL-64-002 finalises.
- Add optional `signatures` array (DSSE over manifest) when Authority signing profile is ready.
## How to produce locally (deterministic)
```
npm ci --ignore-scripts --no-fund --no-audit
npm run sync:spec
npm run build:offline
# compute manifest hashes using sha256sum and fill manifest.json
```
Record generated manifest in sprint evidence when produced; keep caches local to avoid external fetches.

View File

@@ -180,22 +180,38 @@ sequenceDiagram
3. Re-run integrity checks (`mirror verify <path>`).
- **Audit logging:** Export Center logs `mirror.bundle.created`, `mirror.delta.applied`, and `mirror.encryption.enabled` events. Consume them in the central observability pipeline.
## 7. Troubleshooting
| Symptom | Meaning | Action |
|---------|---------|--------|
| `ERR_EXPORT_BASE_MISSING` | Base export not available | Republish base bundle or rebuild as full export. |
| Delta applies but mirror misses entries | Deltas applied out of order | Rebuild from last full bundle and reapply in sequence. |
## 7. Validation checklist (Trivy / mirror bundles)
- Download and verify:
- `stella export download <exportId> --format mirror`
- `stella export verify <exportId>`
- Delta ordering:
- Ensure `manifest.diff.json.baseExportId` exists locally before applying delta.
- Track applied order in `appliedExportIds.log`.
- Trivy adapter (if enabled):
- `stella export trivy-validate --bundle mirror-YYYYMMDD.tar.zst --policy ./policies/export-center.rego`
- Dry-run import:
- `stella export mirror-validate --bundle mirror-YYYYMMDD.tar.zst --dry-run`
- Post-import checks:
- Recompute SHA256 for `manifest.yaml` and a sample data file; compare to manifest.
- Run `mirror verify` (Offline Kit) and confirm zero mismatches.
## 8. Troubleshooting
| Symptom | Meaning | Action |
|---------|---------|--------|
| `ERR_EXPORT_BASE_MISSING` | Base export not available | Republish base bundle or rebuild as full export. |
| Delta applies but mirror misses entries | Deltas applied out of order | Rebuild from last full bundle and reapply in sequence. |
| Decryption fails | Recipient key mismatch or corrupted bundle | Confirm key distribution and re-download bundle. |
| Verification errors | Signature mismatch | Do not import; regenerate bundle and investigate signing pipeline. |
| Manifest hash mismatch | Files changed after extraction | Re-extract bundle and re-run verification; check storage tampering. |
## 8. References
- [Export Center Overview](overview.md)
- [Export Center Architecture](architecture.md)
- [Export Center API reference](api.md)
- [Export Center CLI Guide](cli.md)
| Verification errors | Signature mismatch | Do not import; regenerate bundle and investigate signing pipeline. |
| Manifest hash mismatch | Files changed after extraction | Re-extract bundle and re-run verification; check storage tampering. |
## 9. References
- [Export Center Overview](overview.md)
- [Export Center Architecture](architecture.md)
- [Export Center API reference](api.md)
- [Export Center CLI Guide](cli.md)
- [Concelier mirror runbook](../concelier/operations/mirror.md)
- [Aggregation-Only Contract reference](../../ingestion/aggregation-only-contract.md)

View File

@@ -53,6 +53,7 @@ Graph Indexer + Graph API build the tenant-scoped knowledge graph that powers bl
- [`architecture.md`](architecture.md) — inputs, pipelines, APIs, storage choices, observability, offline handling.
- [`implementation_plan.md`](implementation_plan.md) — phased delivery roadmap, work breakdown, risks, test strategy.
- [`schema.md`](schema.md) — canonical node/edge schema and attribute dictionary (keep in sync with indexer code).
- API surface: `docs/api/graph-gateway-spec-draft.yaml` (NDJSON tiles for `/graph/search|query|paths|diff|export`, budgets, overlays).
- Updates: `docs/updates/2025-10-26-scheduler-graph-jobs.md`, `docs/updates/2025-10-26-authority-graph-scopes.md`, `docs/updates/2025-10-30-devops-governance.md` for the latest decisions/dependencies.
- Index: see `architecture-index.md` for data model, ingestion pipeline, overlays/caches, events, and API/observability pointers.

View File

@@ -12,7 +12,7 @@
- `Advisory` and `VEXStatement` nodes linking to Concelier/Excititor records via digests.
- `PolicyVersion` nodes representing signed policy packs.
- **Edges:** directed, timestamped relationships such as `DEPENDS_ON`, `BUILT_FROM`, `DECLARED_IN`, `AFFECTED_BY`, `VEX_EXEMPTS`, `GOVERNS_WITH`, `OBSERVED_RUNTIME`. Each edge carries provenance (SRM hash, SBOM digest, policy run ID).
- **Overlays:** computed index tables providing fast access to reachability, blast radius, and differential views (e.g., `graph_overlay/vuln/{tenant}/{advisoryKey}`).
- **Overlays:** computed index tables providing fast access to reachability, blast radius, and differential views (e.g., `graph_overlay/vuln/{tenant}/{advisoryKey}`). Runtime endpoints emit overlays inline (`policy.overlay.v1`, `openvex.v1`) with deterministic overlay IDs (`sha256(tenant|nodeId|overlayKind)`) and sampled explain traces on policy overlays.
## 2) Pipelines
@@ -24,11 +24,12 @@
## 3) APIs
- `GET /graph/nodes/{id}` — fetch node with metadata and attached provenance.
- `POST /graph/query/saved` — execute saved query (Cypher-like DSL) with tenant filtering; supports paging, citation metadata, and `explain` traces.
- `GET /graph/impact/{advisoryKey}` — returns impacted artifacts with path context and policy/vex overlays.
- `GET /graph/diff/{snapshotA}/{snapshotB}` streaming API returning diff manifest including new/removed edges, risk summary, and export references.
- `POST /graph/overlay/policy` — create or retrieve overlay for policy version + advisory set, referencing `effective_finding` results.
- `POST /graph/search` — NDJSON node tiles with cursor paging, tenant + scope guards.
- `POST /graph/query` — NDJSON nodes/edges/stats/cursor with budgets (tiles/nodes/edges) and optional inline overlays (`includeOverlays=true`) emitting `policy.overlay.v1` and `openvex.v1` payloads; overlay IDs are `sha256(tenant|nodeId|overlayKind)`; policy overlay may include a sampled `explainTrace`.
- `POST /graph/paths` — bounded BFS (depth ≤6) returning path nodes/edges/stats; honours budgets and overlays.
- `POST /graph/diff` — compares `snapshotA` vs `snapshotB`, streaming node/edge added/removed/changed tiles plus stats; budget enforcement mirrors `/graph/query`.
- `POST /graph/export` — async job producing deterministic manifests (`sha256`, size, format) for `ndjson/csv/graphml/png/svg`; download via `/graph/export/{jobId}`.
- Legacy: `GET /graph/nodes/{id}`, `POST /graph/query/saved`, `GET /graph/impact/{advisoryKey}`, `POST /graph/overlay/policy` remain in spec but should align to the NDJSON surfaces above as they are brought forward.
## 4) Storage considerations

View File

@@ -1,6 +1,6 @@
# Scanner Determinism Score Guide
> **Status:** Draft Sprint 186/202/203
> **Status:** Stable (2025-11)
> **Owners:** Scanner Guild · QA Guild · DevEx/CLI Guild · DevOps Guild
## 1. Goal
@@ -22,36 +22,29 @@ Quantify how repeatable a scanner release is by re-running scans under frozen co
The harness persists the full run set under CAS, allowing regression tests and Offline kit inclusion.
## 3. Output artefacts (`SCAN-DETER-186-010`)
## 3. determinism.json schema (`SCAN-DETER-186-010`)
* `determinism.json` per-image runs, identical counts, score, policy/feed hashes.
* `run_i/*.json` canonicalised outputs for debugging.
* `diffs/` optional diff samples when runs diverge.
Required fields:
Example `determinism.json`:
- `version` (string, default `1`)
- `release` (string; e.g., `scanner-0.14.3`)
- `platform` (`os/arch`)
- `policy_sha`, `feeds_sha`, `scanner_sha` (content-addressed digests)
- `images[]`:
- `digest` (image digest)
- `runs` (int), `identical` (int), `score` (float 01)
- `artifact_hashes` (map filename → sha256)
- `non_deterministic` (array of artefact names when diverged)
- `notes` (optional string)
- `overall_score` (float 01)
- `thresholds` { `image_min`:0.90, `overall_min`:0.95 }
- `generated_at` (UTC ISO-8601)
```json
{
"release": "scanner-0.14.3",
"platform": "linux/amd64",
"policy_sha": "a1b2c3…",
"feeds_sha": "d4e5f6…",
"images": [
{
"digest": "sha256:abc…",
"runs": 10,
"identical": 10,
"score": 1.0,
"artifact_hashes": {
"sbom.cdx.json": "sha256:11…",
"vex.json": "sha256:22…",
"findings.json": "sha256:33…"
}
}
],
"overall_score": 1.0
}
```
Output bundle layout:
- `determinism.json` schema above
- `run_i/*.json` canonicalised artefacts per run
- `diffs/` minimal diffs when divergence occurs
## 4. CI integration (`DEVOPS-SCAN-90-004`)
@@ -71,8 +64,12 @@ Example `determinism.json`:
## 7. Evidence & replay
* Include `determinism.json` and canonical run outputs in Replay bundles (`docs/replay/DETERMINISTIC_REPLAY.md`).
* DSSE-sign determinism results before adding them to Evidence Locker.
Replay instructions (deterministic, offline-friendly):
1. Fetch a published determinism bundle (`determinism.json` + `run_*`) and the corresponding scanner image.
2. Run `scripts/scanner/determinism-run.sh --replay --bundle path/to/bundle --image sha256:...` (uses fixed clock/seed).
3. Verify hashes match the recorded `artifact_hashes`; tool exits non-zero on mismatch.
4. Attach the bundle to Evidence Locker with DSSE signature before export.
## 8. Implementation checklist

View File

@@ -1,6 +1,7 @@
# Entropy Analysis for Executable Layers
> **Status:** Draft Sprint 186/209
> **Imposed rule:** Entropy evidence must be included in scan exports and DSSE attestations; opaque regions without provenance cannot be whitelisted without an explicit policy waiver.
> **Status:** Stable (2025-11)
> **Owners:** Scanner Guild · Policy Guild · UI Guild · Docs Guild
## 1. Overview
@@ -25,10 +26,11 @@ Entropy analysis highlights opaque regions inside container layers (packed binar
* Flag entire files with no symbols or stripped debug info.
* Detect known packer section names (`.UPX*`, `.aspack`, etc.).
* Record offsets, window sizes, and entropy values to support explainability.
* **Outputs:**
* `entropy.report.json` (per-file details, windows, hints).
* `layer_summary.json` (opaque byte ratios per layer and overall image).
* Penalty score contributed to the trust algebra (`entropy_penalty`).
* **Outputs (all canonical, UTF-8, sorted keys):**
* `entropy.report.json` (per-file details, windows, hints; schema `stellaops.entropy/report@1`).
* `layer_summary.json` (opaque byte ratios per layer and overall image; schema `stellaops.entropy/layer-summary@1`).
* `entropy_penalty` scalar injected into trust lattice inputs.
* All outputs are signed within the scan DSSE bundle and exported in Offline/Replay kits.
All JSON output is canonical (sorted keys, UTF-8) and included in DSSE attestations/replay bundles.
@@ -39,6 +41,7 @@ All JSON output is canonical (sorted keys, UTF-8) and included in DSSE attestati
```jsonc
{
"schema": "stellaops.entropy/report@1",
"generatedAt": "2025-11-26T12:00:00Z",
"imageDigest": "sha256:…",
"layerDigest": "sha256:…",
"files": [
@@ -62,6 +65,7 @@ All JSON output is canonical (sorted keys, UTF-8) and included in DSSE attestati
```jsonc
{
"schema": "stellaops.entropy/layer-summary@1",
"generatedAt": "2025-11-26T12:00:00Z",
"imageDigest": "sha256:…",
"layers": [
{
@@ -80,12 +84,13 @@ All JSON output is canonical (sorted keys, UTF-8) and included in DSSE attestati
## 4. Policy integration (`POLICY-RISK-90-001`)
* Policy Engine receives `entropy_penalty` and per-layer ratios via scan evidence.
* Default thresholds:
* Block when `imageOpaqueRatio > 0.15` and provenance unknown.
* Default thresholds (tenant-overridable):
* Block when `imageOpaqueRatio > 0.15` **and** provenance unknown.
* Warn when any executable has `opaqueRatio > 0.30`.
* Penalty weights are configurable per tenant. Policy explanations include:
* Highest-entropy files and offsets.
* Reason code (packed, no symbols, runtime reachable).
* Suppress penalty when symbols are present **and** provenance attested.
* Trust lattice mapping:
* `entropy_penalty` feeds the risk lattice alongside reachability, provenance, and exploitability signals; capped at 0.3.
* Policy explanations include highest-entropy files, offsets, and reason codes (packed, no symbols, runtime reachable).
## 5. UI experience (`UI-ENTROPY-40-001/002`)
@@ -97,21 +102,22 @@ All JSON output is canonical (sorted keys, UTF-8) and included in DSSE attestati
## 6. CLI / API hooks
* CLI `stella scan artifacts --entropy` option prints top opaque files and penalties.
* API `GET /api/v1/scans/{id}/entropy` serves summary + evidence references.
* Notify templates can include entropy penalties to escalate opaque images.
* CLI `stella scan artifacts --entropy --threshold 0.15 --top 10` prints top opaque files and penalty; exits non-zero when penalty exceeds threshold.
* CLI `stella scan export --include entropy` bundles entropy reports with SBOM/VEX for Offline kits.
* API `GET /api/v1/scans/{id}/entropy` serves summary + evidence references; supports `Accept: application/json` or NDJSON stream.
* Notify templates can include entropy penalties to escalate opaque images (channel: Ops/Sec).
## 7. Trust algebra
The penalty is computed as:
\[
\text{entropyPenalty} = K \sum_{\text{layers}} \left( \frac{\text{opaqueBytes}}{\text{totalBytes}} \times \frac{\text{layerBytes}}{\text{imageBytes}} \right)
\text{entropyPenalty} = \min\Bigg(0.3,\; K \sum_{\text{layers}} \big( \frac{\text{opaqueBytes}}{\text{totalBytes}} \times \frac{\text{layerBytes}}{\text{imageBytes}} \big)\Bigg)
\]
* Default `K = 0.5`.
* Cap penalty at 0.3 to avoid over-weighting tiny blobs.
* Combine with other trust signals (reachability, provenance) to prioritise audits.
* Default `K = 0.5`; tenants can override via policy lattice config.
* If symbols are present and attested, apply a 0.5 multiplier to the per-layer contribution.
* Combine with reachability and provenance weights before final risk verdict.
## 8. Implementation checklist

View File

@@ -0,0 +1,10 @@
# SDK Language Support Matrix (Wave B draft · 2025-11-26)
| Language | Package name (staging) | Status | Generator config | Post-processing helpers | Notes |
| --- | --- | --- | --- | --- | --- |
| TypeScript | `@stellaops/sdk` (staging) | Alpha pending OAS freeze | `src/Sdk/StellaOps.Sdk.Generator/ts/config.yaml` | Auth, retry, pagination, telemetry wired | ESM+CJS templates ready; fixture smoke test passes |
| Python | `stellaops-sdk` (staging) | Alpha pending OAS freeze | `src/Sdk/StellaOps.Sdk.Generator/py/config.yaml` | Auth, retry, pagination, telemetry wired | Sync/async clients scaffolded; fixture smoke test passes |
| Go | `github.com/stellaops/sdk-go` | Planned (after Python) | Generator script stub | Helpers planned | Context-first API, streaming helpers; starts after SDKGEN-63-002 |
| Java | `io.stellaops:sdk` | Planned (after Go) | Generator script stub | Helpers planned | HTTP client abstraction TBD; starts after SDKGEN-63-003 |
Evidence for action tracker: Sprint 0208 · Action #2 (Publish SDK language support matrix to CLI/UI guilds).