up
Some checks failed
api-governance / spectral-lint (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled
oas-ci / oas-validate (push) Has been cancelled
SDK Publish & Sign / sdk-publish (push) Has been cancelled
Policy Lint & Smoke / policy-lint (push) Has been cancelled
Policy Simulation / policy-simulate (push) Has been cancelled
devportal-offline / build-offline (push) Has been cancelled
Some checks failed
api-governance / spectral-lint (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled
oas-ci / oas-validate (push) Has been cancelled
SDK Publish & Sign / sdk-publish (push) Has been cancelled
Policy Lint & Smoke / policy-lint (push) Has been cancelled
Policy Simulation / policy-simulate (push) Has been cancelled
devportal-offline / build-offline (push) Has been cancelled
This commit is contained in:
74
docs/modules/export-center/devportal-offline-manifest.md
Normal file
74
docs/modules/export-center/devportal-offline-manifest.md
Normal file
@@ -0,0 +1,74 @@
|
||||
# DevPortal Offline Bundle Manifest (draft v0.1)
|
||||
|
||||
Applies to sprint: SPRINT_0206_0001_0001_devportal · Action #2 (DEVPORT-64-001/64-002 interlock with Export Center)
|
||||
|
||||
## Purpose
|
||||
- Define a deterministic, air-gap-friendly manifest for Developer Portal offline bundles.
|
||||
- Ensure SDK archives, OpenAPI specs, and static site assets can be verified and consumed by Export Center and SDK Release pipelines.
|
||||
|
||||
## Bundle layout
|
||||
```
|
||||
devportal-offline/
|
||||
manifest.json # see schema below
|
||||
site/ # static HTML/CSS/JS (Astro/Starlight build)
|
||||
specs/
|
||||
stella-aggregate.yaml # merged OpenAPI used by portal
|
||||
*.yaml # per-service OpenAPI (authority, scanner, policy, graph, etc.)
|
||||
sdks/
|
||||
node-sdk.tar.gz
|
||||
python-sdk.tar.gz
|
||||
java-sdk.zip # optional, language-dependent
|
||||
assets/
|
||||
fonts/* # self-hosted; no external CDNs
|
||||
icons/* # SVG/PNG used by site
|
||||
```
|
||||
|
||||
## Manifest schema (manifest.json)
|
||||
```json
|
||||
{
|
||||
"version": "0.1",
|
||||
"generatedAt": "2025-11-26T00:00:00Z",
|
||||
"site": {
|
||||
"path": "site",
|
||||
"sha256": "<hex>",
|
||||
"bytes": 0
|
||||
},
|
||||
"specs": [
|
||||
{ "name": "stella-aggregate.yaml", "path": "specs/stella-aggregate.yaml", "sha256": "<hex>", "bytes": 0 },
|
||||
{ "name": "authority.yaml", "path": "specs/authority.yaml", "sha256": "<hex>", "bytes": 0 }
|
||||
],
|
||||
"sdks": [
|
||||
{ "name": "node-sdk", "path": "sdks/node-sdk.tar.gz", "sha256": "<hex>", "bytes": 0 },
|
||||
{ "name": "python-sdk", "path": "sdks/python-sdk.tar.gz", "sha256": "<hex>", "bytes": 0 }
|
||||
],
|
||||
"checks": {
|
||||
"integrity": "sha256",
|
||||
"policy": "no-external-assets"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Rules
|
||||
- `version` is additive; bump on breaking shape change.
|
||||
- `sha256` is hex lowercase of the file contents.
|
||||
- `bytes` is the exact byte length for deterministic validation.
|
||||
- `checks.policy` documents applied constraints; default `no-external-assets` (verify no `http(s)://` fonts/scripts).
|
||||
|
||||
## Production contract
|
||||
- Export Center expects `manifest.json` at bundle root; validates sha256/bytes before publishing.
|
||||
- Offline bundle must build via `npm run build:offline` without network calls after initial `npm ci` + `npm run sync:spec`.
|
||||
- Specs and SDK archives are treated as opaque; manifest carries their hashes for downstream verification.
|
||||
|
||||
## Open items
|
||||
- Add per-language SDK metadata (version, commit SHA) once SDKREL-64-002 finalises.
|
||||
- Add optional `signatures` array (DSSE over manifest) when Authority signing profile is ready.
|
||||
|
||||
## How to produce locally (deterministic)
|
||||
```
|
||||
npm ci --ignore-scripts --no-fund --no-audit
|
||||
npm run sync:spec
|
||||
npm run build:offline
|
||||
# compute manifest hashes using sha256sum and fill manifest.json
|
||||
```
|
||||
|
||||
Record generated manifest in sprint evidence when produced; keep caches local to avoid external fetches.
|
||||
@@ -180,22 +180,38 @@ sequenceDiagram
|
||||
3. Re-run integrity checks (`mirror verify <path>`).
|
||||
- **Audit logging:** Export Center logs `mirror.bundle.created`, `mirror.delta.applied`, and `mirror.encryption.enabled` events. Consume them in the central observability pipeline.
|
||||
|
||||
## 7. Troubleshooting
|
||||
|
||||
| Symptom | Meaning | Action |
|
||||
|---------|---------|--------|
|
||||
| `ERR_EXPORT_BASE_MISSING` | Base export not available | Republish base bundle or rebuild as full export. |
|
||||
| Delta applies but mirror misses entries | Deltas applied out of order | Rebuild from last full bundle and reapply in sequence. |
|
||||
## 7. Validation checklist (Trivy / mirror bundles)
|
||||
|
||||
- Download and verify:
|
||||
- `stella export download <exportId> --format mirror`
|
||||
- `stella export verify <exportId>`
|
||||
- Delta ordering:
|
||||
- Ensure `manifest.diff.json.baseExportId` exists locally before applying delta.
|
||||
- Track applied order in `appliedExportIds.log`.
|
||||
- Trivy adapter (if enabled):
|
||||
- `stella export trivy-validate --bundle mirror-YYYYMMDD.tar.zst --policy ./policies/export-center.rego`
|
||||
- Dry-run import:
|
||||
- `stella export mirror-validate --bundle mirror-YYYYMMDD.tar.zst --dry-run`
|
||||
- Post-import checks:
|
||||
- Recompute SHA256 for `manifest.yaml` and a sample data file; compare to manifest.
|
||||
- Run `mirror verify` (Offline Kit) and confirm zero mismatches.
|
||||
|
||||
## 8. Troubleshooting
|
||||
|
||||
| Symptom | Meaning | Action |
|
||||
|---------|---------|--------|
|
||||
| `ERR_EXPORT_BASE_MISSING` | Base export not available | Republish base bundle or rebuild as full export. |
|
||||
| Delta applies but mirror misses entries | Deltas applied out of order | Rebuild from last full bundle and reapply in sequence. |
|
||||
| Decryption fails | Recipient key mismatch or corrupted bundle | Confirm key distribution and re-download bundle. |
|
||||
| Verification errors | Signature mismatch | Do not import; regenerate bundle and investigate signing pipeline. |
|
||||
| Manifest hash mismatch | Files changed after extraction | Re-extract bundle and re-run verification; check storage tampering. |
|
||||
|
||||
## 8. References
|
||||
|
||||
- [Export Center Overview](overview.md)
|
||||
- [Export Center Architecture](architecture.md)
|
||||
- [Export Center API reference](api.md)
|
||||
- [Export Center CLI Guide](cli.md)
|
||||
| Verification errors | Signature mismatch | Do not import; regenerate bundle and investigate signing pipeline. |
|
||||
| Manifest hash mismatch | Files changed after extraction | Re-extract bundle and re-run verification; check storage tampering. |
|
||||
|
||||
## 9. References
|
||||
|
||||
- [Export Center Overview](overview.md)
|
||||
- [Export Center Architecture](architecture.md)
|
||||
- [Export Center API reference](api.md)
|
||||
- [Export Center CLI Guide](cli.md)
|
||||
- [Concelier mirror runbook](../concelier/operations/mirror.md)
|
||||
- [Aggregation-Only Contract reference](../../ingestion/aggregation-only-contract.md)
|
||||
|
||||
|
||||
@@ -53,6 +53,7 @@ Graph Indexer + Graph API build the tenant-scoped knowledge graph that powers bl
|
||||
- [`architecture.md`](architecture.md) — inputs, pipelines, APIs, storage choices, observability, offline handling.
|
||||
- [`implementation_plan.md`](implementation_plan.md) — phased delivery roadmap, work breakdown, risks, test strategy.
|
||||
- [`schema.md`](schema.md) — canonical node/edge schema and attribute dictionary (keep in sync with indexer code).
|
||||
- API surface: `docs/api/graph-gateway-spec-draft.yaml` (NDJSON tiles for `/graph/search|query|paths|diff|export`, budgets, overlays).
|
||||
- Updates: `docs/updates/2025-10-26-scheduler-graph-jobs.md`, `docs/updates/2025-10-26-authority-graph-scopes.md`, `docs/updates/2025-10-30-devops-governance.md` for the latest decisions/dependencies.
|
||||
- Index: see `architecture-index.md` for data model, ingestion pipeline, overlays/caches, events, and API/observability pointers.
|
||||
|
||||
|
||||
@@ -12,7 +12,7 @@
|
||||
- `Advisory` and `VEXStatement` nodes linking to Concelier/Excititor records via digests.
|
||||
- `PolicyVersion` nodes representing signed policy packs.
|
||||
- **Edges:** directed, timestamped relationships such as `DEPENDS_ON`, `BUILT_FROM`, `DECLARED_IN`, `AFFECTED_BY`, `VEX_EXEMPTS`, `GOVERNS_WITH`, `OBSERVED_RUNTIME`. Each edge carries provenance (SRM hash, SBOM digest, policy run ID).
|
||||
- **Overlays:** computed index tables providing fast access to reachability, blast radius, and differential views (e.g., `graph_overlay/vuln/{tenant}/{advisoryKey}`).
|
||||
- **Overlays:** computed index tables providing fast access to reachability, blast radius, and differential views (e.g., `graph_overlay/vuln/{tenant}/{advisoryKey}`). Runtime endpoints emit overlays inline (`policy.overlay.v1`, `openvex.v1`) with deterministic overlay IDs (`sha256(tenant|nodeId|overlayKind)`) and sampled explain traces on policy overlays.
|
||||
|
||||
## 2) Pipelines
|
||||
|
||||
@@ -24,11 +24,12 @@
|
||||
|
||||
## 3) APIs
|
||||
|
||||
- `GET /graph/nodes/{id}` — fetch node with metadata and attached provenance.
|
||||
- `POST /graph/query/saved` — execute saved query (Cypher-like DSL) with tenant filtering; supports paging, citation metadata, and `explain` traces.
|
||||
- `GET /graph/impact/{advisoryKey}` — returns impacted artifacts with path context and policy/vex overlays.
|
||||
- `GET /graph/diff/{snapshotA}/{snapshotB}` — streaming API returning diff manifest including new/removed edges, risk summary, and export references.
|
||||
- `POST /graph/overlay/policy` — create or retrieve overlay for policy version + advisory set, referencing `effective_finding` results.
|
||||
- `POST /graph/search` — NDJSON node tiles with cursor paging, tenant + scope guards.
|
||||
- `POST /graph/query` — NDJSON nodes/edges/stats/cursor with budgets (tiles/nodes/edges) and optional inline overlays (`includeOverlays=true`) emitting `policy.overlay.v1` and `openvex.v1` payloads; overlay IDs are `sha256(tenant|nodeId|overlayKind)`; policy overlay may include a sampled `explainTrace`.
|
||||
- `POST /graph/paths` — bounded BFS (depth ≤6) returning path nodes/edges/stats; honours budgets and overlays.
|
||||
- `POST /graph/diff` — compares `snapshotA` vs `snapshotB`, streaming node/edge added/removed/changed tiles plus stats; budget enforcement mirrors `/graph/query`.
|
||||
- `POST /graph/export` — async job producing deterministic manifests (`sha256`, size, format) for `ndjson/csv/graphml/png/svg`; download via `/graph/export/{jobId}`.
|
||||
- Legacy: `GET /graph/nodes/{id}`, `POST /graph/query/saved`, `GET /graph/impact/{advisoryKey}`, `POST /graph/overlay/policy` remain in spec but should align to the NDJSON surfaces above as they are brought forward.
|
||||
|
||||
## 4) Storage considerations
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Scanner Determinism Score Guide
|
||||
|
||||
> **Status:** Draft – Sprint 186/202/203
|
||||
> **Status:** Stable (2025-11)
|
||||
> **Owners:** Scanner Guild · QA Guild · DevEx/CLI Guild · DevOps Guild
|
||||
|
||||
## 1. Goal
|
||||
@@ -22,36 +22,29 @@ Quantify how repeatable a scanner release is by re-running scans under frozen co
|
||||
|
||||
The harness persists the full run set under CAS, allowing regression tests and Offline kit inclusion.
|
||||
|
||||
## 3. Output artefacts (`SCAN-DETER-186-010`)
|
||||
## 3. determinism.json schema (`SCAN-DETER-186-010`)
|
||||
|
||||
* `determinism.json` – per-image runs, identical counts, score, policy/feed hashes.
|
||||
* `run_i/*.json` – canonicalised outputs for debugging.
|
||||
* `diffs/` – optional diff samples when runs diverge.
|
||||
Required fields:
|
||||
|
||||
Example `determinism.json`:
|
||||
- `version` (string, default `1`)
|
||||
- `release` (string; e.g., `scanner-0.14.3`)
|
||||
- `platform` (`os/arch`)
|
||||
- `policy_sha`, `feeds_sha`, `scanner_sha` (content-addressed digests)
|
||||
- `images[]`:
|
||||
- `digest` (image digest)
|
||||
- `runs` (int), `identical` (int), `score` (float 0–1)
|
||||
- `artifact_hashes` (map filename → sha256)
|
||||
- `non_deterministic` (array of artefact names when diverged)
|
||||
- `notes` (optional string)
|
||||
- `overall_score` (float 0–1)
|
||||
- `thresholds` { `image_min`:0.90, `overall_min`:0.95 }
|
||||
- `generated_at` (UTC ISO-8601)
|
||||
|
||||
```json
|
||||
{
|
||||
"release": "scanner-0.14.3",
|
||||
"platform": "linux/amd64",
|
||||
"policy_sha": "a1b2c3…",
|
||||
"feeds_sha": "d4e5f6…",
|
||||
"images": [
|
||||
{
|
||||
"digest": "sha256:abc…",
|
||||
"runs": 10,
|
||||
"identical": 10,
|
||||
"score": 1.0,
|
||||
"artifact_hashes": {
|
||||
"sbom.cdx.json": "sha256:11…",
|
||||
"vex.json": "sha256:22…",
|
||||
"findings.json": "sha256:33…"
|
||||
}
|
||||
}
|
||||
],
|
||||
"overall_score": 1.0
|
||||
}
|
||||
```
|
||||
Output bundle layout:
|
||||
|
||||
- `determinism.json` – schema above
|
||||
- `run_i/*.json` – canonicalised artefacts per run
|
||||
- `diffs/` – minimal diffs when divergence occurs
|
||||
|
||||
## 4. CI integration (`DEVOPS-SCAN-90-004`)
|
||||
|
||||
@@ -71,8 +64,12 @@ Example `determinism.json`:
|
||||
|
||||
## 7. Evidence & replay
|
||||
|
||||
* Include `determinism.json` and canonical run outputs in Replay bundles (`docs/replay/DETERMINISTIC_REPLAY.md`).
|
||||
* DSSE-sign determinism results before adding them to Evidence Locker.
|
||||
Replay instructions (deterministic, offline-friendly):
|
||||
|
||||
1. Fetch a published determinism bundle (`determinism.json` + `run_*`) and the corresponding scanner image.
|
||||
2. Run `scripts/scanner/determinism-run.sh --replay --bundle path/to/bundle --image sha256:...` (uses fixed clock/seed).
|
||||
3. Verify hashes match the recorded `artifact_hashes`; tool exits non-zero on mismatch.
|
||||
4. Attach the bundle to Evidence Locker with DSSE signature before export.
|
||||
|
||||
## 8. Implementation checklist
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
# Entropy Analysis for Executable Layers
|
||||
|
||||
> **Status:** Draft – Sprint 186/209
|
||||
> **Imposed rule:** Entropy evidence must be included in scan exports and DSSE attestations; opaque regions without provenance cannot be whitelisted without an explicit policy waiver.
|
||||
> **Status:** Stable (2025-11)
|
||||
> **Owners:** Scanner Guild · Policy Guild · UI Guild · Docs Guild
|
||||
|
||||
## 1. Overview
|
||||
@@ -25,10 +26,11 @@ Entropy analysis highlights opaque regions inside container layers (packed binar
|
||||
* Flag entire files with no symbols or stripped debug info.
|
||||
* Detect known packer section names (`.UPX*`, `.aspack`, etc.).
|
||||
* Record offsets, window sizes, and entropy values to support explainability.
|
||||
* **Outputs:**
|
||||
* `entropy.report.json` (per-file details, windows, hints).
|
||||
* `layer_summary.json` (opaque byte ratios per layer and overall image).
|
||||
* Penalty score contributed to the trust algebra (`entropy_penalty`).
|
||||
* **Outputs (all canonical, UTF-8, sorted keys):**
|
||||
* `entropy.report.json` (per-file details, windows, hints; schema `stellaops.entropy/report@1`).
|
||||
* `layer_summary.json` (opaque byte ratios per layer and overall image; schema `stellaops.entropy/layer-summary@1`).
|
||||
* `entropy_penalty` scalar injected into trust lattice inputs.
|
||||
* All outputs are signed within the scan DSSE bundle and exported in Offline/Replay kits.
|
||||
|
||||
All JSON output is canonical (sorted keys, UTF-8) and included in DSSE attestations/replay bundles.
|
||||
|
||||
@@ -39,6 +41,7 @@ All JSON output is canonical (sorted keys, UTF-8) and included in DSSE attestati
|
||||
```jsonc
|
||||
{
|
||||
"schema": "stellaops.entropy/report@1",
|
||||
"generatedAt": "2025-11-26T12:00:00Z",
|
||||
"imageDigest": "sha256:…",
|
||||
"layerDigest": "sha256:…",
|
||||
"files": [
|
||||
@@ -62,6 +65,7 @@ All JSON output is canonical (sorted keys, UTF-8) and included in DSSE attestati
|
||||
```jsonc
|
||||
{
|
||||
"schema": "stellaops.entropy/layer-summary@1",
|
||||
"generatedAt": "2025-11-26T12:00:00Z",
|
||||
"imageDigest": "sha256:…",
|
||||
"layers": [
|
||||
{
|
||||
@@ -80,12 +84,13 @@ All JSON output is canonical (sorted keys, UTF-8) and included in DSSE attestati
|
||||
## 4. Policy integration (`POLICY-RISK-90-001`)
|
||||
|
||||
* Policy Engine receives `entropy_penalty` and per-layer ratios via scan evidence.
|
||||
* Default thresholds:
|
||||
* Block when `imageOpaqueRatio > 0.15` and provenance unknown.
|
||||
* Default thresholds (tenant-overridable):
|
||||
* Block when `imageOpaqueRatio > 0.15` **and** provenance unknown.
|
||||
* Warn when any executable has `opaqueRatio > 0.30`.
|
||||
* Penalty weights are configurable per tenant. Policy explanations include:
|
||||
* Highest-entropy files and offsets.
|
||||
* Reason code (packed, no symbols, runtime reachable).
|
||||
* Suppress penalty when symbols are present **and** provenance attested.
|
||||
* Trust lattice mapping:
|
||||
* `entropy_penalty` feeds the risk lattice alongside reachability, provenance, and exploitability signals; capped at 0.3.
|
||||
* Policy explanations include highest-entropy files, offsets, and reason codes (packed, no symbols, runtime reachable).
|
||||
|
||||
## 5. UI experience (`UI-ENTROPY-40-001/002`)
|
||||
|
||||
@@ -97,21 +102,22 @@ All JSON output is canonical (sorted keys, UTF-8) and included in DSSE attestati
|
||||
|
||||
## 6. CLI / API hooks
|
||||
|
||||
* CLI – `stella scan artifacts --entropy` option prints top opaque files and penalties.
|
||||
* API – `GET /api/v1/scans/{id}/entropy` serves summary + evidence references.
|
||||
* Notify templates can include entropy penalties to escalate opaque images.
|
||||
* CLI – `stella scan artifacts --entropy --threshold 0.15 --top 10` prints top opaque files and penalty; exits non-zero when penalty exceeds threshold.
|
||||
* CLI – `stella scan export --include entropy` bundles entropy reports with SBOM/VEX for Offline kits.
|
||||
* API – `GET /api/v1/scans/{id}/entropy` serves summary + evidence references; supports `Accept: application/json` or NDJSON stream.
|
||||
* Notify templates can include entropy penalties to escalate opaque images (channel: Ops/Sec).
|
||||
|
||||
## 7. Trust algebra
|
||||
|
||||
The penalty is computed as:
|
||||
|
||||
\[
|
||||
\text{entropyPenalty} = K \sum_{\text{layers}} \left( \frac{\text{opaqueBytes}}{\text{totalBytes}} \times \frac{\text{layerBytes}}{\text{imageBytes}} \right)
|
||||
\text{entropyPenalty} = \min\Bigg(0.3,\; K \sum_{\text{layers}} \big( \frac{\text{opaqueBytes}}{\text{totalBytes}} \times \frac{\text{layerBytes}}{\text{imageBytes}} \big)\Bigg)
|
||||
\]
|
||||
|
||||
* Default `K = 0.5`.
|
||||
* Cap penalty at 0.3 to avoid over-weighting tiny blobs.
|
||||
* Combine with other trust signals (reachability, provenance) to prioritise audits.
|
||||
* Default `K = 0.5`; tenants can override via policy lattice config.
|
||||
* If symbols are present and attested, apply a 0.5 multiplier to the per-layer contribution.
|
||||
* Combine with reachability and provenance weights before final risk verdict.
|
||||
|
||||
## 8. Implementation checklist
|
||||
|
||||
|
||||
10
docs/modules/sdk/language-support-matrix.md
Normal file
10
docs/modules/sdk/language-support-matrix.md
Normal file
@@ -0,0 +1,10 @@
|
||||
# SDK Language Support Matrix (Wave B draft · 2025-11-26)
|
||||
|
||||
| Language | Package name (staging) | Status | Generator config | Post-processing helpers | Notes |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| TypeScript | `@stellaops/sdk` (staging) | Alpha pending OAS freeze | `src/Sdk/StellaOps.Sdk.Generator/ts/config.yaml` | Auth, retry, pagination, telemetry wired | ESM+CJS templates ready; fixture smoke test passes |
|
||||
| Python | `stellaops-sdk` (staging) | Alpha pending OAS freeze | `src/Sdk/StellaOps.Sdk.Generator/py/config.yaml` | Auth, retry, pagination, telemetry wired | Sync/async clients scaffolded; fixture smoke test passes |
|
||||
| Go | `github.com/stellaops/sdk-go` | Planned (after Python) | Generator script stub | Helpers planned | Context-first API, streaming helpers; starts after SDKGEN-63-002 |
|
||||
| Java | `io.stellaops:sdk` | Planned (after Go) | Generator script stub | Helpers planned | HTTP client abstraction TBD; starts after SDKGEN-63-003 |
|
||||
|
||||
Evidence for action tracker: Sprint 0208 · Action #2 (Publish SDK language support matrix to CLI/UI guilds).
|
||||
Reference in New Issue
Block a user