stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity

docs consolidation work

Browse Source
This commit is contained in:
StellaOps Bot
2025-12-25 10:53:53 +02:00
parent b9f71fc7e9
commit deb82b4f03
117 changed files with 852 additions and 847 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/modules/export-center/AGENTS.md
View File

@@ -17,7 +17,7 @@ Export Center packages reproducible evidence bundles (JSON, Trivy DB, mirror) wi
4. Coordinate cross-module changes in the main /AGENTS.md description and through the sprint plan.
## Guardrails
- Honour the Aggregation-Only Contract where applicable (see ../../ingestion/aggregation-only-contract.md).
- Honour the Aggregation-Only Contract where applicable (see ../../aoc/aggregation-only-contract.md).
- Preserve determinism: sort outputs, normalise timestamps (UTC ISO-8601), and avoid machine-specific artefacts.
- Keep Offline Kit parity in mind—document air-gapped workflows for any new feature.
- Update runbooks/observability assets when operational characteristics change.

132
docs/modules/export-center/api.md
View File

@@ -16,8 +16,8 @@ This reference describes the Export Center API introduced in Export Center Phase
- `export:download` for bundle downloads and manifests.
- **Tenant context:** Provide `X-Stella-Tenant` when the token carries multiple tenants; defaults to token tenant otherwise.
- **Idempotency:** Mutating endpoints accept `Idempotency-Key` (UUID). Retrying with the same key returns the original result.
- **Rate limits and quotas:** Responses include `X-Stella-Quota-Limit`, `X-Stella-Quota-Remaining`, and `X-Stella-Quota-Reset`. Exceeding quotas returns `429 Too Many Requests` with `ERR_EXPORT_QUOTA`.
- **Integrity headers (downloads):** `Digest: sha-256=<base64>`, `X-Stella-Signature: dsse-b64=<payload>`, and `X-Stella-Immutability: true` accompany bundle/manifest downloads; clients must validate before use.
- **Rate limits and quotas:** Responses include `X-Stella-Quota-Limit`, `X-Stella-Quota-Remaining`, and `X-Stella-Quota-Reset`. Exceeding quotas returns `429 Too Many Requests` with `ERR_EXPORT_QUOTA`.
- **Integrity headers (downloads):** `Digest: sha-256=<base64>`, `X-Stella-Signature: dsse-b64=<payload>`, and `X-Stella-Immutability: true` accompany bundle/manifest downloads; clients must validate before use.
- **Content negotiation:** Requests and responses use `application/json; charset=utf-8` unless otherwise stated. Downloads stream binary content with profile-specific media types.
- **SSE:** Event streams set `Content-Type: text/event-stream` and keep connections alive with comment heartbeats every 15 seconds.
@@ -101,29 +101,29 @@ Scopes: export:profile:manage
**Request**
```json
{
"profileId": "prof-airgap-mirror",
"name": "Airgap Mirror Weekly",
"kind": "mirror",
"variant": "full",
"include": ["advisories", "vex", "sboms", "policy"],
"distribution": ["http", "object"],
"encryption": {
"enabled": true,
"recipientKeys": ["age1tenantkey..."],
"strict": false
},
"retention": {"mode": "days", "value": 30},
"limits": {
"maxActiveRuns": 4,
"maxQueuedRuns": 50,
"backpressureMode": "reject"
},
"approval": {
"required": false
}
}
```
{
"profileId": "prof-airgap-mirror",
"name": "Airgap Mirror Weekly",
"kind": "mirror",
"variant": "full",
"include": ["advisories", "vex", "sboms", "policy"],
"distribution": ["http", "object"],
"encryption": {
"enabled": true,
"recipientKeys": ["age1tenantkey..."],
"strict": false
},
"retention": {"mode": "days", "value": 30},
"limits": {
"maxActiveRuns": 4,
"maxQueuedRuns": 50,
"backpressureMode": "reject"
},
"approval": {
"required": false
}
}
```
**Response 201**
@@ -192,24 +192,24 @@ Scopes: export:run
{
"runId": "run-20251029-01",
"status": "pending",
"profileId": "prof-json-raw",
"createdAt": "2025-10-29T12:12:11Z",
"createdBy": "user:ops",
"selectors": { "...": "..." },
"links": {
"self": "/api/export/runs/run-20251029-01",
"events": "/api/export/runs/run-20251029-01/events"
},
"quotas": {
"maxActiveRuns": 4,
"maxQueuedRuns": 50,
"backpressureMode": "reject"
},
"approval": {
"required": false
}
}
```
"profileId": "prof-json-raw",
"createdAt": "2025-10-29T12:12:11Z",
"createdBy": "user:ops",
"selectors": { "...": "..." },
"links": {
"self": "/api/export/runs/run-20251029-01",
"events": "/api/export/runs/run-20251029-01/events"
},
"quotas": {
"maxActiveRuns": 4,
"maxQueuedRuns": 50,
"backpressureMode": "reject"
},
"approval": {
"required": false
}
}
```
### 4.2 List runs
@@ -231,15 +231,15 @@ Response fields:
| Field | Description |
|-------|-------------|
| `status` | `pending`, `running`, `success`, `failed`, `canceled`. |
| `progress` | Object with `adapters`, `bytesWritten`, `recordsProcessed`. |
| `errorCode` | Populated when `status=failed` (`signing`, `distribution`, etc). |
| `policySnapshotId` | Returned for policy-aware profiles. |
| `distributions` | List of available distribution descriptors (type, location, sha256, expiresAt). |
| `rerunHash` | SHA-256 over sorted `contents[*].digest`; used for determinism checks. |
| `integrity` | Expected HTTP headers (`Digest`, `X-Stella-Signature`, `X-Stella-Immutability`) and OCI annotations (`io.stellaops.export.*`). |
| `quotas` | Active limits/backpressure settings returned with the run. |
| `approval` | Cross-tenant approval ticket when selectors span multiple tenants/wildcards. |
| `status` | `pending`, `running`, `success`, `failed`, `canceled`. |
| `progress` | Object with `adapters`, `bytesWritten`, `recordsProcessed`. |
| `errorCode` | Populated when `status=failed` (`signing`, `distribution`, etc). |
| `policySnapshotId` | Returned for policy-aware profiles. |
| `distributions` | List of available distribution descriptors (type, location, sha256, expiresAt). |
| `rerunHash` | SHA-256 over sorted `contents[*].digest`; used for determinism checks. |
| `integrity` | Expected HTTP headers (`Digest`, `X-Stella-Signature`, `X-Stella-Immutability`) and OCI annotations (`io.stellaops.export.*`). |
| `quotas` | Active limits/backpressure settings returned with the run. |
| `approval` | Cross-tenant approval ticket when selectors span multiple tenants/wildcards. |
### 4.4 Cancel a run
@@ -294,16 +294,16 @@ GET /api/export/runs/{runId}/download
Scopes: export:download
```
Streams the primary bundle (tarball, zip, or profile-specific layout). Headers:
- `Content-Disposition: attachment; filename="export-run-20251029-01.tar.zst"`
- `Digest: sha-256=<base64>` (EC5)
- `X-Stella-Signature: dsse-b64:<payload>` (EC3/EC5)
- `X-Stella-Immutability: true`
- `X-Export-Size: 73482019`
- `X-Export-Encryption: age` (when mirror encryption enabled)
Supports HTTP range requests for resume functionality. If no bundle exists yet, responds `409` with `ERR_EXPORT_007`.
Streams the primary bundle (tarball, zip, or profile-specific layout). Headers:
- `Content-Disposition: attachment; filename="export-run-20251029-01.tar.zst"`
- `Digest: sha-256=<base64>` (EC5)
- `X-Stella-Signature: dsse-b64:<payload>` (EC3/EC5)
- `X-Stella-Immutability: true`
- `X-Export-Size: 73482019`
- `X-Export-Encryption: age` (when mirror encryption enabled)
Supports HTTP range requests for resume functionality. If no bundle exists yet, responds `409` with `ERR_EXPORT_007`.
### 6.2 Manifest download
@@ -312,8 +312,8 @@ GET /api/export/runs/{runId}/manifest
Scopes: export:download
```
Returns signed `export.json`. To fetch the detached signature, append `?signature=true`.
- Integrity annotations are mirrored in response headers (`Digest`, `X-Stella-Signature`, `X-Stella-Immutability`) and in the manifest `integrity` block to keep rerun-hash deterministic.
Returns signed `export.json`. To fetch the detached signature, append `?signature=true`.
- Integrity annotations are mirrored in response headers (`Digest`, `X-Stella-Signature`, `X-Stella-Immutability`) and in the manifest `integrity` block to keep rerun-hash deterministic.
### 6.3 Provenance download
@@ -322,7 +322,7 @@ GET /api/export/runs/{runId}/provenance
Scopes: export:download
```
Returns signed `provenance.json`. Supports `?signature=true`. Provenance includes attestation subject digests, policy snapshot ids, adapter versions, and KMS key identifiers.
Returns signed `provenance.json`. Supports `?signature=true`. Provenance includes attestation subject digests, policy snapshot ids, adapter versions, and KMS key identifiers.
### 6.4 Distribution descriptors
@@ -356,6 +356,6 @@ Payload includes `targetUrl`, `events` (e.g., `run.succeeded`), and optional sec
- [Export Center Architecture](architecture.md)
- [Export Center Profiles](profiles.md)
- [Export Center CLI Guide](cli.md) *(companion document)*
- [Aggregation-Only Contract reference](../../ingestion/aggregation-only-contract.md)
- [Aggregation-Only Contract reference](../../aoc/aggregation-only-contract.md)
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.

30
docs/modules/export-center/cli.md
View File

@@ -144,20 +144,20 @@ stella export provenance run-20251029-01 --output manifests/provenance.json
Retrieves the signed provenance file. `--signature` behaves like the manifest command.
### 4.4 `stella export verify`
```
stella export verify run-20251029-01 \
--manifest manifests/export.json \
--provenance manifests/provenance.json \
--key keys/acme-export.pub
```
Wrapper around `cosign verify`. Returns exit `0` when signatures and digests validate. Exit `20` when verification fails.
Integrity and determinism checks (EC1–EC10):
- `stella export manifest` and `provenance` commands emit `Digest`/`X-Stella-Signature` headers; cache them for rerun-hash validation.
- Offline kits: run `docs/modules/export-center/operations/verify-export-kit.sh <kit_dir>` to assert rerunHash, integrity headers vs OCI annotations, quotas/backpressure block, approvals, and log metadata in provenance.
### 4.4 `stella export verify`
```
stella export verify run-20251029-01 \
--manifest manifests/export.json \
--provenance manifests/provenance.json \
--key keys/acme-export.pub
```
Wrapper around `cosign verify`. Returns exit `0` when signatures and digests validate. Exit `20` when verification fails.
Integrity and determinism checks (EC1–EC10):
- `stella export manifest` and `provenance` commands emit `Digest`/`X-Stella-Signature` headers; cache them for rerun-hash validation.
- Offline kits: run `docs/modules/export-center/operations/verify-export-kit.sh <kit_dir>` to assert rerunHash, integrity headers vs OCI annotations, quotas/backpressure block, approvals, and log metadata in provenance.
## 5. CI recipe (GitHub Actions example)
@@ -230,6 +230,6 @@ Exit codes above 100 are reserved for future profile-specific tooling.
- [Export Center Profiles](profiles.md)
- [Export Center API reference](api.md)
- [Export Center Architecture](architecture.md)
- [Aggregation-Only Contract reference](../../ingestion/aggregation-only-contract.md)
- [Aggregation-Only Contract reference](../../aoc/aggregation-only-contract.md)
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.

76
docs/modules/export-center/mirror-bundles.md
View File

@@ -92,11 +92,11 @@ delta/
manifest.diff.json # summary of counts, hashes, base export metadata
```
- **Base lookup:** The worker verifies that the base export is reachable (download path or OCI reference). If missing, the run fails with `ERR_EXPORT_BASE_MISSING`.
- **Change detection:** Uses deterministic hashing of normalized records to compute additions/updates. Indexes are regenerated only for affected subjects.
- **Application order:** Consumers apply deltas sequentially. A `resetBaseline=true` flag instructs them to drop cached state and apply the bundle as a full refresh.
- **Tombstones required:** Every removal must emit a tombstone entry plus the `removed` list; deltas without tombstones fail verification (`verify-export-kit.sh`).
- **Integrity headers:** Each delta bundle exports `Digest`, `X-Stella-Signature`, and `X-Stella-Immutability` derived from the OCI annotation `io.stellaops.export.manifest-digest`. Consumers must validate before applying.
- **Base lookup:** The worker verifies that the base export is reachable (download path or OCI reference). If missing, the run fails with `ERR_EXPORT_BASE_MISSING`.
- **Change detection:** Uses deterministic hashing of normalized records to compute additions/updates. Indexes are regenerated only for affected subjects.
- **Application order:** Consumers apply deltas sequentially. A `resetBaseline=true` flag instructs them to drop cached state and apply the bundle as a full refresh.
- **Tombstones required:** Every removal must emit a tombstone entry plus the `removed` list; deltas without tombstones fail verification (`verify-export-kit.sh`).
- **Integrity headers:** Each delta bundle exports `Digest`, `X-Stella-Signature`, and `X-Stella-Immutability` derived from the OCI annotation `io.stellaops.export.manifest-digest`. Consumers must validate before applying.
Example `manifest.diff.json` (delta):
@@ -182,40 +182,40 @@ sequenceDiagram
3. Re-run integrity checks (`mirror verify <path>`).
- **Audit logging:** Export Center logs `mirror.bundle.created`, `mirror.delta.applied`, and `mirror.encryption.enabled` events. Consume them in the central observability pipeline.
## 7. Validation checklist (Trivy / mirror bundles)
- Download and verify:
- `stella export download <exportId> --format mirror`
- `stella export verify <exportId>`
- Delta ordering:
- Ensure `manifest.diff.json.baseExportId` exists locally before applying delta.
- Track applied order in `appliedExportIds.log`.
- Trivy adapter (if enabled):
- `stella export trivy-validate --bundle mirror-YYYYMMDD.tar.zst --policy ./policies/export-center.rego`
- Dry-run import:
- `stella export mirror-validate --bundle mirror-YYYYMMDD.tar.zst --dry-run`
- Post-import checks:
- Recompute SHA256 for `manifest.yaml` and a sample data file; compare to manifest.
- Run `mirror verify` (Offline Kit) and confirm zero mismatches.
- Confirm OCI annotations `io.stellaops.export.profile/run/manifest-digest/provenance-ref` match the bundle being applied.
## 8. Troubleshooting
| Symptom | Meaning | Action |
|---------|---------|--------|
| `ERR_EXPORT_BASE_MISSING` | Base export not available | Republish base bundle or rebuild as full export. |
| Delta applies but mirror misses entries | Deltas applied out of order | Rebuild from last full bundle and reapply in sequence. |
## 7. Validation checklist (Trivy / mirror bundles)
- Download and verify:
- `stella export download <exportId> --format mirror`
- `stella export verify <exportId>`
- Delta ordering:
- Ensure `manifest.diff.json.baseExportId` exists locally before applying delta.
- Track applied order in `appliedExportIds.log`.
- Trivy adapter (if enabled):
- `stella export trivy-validate --bundle mirror-YYYYMMDD.tar.zst --policy ./policies/export-center.rego`
- Dry-run import:
- `stella export mirror-validate --bundle mirror-YYYYMMDD.tar.zst --dry-run`
- Post-import checks:
- Recompute SHA256 for `manifest.yaml` and a sample data file; compare to manifest.
- Run `mirror verify` (Offline Kit) and confirm zero mismatches.
- Confirm OCI annotations `io.stellaops.export.profile/run/manifest-digest/provenance-ref` match the bundle being applied.
## 8. Troubleshooting
| Symptom | Meaning | Action |
|---------|---------|--------|
| `ERR_EXPORT_BASE_MISSING` | Base export not available | Republish base bundle or rebuild as full export. |
| Delta applies but mirror misses entries | Deltas applied out of order | Rebuild from last full bundle and reapply in sequence. |
| Decryption fails | Recipient key mismatch or corrupted bundle | Confirm key distribution and re-download bundle. |
| Verification errors | Signature mismatch | Do not import; regenerate bundle and investigate signing pipeline. |
| Manifest hash mismatch | Files changed after extraction | Re-extract bundle and re-run verification; check storage tampering. |
## 9. References
- [Export Center Overview](overview.md)
- [Export Center Architecture](architecture.md)
- [Export Center API reference](api.md)
- [Export Center CLI Guide](cli.md)
| Verification errors | Signature mismatch | Do not import; regenerate bundle and investigate signing pipeline. |
| Manifest hash mismatch | Files changed after extraction | Re-extract bundle and re-run verification; check storage tampering. |
## 9. References
- [Export Center Overview](overview.md)
- [Export Center Architecture](architecture.md)
- [Export Center API reference](api.md)
- [Export Center CLI Guide](cli.md)
- [Concelier mirror runbook](../concelier/operations/mirror.md)
- [Aggregation-Only Contract reference](../../ingestion/aggregation-only-contract.md)
- [Aggregation-Only Contract reference](../../aoc/aggregation-only-contract.md)
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.

24
docs/modules/export-center/operations/runbook.md
View File

@@ -11,17 +11,17 @@ The Export Center packages StellaOps evidence and policy overlays into reproduci
- Runbook execution for recovery, retention, and compliance.
- Coordination with DevOps validation (cosign + `trivy module db import` smoke tests).
Related documentation:
- `docs/modules/export-center/overview.md`
- `docs/modules/export-center/architecture.md`
- `docs/modules/export-center/profiles.md`
- `docs/modules/export-center/trivy-adapter.md`
- `docs/modules/export-center/mirror-bundles.md`
- `docs/modules/export-center/api.md`
- `docs/modules/export-center/cli.md`
- `docs/modules/export-center/operations/kms-envelope-pattern.md`
- `docs/modules/export-center/operations/risk-bundle-provider-matrix.md`
Related documentation:
- `docs/modules/export-center/overview.md`
- `docs/modules/export-center/architecture.md`
- `docs/modules/export-center/profiles.md`
- `docs/modules/export-center/trivy-adapter.md`
- `docs/modules/export-center/mirror-bundles.md`
- `docs/modules/export-center/api.md`
- `docs/modules/export-center/cli.md`
- `docs/modules/export-center/operations/kms-envelope-pattern.md`
- `docs/modules/export-center/operations/risk-bundle-provider-matrix.md`
## 2. Contacts & tooling
@@ -199,7 +199,7 @@ If encryption enabled, decrypt using age or AES key before verification.
- `docs/modules/export-center/trivy-adapter.md`
- `docs/modules/export-center/mirror-bundles.md`
- `ops/devops/TASKS.md` (`DEVOPS-EXPORT-36-001`, `DEVOPS-EXPORT-37-001`)
- `docs/ingestion/aggregation-only-contract.md`
- `docs/aoc/aggregation-only-contract.md`
- `docs/24_OFFLINE_KIT.md`
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.

2
docs/modules/export-center/overview.md
View File

@@ -33,7 +33,7 @@ Refer to `docs/modules/export-center/architecture.md` (Sprint 35 task) for compo
- **Signing and encryption.** Manifests and payloads are signed using the platform KMS. Mirror profiles support optional in-bundle encryption (age/AES-GCM) with key wrapping.
- **Determinism.** Identical inputs yield identical bundles. Timestamps serialize in UTC ISO-8601; manifests include content hashes for audit replay.
See `docs/security/policy-governance.md` and `docs/ingestion/aggregation-only-contract.md` for broader guardrail context.
See `docs/security/policy-governance.md` and `docs/aoc/aggregation-only-contract.md` for broader guardrail context.
## Operating it offline
- **Offline Kit integration.** Air-gapped deployments receive pre-built export profiles and object storage layout templates through the Offline Kit bundles.