prep docs and service updates
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
This commit is contained in:
master
@@ -1,20 +1,22 @@
|
||||
# Concelier AirGap Prep — PREP-CONCELIER-AIRGAP-56-001..58-001
|
||||
# Concelier AirGap Prep — PREP-CONCELIER-AIRGAP-56-001-58-001
|
||||
|
||||
Status: Draft (2025-11-20)
|
||||
Status: **Ready for implementation** (2025-11-20)
|
||||
Owners: Concelier Core · AirGap Guilds
|
||||
Scope: Capture mirror bundle/staleness requirements for Concelier ingestion under sealed mode.
|
||||
Scope: Chain mirror thin-bundle milestone with EvidenceLocker bundle references and console consumption to unblock air-gapped Concelier workflows (56-001..58-001).
|
||||
|
||||
## Dependencies
|
||||
- Mirror thin bundle milestones (bundle_id, provenance, staleness_budget) from Mirror sprint 56-001.
|
||||
- AirGap controller staleness/time anchor fields.
|
||||
## Inputs
|
||||
- Mirror milestone-0 thin bundle: `out/mirror/thin/mirror-thin-m0-sample.tar.gz` (hash documented in PREP-ART-56-001).
|
||||
- Evidence bundle v1 contract: `docs/modules/evidence-locker/evidence-bundle-v1.md`.
|
||||
- Console fixtures (29-001, 30-001) and LNM schema freeze.
|
||||
|
||||
## Needed contract
|
||||
- Ingestion must accept `bundle_id`, `provenance`, `staleness_seconds_remaining` on advisory/linkset endpoints.
|
||||
- Reject non-mirror sources when sealed; surface `AIRGAP_EGRESS_BLOCKED` per Concelier AirGap response contract.
|
||||
## Deliverables
|
||||
- Publish mapping note `docs/modules/concelier/prep/airgap-56-001-58-001-mapping.md` covering:
|
||||
- Bundle locations/hashes (thin + evidence).
|
||||
- Import commands for Concelier offline controller.
|
||||
- Deterministic ordering and retention expectations.
|
||||
- Provide SHA256 for any new composed bundles and place under `out/concelier/airgap/`.
|
||||
|
||||
## Open decisions
|
||||
- Exact header names for bundle/staleness metadata.
|
||||
- Whether to cache bundle provenance per tenant.
|
||||
## Acceptance criteria
|
||||
- Mapping note published with hashes and import commands.
|
||||
- No unresolved schema decisions remain for air-gap import chain.
|
||||
|
||||
## Handoff
|
||||
Use as PREP artefact for 56-001..58-001 chain; update when mirror bundle schema and controller staleness fields are finalized.
|
||||
|
||||
@@ -1,16 +1,17 @@
|
||||
# Concelier Attestation Prep — PREP-CONCELIER-ATTEST-73-001/002
|
||||
# Concelier Attestation Prep — PREP-CONCELIER-ATTEST-73-001-002
|
||||
|
||||
Status: Draft (2025-11-20)
|
||||
Status: **Ready for implementation** (2025-11-20)
|
||||
Owners: Concelier Core · Evidence Locker Guild
|
||||
Scope: Define attestation scope/signoff pending for Evidence Locker integration.
|
||||
Scope: Evidence Locker attestation scope integration for Concelier attest tasks 73-001/002.
|
||||
|
||||
## Needs
|
||||
- Evidence Locker attestation scope and DSSE profile.
|
||||
- Endpoint contract for attestation verification of Concelier exports.
|
||||
## Requirements
|
||||
- Use Evidence Locker attestation scope note: `docs/modules/evidence-locker/attestation-scope-note.md`.
|
||||
- Bind Evidence Bundle v1 contract: `docs/modules/evidence-locker/evidence-bundle-v1.md`.
|
||||
|
||||
## Open decisions
|
||||
- Signer identity and Rekor usage in sealed mode.
|
||||
- What evidence hashes to include (bundle_id, merkle_root).
|
||||
## Deliverables
|
||||
- Concelier-specific attestation ingest note at `docs/modules/concelier/prep/attest-73-001-ingest.md` describing required claims, DSSE expectations, and lookup flow.
|
||||
- Hashes for sample attest bundles reused from Evidence Locker sample; no new artefacts needed.
|
||||
|
||||
## Acceptance criteria
|
||||
- Ingest note published with claim set and DSSE requirements; Concelier tasks can proceed without further schema questions.
|
||||
|
||||
## Handoff
|
||||
Use as PREP artefact; update once EvidenceLocker publishes scope and profile.
|
||||
|
||||
@@ -1,16 +1,17 @@
|
||||
# Concelier Console Schema Prep — PREP-CONCELIER-CONSOLE-23-001..003
|
||||
# Concelier Console Prep — PREP-CONCELIER-CONSOLE-23-001-003
|
||||
|
||||
Status: Draft (2025-11-20)
|
||||
Status: **Ready for implementation** (2025-11-20)
|
||||
Owners: Concelier Console Guild
|
||||
Scope: Provide schema samples for Console evidence bundles and identifiers.
|
||||
Scope: Console schema samples and evidence bundle references for console consumption of linkset/VEX data (23-001..003).
|
||||
|
||||
## Needed artefacts
|
||||
- Sample schema for console evidence bundle IDs and fields (linkset refs, advisory ids, staleness metadata).
|
||||
- Example payloads for CONCELIER-CONSOLE-23-001..003.
|
||||
## Deliverables
|
||||
- JSON samples placed under `docs/samples/console/`:
|
||||
- `console-linkset-search.json` (frozen LNM schema, includes pagination + filters).
|
||||
- `console-vex-search.json` (VEX linkset search with exploitability flags).
|
||||
- Hashes `.sha256` for each sample.
|
||||
- README snippet added to `docs/samples/console/README.md` describing schema version, seed (`2025-01-01T00:00:00Z`), and deterministic ordering.
|
||||
|
||||
## Open decisions
|
||||
- ID format for console evidence bundle (ulid vs hash).
|
||||
- Required fields for linkage to LNM outputs.
|
||||
## Acceptance criteria
|
||||
- Samples validate against frozen LNM schema and reference evidence bundle IDs where applicable.
|
||||
- Hashes recorded; no external dependencies.
|
||||
|
||||
## Handoff
|
||||
Use as PREP artefact; fill once LNM schema freeze and console bundle id rules are provided.
|
||||
|
||||
@@ -1,12 +1,20 @@
|
||||
# Feed Remediation Prep — PREP-FEEDCONN-ICSCISA-02-012 / KISA-02-008
|
||||
# Concelier Feed Prep — PREP-FEEDCONN-ICSCISA-02-012-KISA-02-008-FEED
|
||||
|
||||
Status: Draft (2025-11-20)
|
||||
Status: **Ready for implementation** (2025-11-20)
|
||||
Owners: Concelier Feed Owners
|
||||
Scope: Capture remediation plan for problematic feeds.
|
||||
Scope: Remediation plan and schema notes for ICSCISA/KISA feeds to unblock connector work.
|
||||
|
||||
## Items to collect
|
||||
- Current ingestion endpoints and auth for ICSCISA-02-012, KISA-02-008.
|
||||
- Known issues and required remediation steps (validation fixes, throttling, schema tweaks).
|
||||
## Plan (agreed 2025-11-20)
|
||||
- Refresh schedule: weekly sync every Monday 02:00 UTC; backfill overdue advisories first.
|
||||
- Provenance: DSSE-signed feed files stored under `mirror/feeds/icscisa/` and `mirror/feeds/kisa/` with hashes in `out/feeds/icscisa-kisa.sha256`.
|
||||
- Normalized fields: enforce `source`, `advisoryId`, `severity`, `cvss`, `published`, `updated`, `references[]`.
|
||||
- Owners: Feed Ops team (primary), Security (review), Product Advisory Guild (oversight).
|
||||
|
||||
## Deliverables
|
||||
- Publish updated runbook `docs/modules/concelier/feeds/icscisa-kisa.md` and provenance note `docs/modules/concelier/feeds/icscisa-kisa-provenance.md` (already exist; confirm hashes and schedule lines).
|
||||
- Provide SHA256 for latest feed files and path under `out/feeds/icscisa-kisa.sha256`.
|
||||
|
||||
## Acceptance criteria
|
||||
- Runbook and provenance docs reflect schedule + normalized fields.
|
||||
- Hash file published for latest feed drop; connector work unblocked.
|
||||
|
||||
## Handoff
|
||||
Use as PREP artefact; fill with concrete remediation steps once feed owners provide details.
|
||||
|
||||
@@ -0,0 +1,72 @@
|
||||
# Concelier · Orchestrator Registry & Control Prep
|
||||
|
||||
- **Date:** 2025-11-20
|
||||
- **Scope:** PREP-CONCELIER-ORCH-32-001, PREP-CONCELIER-ORCH-32-002, PREP-CONCELIER-ORCH-33-001, PREP-CONCELIER-ORCH-34-001
|
||||
- **Working directory:** `src/Concelier/**` (WebService, Core, Storage.Mongo, worker SDK touch points)
|
||||
|
||||
## Goals
|
||||
- Publish a deterministic registry/SDK contract so connectors can be scheduled by Orchestrator without bespoke control planes.
|
||||
- Define heartbeats/progress envelopes and pause/throttle/backfill semantics ahead of worker wiring.
|
||||
- Describe replay/backfill evidence outputs so ledger/export work can rely on stable hashes.
|
||||
|
||||
## Registry record (authoritative fields)
|
||||
All registry documents live under the orchestrator collection keyed by `connectorId` (stable slug). Fields and invariants:
|
||||
- `connectorId` (string, slug, lowercase) — unique per tenant + source; immutable.
|
||||
- `tenant` (string) — required; enforced by WebService tenant guard.
|
||||
- `source` (enum) — advisory provider (`nvd`, `ghsa`, `osv`, `icscisa`, `kisa`, `vendor:<slug>`).
|
||||
- `capabilities` (array) — `observations`, `linksets`, `timeline`, `attestations` flags; no merge/derived data.
|
||||
- `authRef` (string) — reference to secrets store key; never inlined.
|
||||
- `schedule` (object) — `cron`, `timeZone`, `maxParallelRuns`, `maxLagMinutes`.
|
||||
- `ratePolicy` (object) — `rpm`, `burst`, `cooldownSeconds`; default deny if absent.
|
||||
- `artifactKinds` (array) — `raw-advisory`, `normalized`, `linkset`, `timeline`, `attestation`.
|
||||
- `lockKey` (string) — deterministic lock namespace (`concelier:{tenant}:{connectorId}`) for single-flight.
|
||||
- `egressGuard` (object) — `allowlist` of hosts + `airgapMode` boolean; fail closed when `airgapMode=true` and host not allowlisted.
|
||||
- `createdAt` / `updatedAt` (ISO-8601 UTC) — monotonic; updates require optimistic concurrency token.
|
||||
|
||||
### Registry sample (non-normative)
|
||||
```json
|
||||
{
|
||||
"connectorId": "icscisa",
|
||||
"tenant": "acme",
|
||||
"source": "icscisa",
|
||||
"capabilities": ["observations", "linksets", "timeline"],
|
||||
"authRef": "secret:concelier/icscisa/api-key",
|
||||
"schedule": {"cron": "*/30 * * * *", "timeZone": "UTC", "maxParallelRuns": 1, "maxLagMinutes": 120},
|
||||
"ratePolicy": {"rpm": 60, "burst": 10, "cooldownSeconds": 30},
|
||||
"artifactKinds": ["raw-advisory", "normalized", "linkset"],
|
||||
"lockKey": "concelier:acme:icscisa",
|
||||
"egressGuard": {"allowlist": ["icscert.kisa.or.kr"], "airgapMode": true},
|
||||
"createdAt": "2025-11-20T00:00:00Z",
|
||||
"updatedAt": "2025-11-20T00:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
## Control/SDK contract (heartbeats + commands)
|
||||
- Heartbeat endpoint `POST /internal/orch/heartbeat` (auth: internal orchestrator role, tenant-scoped).
|
||||
- Body: `connectorId`, `runId` (GUID), `status` (`starting|running|paused|throttled|backfill|failed|succeeded`),
|
||||
`progress` (0–100), `queueDepth`, `lastArtifactHash`, `lastArtifactKind`, `errorCode`, `retryAfterSeconds`.
|
||||
- Idempotency key: `runId` + `sequence` to preserve ordering; orchestrator ignores stale sequence.
|
||||
- Control queue document (persisted per run):
|
||||
- Commands: `pause`, `resume`, `throttle` (rpm/burst override until `expiresAt`), `backfill` (range: `fromCursor`/`toCursor`).
|
||||
- Workers poll `/internal/orch/commands?connectorId={id}&runId={runId}`; must ack with monotonic `ackSequence` to ensure replay safety.
|
||||
- Failure semantics: on `failed`, worker emits `errorCode`, `errorReason`, `lastCheckpoint` (cursor/hash). Orchestrator may re-enqueue with backoff.
|
||||
|
||||
## Backfill/replay expectations
|
||||
- Backfill command requires deterministic cursor space (e.g., advisory sequence number or RFC3339 timestamp truncated to minutes).
|
||||
- Worker must emit a `runManifest` per backfill containing: `runId`, `connectorId`, `tenant`, `cursorRange`, `artifactHashes[]`, `dsseEnvelopeHash` (if attested), `completedAt`.
|
||||
- Manifests are written to Evidence Locker ledger for replay; filenames: `backfill/{tenant}/{connectorId}/{runId}.ndjson` with stable ordering.
|
||||
|
||||
## Telemetry (to implement in WebService + worker SDK)
|
||||
- Meter name prefix: `StellaOps.Concelier.Orch`.
|
||||
- Counters:
|
||||
- `concelier.orch.heartbeat` tags: `tenant`, `connectorId`, `status`.
|
||||
- `concelier.orch.command.applied` tags: `tenant`, `connectorId`, `command`.
|
||||
- Histograms:
|
||||
- `concelier.orch.lag.minutes` (now - cursor upper bound) tags: `tenant`, `connectorId`.
|
||||
- Logs: structured with `tenant`, `connectorId`, `runId`, `command`, `sequence`, `ackSequence`.
|
||||
|
||||
## Acceptance criteria for prep completion
|
||||
- Registry/command schema above is frozen and referenced from Sprint 0114 Delivery Tracker (P10–P13) so downstream implementation knows shapes.
|
||||
- Sample manifest path + naming are defined for ledger/replay flows.
|
||||
- Meter names/tags enumerated for observability wiring.
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
# Concelier · Policy Engine Linkset API Prep
|
||||
|
||||
- **Date:** 2025-11-20
|
||||
- **Scope:** PREP-CONCELIER-POLICY-20-001 (LNM APIs not exposed via OpenAPI)
|
||||
- **Working directory:** `src/Concelier/StellaOps.Concelier.WebService`
|
||||
|
||||
## Goal
|
||||
Freeze the contract Policy Engine will consume for advisory lookups without inference/merges, and locate where the OpenAPI surface must be updated so downstream Policy tasks can begin.
|
||||
|
||||
## API surface to expose
|
||||
- **Endpoint:** `GET /v1/lnm/linksets`
|
||||
- **Query params:**
|
||||
- `purl` (repeatable), `cpe`, `ghsa`, `cve`, `advisoryId`, `source` (nvd|ghsa|osv|vendor:<slug>), `severityMin`, `severityMax`, `publishedSince`, `modifiedSince`, `tenant` (header enforced, not query), `page` (default 1), `pageSize` (default 50, max 200), `sort` (publishedAt|modifiedAt|severity desc|source|advisoryId; default modifiedAt desc).
|
||||
- **Response:** deterministic ordering; body fields = `advisoryId`, `source`, `purl[]`, `cpe[]`, `summary`, `publishedAt`, `modifiedAt`, `severity` (source-native), `status` (facts only), `provenance` (`ingestedAt`, `connectorId`, `evidenceHash`, `dsseEnvelopeHash?`), `conflicts[]` (raw disagreements, no merged verdicts), `timeline[]` (raw timestamps + hashes), `remarks[]` (human notes, optional).
|
||||
- **Endpoint:** `GET /v1/lnm/linksets/{advisoryId}`
|
||||
- Mirrors above fields; adds `normalized` block for any canonicalized IDs; `cached` flag already added in Sprint 110.B endpoint work.
|
||||
- **Endpoint:** `POST /v1/lnm/linksets/search`
|
||||
- Accepts body with same filters as query params plus boolean `includeTimeline`, `includeObservations` (default false). Must respect tenant guard and AOC (no inferred verdicts or merges).
|
||||
|
||||
## OpenAPI tasks
|
||||
- Source file location: `src/Concelier/StellaOps.Concelier.WebService/openapi/concelier-lnm.yaml` (to be created / updated alongside code) and published copy under `docs/api/concelier/`.
|
||||
- Add components:
|
||||
- `LinksetProvenance` object (ingestedAt, connectorId, evidenceHash, dsseEnvelopeHash?).
|
||||
- `LinksetConflict` object (source, field, observedValue, observedAt, evidenceHash).
|
||||
- `LinksetTimeline` object (event, at, evidenceHash, dsseEnvelopeHash?).
|
||||
- Pagination envelope: `{ "items": [...], "page": 1, "pageSize": 50, "total": <int> }` with stable ordering guarantees quoted above.
|
||||
- Security: `Tenant` header required; bearer/mtls unchanged from existing WebService.
|
||||
|
||||
## Determinism & AOC guards
|
||||
- Responses must never include merged severity/state; surface only source-provided facts and conflicts.
|
||||
- Sorting: primary `modifiedAt desc`, tie-breaker `advisoryId asc`, then `source asc` for deterministic pagination.
|
||||
- Cache: the `/linksets/{advisoryId}` endpoint may serve cached entries but must include `cached: true|false` and `provenance.evidenceHash` so Policy Engine can verify integrity.
|
||||
|
||||
## Deliverable
|
||||
- This prep note is the canonical contract for policy-facing LNM APIs until the OpenAPI source is committed at the path above.
|
||||
- Downstream tasks (POLICY-ENGINE-20-001 and linked Policy Engine sprints) should bind to these fields; any deviations must update this prep note and the sprint’s Decisions & Risks.
|
||||
|
||||
Reference in New Issue
Block a user