From 691028fe6982aa129aebb8ad201da450b8df0a10 Mon Sep 17 00:00:00 2001 From: Vladimir Moushkov Date: Thu, 30 Oct 2025 18:20:31 +0200 Subject: [PATCH] feat: Document completed tasks across multiple components - Added completed tasks documentation for Scheduler WebService, ImpactIndex, Models, Queue, Storage.Mongo, Worker, Signals, Signer, UI, Zastava.Observer, Zastava.Webhook, Zastava.Core, Cryptography.Kms, Cryptography, and Plugin. - Each task includes ID, status, owners, dependencies, descriptions, and exit criteria to ensure clarity and traceability. - Enhanced integration and unit testing coverage across various components to validate functionality and compliance with specifications. --- .gitea/workflows/build-test-deploy.yml | 3 + .gitea/workflows/release.yml | 3 + NuGet.config | 1 + deploy/telemetry/storage/README.md | 3 + docs/01_WHAT_IS_IT.md | 77 -- docs/02_WHY.md | 121 --- docs/03_QUICKSTART.md | 156 ---- docs/04_FEATURE_MATRIX.md | 6 +- docs/05_SYSTEM_REQUIREMENTS_SPEC.md | 8 +- docs/07_HIGH_LEVEL_ARCHITECTURE.md | 4 +- docs/10_OFFLINE_KIT.md | 8 +- docs/11_GOVERNANCE.md | 20 +- docs/12_CODE_OF_CONDUCT.md | 2 +- docs/12_PERFORMANCE_WORKBOOK.md | 2 +- docs/13_SECURITY_POLICY.md | 6 +- docs/README.md | 173 +---- docs/TASKS.completed.md | 88 +++ docs/TASKS.md | 81 +- docs/aoc/guard-library.md | 111 +++ docs/backlog/2025-10-cleanup.md | 2 +- docs/dev/cartographer-graph-handshake.md | 2 + docs/devops/contracts-and-rules.md | 24 + docs/evaluate/checklist.md | 38 + docs/events/README.md | 2 +- docs/events/orchestrator-scanner-events.md | 12 +- .../scanner.event.report.ready@1.sample.json | 20 +- ...scanner.event.scan.completed@1.sample.json | 20 +- docs/events/scanner.event.report.ready@1.json | 31 +- .../scanner.event.scan.completed@1.json | 31 +- docs/examples/ui-tours.md | 30 +- docs/faq/policy-faq.md | 2 +- docs/high-level-architecture.md | 49 ++ docs/implplan/SPRINTS.md | 2 +- docs/ingestion/aggregation-only-contract.md | 2 +- docs/key-features.md | 33 + .../authority/operations/backup-restore.md | 4 +- docs/modules/cli/guides/cli-reference.md | 6 +- .../modules/platform/architecture-overview.md | 2 +- .../modules/telemetry/operations/collector.md | 2 +- docs/modules/telemetry/operations/storage.md | 2 + docs/notifications/overview.md | 2 +- docs/observability/observability.md | 4 +- docs/overview.md | 39 + docs/quickstart.md | 93 +++ docs/security/authority-scopes.md | 2 +- docs/technical/README.md | 17 + docs/technical/architecture/README.md | 44 ++ docs/technical/architecture/component-map.md | 77 ++ docs/technical/development/README.md | 33 + docs/technical/interfaces/README.md | 48 ++ docs/technical/observability/README.md | 29 + docs/technical/operations/README.md | 47 ++ docs/technical/process/README.md | 25 + docs/technical/security/README.md | 35 + docs/technical/strategy/README.md | 22 + docs/ui/admin.md | 9 +- docs/ui/advisories-and-vex.md | 397 +++++----- docs/ui/console-overview.md | 260 +++---- docs/ui/downloads.md | 424 +++++------ docs/ui/findings.md | 7 +- docs/ui/navigation.md | 326 ++++----- docs/ui/policies.md | 6 +- docs/ui/policy-editor.md | 2 +- docs/ui/runs.md | 4 +- docs/ui/sbom-explorer.md | 4 +- docs/updates/2025-10-27-task-packs-docs.md | 4 +- docs/updates/2025-10-30-devops-governance.md | 17 + etc/authority.yaml | 13 +- etc/authority.yaml.sample | 23 +- ops/authority/TASKS.completed.md | 5 + ops/authority/TASKS.md | 1 - ops/deployment/TASKS.completed.md | 5 + ops/deployment/TASKS.md | 5 +- ops/devops/TASKS.completed.md | 27 + ops/devops/TASKS.md | 24 +- .../telemetry/validate_storage_stack.py | 83 +++ ops/licensing/TASKS.completed.md | 5 + ops/licensing/TASKS.md | 9 +- ops/offline-kit/TASKS.completed.md | 8 + ops/offline-kit/TASKS.md | 8 +- samples/TASKS.completed.md | 8 + samples/TASKS.md | 12 +- samples/api/reports/report-sample.dsse.json | 69 +- .../Results/AocHttpResults.cs | 95 +++ .../Routing/AocGuardEndpointFilter.cs | 91 +++ .../StellaOps.Aoc.AspNetCore.csproj | 14 + .../AocHttpResultsTests.cs | 49 ++ .../StellaOps.Aoc.AspNetCore.Tests.csproj | 32 + .../StellaOps.Attestor/TASKS.completed.md | 7 + src/Attestor/StellaOps.Attestor/TASKS.md | 13 +- .../authority/openapi.yaml | 689 ++++++++++++++++++ .../StellaOpsClaimTypes.cs | 28 +- .../TASKS.completed.md | 8 + .../TASKS.md | 36 +- .../ClientCredentialsAndTokenHandlersTests.cs | 17 +- .../Signing/AuthorityJwksServiceTests.cs | 251 +++++++ .../AuthoritySigningKeyManagerTests.cs | 16 +- .../Handlers/ClientCredentialsHandlers.cs | 14 + .../StellaOps.Authority/Program.cs | 50 +- .../Signing/AuthorityJwksService.cs | 78 +- .../Signing/AuthoritySigningKeyManager.cs | 8 +- .../Signing/KmsAuthoritySigningKeySource.cs | 31 +- .../StellaOps.Authority/TASKS.completed.md | 33 + src/Authority/StellaOps.Authority/TASKS.md | 29 - src/Bench/StellaOps.Bench/TASKS.completed.md | 10 + src/Bench/StellaOps.Bench/TASKS.md | 6 - .../StellaOps.Cartographer/TASKS.completed.md | 5 + .../StellaOps.Cartographer/TASKS.md | 11 +- src/Cli/StellaOps.Cli/TASKS.completed.md | 9 + src/Cli/StellaOps.Cli/TASKS.md | 5 - .../StellaOps.Concelier.WebService/Program.cs | 35 +- .../StellaOps.Concelier.WebService.csproj | 3 +- .../TASKS.completed.md | 5 + .../StellaOps.Concelier.WebService/TASKS.md | 189 +++-- .../TASKS.completed.md | 18 + .../TASKS.md | 14 +- .../TASKS.completed.md | 18 + .../TASKS.md | 8 - .../TASKS.completed.md | 20 + .../TASKS.md | 9 - .../TASKS.completed.md | 24 + .../TASKS.md | 17 +- .../TASKS.completed.md | 18 + .../TASKS.md | 14 +- .../TASKS.completed.md | 16 + .../TASKS.md | 13 +- .../TASKS.completed.md | 32 + .../TASKS.md | 21 +- .../TASKS.completed.md | 20 + .../TASKS.md | 9 - .../TASKS.completed.md | 26 + .../TASKS.md | 18 +- .../TASKS.completed.md | 10 + .../TASKS.md | 14 +- .../TASKS.completed.md | 34 + .../TASKS.md | 16 - .../TASKS.completed.md | 24 + .../TASKS.md | 11 - .../TASKS.completed.md | 16 + .../TASKS.md | 13 +- .../TASKS.completed.md | 22 + .../TASKS.md | 16 +- .../TASKS.completed.md | 20 + .../TASKS.md | 9 - .../TASKS.completed.md | 16 + .../TASKS.md | 15 +- .../TASKS.completed.md | 32 + .../TASKS.md | 15 - .../TASKS.completed.md | 36 + .../TASKS.md | 17 - .../TASKS.completed.md | 18 + .../TASKS.md | 8 - .../TASKS.completed.md | 18 + .../TASKS.md | 8 - .../TASKS.completed.md | 7 + .../TASKS.md | 3 - .../TASKS.completed.md | 20 + .../TASKS.md | 15 +- .../TASKS.completed.md | 18 + .../TASKS.md | 8 - .../TASKS.completed.md | 22 + .../TASKS.md | 24 +- .../TASKS.completed.md | 18 + .../TASKS.md | 8 - .../TASKS.completed.md | 18 + .../TASKS.md | 8 - .../TASKS.completed.md | 22 + .../TASKS.md | 16 +- .../TASKS.completed.md | 22 + .../TASKS.md | 24 +- .../TASKS.completed.md | 11 + .../StellaOps.Concelier.Core/TASKS.md | 222 +++--- .../TASKS.completed.md | 22 + .../TASKS.md | 10 - .../TASKS.completed.md | 26 + .../TASKS.md | 12 - .../TASKS.completed.md | 37 + .../StellaOps.Concelier.Merge/TASKS.md | 42 +- .../TASKS.completed.md | 34 + .../StellaOps.Concelier.Models/TASKS.md | 22 +- .../TASKS.completed.md | 16 + .../TASKS.md | 13 +- .../TASKS.completed.md | 8 + .../TASKS.md | 56 +- .../TASKS.completed.md | 5 + .../TASKS.md | 13 +- .../TASKS.completed.md | 7 + .../StellaOps.Excititor.Worker/TASKS.md | 35 +- .../TASKS.completed.md | 6 + .../StellaOps.Excititor.Attestation/TASKS.md | 2 - .../VexAttestationVerificationOptions.cs | 27 + .../Verification/VexAttestationVerifier.cs | 92 ++- .../TASKS.completed.md | 8 + .../TASKS.md | 3 - .../TASKS.completed.md | 8 + .../TASKS.md | 3 - .../TASKS.completed.md | 6 + .../TASKS.md | 2 - .../TASKS.completed.md | 8 + .../TASKS.md | 3 - .../TASKS.completed.md | 6 + .../TASKS.md | 2 - .../TASKS.completed.md | 14 + .../TASKS.md | 6 - .../TASKS.completed.md | 6 + .../TASKS.md | 2 - .../TASKS.completed.md | 6 + .../TASKS.md | 2 - .../TASKS.completed.md | 9 + .../StellaOps.Excititor.Core/TASKS.md | 192 +++-- .../TASKS.completed.md | 16 + .../StellaOps.Excititor.Export/TASKS.md | 7 - .../TASKS.completed.md | 8 + .../StellaOps.Excititor.Formats.CSAF/TASKS.md | 3 - .../TASKS.completed.md | 8 + .../TASKS.md | 3 - .../TASKS.completed.md | 8 + .../TASKS.md | 3 - .../TASKS.completed.md | 14 + .../StellaOps.Excititor.Policy/TASKS.md | 6 - .../StellaOps.Notifier/TASKS.completed.md | 5 + src/Notifier/StellaOps.Notifier/TASKS.md | 147 ++-- .../TASKS.completed.md | 13 + src/Policy/StellaOps.Policy.Engine/TASKS.md | 9 - .../StellaOps.Policy/TASKS.completed.md | 5 + .../__Libraries/StellaOps.Policy/TASKS.md | 1 - .../TASKS.completed.md | 9 + .../TASKS.md | 13 +- .../Contracts/OrchestratorEventContracts.cs | 56 +- .../Endpoints/PolicyEndpoints.cs | 10 +- .../Endpoints/ReportEndpoints.cs | 24 +- .../OrchestratorEventSerializer.cs | 93 ++- .../Services/ReportEventDispatcher.cs | 44 +- .../TASKS.completed.md | 19 + .../StellaOps.Scanner.WebService/TASKS.md | 16 +- .../TASKS.completed.md | 10 + src/Scanner/StellaOps.Scanner.Worker/TASKS.md | 14 +- .../TASKS.completed.md | 14 + .../TASKS.md | 32 +- .../TASKS.completed.md | 18 + .../TASKS.md | 16 +- .../TASKS.completed.md | 8 + .../TASKS.md | 52 +- .../TASKS.completed.md | 14 + .../TASKS.md | 56 +- .../TASKS.completed.md | 14 + .../TASKS.md | 56 +- .../TASKS.completed.md | 10 + .../TASKS.md | 16 +- .../TASKS.completed.md | 13 + .../StellaOps.Scanner.Analyzers.Lang/TASKS.md | 17 +- .../TASKS.completed.md | 11 + .../StellaOps.Scanner.Analyzers.OS/TASKS.md | 15 +- .../TASKS.completed.md | 8 + .../StellaOps.Scanner.Cache/TASKS.md | 16 +- .../StellaOps.Scanner.Core/TASKS.completed.md | 7 + .../StellaOps.Scanner.Core/TASKS.md | 3 - .../StellaOps.Scanner.Diff/TASKS.completed.md | 7 + .../StellaOps.Scanner.Diff/TASKS.md | 11 +- .../StellaOps.Scanner.Emit/TASKS.completed.md | 12 + .../StellaOps.Scanner.Emit/TASKS.md | 16 +- .../TASKS.completed.md | 12 + .../StellaOps.Scanner.EntryTrace/TASKS.md | 16 +- .../TASKS.completed.md | 7 + .../StellaOps.Scanner.Queue/TASKS.md | 11 +- .../TASKS.completed.md | 9 + .../StellaOps.Scanner.Storage/TASKS.md | 13 +- .../PlatformEventSamplesTests.cs | 44 +- .../ReportEventDispatcherTests.cs | 23 +- .../ReportsEndpointsTests.cs | 19 +- .../scanner.event.report.ready@1.sample.json | 101 +++ ...scanner.event.scan.completed@1.sample.json | 107 +++ .../Events/GraphJobEventPublisher.cs | 226 ++++-- .../Events/IRedisConnectionFactory.cs | 8 + .../Events/RedisConnectionFactory.cs | 26 + .../Options/SchedulerEventsOptions.cs | 47 +- .../StellaOps.Scheduler.WebService/Program.cs | 13 +- .../StellaOps.Scheduler.WebService.csproj | 5 +- .../TASKS.completed.md | 12 + .../StellaOps.Scheduler.WebService/TASKS.md | 9 +- .../TASKS.completed.md | 7 + .../StellaOps.Scheduler.ImpactIndex/TASKS.md | 3 - .../TASKS.completed.md | 11 + .../StellaOps.Scheduler.Models/TASKS.md | 7 - .../TASKS.completed.md | 7 + .../StellaOps.Scheduler.Queue/TASKS.md | 15 +- .../TASKS.completed.md | 7 + .../TASKS.md | 15 +- .../TASKS.completed.md | 14 + .../StellaOps.Scheduler.Worker/TASKS.md | 10 - .../GraphJobEventPublisherTests.cs | 65 +- .../StellaOps.Signals/TASKS.completed.md | 6 + src/Signals/StellaOps.Signals/TASKS.md | 24 +- .../StellaOps.Signer/TASKS.completed.md | 7 + src/Signer/StellaOps.Signer/TASKS.md | 3 - src/StellaOps.sln | 66 +- src/UI/StellaOps.UI/TASKS.completed.md | 6 + src/UI/StellaOps.UI/TASKS.md | 188 +++-- src/Web/StellaOps.Web/TASKS.md | 6 +- .../TASKS.completed.md | 9 + .../StellaOps.Zastava.Observer/TASKS.md | 13 +- .../TASKS.completed.md | 8 + .../StellaOps.Zastava.Webhook/TASKS.md | 16 +- .../StellaOps.Zastava.Core/TASKS.completed.md | 8 + .../StellaOps.Zastava.Core/TASKS.md | 16 +- .../AuthoritySigningOptions.cs | 10 + .../TASKS.completed.md | 6 + .../StellaOps.Cryptography.Kms/TASKS.md | 24 +- .../StellaOps.Cryptography/TASKS.completed.md | 27 + .../StellaOps.Cryptography/TASKS.md | 82 +-- .../StellaOps.Plugin/TASKS.completed.md | 14 + src/__Libraries/StellaOps.Plugin/TASKS.md | 6 - 312 files changed, 6399 insertions(+), 3319 deletions(-) delete mode 100755 docs/01_WHAT_IS_IT.md delete mode 100755 docs/02_WHY.md delete mode 100755 docs/03_QUICKSTART.md create mode 100644 docs/TASKS.completed.md create mode 100644 docs/aoc/guard-library.md create mode 100644 docs/devops/contracts-and-rules.md create mode 100644 docs/evaluate/checklist.md create mode 100644 docs/high-level-architecture.md create mode 100644 docs/key-features.md create mode 100644 docs/overview.md create mode 100644 docs/quickstart.md create mode 100644 docs/technical/README.md create mode 100644 docs/technical/architecture/README.md create mode 100644 docs/technical/architecture/component-map.md create mode 100644 docs/technical/development/README.md create mode 100644 docs/technical/interfaces/README.md create mode 100644 docs/technical/observability/README.md create mode 100644 docs/technical/operations/README.md create mode 100644 docs/technical/process/README.md create mode 100644 docs/technical/security/README.md create mode 100644 docs/technical/strategy/README.md create mode 100644 docs/updates/2025-10-30-devops-governance.md create mode 100644 ops/authority/TASKS.completed.md create mode 100644 ops/deployment/TASKS.completed.md create mode 100644 ops/devops/TASKS.completed.md create mode 100644 ops/devops/telemetry/validate_storage_stack.py create mode 100644 ops/licensing/TASKS.completed.md create mode 100644 ops/offline-kit/TASKS.completed.md create mode 100644 samples/TASKS.completed.md create mode 100644 src/Aoc/__Libraries/StellaOps.Aoc.AspNetCore/Results/AocHttpResults.cs create mode 100644 src/Aoc/__Libraries/StellaOps.Aoc.AspNetCore/Routing/AocGuardEndpointFilter.cs create mode 100644 src/Aoc/__Libraries/StellaOps.Aoc.AspNetCore/StellaOps.Aoc.AspNetCore.csproj create mode 100644 src/Aoc/__Tests/StellaOps.Aoc.AspNetCore.Tests/AocHttpResultsTests.cs create mode 100644 src/Aoc/__Tests/StellaOps.Aoc.AspNetCore.Tests/StellaOps.Aoc.AspNetCore.Tests.csproj create mode 100644 src/Attestor/StellaOps.Attestor/TASKS.completed.md create mode 100644 src/Authority/StellaOps.Api.OpenApi/authority/openapi.yaml create mode 100644 src/Authority/StellaOps.Authority/StellaOps.Authority.Plugin.Standard/TASKS.completed.md create mode 100644 src/Authority/StellaOps.Authority/StellaOps.Authority.Tests/Signing/AuthorityJwksServiceTests.cs create mode 100644 src/Authority/StellaOps.Authority/TASKS.completed.md create mode 100644 src/Bench/StellaOps.Bench/TASKS.completed.md create mode 100644 src/Cartographer/StellaOps.Cartographer/TASKS.completed.md create mode 100644 src/Cli/StellaOps.Cli/TASKS.completed.md create mode 100644 src/Concelier/StellaOps.Concelier.WebService/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Connector.Acsc/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Connector.Cccs/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertBund/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertCc/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertFr/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertIn/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Connector.Common/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Connector.Cve/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Connector.Distro.RedHat/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Connector.Distro.Ubuntu/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ghsa/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ics.Cisa/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ics.Kaspersky/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Connector.Jvn/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Connector.Kev/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Connector.Kisa/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Connector.Nvd/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Connector.Osv/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ru.Bdu/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ru.Nkcki/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Connector.StellaOpsMirror/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Adobe/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Apple/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Chromium/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Cisco/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Msrc/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Oracle/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Vmware/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Core/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Exporter.Json/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Exporter.TrivyDb/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Models/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Normalization/TASKS.completed.md create mode 100644 src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.completed.md create mode 100644 src/Excititor/StellaOps.Excititor.Connectors.StellaOpsMirror/TASKS.completed.md create mode 100644 src/Excititor/StellaOps.Excititor.Worker/TASKS.completed.md create mode 100644 src/Excititor/__Libraries/StellaOps.Excititor.Attestation/TASKS.completed.md create mode 100644 src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Abstractions/TASKS.completed.md create mode 100644 src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Cisco.CSAF/TASKS.completed.md create mode 100644 src/Excititor/__Libraries/StellaOps.Excititor.Connectors.MSRC.CSAF/TASKS.completed.md create mode 100644 src/Excititor/__Libraries/StellaOps.Excititor.Connectors.OCI.OpenVEX.Attest/TASKS.completed.md create mode 100644 src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Oracle.CSAF/TASKS.completed.md create mode 100644 src/Excititor/__Libraries/StellaOps.Excititor.Connectors.RedHat.CSAF/TASKS.completed.md create mode 100644 src/Excititor/__Libraries/StellaOps.Excititor.Connectors.SUSE.RancherVEXHub/TASKS.completed.md create mode 100644 src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Ubuntu.CSAF/TASKS.completed.md create mode 100644 src/Excititor/__Libraries/StellaOps.Excititor.Core/TASKS.completed.md create mode 100644 src/Excititor/__Libraries/StellaOps.Excititor.Export/TASKS.completed.md create mode 100644 src/Excititor/__Libraries/StellaOps.Excititor.Formats.CSAF/TASKS.completed.md create mode 100644 src/Excititor/__Libraries/StellaOps.Excititor.Formats.CycloneDX/TASKS.completed.md create mode 100644 src/Excititor/__Libraries/StellaOps.Excititor.Formats.OpenVEX/TASKS.completed.md create mode 100644 src/Excititor/__Libraries/StellaOps.Excititor.Policy/TASKS.completed.md create mode 100644 src/Notifier/StellaOps.Notifier/TASKS.completed.md create mode 100644 src/Policy/StellaOps.Policy.Engine/TASKS.completed.md create mode 100644 src/Policy/__Libraries/StellaOps.Policy/TASKS.completed.md create mode 100644 src/Scanner/StellaOps.Scanner.Sbomer.BuildXPlugin/TASKS.completed.md create mode 100644 src/Scanner/StellaOps.Scanner.WebService/TASKS.completed.md create mode 100644 src/Scanner/StellaOps.Scanner.Worker/TASKS.completed.md create mode 100644 src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.DotNet/TASKS.completed.md create mode 100644 src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Go/TASKS.completed.md create mode 100644 src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Java/TASKS.completed.md create mode 100644 src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Node/TASKS.completed.md create mode 100644 src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Python/TASKS.completed.md create mode 100644 src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Rust/TASKS.completed.md create mode 100644 src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang/TASKS.completed.md create mode 100644 src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.OS/TASKS.completed.md create mode 100644 src/Scanner/__Libraries/StellaOps.Scanner.Cache/TASKS.completed.md create mode 100644 src/Scanner/__Libraries/StellaOps.Scanner.Core/TASKS.completed.md create mode 100644 src/Scanner/__Libraries/StellaOps.Scanner.Diff/TASKS.completed.md create mode 100644 src/Scanner/__Libraries/StellaOps.Scanner.Emit/TASKS.completed.md create mode 100644 src/Scanner/__Libraries/StellaOps.Scanner.EntryTrace/TASKS.completed.md create mode 100644 src/Scanner/__Libraries/StellaOps.Scanner.Queue/TASKS.completed.md create mode 100644 src/Scanner/__Libraries/StellaOps.Scanner.Storage/TASKS.completed.md create mode 100644 src/Scanner/docs/events/samples/scanner.event.report.ready@1.sample.json create mode 100644 src/Scanner/docs/events/samples/scanner.event.scan.completed@1.sample.json create mode 100644 src/Scheduler/StellaOps.Scheduler.WebService/GraphJobs/Events/IRedisConnectionFactory.cs create mode 100644 src/Scheduler/StellaOps.Scheduler.WebService/GraphJobs/Events/RedisConnectionFactory.cs create mode 100644 src/Scheduler/StellaOps.Scheduler.WebService/TASKS.completed.md create mode 100644 src/Scheduler/__Libraries/StellaOps.Scheduler.ImpactIndex/TASKS.completed.md create mode 100644 src/Scheduler/__Libraries/StellaOps.Scheduler.Models/TASKS.completed.md create mode 100644 src/Scheduler/__Libraries/StellaOps.Scheduler.Queue/TASKS.completed.md create mode 100644 src/Scheduler/__Libraries/StellaOps.Scheduler.Storage.Mongo/TASKS.completed.md create mode 100644 src/Scheduler/__Libraries/StellaOps.Scheduler.Worker/TASKS.completed.md create mode 100644 src/Signals/StellaOps.Signals/TASKS.completed.md create mode 100644 src/Signer/StellaOps.Signer/TASKS.completed.md create mode 100644 src/UI/StellaOps.UI/TASKS.completed.md create mode 100644 src/Zastava/StellaOps.Zastava.Observer/TASKS.completed.md create mode 100644 src/Zastava/StellaOps.Zastava.Webhook/TASKS.completed.md create mode 100644 src/Zastava/__Libraries/StellaOps.Zastava.Core/TASKS.completed.md create mode 100644 src/__Libraries/StellaOps.Cryptography.Kms/TASKS.completed.md create mode 100644 src/__Libraries/StellaOps.Cryptography/TASKS.completed.md create mode 100644 src/__Libraries/StellaOps.Plugin/TASKS.completed.md diff --git a/.gitea/workflows/build-test-deploy.yml b/.gitea/workflows/build-test-deploy.yml index 790cd7bd..1d4770dc 100644 --- a/.gitea/workflows/build-test-deploy.yml +++ b/.gitea/workflows/build-test-deploy.yml @@ -70,6 +70,9 @@ jobs: - name: Validate NuGet restore source ordering run: python3 ops/devops/validate_restore_sources.py + - name: Validate telemetry storage configuration + run: python3 ops/devops/telemetry/validate_storage_stack.py + - name: Setup .NET ${{ env.DOTNET_VERSION }} uses: actions/setup-dotnet@v4 with: diff --git a/.gitea/workflows/release.yml b/.gitea/workflows/release.yml index 3b9c3ef3..a1981fc2 100644 --- a/.gitea/workflows/release.yml +++ b/.gitea/workflows/release.yml @@ -47,6 +47,9 @@ jobs: - name: Validate NuGet restore source ordering run: python3 ops/devops/validate_restore_sources.py + - name: Validate telemetry storage configuration + run: python3 ops/devops/telemetry/validate_storage_stack.py + - name: Set up Docker Buildx uses: docker/setup-buildx-action@v3 diff --git a/NuGet.config b/NuGet.config index 0706b670..024f15b2 100644 --- a/NuGet.config +++ b/NuGet.config @@ -38,6 +38,7 @@ + diff --git a/deploy/telemetry/storage/README.md b/deploy/telemetry/storage/README.md index 1fab323f..3ffce99b 100644 --- a/deploy/telemetry/storage/README.md +++ b/deploy/telemetry/storage/README.md @@ -24,6 +24,9 @@ These configurations are referenced by the Docker Compose overlay connected environments. Place the Prometheus bearer token in `auth/token` when using the Compose overlay (the directory contains a `.gitkeep` placeholder and is gitignored by default). +Run `python ops/devops/telemetry/validate_storage_stack.py` after editing any of these files to +ensure TLS, multitenancy, and override references remain intact. + ## Security - Both Tempo and Loki require mutual TLS. diff --git a/docs/01_WHAT_IS_IT.md b/docs/01_WHAT_IS_IT.md deleted file mode 100755 index 90e2bf86..00000000 --- a/docs/01_WHAT_IS_IT.md +++ /dev/null @@ -1,77 +0,0 @@ -# 1 · What Is - **Stella Ops**? - -Stella Ops is a **self‑hosted, SBOM‑first DevSecOps platform** that gives engineering and security teams instant (< 5 s) feedback on container and artifact risk—even when they run completely offline. -It is built around five design pillars: **modular, open, fast, local, and UI‑controllable**. - ---- - -## 1. What the Product Does — 7‑Point Snapshot - -| # | Capability | What It Means in Practice | -|---|------------|---------------------------| -| **1** | **SBOM‑Centric Scanning** | Generates and scans *Software Bills of Materials* (Trivy JSON, SPDX‑JSON, CycloneDX‑JSON); auto‑detects format and stores each SBOM as a blob. | -| **2** | **Delta‑SBOM Engine** | Uploads SBOM only for *new* layers; warm‑cache image rescans complete in < 1 s. | -| **3** | **Anonymous Internal Registry** | Ships a built‑in `StellaOps.Registry` so agents (`Stella CLI`, `Zastava`, SBOM‑builder) can be pulled inside air‑gapped networks without external credentials. | -| **4** | **Policy‑as‑Code** | Supports YAML rules today and OPA/Rego (`StellaOps.MutePolicies`) tomorrow—edit in the web UI, versioned in Mongo, enforce at scan time. | -| **5** | **Pluggable Modules** | Every scanner, exporter, or attestor is a hot‑load .NET plug‑in (e.g., `StellaOpsAttestor` for SLSA/Rekor in the roadmap). | -| **6** | **Horizontally Scalable** | Stateless API backed by Redis & Mongo; optional Kubernetes charts for multi‑node performance. | -| **7** | **Sovereign & Localized** | Localized UI, optional connectors to regional catalogues, and zero telemetry by default—ready for high‑compliance, air‑gapped deployments. | - -> **🆓 Free tier update (July 2025)** – Every self‑hosted instance now includes **{{ quota_token }} scans per UTC day**. -> A yellow banner appears once you cross **200 scans** (≈ 60 % of quota). -> Past {{ quota_token }} , `/scan` responds with soft 5 s waits (graceful back‑off), and may return **429 + Retry‑After (to UTC midnight)** after repeated hits. - ---- - -## 2. How It Works — End‑to‑End Flow (30 sec tour) - -1. **Build Phase** - `sbom‑builder` container runs inside CI, pulls base layers metadata, and queries `/layers/missing`—receiving in ~20 ms which layers still need SBOMs. - • New layers ➟ SBOM generated ➟ `*.sbom.` + `*.sbom.type` dropped next to image tarball. - -2. **Push to Registry** - Image and SBOM blobs are pushed to the **anonymous internal registry** (`StellaOps.Registry`). Cosign tags are attached if enabled. - -3. **Scan Phase** - `Stella CLI` agent pulls the SBOM blob, sends `/scan?sbomType=spdx-json` to backend. If flag is absent, backend auto‑detects. - • Free‑tier tokens inherit the **333‑scan/day quota**; response headers expose remaining scans and reset time. - -4. **Policy & Risk Evaluation** - Backend hydrates CVE data, merges any cached layer scores, and calls the **Policy‑as‑Code engine**: - * YAML rules → built‑in interpreter; - * Rego policies (future) → embedded OPA. - -5. **Attestation & Transparency** *(Roadmap)* - `StellaOpsAttestor` signs results with SLSA provenance and records them in a local **Rekor** mirror for tamper‑proof history. - -6. **Feedback Loop** - • CLI exits with non‑zero on policy block. - • UI dashboard shows findings, quota banner, and per‑token scan counters; triagers can mute or set expiry dates directly. - ---- - -## 3. Why Such a Product Is Needed - -> *“Software supply‑chain attacks have increased **742 %** over the past three years.”* – Sonatype 2024 State of the Software Supply Chain - -### Key Drivers & Regulations - -| Driver | Detail & Obligation | -|--------|--------------------| -| **Government SBOM Mandates** | • **US EO 14028** & NIST SP 800‑218 require suppliers to provide SBOMs.
• EU **Cyber Resilience Act (CRA)** will demand attestations of secure development by 2026. | -| **SLSA & SSDF Frameworks** | Industry pushes toward **SLSA v1.0** levels 2‑3 and NIST **SSDF 1.1** controls, emphasising provenance and policy enforcement. | -| **Transparency Logs** | **Sigstore Rekor** gains traction as a standard for tamper‑evident signatures—even for air‑gapped replicas. | -| **Offline & Sovereign Deployments** | Critical‑infra operators (finance, telecom, defence) must run security tooling without Internet and with local language/VDB support. | -| **Performance Expectations** | Modern CI/CD pipelines trigger hundreds of image builds daily; waiting 30‑60 s per scan is no longer acceptable—and now **must be achieved within a 333‑scan/day free quota**. | - -### Gap in Existing Tools - -* SaaS‑only scanners can’t run in regulated or disconnected environments. -* Monolithic open‑source scanners are hard‑wired to Trivy or Syft formats, lacking delta optimisation. -* Few products expose **Policy‑as‑Code** with full UI editing **and** history audit in a single package. -* None address quota‑aware throttling without hidden paywalls. - -**Stella Ops** fills this gap by combining *speed*, *modular openness*, *sovereign readiness* **and transparent quota limits**—making thorough supply‑chain security attainable for every team, not just cloud‑native startups. - ---- -*Last updated: 14 Jul 2025* diff --git a/docs/02_WHY.md b/docs/02_WHY.md deleted file mode 100755 index 21b83e94..00000000 --- a/docs/02_WHY.md +++ /dev/null @@ -1,121 +0,0 @@ -# 2 · WHY — Why Stella Ops Exists - -> Explaining the concrete pain we solve, why the world needs **one more** DevSecOps -> platform, and the success signals that prove we are on the right track. - -Software‑supply‑chain attacks, licence‑risk, and incomplete SBOM coverage slow -teams and compliance audits to a crawl. Most existing scanners: - -* **Assume Internet** access for CVE feeds or SaaS back‑ends. -* **Parse an entire image** every build (no layer‑delta optimisation). -* **Accept a single SBOM format** (usually Trivy JSON) and choke on anything else. -* Offer **no built‑in policy history / audit trail**. -* Require 30‑60 s wall‑time per scan, an order of magnitude slower than modern CI - expectations. -* **Hide quota limits** or throttle without warning once you move past free trials. - ---- -# 1 Free‑Tier Quota — Why **{{ quota_token }} **? - -The limit of **{{ quota_token }} SBOM scans per UTC day** was not chosen at random. - -| Constraint | Analysis | Outcome | -|------------|----------|---------| -| **SMB workload** | Internal survey across 37 SMBs shows median **210** container builds/day (p95 ≈ 290). | {{ quota_token }} gives ≈ 1.6 × head‑room without forcing a paid tier. | -| **Cost of feeds** | Hosting, Trivy DB mirrors & CVE merge traffic average **≈ $14 / 1 000 scans**. | {{ quota_token }} /day yields <$5 infra cost per user — sustainable for an OSS project. | -| **Incentive to upgrade** | Larger orgs (> 300 builds/day) gain ROI from Plus/Pro tiers anyway. | Clear upsell path without hurting hobbyists. | - -> **In one sentence:**  *{{ quota_token }} scans cover the daily needs of a typical small / -> medium business, keep free usage genuinely useful and still leave a financial -> runway for future development*. - -## 1.1 How the Quota Is Enforced (1‑minute view) - -* Backend loads the **Quota plug‑in** at startup. -* Every `/scan` call passes the caller’s **Client‑JWT** to the plug‑in. -* The plug‑in **increments a counter in Redis** under - `quota::` (expires at UTC midnight). -* Soft wait‑wall (5 s) after limit; hard wait‑wall (60 s) after 30 blocked calls. -* For **offline installs**, a *1‑month validity Client‑JWT* ships inside every - **Offline Update Kit (OUK)** tarball. Uploading the OUK refreshes the token - automatically. - -Detailed sequence living in **30_QUOTA_ENFORCEMENT_FLOW.md**. - - - ---- - -## 2 · Why *Another* DevSecOps Product? — Macro Drivers - -| Driver | Evidence | Implication for Tooling | -|--------|----------|-------------------------| -| **Exploding supply‑chain attacks** | Sonatype 2024 report shows **742 %** growth since 2020. | SBOMs & provenance checks must be default, not “best‑practice”. | -| **Regulation tsunami** | • US EO 14028 & NIST SP 800‑218
• EU Cyber‑Resilience Act (CRA) in force 2026
• Local critical‑infrastructure rules in some jurisdictions | Vendors must *attest* build provenance (SLSA) and store tamper‑proof SBOMs. | -| **Runtime‑cost intolerance** | Pipelines build hundreds of images/day; waiting > 10 s per scan breaks SLA. | Need **delta‑aware** engines that reuse layer analyses (< 1 s warm scans). | -| **Air‑gap & sovereignty demands** | Finance/defence prohibit outbound traffic; data must stay on‑prem. | Ship **self‑contained registry + CVE DB** and run offline. | -| **Predictable free‑tier limits** | Teams want clarity, not surprise throttling. | Provide **transparent {{ quota_token }} scans/day quota**, early banner & graceful wait‑wall. | - -> **Therefore:** The market demands a **modular, SBOM‑first, sub‑5 s, 100 % self‑hosted** -> platform **with a transparent free‑tier quota**—precisely the niche Stella Ops targets. - ---- - -## 3 · Gap in Current Tooling - -* Trivy / Syft create SBOMs but re‑analyse **every** layer → wasted minutes/day. -* Policy engines (OPA/Rego) are separate binaries, with no UI or change history. -* No mainstream OSS bundle ships an **anonymous internal registry** for air‑gapped pulls. -* Provenance attestation (SLSA) and Rekor transparency logs remain “bring‑your‑own”. -* Free tiers either stop at 100 scans **or** silently throttle; none announce a **clear {{ quota_token }} /day allowance**. - ---- - -## 4 · Why Stella Ops Can Win - -1. **Speed First** — Delta‑SBOM flow uses cached layers to hit `< 1 s` warm scans. -2. **Multi‑Format Ready** — Auto‑detects Trivy‑JSON, SPDX‑JSON, CycloneDX‑JSON; UI - lets teams choose per‑project defaults. -3. **Offline by Default** — Ships an **anonymous internal Docker registry** - (`StellaOps.Registry`) plus Redis, Mongo, CVE DB, and UI in a single compose up. -4. **Open & Modular** — .NET hot‑load plug‑ins (`StellaOpsAttestor`, future scanners) - under AGPL; anyone can extend. -5. **Policy as Code** — YAML rules today, upgrade path to OPA/Rego with history stored - in Mongo via `StellaOps.MutePolicies`. -6. **Sovereign‑Ready** — Russian‑language UI, local vulnerability mirrors, zero - telemetry by default. -7. **Honest Free‑tier Boundaries** — Clear **{{ quota_token }} scans/day** limit, early banner at 200 and predictable wait‑wall—no hidden throttling. - ---- - -## 5 · Success Criteria — Signals We Solve the Problem - -* **Performance:** P95 scan < 5 s on first pass; `< 1 s` for warm delta scans. -* **Compatibility:** SBOMs in at least three formats consumed by ≥ 3 downstream tools. -* **Adoption:** ≥ 1 000 reported installs & ≥ 2 000 binary downloads by Q2‑2026. -* **Compliance:** Positive audits referencing CRA / NIST / SLSA readiness. -* **Community:** ≥ 15 first‑time contributors merged per quarter by 2026. -* **Transparency:** 0 support tickets complaining about “mystery throttling”. - ---- - -## 6 · Non‑Goals (2025‑2027) - -* Multi‑tenant SaaS offering. -* Automatic “fix‑PR” generation (left to ecosystem). -* Windows container **scanning** (Windows *agents* are on the 12‑month roadmap). - ---- - -## 7 · Stakeholder Pain‑Point Recap - -| Persona | Pain Today | Stella Ops Solution | -|---------|------------|---------------------| -| **Dev** | “My CI fails for 45 s on every push.” | < 5 s initial, < 1 s warm scans. | -| **Sec‑Ops** | Separate tools for SBOM, policy, and audit. | Unified UI + YAML / Rego policies with history. | -| **Infra** | Internet‑blocked site; no public pulls allowed. | Offline compose bundle + internal registry. | -| **Compliance** | Need CRA‑ready provenance by 2026. | Future `StellaOpsAttestor` SLSA + Rekor integration. | -| **Budget owner** | Fears hidden overage charges in “free” tiers. | Transparent {{ quota_token }} scans/day limit, visible in UI/API. | - ---- -*Last updated: 14 Jul 2025 (sync with free‑tier quota rev 2.0).* diff --git a/docs/03_QUICKSTART.md b/docs/03_QUICKSTART.md deleted file mode 100755 index 900954cb..00000000 --- a/docs/03_QUICKSTART.md +++ /dev/null @@ -1,156 +0,0 @@ -# Five‑Minute Quick‑Start ⚡ -Run your first container scan locally - -> **Heads‑up** – the public α `v0.1.0` image drops **late 2025**. -> Once it is published as -> `registry.stella-ops.org/stella-ops/stella-ops:0.1.0‑alpha` -> every command on this page works without changes. - ---- - -## 0 · What you need 🔧 - -| Requirement | Minimum | Notes | -|-------------|---------|-------| -| OS | Ubuntu 22.04 • Alma 9 | x86‑64 or arm64 | -| Docker | Engine 25 • Compose v2 | `docker -v` | -| CPU / RAM | 2 vCPU / 2 GiB | Dev‑laptop baseline | -| Disk | 10 GiB SSD | SBOM cache | - -> **Tip –** If you already have Redis & MongoDB, skip the infra -> compose file and point Stella Ops at those hosts via `.env`. - ---- - -## 1 · Fetch the signed Compose bundles 📦 - -```bash -# Infrastructure (Redis + MongoDB) -curl -LO https://get.stella-ops.org/docker-compose.infrastructure.yml -curl -LO https://get.stella-ops.org/docker-compose.infrastructure.yml.sig - -# Core scanner stack -curl -LO https://get.stella-ops.org/docker-compose.stella-ops.yml -curl -LO https://get.stella-ops.org/docker-compose.stella-ops.yml.sig - -# Verify signatures (supply‑chain 101) -cosign verify-blob --key https://stella-ops.org/keys/cosign.pub \ - --signature docker-compose.infrastructure.yml.sig docker-compose.infrastructure.yml -cosign verify-blob --key https://stella-ops.org/keys/cosign.pub \ - --signature docker-compose.stella-ops.yml.sig docker-compose.stella-ops.yml -```` - ---- - -## 2 · Create `.env` 🗝️ - -```bash - -# ─── Identity (shows in reports) ─────────────────────────── -STELLA_OPS_COMPANY_NAME="Acme Corp" -STELLA_OPS_ISSUER_EMAIL="ops@acme.example" -STELLA_OPS_DEFAULT_ADMIN_USERNAME="admin" -STELLA_OPS_DEFAULT_ADMIN_PASSWORD="changeme!" -STELLA_OPS_DEFAULT_JWT="" # or load it later with -# docker --env-file .env compose -f docker-compose.stella-ops.yml exec stella set-jwt - - -# ─── Database secrets ────────────────────────────────────── -MONGO_INITDB_ROOT_USERNAME=stella_admin -MONGO_INITDB_ROOT_PASSWORD=$(openssl rand -base64 18) -MONGO_URL=mongodb - -REDIS_PASSWORD=$(openssl rand -base64 18) -REDIS_URL=redis - - - -``` - ---- - -## 3 · Start the supporting services 🗄️ - -```bash -docker compose --env-file .env -f docker-compose.infrastructure.yml pull -docker compose --env-file .env -f docker-compose.infrastructure.yml up -d -``` - ---- - -## 4 · Launch Stella Ops 🚀 - -```bash -docker compose --env-file .env -f docker-compose.stella-ops.yml pull -docker compose --env-file .env -f docker-compose.stella-ops.yml up -d -``` - -*Point your browser at* **`https://:8443`** – the certificate is -self‑signed in the alpha. -Default credentials: **`admin / changeme`** (rotate immediately!). - ---- - -## 5 · Run a scan 🔍 - -```bash -docker compose --env-file .env -f docker-compose.stella-ops.yml \ - exec stella-ops stella scan alpine:3.20 -``` - -* First scan downloads CVE feeds (\~ 50 MB). -* Warm scans finish in **≈ 5 s** on a 4‑vCPU host thanks to the Δ‑SBOM engine. - ---- - -## 6 · Reload or add a token later 🔄 - -```bash -# After adding STELLA_JWT to .env … -docker compose --env-file .env -f docker-compose.stella-ops.yml \ - exec stella-ops stella jwt -``` - -*Anonymous mode* → **{{ quota_anon }} scans/day** -*Token mode* → **{{ quota_token }} scans/day** -At **10 % of the daily max** a polite reminder appears; after {{ quota_token }} the server applies a **soft 5 s back‑off** and may return **429 + Retry‑After** until the daily reset. - ---- - -## 7 · Typical next steps ➡️ - -| Task | Where to look | -| ---------------------------------------- | ------------------------------------------------------------------- | -| CI pipelines (GitHub / GitLab / Jenkins) | [`docs/ci/`](ci/) | -| Air‑gapped install | [Offline Update Kit](10_OFFLINE_KIT.md) | -| Feature overview | [20\_FEATURES.md](20_FEATURES.md) | -| Governance & licence | [`LICENSE.md`](LICENSE.md) • [`11_GOVERNANCE.md`](11_GOVERNANCE.md) | - ---- - -## 8 · Uninstall / cleanup 🧹 - -```bash -docker compose --env-file .env -f docker-compose.stella-ops.yml down -v -docker compose --env-file .env -f docker-compose.infrastructure.yml down -v -rm compose-*.yml compose-*.yml.sig .env -``` - ---- - -### Licence & provenance 📜 - -Stella Ops is **AGPL‑3.0‑or‑later**. Every release ships: - -* **Cosign‑signed** container images -* A full **SPDX 2.3** SBOM - -```bash -cosign verify \ - --key https://stella-ops.org/keys/cosign.pub \ - registry.stella-ops.org/stella-ops/stella-ops: -``` - ---- - -© 2025‑2026 Stella Ops – free / libre / open‑source. diff --git a/docs/04_FEATURE_MATRIX.md b/docs/04_FEATURE_MATRIX.md index 4e4efb70..5af6cea9 100755 --- a/docs/04_FEATURE_MATRIX.md +++ b/docs/04_FEATURE_MATRIX.md @@ -1,5 +1,7 @@ -# 4 · Feature Matrix — **Stella Ops** -*(rev 2.0 · 14 Jul 2025)* +# 4 · Feature Matrix — **Stella Ops** +*(rev 2.0 · 14 Jul 2025)* + +> **Looking for a quick read?** Check [`key-features.md`](key-features.md) for the short capability cards; this matrix keeps full tier-by-tier detail. | Category | Capability | Free Tier (≤ 333 scans / day) | Community Plug‑in | Commercial Add‑On | Notes / ETA | | ---------------------- | ------------------------------------- | ----------------------------- | ----------------- | ------------------- | ------------------------------------------ | diff --git a/docs/05_SYSTEM_REQUIREMENTS_SPEC.md b/docs/05_SYSTEM_REQUIREMENTS_SPEC.md index ddaa6a26..181a84d2 100755 --- a/docs/05_SYSTEM_REQUIREMENTS_SPEC.md +++ b/docs/05_SYSTEM_REQUIREMENTS_SPEC.md @@ -11,18 +11,18 @@ Stella Ops · self‑hosted supply‑chain‑security platform ## 1 · Purpose & Scope -This SRS defines everything the **v0.8‑beta** release of _Stella Ops_ must do, **including the Free‑tier daily quota of {{ quota_token }} SBOM scans per token**. +This SRS defines everything the **v0.1.0‑alpha** release of _Stella Ops_ must do, **including the Free‑tier daily quota of {{ quota_token }} SBOM scans per token**. Scope includes core platform, CLI, UI, quota layer, and plug‑in host; commercial or closed‑source extensions are explicitly out‑of‑scope. --- ## 2 · References -* [02_WHY.md](02_WHY.md) – market gap & problem statement +* [overview.md](overview.md) – market gap & problem statement * [03_VISION.md](03_VISION.md) – north‑star, KPIs, quarterly themes * [07_HIGH_LEVEL_ARCHITECTURE.md](07_HIGH_LEVEL_ARCHITECTURE.md) – context & data flow diagrams -* [08_MODULE_SPECIFICATIONS.md](08_MODULE_SPECIFICATIONS.md) – component APIs & plug‑in contracts -* [09_API_CLI_REFERENCE.md](09_API_CLI_REFERENCE.md) – REST & CLI surface +* [modules/platform/architecture-overview.md](modules/platform/architecture-overview.md) – component APIs & plug‑in contracts +* [09_API_CLI_REFERENCE.md](09_API_CLI_REFERENCE.md) – REST & CLI surface --- diff --git a/docs/07_HIGH_LEVEL_ARCHITECTURE.md b/docs/07_HIGH_LEVEL_ARCHITECTURE.md index 9847259a..63613f45 100755 --- a/docs/07_HIGH_LEVEL_ARCHITECTURE.md +++ b/docs/07_HIGH_LEVEL_ARCHITECTURE.md @@ -1,4 +1,6 @@ -# High‑Level Architecture — **Stella Ops** (Consolidated • 2025Q4) +# High‑Level Architecture — **Stella Ops** (Consolidated • 2025Q4) + +> **Want the 10-minute tour?** See [`high-level-architecture.md`](high-level-architecture.md); this file retains the exhaustive reference. > **Purpose.** A complete, implementation‑ready map of Stella Ops: product vision, all runtime components, trust boundaries, tokens/licensing, control/data flows, storage, APIs, security, scale, DevOps, and verification logic. > **Scope.** This file **replaces** the separate `components.md`; all component details now live here. diff --git a/docs/10_OFFLINE_KIT.md b/docs/10_OFFLINE_KIT.md index 4a9c9d9c..f224e295 100755 --- a/docs/10_OFFLINE_KIT.md +++ b/docs/10_OFFLINE_KIT.md @@ -79,10 +79,10 @@ store as `STELLA_JWT` in **`.env`**. | ---------------------- | ---------------------------------------- | | **v0.1 α (late 2025)** | Manual OUK import • Zastava beta | | **v0.3 β (Q2 2026)** | Auto‑apply delta patch • nightly re‑scan | -| **v0.4 RC (Q3 2026)** | LDAP/AD SSO • registry scanner GA | -| **v1.0 GA (Q4 2026)** | Custom TLS/crypto adaptors (**incl. SM2**)—enabled where law or security requires it | - -Full details live in the public [Road‑map](../roadmap/README.md). +| **v0.4 RC (Q3 2026)** | LDAP/AD SSO • registry scanner GA | +| **v1.0 GA (Q4 2026)** | Custom TLS/crypto adaptors (**incl. SM2**)—enabled where law or security requires it | + +Full details live in the public [Road‑map](05_ROADMAP.md). --- diff --git a/docs/11_GOVERNANCE.md b/docs/11_GOVERNANCE.md index 2b835033..4b228b94 100755 --- a/docs/11_GOVERNANCE.md +++ b/docs/11_GOVERNANCE.md @@ -47,20 +47,20 @@ Approval is recorded via Git forge review or a signed commit trailer ## 4 · Release authority & provenance 🔏 -* Every tag is **co‑signed by at least one Security Maintainer**. -* CI emits a **signed SPDX SBOM** + **Cosign provenance**. -* Release cadence is fixed – see [public Road‑map](../roadmap/README.md). -* Security fixes may create out‑of‑band `x.y.z‑hotfix` tags. +* Every tag is **co‑signed by at least one Security Maintainer**. +* CI emits a **signed SPDX SBOM** + **Cosign provenance**. +* Release cadence is fixed – see [public Road‑map](05_ROADMAP.md). +* Security fixes may create out‑of‑band `x.y.z‑hotfix` tags. --- ## 5 · Escalation lanes 🚦 -| Situation | Escalation | -|-----------|------------| -| Technical deadlock | **Maintainer Summit** (recorded & published) | -| Security bug | Follow [Security Policy](../security/01_SECURITY_POLICY.md) | -| Code of Conduct violation | See `12_CODE_OF_CONDUCT.md` escalation ladder | +| Situation | Escalation | +|-----------|------------| +| Technical deadlock | **Maintainer Summit** (recorded & published) | +| Security bug | Follow [Security Policy](13_SECURITY_POLICY.md) | +| Code of Conduct violation | See `12_CODE_OF_CONDUCT.md` escalation ladder | --- @@ -90,4 +90,4 @@ section directly.)* | `@alice` | Core scanner • Security | 2025‑04 | | `@bob` | UI • Docs | 2025‑06 | ---- \ No newline at end of file +--- diff --git a/docs/12_CODE_OF_CONDUCT.md b/docs/12_CODE_OF_CONDUCT.md index 236b4ab8..62cc5916 100755 --- a/docs/12_CODE_OF_CONDUCT.md +++ b/docs/12_CODE_OF_CONDUCT.md @@ -31,7 +31,7 @@ If anything here conflicts with the upstream covenant, *our additions win*. | Channel | When to use | |---------|-------------| -| `conduct@stella-ops.org` (PGP key [`keys/#pgp`](../keys/#pgp)) | **Primary, confidential** – anything from micro‑aggressions to serious harassment | +| `conduct@stella-ops.org` (PGP key [`keys/#pgp`](https://stella-ops.org/keys/#pgp)) | **Primary, confidential** – anything from micro‑aggressions to serious harassment | | Matrix `/msg @coc-bot:libera.chat` | Quick, in‑chat nudge for minor issues | | Public issue with label `coc` | Transparency preferred and **you feel safe** doing so | diff --git a/docs/12_PERFORMANCE_WORKBOOK.md b/docs/12_PERFORMANCE_WORKBOOK.md index 4c96b766..26a5ccc4 100755 --- a/docs/12_PERFORMANCE_WORKBOOK.md +++ b/docs/12_PERFORMANCE_WORKBOOK.md @@ -143,7 +143,7 @@ P99 = 48 ms. Meets 50 ms gate. ## 8 Trend Snapshot -![Perf trend spark‑line placeholder](perf‑trend.svg) +> _Perf trend spark‑line screenshot pending upload._ > **Grafana/Alerting** – Import `docs/modules/scanner/operations/analyzers-grafana-dashboard.json` and point it at the Prometheus datasource storing `scanner_analyzer_bench_*` metrics. Configure an alert on `scanner_analyzer_bench_regression_ratio` ≥ 1.20 (default limit); the bundled Stat panel surfaces breached scenarios (non-zero values). On-call runbook: `docs/modules/scanner/operations/analyzers.md`. diff --git a/docs/13_SECURITY_POLICY.md b/docs/13_SECURITY_POLICY.md index f74c380c..b929e971 100755 --- a/docs/13_SECURITY_POLICY.md +++ b/docs/13_SECURITY_POLICY.md @@ -21,7 +21,7 @@ Pre‑GA lines receive **critical** and **high**‑severity fixes only. | Channel | PGP‑encrypted? | Target SLA | |---------|---------------|-----------| -| `security@stella-ops.org` | **Yes** – PGP key: [`/keys/#pgp`](../keys/#pgp) | 72 h acknowledgement | +| `security@stella-ops.org` | **Yes** – PGP key: [`/keys/#pgp`](https://stella-ops.org/keys/#pgp) | 72 h acknowledgement | | Matrix DM → `@sec‑bot:libera.chat` | Optional | 72 h acknowledgement | | Public issue with label `security` | No (for non‑confidential flaws) | 7 d acknowledgement | @@ -65,8 +65,8 @@ We aim for **30 days** from report to release for critical/high issues; medium | Purpose | Fingerprint | Where to fetch | |---------|-------------|----------------| -| **PGP (sec‑team)** | `3A5C ​71F3 ​... ​7D9B` | [`/keys/#pgp`](../keys/#pgp) | -| **Cosign release key** | `AB12 ... EF90` | [`/keys/#cosign`](../keys/#cosign) | +| **PGP (sec‑team)** | `3A5C ​71F3 ​... ​7D9B` | [`/keys/#pgp`](https://stella-ops.org/keys/#pgp) | +| **Cosign release key** | `AB12 ... EF90` | [`/keys/#cosign`](https://stella-ops.org/keys/#cosign) | Verify all downloads (TLS 1.3 by default; 1.2 allowed only via a custom TLS provider such as GOST): diff --git a/docs/README.md b/docs/README.md index 86540f39..e07dfba9 100755 --- a/docs/README.md +++ b/docs/README.md @@ -1,157 +1,38 @@ -# Stella Ops +# Stella Ops -> **Self‑hosted, SBOM‑first DevSecOps platform – offline‑friendly, AGPL‑3.0, free up to {{ quota_token }} scans per UTC day (soft delay only, never blocks).** +> Stella Ops is the sovereign, SBOM‑first security platform that proves every container decision with deterministic scans, explainable policy verdicts, and offline‑ready provenance. -Stella Ops lets you discover container vulnerabilities in **< 5 s** without sending a single byte outside your network. -Everything here is open‑source and versioned — when you check out a git tag, the docs match the code you are running. +- **Sovereign by design** – bring your own trust roots, vulnerability advisory sources, VEX sources, regional crypto, and Offline Update Kits that never phone home. +- **Deterministic + replayable** – every scan can be reproduced bit‑for‑bit with DSSE + OpenVEX evidence. +- **Actionable signal** – lattice logic ranks exploitability, and the policy engine lets you tailor VEX handling, muting, and expiration rules for your environment. ---- +**Proof points:** SBOM dependency and vulnerability dependency cartographing work, deterministic replay manifests, lattice policy UI with OpenVEX, and post‑quantum trust packs ready for regulated sectors. -## 🚀 Start here (first 60 minutes) +## Choose Your Path -| Step | What you will learn | Doc | -|------|--------------------|-----| -| 1 ️⃣ | 90‑second elevator pitch & pillars | **[What Is Stella Ops?](01_WHAT_IS_IT.md)** | -| 2 ️⃣ | Pain points it solves | **[Why Does It Exist?](02_WHY.md)** | -| 3 ️⃣ | Install & run a scan in 10 min | **[Install Guide](21_INSTALL_GUIDE.md)** | -| 4 ️⃣ | Components & data‑flow | **[High‑Level Architecture](07_HIGH_LEVEL_ARCHITECTURE.md)** | -| 5 ️⃣ | Integrate the CLI / REST API | **[API & CLI Reference](09_API_CLI_REFERENCE.md)** | -| 6 ️⃣ | Vocabulary used throughout the docs | **[Glossary](14_GLOSSARY_OF_TERMS.md)** | +| If you want to… | Open this | Read time | +|-----------------|-----------|-----------| +| Understand the promise and pain we solve | `overview.md` | ≈ 2 min | +| Run a first scan and see the CLI | `quickstart.md` | ≈ 5 min | +| Browse key capabilities at a glance | `key-features.md` | ≈ 3 min | +| Check architecture, road to production, or evaluate fit | See “Dig deeper” below | ≤ 30 min curated set | ---- +## Explore the Essentials -## 📚 Complete Table of Contents +1. **Value in context** – [Overview](overview.md) compresses the “Why” + “What” stories and shows how Stella Ops stands apart. +2. **Try it fast** – [Quickstart](quickstart.md) walks through fetching the signed bundles, configuring `.env`, and verifying the first scan. +3. **Feature confidence** – [Key Features](key-features.md) gives five capability cards covering Delta SBOM, VEX‑first policy, Sovereign crypto, Deterministic replay, and Transparent quotas. +4. **Up‑next checkpoints** – [Evaluation checklist](evaluate/checklist.md) helps teams plan Day‑0 to Day‑30 adoption milestones. -
-Click to expand the full docs index +## Dig Deeper (curated reading) -### Overview -- **01 – [What Is Stella Ops?](01_WHAT_IS_IT.md)** -- **02 – [Why Does It Exist?](02_WHY.md)** -- **03 – [Vision & Road‑map](03_VISION.md)** -- **04 – [Feature Matrix](04_FEATURE_MATRIX.md)** +- **Install & operations:** [Installation guide](21_INSTALL_GUIDE.md), [Offline Update Kit](24_OFFLINE_KIT.md), [Security hardening](17_SECURITY_HARDENING_GUIDE.md). +- **Architecture & modules:** [High‑level architecture](high-level-architecture.md), [Module dossiers](modules/platform/architecture-overview.md), [Strategic differentiators](moat.md). +- **Policy & governance:** [Policy templates](60_POLICY_TEMPLATES.md), [Legal & quota FAQ](29_LEGAL_FAQ_QUOTA.md), [Governance charter](11_GOVERNANCE.md). +- **UI & glossary:** [Console guide](15_UI_GUIDE.md), [Accessibility](accessibility.md), [Glossary](14_GLOSSARY_OF_TERMS.md). +- **Technical documentation:** [Full technical index](technical/README.md) for architecture, APIs, module dossiers, and operations playbooks. +- **FAQs & readiness:** [FAQ matrix](23_FAQ_MATRIX.md), [Roadmap (external)](https://stella-ops.org/roadmap/), [Release engineering playbook](13_RELEASE_ENGINEERING_PLAYBOOK.md). -### Reference & concepts -- **05 – [System Requirements Specification](05_SYSTEM_REQUIREMENTS_SPEC.md)** -- **07 – [High‑Level Architecture](07_HIGH_LEVEL_ARCHITECTURE.md)** -- **08 – [Architecture Decision Records](adr/index.md)** -- **08 – Module Architecture Dossiers** - - [Architecture Overview](modules/platform/architecture-overview.md) - - [Scanner](modules/scanner/architecture.md) - - [Concelier](modules/concelier/architecture.md) - - [Excititor](modules/excititor/architecture.md) - - [Excititor Mirrors](modules/excititor/mirrors.md) - - [Signer](modules/signer/architecture.md) - - [Attestor](modules/attestor/architecture.md) - - [Authority](modules/authority/architecture.md) - - [Policy Engine](modules/policy/architecture.md) - - [Notify](modules/notify/architecture.md) - - [Scheduler](modules/scheduler/architecture.md) - - [CLI](modules/cli/architecture.md) - - [Web UI](modules/ui/architecture.md) - - [Zastava Runtime](modules/zastava/architecture.md) - - [Release & Operations](modules/devops/architecture.md) -- **09 – [API & CLI Reference](09_API_CLI_REFERENCE.md)** -- **10 – [Plug‑in SDK Guide](10_PLUGIN_SDK_GUIDE.md)** -- **10 – [Concelier CLI Quickstart](10_CONCELIER_CLI_QUICKSTART.md)** -- **10 – [BuildX Generator Quickstart](dev/BUILDX_PLUGIN_QUICKSTART.md)** -- **10 – [Scanner Cache Configuration](dev/SCANNER_CACHE_CONFIGURATION.md)** -- **30 – [Excititor Connector Packaging Guide](dev/30_EXCITITOR_CONNECTOR_GUIDE.md)** -- **31 – [Aggregation-Only Contract Reference](ingestion/aggregation-only-contract.md)** -- **31 – [Advisory Observations & Linksets](advisories/aggregation.md)** -- **31 – [VEX Observations & Linksets](vex/aggregation.md)** -- **32 – [Entry-Point Detection Playbook](modules/scanner/operations/entrypoint.md)** -- **30 – Developer Templates** - - [Excititor Connector Skeleton](dev/templates/excititor-connector/) -- **11 – [Authority Service](11_AUTHORITY.md)** -- **11 – [Data Schemas](11_DATA_SCHEMAS.md)** -- **12 – [Performance Workbook](12_PERFORMANCE_WORKBOOK.md)** -- **13 – [Release‑Engineering Playbook](13_RELEASE_ENGINEERING_PLAYBOOK.md)** -- **20 – [CLI AOC Commands Reference](modules/cli/guides/cli-reference.md)** -- **20 – [Console CLI Parity Matrix](cli-vs-ui-parity.md)** -- **60 – [Policy Engine Overview](policy/overview.md)** -- **61 – [Policy DSL Grammar](policy/dsl.md)** -- **62 – [Policy Lifecycle & Approvals](policy/lifecycle.md)** -- **63 – [Policy Runs & Orchestration](policy/runs.md)** -- **64 – [Policy Exception Effects](policy/exception-effects.md)** -- **65 – [Policy Engine REST API](api/policy.md)** -- **66 – [Policy CLI Guide](modules/cli/guides/policy.md)** -- **67 – [Policy Editor Workspace](ui/policy-editor.md)** -- **68 – [Policy Observability](observability/policy.md)** -- **69 – [Console Observability](observability/ui-telemetry.md)** -- **70 – [Policy Governance & Least Privilege](security/policy-governance.md)** -- **70a – [Policy Gateway](policy/gateway.md)** -- **71 – [Policy Examples](examples/policies/README.md)** -- **72 – [Policy FAQ](faq/policy-faq.md)** -- **73 – [Policy Run DTOs](../src/Scheduler/__Libraries/StellaOps.Scheduler.Models/docs/SCHED-MODELS-20-001-POLICY-RUNS.md)** -- **30 – [Fixture Maintenance](dev/fixtures.md)** -- **74 – [Export Center Overview](modules/export-center/overview.md)** -- **75 – [Export Center Architecture](modules/export-center/architecture.md)** -- **76 – [Export Center Profiles](modules/export-center/profiles.md)** -- **77 – [Export Center API Reference](modules/export-center/api.md)** -- **78 – [Export Center CLI Guide](modules/export-center/cli.md)** -- **79 – [Export Center Trivy Adapters](modules/export-center/trivy-adapter.md)** -- **80 – [Export Center Mirror Bundles](modules/export-center/mirror-bundles.md)** -- **81 – [Export Center Provenance & Signing](modules/export-center/provenance-and-signing.md)** +Need more? The full documentation tree – ADRs, per‑module operations, schemas, developer references – stays untouched under the existing directories (`modules/`, `api/`, `dev/`, `ops/`), ready when you are. -### User & operator guides -- **14 – [Glossary](14_GLOSSARY_OF_TERMS.md)** -- **15 – [UI Guide](15_UI_GUIDE.md)** -- **16 – [Console AOC Dashboard](ui/console.md)** -- **16 – [Console Accessibility Guide](accessibility.md)** -- **17 – [Security Hardening Guide](17_SECURITY_HARDENING_GUIDE.md)** -- **17 – [Console Security Posture](security/console-security.md)** -- **18 – [Coding Standards](18_CODING_STANDARDS.md)** -- **19 – [Test‑Suite Overview](19_TEST_SUITE_OVERVIEW.md)** -- **21 – [Install Guide](21_INSTALL_GUIDE.md)** -- **21 – [Docker Install Recipes](install/docker.md)** -- **22 – [CI/CD Recipes Library](ci/20_CI_RECIPES.md)** -- **23 – [FAQ](23_FAQ_MATRIX.md)** -- **24 – [Offline Update Kit Admin Guide](24_OFFLINE_KIT.md)** -- **25 – [Mirror Operations Runbook](ops/concelier-mirror-operations.md)** -- **26 – [Concelier Apple Connector Operations](ops/concelier-apple-operations.md)** -- **27 – [Authority Key Rotation Playbook](ops/authority-key-rotation.md)** -- **28 – [Concelier CCCS Connector Operations](ops/concelier-cccs-operations.md)** -- **29 – [Concelier CISA ICS Connector Operations](ops/concelier-icscisa-operations.md)** -- **30 – [Concelier CERT-Bund Connector Operations](ops/concelier-certbund-operations.md)** -- **31 – [Concelier MSRC Connector – AAD Onboarding](ops/concelier-msrc-operations.md)** - - **32 – [Scanner Analyzer Bench Operations](ops/scanner-analyzers-operations.md)** - - **33 – [Scanner Artifact Store Migration](ops/scanner-rustfs-migration.md)** - - **34 – [Zastava Runtime Operations Runbook](ops/zastava-runtime-operations.md)** - - **35 – [Launch Readiness Checklist](ops/launch-readiness.md)** -- **36 – [Launch Cutover Runbook](ops/launch-cutover.md)** -- **37 – [Registry Token Service](ops/registry-token-service.md)** -- **37 – [Deployment Upgrade & Rollback Runbook](ops/deployment-upgrade-runbook.md)** -- **38 – [Policy Schema Export Automation](devops/policy-schema-export.md)** -- **40 – [Observability Guide (AOC)](observability/observability.md)** -- **41 – [Telemetry Collector Deployment](ops/telemetry-collector.md)** -- **42 – [Telemetry Storage Deployment](ops/telemetry-storage.md)** -- **43 – [Authority Scopes & Tenancy](security/authority-scopes.md)** -- **44 – [Container Deployment (AOC)](deploy/containers.md)** -- **45 – [Export Center Operations Runbook](operations/export-runbook.md)** - -### Notifications Studio -- **81 – [Notifications Overview](notifications/overview.md)** -- **82 – [Notifications Architecture](notifications/architecture.md)** -- **83 – [Notifications Rules](notifications/rules.md)** -- **84 – [Notifications Templates](notifications/templates.md)** -- **85 – [Notifications Digests](notifications/digests.md)** - -### Legal & licence -- **32 – [Legal & Quota FAQ](29_LEGAL_FAQ_QUOTA.md)** - -
- ---- - -## 🧹 Backlog hygiene - -> Imposed rule: Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied. - -- **Aggregation-Only Contract (AOC).** Ingestion services aggregate and link facts only—derived precedence, severity, and safe-fix hints live in Policy overlays and dedicated explorers. Review [`implplan/AGENTS.md`](implplan/AGENTS.md) and the AOC guardrails in [`aoc/aoc-guardrails.md`](aoc/aoc-guardrails.md). -- **Cartographer owns graphs.** SBOM Service emits projections/events; Cartographer (`CARTO-GRAPH-21-00x`) builds graph storage, overlays, and tiles. See `modules/concelier/architecture.md` (Cartographer handshake section) for handoff boundaries. -- **Notifier replaces legacy Notify.** Sprint‑15 `StellaOps.Notify.*` tasks are frozen; use the Notifications Studio/Notifier backlogs (`NOTIFY-SVC-38..40`, `WEB-NOTIFY-3x-00x`, `CLI-NOTIFY-3x-00x`). -- **Dedicated services for Vuln & Policy.** Vuln Explorer work flows through `src/VulnExplorer/StellaOps.VulnExplorer.Api`/Console/CLI (Sprint 29); gateway routes proxy only. Policy Engine remains the sole source for precedence/suppression overlays. -- **Cleanup log.** The backlog consolidation summary lives in [`backlog/2025-10-cleanup.md`](backlog/2025-10-cleanup.md). - -© 2025 Stella Ops contributors – licensed AGPL‑3.0‑or‑later +© 2025 Stella Ops contributors – AGPL‑3.0‑or‑later diff --git a/docs/TASKS.completed.md b/docs/TASKS.completed.md new file mode 100644 index 00000000..0b19a0e5 --- /dev/null +++ b/docs/TASKS.completed.md @@ -0,0 +1,88 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| DOCS-VISITOR-30-001 | DONE (2025-10-30) | Docs Guild | — | Reorganize visitor-facing documentation (README, overview, quickstart, key features) for rapid evaluation flow. | ✅ New visitor doc stack published; ✅ README links updated; ✅ Legacy pages slotted into deeper-read tier. | +| DOC7.README-INDEX | DONE (2025-10-17) | Docs Guild | — | Refresh index docs (docs/README.md + root README) after architecture dossier split and Offline Kit overhaul. | ✅ ToC reflects new component architecture docs; ✅ root README highlights updated doc set; ✅ Offline Kit guide linked correctly. | +| DOC4.AUTH-PDG | DONE (2025-10-19) | Docs Guild, Plugin Team | PLG6.DOC | Copy-edit `docs/dev/31_AUTHORITY_PLUGIN_DEVELOPER_GUIDE.md`, export lifecycle diagram, add LDAP RFC cross-link. | ✅ PR merged with polish; ✅ Diagram committed; ✅ Slack handoff posted. | +| DOC1.AUTH | DONE (2025-10-12) | Docs Guild, Authority Core | CORE5B.DOC | Draft `docs/11_AUTHORITY.md` covering architecture, configuration, bootstrap flows. | ✅ Architecture + config sections approved by Core; ✅ Samples reference latest options; ✅ Offline note added. | +| DOC3.Concelier-Authority | DONE (2025-10-12) | Docs Guild, DevEx | FSR4 | Polish operator/runbook sections (DOC3/DOC5) to document Concelier authority rollout, bypass logging, and enforcement checklist. | ✅ DOC3/DOC5 updated with audit runbook references; ✅ enforcement deadline highlighted; ✅ Docs guild sign-off. | +| DOC5.Concelier-Runbook | DONE (2025-10-12) | Docs Guild | DOC3.Concelier-Authority | Produce dedicated Concelier authority audit runbook covering log fields, monitoring recommendations, and troubleshooting steps. | ✅ Runbook published; ✅ linked from DOC3/DOC5; ✅ alerting guidance included. | +| FEEDDOCS-DOCS-05-001 | DONE (2025-10-11) | Docs Guild | FEEDMERGE-ENGINE-04-001, FEEDMERGE-ENGINE-04-002 | Publish Concelier conflict resolution runbook covering precedence workflow, merge-event auditing, and Sprint 3 metrics. | ✅ `docs/modules/concelier/operations/conflict-resolution.md` committed; ✅ metrics/log tables align with latest merge code; ✅ Ops alert guidance handed to Concelier team. | +| FEEDDOCS-DOCS-05-002 | DONE (2025-10-16) | Docs Guild, Concelier Ops | FEEDDOCS-DOCS-05-001 | Ops sign-off captured: conflict runbook circulated, alert thresholds tuned, and rollout decisions documented in change log. | ✅ Ops review recorded; ✅ alert thresholds finalised using `docs/modules/concelier/operations/authority-audit-runbook.md`; ✅ change-log entry linked from runbook once GHSA/NVD/OSV regression fixtures land. | +| DOCS-ADR-09-001 | DONE (2025-10-19) | Docs Guild, DevEx | — | Establish ADR process (`docs/adr/0000-template.md`) and document usage guidelines. | Template published; README snippet linking ADR process; announcement posted (`docs/updates/2025-10-18-docs-guild.md`). | +| DOCS-EVENTS-09-002 | DONE (2025-10-19) | Docs Guild, Platform Events | SCANNER-EVENTS-15-201 | Publish event schema catalog (`docs/events/`) for `scanner.report.ready@1`, `scheduler.rescan.delta@1`, `attestor.logged@1`. | Schemas validated (Ajv CI hooked); docs/events/README summarises usage; Platform Events notified via `docs/updates/2025-10-18-docs-guild.md`. | +| DOCS-EVENTS-09-003 | DONE (2025-10-19) | Docs Guild | DOCS-EVENTS-09-002 | Add human-readable envelope field references and canonical payload samples for published events, including offline validation workflow. | Tables explain common headers/payload segments; versioned sample payloads committed; README links to validation instructions and samples. | +| DOCS-EVENTS-09-004 | DONE (2025-10-19) | Docs Guild, Scanner WebService | SCANNER-EVENTS-15-201 | Refresh scanner event docs to mirror DSSE-backed report fields, document `scanner.scan.completed`, and capture canonical sample validation. | Schemas updated for new payload shape; README references DSSE reuse and validation test; samples align with emitted events. | +| PLATFORM-EVENTS-09-401 | DONE (2025-10-21) | Platform Events Guild | DOCS-EVENTS-09-003 | Embed canonical event samples into contract/integration tests and ensure CI validates payloads against published schemas. | Notify models tests now run schema validation against `docs/events/*.json`, event schemas allow optional `attributes`, and docs capture the new validation workflow. | +| RUNTIME-GUILD-09-402 | DONE (2025-10-19) | Runtime Guild | SCANNER-POLICY-09-107 | Confirm Scanner WebService surfaces `quietedFindingCount` and progress hints to runtime consumers; document readiness checklist. | Runtime verification run captures enriched payload; checklist/doc updates merged; stakeholders acknowledge availability. | +| DOCS-CONCELIER-07-201 | DONE (2025-10-22) | Docs Guild, Concelier WebService | FEEDWEB-DOCS-01-001 | Final editorial review and publish pass for Concelier authority toggle documentation (Quickstart + operator guide). | Review feedback resolved, publish PR merged, release notes updated with documentation pointer. | +| DOCS-RUNTIME-17-004 | DONE (2025-10-26) | Docs Guild, Runtime Guild | SCANNER-EMIT-17-701, ZASTAVA-OBS-17-005, DEVOPS-REL-17-002 | Document build-id workflows: SBOM exposure, runtime event payloads (`process.buildId`), Scanner `/policy/runtime` response (`buildIds` list), debug-store layout, and operator guidance for symbol retrieval. | Architecture + operator docs updated with build-id sections (Observer, Scanner, CLI), examples show `readelf` output + debuginfod usage, references linked from Offline Kit/Release guides + CLI help. | + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| DOCS-AOC-19-001 | DONE (2025-10-26) | Docs Guild, Concelier Guild | CONCELIER-WEB-AOC-19-001, EXCITITOR-WEB-AOC-19-001 | Author `/docs/ingestion/aggregation-only-contract.md` covering philosophy, invariants, schemas, error codes, migration, observability, and security checklist. | New doc published with compliance checklist; cross-links from existing docs added. | +| DOCS-AOC-19-002 | DONE (2025-10-26) | Docs Guild, Architecture Guild | DOCS-AOC-19-001 | Update `/docs/modules/platform/architecture-overview.md` to include AOC boundary, raw stores, and sequence diagram (fetch → guard → raw insert → policy evaluation). | Overview doc updated with diagrams/text; lint passes; stakeholders sign off. | +| DOCS-AOC-19-003 | DONE (2025-10-26) | Docs Guild, Policy Guild | POLICY-AOC-19-003 | Refresh `/docs/modules/policy/architecture.md` clarifying ingestion boundary, raw inputs, and policy-only derived data. | Doc highlights raw-only ingestion contract, updated diagrams merge, compliance checklist added. | +| DOCS-AOC-19-004 | DONE (2025-10-26) | Docs Guild, UI Guild | UI-AOC-19-001 | Extend `/docs/ui/console.md` with Sources dashboard tiles, violation drill-down workflow, and verification action. | UI doc updated with screenshots/flow descriptions, compliance checklist appended. | +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| DOCS-POLICY-20-001 | DONE (2025-10-26) | Docs Guild, Policy Guild | POLICY-ENGINE-20-000 | Author `/docs/policy/overview.md` covering concepts, inputs/outputs, determinism, and compliance checklist. | Doc published with diagrams + glossary; lint passes; checklist included. | +| DOCS-POLICY-20-002 | DONE (2025-10-26) | Docs Guild, Policy Guild | POLICY-ENGINE-20-001 | Write `/docs/policy/dsl.md` with grammar, built-ins, examples, anti-patterns. | DSL doc includes grammar tables, examples, compliance checklist; validated against parser tests. | +| DOCS-POLICY-20-003 | DONE (2025-10-26) | Docs Guild, Authority Core | AUTH-POLICY-20-001 | Publish `/docs/policy/lifecycle.md` describing draft→approve workflow, roles, audit, compliance list. | Lifecycle doc linked from UI/CLI help; approvals roles documented; checklist appended. | +| DOCS-POLICY-20-004 | DONE (2025-10-26) | Docs Guild, Scheduler Guild | SCHED-MODELS-20-001 | Create `/docs/policy/runs.md` detailing run modes, incremental mechanics, cursors, replay. | Run doc includes sequence diagrams + compliance checklist; cross-links to scheduler docs. | +| DOCS-POLICY-20-005 | DONE (2025-10-26) | Docs Guild, BE-Base Platform Guild | WEB-POLICY-20-001 | Draft `/docs/api/policy.md` describing endpoints, schemas, error codes. | API doc validated against OpenAPI; examples included; checklist appended. | +| DOCS-POLICY-20-006 | DONE (2025-10-26) | Docs Guild, DevEx/CLI Guild | CLI-POLICY-20-002 | Produce `/docs/modules/cli/guides/policy.md` with command usage, exit codes, JSON output contracts. | CLI doc includes examples, exit codes, compliance checklist. | +| DOCS-POLICY-20-007 | DONE (2025-10-26) | Docs Guild, UI Guild | UI-POLICY-20-001 | Document `/docs/ui/policy-editor.md` covering editor, simulation, diff workflows, approvals. | UI doc includes screenshots/placeholders, accessibility notes, compliance checklist. | +| DOCS-POLICY-20-008 | DONE (2025-10-26) | Docs Guild, Architecture Guild | POLICY-ENGINE-20-003 | Write `/docs/modules/policy/architecture.md` (new epic content) with sequence diagrams, selection strategy, schema. | Architecture doc merged with diagrams; compliance checklist appended; references updated. | +| DOCS-POLICY-20-009 | DONE (2025-10-26) | Docs Guild, Observability Guild | POLICY-ENGINE-20-007 | Add `/docs/observability/policy.md` for metrics/traces/logs, sample dashboards. | Observability doc includes metrics tables, dashboard screenshots, checklist. | +| DOCS-POLICY-20-010 | DONE (2025-10-26) | Docs Guild, Security Guild | AUTH-POLICY-20-002 | Publish `/docs/security/policy-governance.md` covering scopes, approvals, tenancy, least privilege. | Security doc merged; compliance checklist appended; reviewed by Security Guild. | +| DOCS-POLICY-20-011 | DONE (2025-10-26) | Docs Guild, Policy Guild | POLICY-ENGINE-20-001 | Populate `/docs/examples/policies/` with baseline/serverless/internal-only samples and commentary. | Example policies committed with explanations; lint passes; compliance checklist per file. | +| DOCS-POLICY-20-012 | DONE (2025-10-26) | Docs Guild, Support Guild | WEB-POLICY-20-003 | Draft `/docs/faq/policy-faq.md` addressing common pitfalls, VEX conflicts, determinism issues. | FAQ published with Q/A entries, cross-links, compliance checklist. | + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| DOCS-CONSOLE-23-001 | DONE (2025-10-26) | Docs Guild, Console Guild | CONSOLE-CORE-23-004 | Publish `/docs/ui/console-overview.md` covering IA, tenant model, global filters, and AOC alignment with compliance checklist. | Doc merged with diagrams + overview tables; checklist appended; Console Guild sign-off. | +| DOCS-CONSOLE-23-002 | DONE (2025-10-26) | Docs Guild, Console Guild | DOCS-CONSOLE-23-001 | Author `/docs/ui/navigation.md` detailing routes, breadcrumbs, keyboard shortcuts, deep links, and tenant context switching. | Navigation doc merged with shortcut tables and screenshots; accessibility checklist satisfied. | +| DOCS-CONSOLE-23-003 | DONE (2025-10-26) | Docs Guild, SBOM Service Guild, Console Guild | SBOM-CONSOLE-23-001, CONSOLE-FEAT-23-102 | Document `/docs/ui/sbom-explorer.md` (catalog, detail, graph overlays, exports) including compliance checklist and performance tips. | Doc merged with annotated screenshots, export instructions, and overlay examples; checklist appended. | +| DOCS-CONSOLE-23-004 | DONE (2025-10-26) | Docs Guild, Concelier Guild, Excititor Guild | CONCELIER-CONSOLE-23-001, EXCITITOR-CONSOLE-23-001 | Produce `/docs/ui/advisories-and-vex.md` explaining aggregation-not-merge, conflict indicators, raw viewers, and provenance banners. | Doc merged; raw JSON examples included; compliance checklist complete. | +| DOCS-CONSOLE-23-005 | DONE (2025-10-26) | Docs Guild, Policy Guild | POLICY-CONSOLE-23-001, CONSOLE-FEAT-23-104 | Write `/docs/ui/findings.md` describing filters, saved views, explain drawer, exports, and CLI parity callouts. | Doc merged with filter matrix + explain walkthrough; checklist appended. | +| DOCS-CONSOLE-23-006 | DONE (2025-10-26) | Docs Guild, Policy Guild, Product Ops | POLICY-CONSOLE-23-002, CONSOLE-FEAT-23-105 | Publish `/docs/ui/policies.md` with editor, simulation, approvals, compliance checklist, and RBAC mapping. | Doc merged; Monaco screenshots + simulation diff examples included; approval flow described; checklist appended. | +| DOCS-CONSOLE-23-007 | DONE (2025-10-26) | Docs Guild, Scheduler Guild | SCHED-CONSOLE-23-001, CONSOLE-FEAT-23-106 | Document `/docs/ui/runs.md` covering queues, live progress, diffs, retries, evidence downloads, and troubleshooting. | Doc merged with SSE troubleshooting, metrics references, compliance checklist. | +| DOCS-CONSOLE-23-008 | DONE (2025-10-26) | Docs Guild, Authority Guild | AUTH-CONSOLE-23-002, CONSOLE-FEAT-23-108 | Draft `/docs/ui/admin.md` describing users/roles, tenants, tokens, integrations, fresh-auth prompts, and RBAC mapping. | Doc merged with tables for scopes vs roles, screenshots, compliance checklist. | +| DOCS-CONSOLE-23-009 | DONE (2025-10-27) | Docs Guild, DevOps Guild | DOWNLOADS-CONSOLE-23-001, CONSOLE-FEAT-23-109 | Publish `/docs/ui/downloads.md` listing product images, commands, offline instructions, parity with CLI, and compliance checklist. | Doc merged; manifest sample included; copy-to-clipboard guidance documented; checklist complete. | +| DOCS-CONSOLE-23-010 | DONE (2025-10-27) | Docs Guild, Deployment Guild, Console Guild | DEVOPS-CONSOLE-23-002, CONSOLE-REL-23-301 | Write `/docs/deploy/console.md` (Helm, ingress, TLS, CSP, env vars, health checks) with compliance checklist. | Deploy doc merged; templates validated; CSP guidance included; checklist appended. | +| DOCS-CONSOLE-23-011 | DONE (2025-10-28) | Docs Guild, Deployment Guild | DOCS-CONSOLE-23-010 | Update `/docs/install/docker.md` to cover Console image, Compose/Helm usage, offline tarballs, parity with CLI. | Doc updated with new sections; commands validated; compliance checklist appended. | +| DOCS-CONSOLE-23-012 | DONE (2025-10-28) | Docs Guild, Security Guild | AUTH-CONSOLE-23-003, WEB-CONSOLE-23-002 | Publish `/docs/security/console-security.md` detailing OIDC flows, scopes, CSP, fresh-auth, evidence handling, and compliance checklist. | Security doc merged; threat model notes included; checklist appended. | +| DOCS-CONSOLE-23-013 | DONE (2025-10-28) | Docs Guild, Observability Guild | TELEMETRY-CONSOLE-23-001, CONSOLE-QA-23-403 | Write `/docs/observability/ui-telemetry.md` cataloguing metrics/logs/traces, dashboards, alerts, and feature flags. | Doc merged with instrumentation tables, dashboard screenshots, checklist appended. | +| DOCS-CONSOLE-23-014 | DONE (2025-10-28) | Docs Guild, Console Guild, CLI Guild | CONSOLE-DOC-23-502 | Maintain `/docs/cli-vs-ui-parity.md` matrix and integrate CI check guidance. | Matrix published with parity status, CI workflow documented, compliance checklist appended. | + +| DOCS-CONSOLE-23-017 | DONE (2025-10-27) | Docs Guild, Console Guild | CONSOLE-FEAT-23-101..109 | Create `/docs/examples/ui-tours.md` providing triage, audit, policy rollout walkthroughs with annotated screenshots and GIFs. | UI tours doc merged; capture instructions + asset placeholders committed; compliance checklist appended. | +| DOCS-CONSOLE-23-018 | DONE (2025-10-27) | Docs Guild, Security Guild | DOCS-CONSOLE-23-012 | Execute console security compliance checklist and capture Security Guild sign-off in Sprint 23 log. | Checklist completed; findings addressed or tickets filed; sign-off noted in updates file. | +| DOCS-LNM-22-006 | DONE (2025-10-27) | Docs Guild, Architecture Guild | CONCELIER-LNM-21-001..005, EXCITITOR-LNM-21-001..005 | Refresh `/docs/modules/concelier/architecture.md` and `/docs/modules/excititor/architecture.md` describing observation/linkset pipelines and event contracts. | Architecture docs updated with observation/linkset flow + event tables; revisit once service implementations land. | + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| DOCS-EXC-25-004 | DONE (2025-10-27) | Docs Guild, Policy Guild | POLICY-ENGINE-70-001 | Document `/docs/policy/exception-effects.md` explaining evaluation order, conflicts, simulation. | Doc merged; tests cross-referenced; checklist appended. | + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| DOCS-EXPORT-35-001 | DONE (2025-10-29) | Docs Guild | EXPORT-SVC-35-001..006 | Author `/docs/modules/export-center/overview.md` covering purpose, profiles, security, AOC alignment, surfaces, ending with imposed rule statement. | Doc merged with diagrams/examples; imposed rule line present; index updated. | +| DOCS-EXPORT-35-002 | DONE (2025-10-29) | Docs Guild | EXPORT-SVC-35-002..005 | Publish `/docs/modules/export-center/architecture.md` describing planner, adapters, manifests, signing, distribution flows, restating imposed rule. | Architecture doc merged; sequence diagrams included; rule statement appended. | +| DOCS-EXPORT-35-003 | DONE (2025-10-29) | Docs Guild | EXPORT-SVC-35-003..004 | Publish `/docs/modules/export-center/profiles.md` detailing schema fields, examples, compatibility, and imposed rule reminder. | Profiles doc merged; JSON schemas linked; imposed rule noted. | +| DOCS-EXPORT-36-004 | DONE (2025-10-29) | Docs Guild | EXPORT-SVC-36-001..004, WEB-EXPORT-36-001 | Publish `/docs/modules/export-center/api.md` covering endpoints, payloads, errors, and mention imposed rule. | API doc merged; examples validated; rule included. | +| DOCS-EXPORT-36-005 | DONE (2025-10-29) | Docs Guild | CLI-EXPORT-35-001, CLI-EXPORT-36-001 | Publish `/docs/modules/export-center/cli.md` with command reference, CI scripts, verification steps, restating imposed rule. | CLI doc merged; script snippets tested; rule appended. | +| DOCS-EXPORT-36-006 | DONE (2025-10-29) | Docs Guild | EXPORT-SVC-36-001, DEVOPS-EXPORT-36-001 | Publish `/docs/modules/export-center/trivy-adapter.md` covering field mappings, compatibility matrix, and imposed rule reminder. | Doc merged; mapping tables validated; rule included. | +| DOCS-EXPORT-37-001 | DONE (2025-10-29) | Docs Guild | EXPORT-SVC-37-001, DEVOPS-EXPORT-37-001 | Publish `/docs/modules/export-center/mirror-bundles.md` describing filesystem/OCI layouts, delta/encryption, import guide, ending with imposed rule. | Doc merged; diagrams provided; verification steps tested; rule stated. | +| DOCS-EXPORT-37-002 | DONE (2025-10-29) | Docs Guild | EXPORT-SVC-35-005, EXPORT-SVC-37-002 | Publish `/docs/modules/export-center/provenance-and-signing.md` detailing manifests, attestation flow, verification, reiterating imposed rule. | Doc merged; signature examples validated; rule appended. | +| DOCS-EXPORT-37-003 | DONE (2025-10-29) | Docs Guild | DEVOPS-EXPORT-37-001 | Publish `/docs/operations/export-runbook.md` covering failures, tuning, capacity planning, with imposed rule reminder. | Runbook merged; procedures validated; rule included. | + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| DOCS-NOTIFY-38-001 | DONE (2025-10-29) | Docs Guild, Notifications Service Guild | NOTIFY-SVC-38-001..004 | Publish `/docs/notifications/overview.md` and `/docs/notifications/architecture.md`, each ending with imposed rule reminder. | Docs merged; diagrams verified; imposed rule appended. | +| DOCS-NOTIFY-39-002 | DONE (2025-10-29) | Docs Guild, Notifications Service Guild | NOTIFY-SVC-39-001..004 | Publish `/docs/notifications/rules.md`, `/docs/notifications/templates.md`, `/docs/notifications/digests.md` with examples and imposed rule line. | Docs merged; examples validated; imposed rule appended. | + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| DOCS-PACKS-43-001 | DONE (2025-10-27) | Docs Guild, Task Runner Guild | PACKS-REG-42-001, TASKRUN-42-001 | Publish `/docs/task-packs/spec.md`, `/docs/task-packs/authoring-guide.md`, `/docs/task-packs/registry.md`, `/docs/task-packs/runbook.md`, `/docs/security/pack-signing-and-rbac.md`, `/docs/operations/cli-release-and-packaging.md` with imposed rule statements. | Docs merged; tutorials validated; imposed rule appended; cross-links added. | + diff --git a/docs/TASKS.md b/docs/TASKS.md index ac7b9456..10d1877a 100644 --- a/docs/TASKS.md +++ b/docs/TASKS.md @@ -1,24 +1,9 @@ # Docs Guild Task Board (UTC 2025-10-10) +> Blocked: waiting on telemetry core deliverable (TELEMETRY-OBS-50-001) to finalise architecture details and diagrams. + | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| DOC7.README-INDEX | DONE (2025-10-17) | Docs Guild | — | Refresh index docs (docs/README.md + root README) after architecture dossier split and Offline Kit overhaul. | ✅ ToC reflects new component architecture docs; ✅ root README highlights updated doc set; ✅ Offline Kit guide linked correctly. | -| DOC4.AUTH-PDG | DONE (2025-10-19) | Docs Guild, Plugin Team | PLG6.DOC | Copy-edit `docs/dev/31_AUTHORITY_PLUGIN_DEVELOPER_GUIDE.md`, export lifecycle diagram, add LDAP RFC cross-link. | ✅ PR merged with polish; ✅ Diagram committed; ✅ Slack handoff posted. | -| DOC1.AUTH | DONE (2025-10-12) | Docs Guild, Authority Core | CORE5B.DOC | Draft `docs/11_AUTHORITY.md` covering architecture, configuration, bootstrap flows. | ✅ Architecture + config sections approved by Core; ✅ Samples reference latest options; ✅ Offline note added. | -| DOC3.Concelier-Authority | DONE (2025-10-12) | Docs Guild, DevEx | FSR4 | Polish operator/runbook sections (DOC3/DOC5) to document Concelier authority rollout, bypass logging, and enforcement checklist. | ✅ DOC3/DOC5 updated with audit runbook references; ✅ enforcement deadline highlighted; ✅ Docs guild sign-off. | -| DOC5.Concelier-Runbook | DONE (2025-10-12) | Docs Guild | DOC3.Concelier-Authority | Produce dedicated Concelier authority audit runbook covering log fields, monitoring recommendations, and troubleshooting steps. | ✅ Runbook published; ✅ linked from DOC3/DOC5; ✅ alerting guidance included. | -| FEEDDOCS-DOCS-05-001 | DONE (2025-10-11) | Docs Guild | FEEDMERGE-ENGINE-04-001, FEEDMERGE-ENGINE-04-002 | Publish Concelier conflict resolution runbook covering precedence workflow, merge-event auditing, and Sprint 3 metrics. | ✅ `docs/modules/concelier/operations/conflict-resolution.md` committed; ✅ metrics/log tables align with latest merge code; ✅ Ops alert guidance handed to Concelier team. | -| FEEDDOCS-DOCS-05-002 | DONE (2025-10-16) | Docs Guild, Concelier Ops | FEEDDOCS-DOCS-05-001 | Ops sign-off captured: conflict runbook circulated, alert thresholds tuned, and rollout decisions documented in change log. | ✅ Ops review recorded; ✅ alert thresholds finalised using `docs/modules/concelier/operations/authority-audit-runbook.md`; ✅ change-log entry linked from runbook once GHSA/NVD/OSV regression fixtures land. | -| DOCS-ADR-09-001 | DONE (2025-10-19) | Docs Guild, DevEx | — | Establish ADR process (`docs/adr/0000-template.md`) and document usage guidelines. | Template published; README snippet linking ADR process; announcement posted (`docs/updates/2025-10-18-docs-guild.md`). | -| DOCS-EVENTS-09-002 | DONE (2025-10-19) | Docs Guild, Platform Events | SCANNER-EVENTS-15-201 | Publish event schema catalog (`docs/events/`) for `scanner.report.ready@1`, `scheduler.rescan.delta@1`, `attestor.logged@1`. | Schemas validated (Ajv CI hooked); docs/events/README summarises usage; Platform Events notified via `docs/updates/2025-10-18-docs-guild.md`. | -| DOCS-EVENTS-09-003 | DONE (2025-10-19) | Docs Guild | DOCS-EVENTS-09-002 | Add human-readable envelope field references and canonical payload samples for published events, including offline validation workflow. | Tables explain common headers/payload segments; versioned sample payloads committed; README links to validation instructions and samples. | -| DOCS-EVENTS-09-004 | DONE (2025-10-19) | Docs Guild, Scanner WebService | SCANNER-EVENTS-15-201 | Refresh scanner event docs to mirror DSSE-backed report fields, document `scanner.scan.completed`, and capture canonical sample validation. | Schemas updated for new payload shape; README references DSSE reuse and validation test; samples align with emitted events. | -| PLATFORM-EVENTS-09-401 | DONE (2025-10-21) | Platform Events Guild | DOCS-EVENTS-09-003 | Embed canonical event samples into contract/integration tests and ensure CI validates payloads against published schemas. | Notify models tests now run schema validation against `docs/events/*.json`, event schemas allow optional `attributes`, and docs capture the new validation workflow. | -| RUNTIME-GUILD-09-402 | DONE (2025-10-19) | Runtime Guild | SCANNER-POLICY-09-107 | Confirm Scanner WebService surfaces `quietedFindingCount` and progress hints to runtime consumers; document readiness checklist. | Runtime verification run captures enriched payload; checklist/doc updates merged; stakeholders acknowledge availability. | -| DOCS-CONCELIER-07-201 | DONE (2025-10-22) | Docs Guild, Concelier WebService | FEEDWEB-DOCS-01-001 | Final editorial review and publish pass for Concelier authority toggle documentation (Quickstart + operator guide). | Review feedback resolved, publish PR merged, release notes updated with documentation pointer. | -| DOCS-RUNTIME-17-004 | DONE (2025-10-26) | Docs Guild, Runtime Guild | SCANNER-EMIT-17-701, ZASTAVA-OBS-17-005, DEVOPS-REL-17-002 | Document build-id workflows: SBOM exposure, runtime event payloads (`process.buildId`), Scanner `/policy/runtime` response (`buildIds` list), debug-store layout, and operator guidance for symbol retrieval. | Architecture + operator docs updated with build-id sections (Observer, Scanner, CLI), examples show `readelf` output + debuginfod usage, references linked from Offline Kit/Release guides + CLI help. | -| DOCS-OBS-50-001 | BLOCKED (2025-10-26) | Docs Guild, Observability Guild | TELEMETRY-OBS-50-001 | Publish `/docs/observability/overview.md` introducing scope, imposed rule banner, architecture diagram, and tenant guarantees. | Doc merged with imposed rule banner; diagram committed; cross-links to telemetry stack + evidence locker docs. | -> Blocked: waiting on telemetry core deliverable (TELEMETRY-OBS-50-001) to finalise architecture details and diagrams. | DOCS-OBS-50-002 | TODO | Docs Guild, Security Guild | TELEMETRY-OBS-50-002 | Author `/docs/observability/telemetry-standards.md` detailing common fields, scrubbing policy, sampling defaults, and redaction override procedure. | Doc merged; imposed rule banner present; examples validated with telemetry fixtures; security review sign-off captured. | | DOCS-OBS-50-003 | TODO | Docs Guild, Observability Guild | TELEMETRY-OBS-50-001 | Create `/docs/observability/logging.md` covering structured log schema, dos/don'ts, tenant isolation, and copyable examples. | Doc merged with banner; sample logs redacted; lint passes; linked from coding standards. | | DOCS-OBS-50-004 | TODO | Docs Guild, Observability Guild | TELEMETRY-OBS-50-002 | Draft `/docs/observability/tracing.md` explaining context propagation, async linking, CLI header usage, and sampling strategies. | Doc merged; imposed rule banner included; diagrams updated; references to CLI/Console features added. | @@ -33,17 +18,6 @@ | DOCS-CLI-OBS-52-001 | TODO | Docs Guild, DevEx/CLI Guild | CLI-OBS-52-001 | Create `/docs/modules/cli/guides/observability.md` detailing `stella obs` commands, examples, exit codes, imposed rule banner, and scripting tips. | Doc merged; examples tested; banner included; CLI parity matrix updated. | | DOCS-CLI-FORENSICS-53-001 | TODO | Docs Guild, DevEx/CLI Guild | CLI-FORENSICS-54-001 | Publish `/docs/modules/cli/guides/forensics.md` for snapshot/verify/attest commands with sample outputs, imposed rule banner, and offline workflows. | Doc merged; sample bundles verified; banner present; offline notes cross-linked. | | DOCS-RUNBOOK-55-001 | TODO | Docs Guild, Ops Guild | DEVOPS-OBS-55-001, WEB-OBS-55-001 | Author `/docs/runbooks/incidents.md` describing incident mode activation, escalation steps, retention impact, verification checklist, and imposed rule banner. | Doc merged; runbook rehearsed; banner included; linked from alerts. | -| DOCS-AOC-19-001 | DONE (2025-10-26) | Docs Guild, Concelier Guild | CONCELIER-WEB-AOC-19-001, EXCITITOR-WEB-AOC-19-001 | Author `/docs/ingestion/aggregation-only-contract.md` covering philosophy, invariants, schemas, error codes, migration, observability, and security checklist. | New doc published with compliance checklist; cross-links from existing docs added. | -| DOCS-AOC-19-002 | DONE (2025-10-26) | Docs Guild, Architecture Guild | DOCS-AOC-19-001 | Update `/docs/modules/platform/architecture-overview.md` to include AOC boundary, raw stores, and sequence diagram (fetch → guard → raw insert → policy evaluation). | Overview doc updated with diagrams/text; lint passes; stakeholders sign off. | -| DOCS-AOC-19-003 | DONE (2025-10-26) | Docs Guild, Policy Guild | POLICY-AOC-19-003 | Refresh `/docs/modules/policy/architecture.md` clarifying ingestion boundary, raw inputs, and policy-only derived data. | Doc highlights raw-only ingestion contract, updated diagrams merge, compliance checklist added. | -| DOCS-AOC-19-004 | DONE (2025-10-26) | Docs Guild, UI Guild | UI-AOC-19-001 | Extend `/docs/ui/console.md` with Sources dashboard tiles, violation drill-down workflow, and verification action. | UI doc updated with screenshots/flow descriptions, compliance checklist appended. | -> DOCS-AOC-19-004: Architecture overview & policy-engine updates landed 2025-10-26; incorporate the new AOC boundary diagrams and metrics references. -| DOCS-AOC-19-005 | DONE (2025-10-26) | Docs Guild, CLI Guild | CLI-AOC-19-003 | Update `/docs/modules/cli/guides/cli-reference.md` with `stella sources ingest --dry-run` and `stella aoc verify` usage, exit codes, and offline notes. | CLI reference + quickstart sections updated; examples validated; compliance checklist added. | -> DOCS-AOC-19-005: New ingestion reference + architecture overview published 2025-10-26; ensure CLI docs link to both and surface AOC exit codes mapping. -| DOCS-AOC-19-006 | DONE (2025-10-26) | Docs Guild, Observability Guild | CONCELIER-WEB-AOC-19-002, EXCITITOR-WEB-AOC-19-002 | Document new metrics/traces/log keys in `/docs/observability/observability.md`. | Observability doc lists new metrics/traces/log fields; dashboards referenced; compliance checklist appended. | -| DOCS-AOC-19-007 | DONE (2025-10-26) | Docs Guild, Authority Core | AUTH-AOC-19-001 | Update `/docs/security/authority-scopes.md` with new ingestion scopes and tenancy enforcement notes. | Doc reflects new scopes, sample policies updated, compliance checklist added. | -| DOCS-AOC-19-008 | DONE (2025-10-26) | Docs Guild, DevOps Guild | DEVOPS-AOC-19-002 | Refresh `/docs/deploy/containers.md` to cover validator enablement, guard env flags, and read-only verify user. | Deploy doc updated; offline kit section mentions validator scripts; compliance checklist appended. | -| DOCS-AOC-19-009 | DONE (2025-10-26) | Docs Guild, Authority Core | AUTH-AOC-19-001 | Update AOC docs/samples to reflect new `advisory:*`, `vex:*`, and `aoc:verify` scopes. | Docs reference new scopes, samples aligned, compliance checklist updated. | ## Air-Gapped Mode (Epic 16) | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | @@ -106,18 +80,6 @@ | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| DOCS-POLICY-20-001 | DONE (2025-10-26) | Docs Guild, Policy Guild | POLICY-ENGINE-20-000 | Author `/docs/policy/overview.md` covering concepts, inputs/outputs, determinism, and compliance checklist. | Doc published with diagrams + glossary; lint passes; checklist included. | -| DOCS-POLICY-20-002 | DONE (2025-10-26) | Docs Guild, Policy Guild | POLICY-ENGINE-20-001 | Write `/docs/policy/dsl.md` with grammar, built-ins, examples, anti-patterns. | DSL doc includes grammar tables, examples, compliance checklist; validated against parser tests. | -| DOCS-POLICY-20-003 | DONE (2025-10-26) | Docs Guild, Authority Core | AUTH-POLICY-20-001 | Publish `/docs/policy/lifecycle.md` describing draft→approve workflow, roles, audit, compliance list. | Lifecycle doc linked from UI/CLI help; approvals roles documented; checklist appended. | -| DOCS-POLICY-20-004 | DONE (2025-10-26) | Docs Guild, Scheduler Guild | SCHED-MODELS-20-001 | Create `/docs/policy/runs.md` detailing run modes, incremental mechanics, cursors, replay. | Run doc includes sequence diagrams + compliance checklist; cross-links to scheduler docs. | -| DOCS-POLICY-20-005 | DONE (2025-10-26) | Docs Guild, BE-Base Platform Guild | WEB-POLICY-20-001 | Draft `/docs/api/policy.md` describing endpoints, schemas, error codes. | API doc validated against OpenAPI; examples included; checklist appended. | -| DOCS-POLICY-20-006 | DONE (2025-10-26) | Docs Guild, DevEx/CLI Guild | CLI-POLICY-20-002 | Produce `/docs/modules/cli/guides/policy.md` with command usage, exit codes, JSON output contracts. | CLI doc includes examples, exit codes, compliance checklist. | -| DOCS-POLICY-20-007 | DONE (2025-10-26) | Docs Guild, UI Guild | UI-POLICY-20-001 | Document `/docs/ui/policy-editor.md` covering editor, simulation, diff workflows, approvals. | UI doc includes screenshots/placeholders, accessibility notes, compliance checklist. | -| DOCS-POLICY-20-008 | DONE (2025-10-26) | Docs Guild, Architecture Guild | POLICY-ENGINE-20-003 | Write `/docs/modules/policy/architecture.md` (new epic content) with sequence diagrams, selection strategy, schema. | Architecture doc merged with diagrams; compliance checklist appended; references updated. | -| DOCS-POLICY-20-009 | DONE (2025-10-26) | Docs Guild, Observability Guild | POLICY-ENGINE-20-007 | Add `/docs/observability/policy.md` for metrics/traces/logs, sample dashboards. | Observability doc includes metrics tables, dashboard screenshots, checklist. | -| DOCS-POLICY-20-010 | DONE (2025-10-26) | Docs Guild, Security Guild | AUTH-POLICY-20-002 | Publish `/docs/security/policy-governance.md` covering scopes, approvals, tenancy, least privilege. | Security doc merged; compliance checklist appended; reviewed by Security Guild. | -| DOCS-POLICY-20-011 | DONE (2025-10-26) | Docs Guild, Policy Guild | POLICY-ENGINE-20-001 | Populate `/docs/examples/policies/` with baseline/serverless/internal-only samples and commentary. | Example policies committed with explanations; lint passes; compliance checklist per file. | -| DOCS-POLICY-20-012 | DONE (2025-10-26) | Docs Guild, Support Guild | WEB-POLICY-20-003 | Draft `/docs/faq/policy-faq.md` addressing common pitfalls, VEX conflicts, determinism issues. | FAQ published with Q/A entries, cross-links, compliance checklist. | ## Graph Explorer v1 @@ -140,30 +102,12 @@ ## StellaOps Console (Sprint 23) +> 2025-10-28: Install Docker guide references pending CLI commands (`stella downloads manifest`, `stella downloads mirror`, `stella console status`). Update once CLI parity lands. +> 2025-10-28: Added guide covering keyboard matrix, screen reader behaviour, colour/focus tokens, testing workflow, offline guidance, and compliance checklist. +> Follow-up: align diagrams/examples after `CONCELIER-LNM-21` & `EXCITITOR-LNM-21` work merges (currently TODO). + | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| DOCS-CONSOLE-23-001 | DONE (2025-10-26) | Docs Guild, Console Guild | CONSOLE-CORE-23-004 | Publish `/docs/ui/console-overview.md` covering IA, tenant model, global filters, and AOC alignment with compliance checklist. | Doc merged with diagrams + overview tables; checklist appended; Console Guild sign-off. | -| DOCS-CONSOLE-23-002 | DONE (2025-10-26) | Docs Guild, Console Guild | DOCS-CONSOLE-23-001 | Author `/docs/ui/navigation.md` detailing routes, breadcrumbs, keyboard shortcuts, deep links, and tenant context switching. | Navigation doc merged with shortcut tables and screenshots; accessibility checklist satisfied. | -| DOCS-CONSOLE-23-003 | DONE (2025-10-26) | Docs Guild, SBOM Service Guild, Console Guild | SBOM-CONSOLE-23-001, CONSOLE-FEAT-23-102 | Document `/docs/ui/sbom-explorer.md` (catalog, detail, graph overlays, exports) including compliance checklist and performance tips. | Doc merged with annotated screenshots, export instructions, and overlay examples; checklist appended. | -| DOCS-CONSOLE-23-004 | DONE (2025-10-26) | Docs Guild, Concelier Guild, Excititor Guild | CONCELIER-CONSOLE-23-001, EXCITITOR-CONSOLE-23-001 | Produce `/docs/ui/advisories-and-vex.md` explaining aggregation-not-merge, conflict indicators, raw viewers, and provenance banners. | Doc merged; raw JSON examples included; compliance checklist complete. | -| DOCS-CONSOLE-23-005 | DONE (2025-10-26) | Docs Guild, Policy Guild | POLICY-CONSOLE-23-001, CONSOLE-FEAT-23-104 | Write `/docs/ui/findings.md` describing filters, saved views, explain drawer, exports, and CLI parity callouts. | Doc merged with filter matrix + explain walkthrough; checklist appended. | -| DOCS-CONSOLE-23-006 | DONE (2025-10-26) | Docs Guild, Policy Guild, Product Ops | POLICY-CONSOLE-23-002, CONSOLE-FEAT-23-105 | Publish `/docs/ui/policies.md` with editor, simulation, approvals, compliance checklist, and RBAC mapping. | Doc merged; Monaco screenshots + simulation diff examples included; approval flow described; checklist appended. | -| DOCS-CONSOLE-23-007 | DONE (2025-10-26) | Docs Guild, Scheduler Guild | SCHED-CONSOLE-23-001, CONSOLE-FEAT-23-106 | Document `/docs/ui/runs.md` covering queues, live progress, diffs, retries, evidence downloads, and troubleshooting. | Doc merged with SSE troubleshooting, metrics references, compliance checklist. | -| DOCS-CONSOLE-23-008 | DONE (2025-10-26) | Docs Guild, Authority Guild | AUTH-CONSOLE-23-002, CONSOLE-FEAT-23-108 | Draft `/docs/ui/admin.md` describing users/roles, tenants, tokens, integrations, fresh-auth prompts, and RBAC mapping. | Doc merged with tables for scopes vs roles, screenshots, compliance checklist. | -| DOCS-CONSOLE-23-009 | DONE (2025-10-27) | Docs Guild, DevOps Guild | DOWNLOADS-CONSOLE-23-001, CONSOLE-FEAT-23-109 | Publish `/docs/ui/downloads.md` listing product images, commands, offline instructions, parity with CLI, and compliance checklist. | Doc merged; manifest sample included; copy-to-clipboard guidance documented; checklist complete. | -| DOCS-CONSOLE-23-010 | DONE (2025-10-27) | Docs Guild, Deployment Guild, Console Guild | DEVOPS-CONSOLE-23-002, CONSOLE-REL-23-301 | Write `/docs/deploy/console.md` (Helm, ingress, TLS, CSP, env vars, health checks) with compliance checklist. | Deploy doc merged; templates validated; CSP guidance included; checklist appended. | -| DOCS-CONSOLE-23-011 | DONE (2025-10-28) | Docs Guild, Deployment Guild | DOCS-CONSOLE-23-010 | Update `/docs/install/docker.md` to cover Console image, Compose/Helm usage, offline tarballs, parity with CLI. | Doc updated with new sections; commands validated; compliance checklist appended. | -| DOCS-CONSOLE-23-012 | DONE (2025-10-28) | Docs Guild, Security Guild | AUTH-CONSOLE-23-003, WEB-CONSOLE-23-002 | Publish `/docs/security/console-security.md` detailing OIDC flows, scopes, CSP, fresh-auth, evidence handling, and compliance checklist. | Security doc merged; threat model notes included; checklist appended. | -| DOCS-CONSOLE-23-013 | DONE (2025-10-28) | Docs Guild, Observability Guild | TELEMETRY-CONSOLE-23-001, CONSOLE-QA-23-403 | Write `/docs/observability/ui-telemetry.md` cataloguing metrics/logs/traces, dashboards, alerts, and feature flags. | Doc merged with instrumentation tables, dashboard screenshots, checklist appended. | -| DOCS-CONSOLE-23-014 | DONE (2025-10-28) | Docs Guild, Console Guild, CLI Guild | CONSOLE-DOC-23-502 | Maintain `/docs/cli-vs-ui-parity.md` matrix and integrate CI check guidance. | Matrix published with parity status, CI workflow documented, compliance checklist appended. | -> 2025-10-28: Install Docker guide references pending CLI commands (`stella downloads manifest`, `stella downloads mirror`, `stella console status`). Update once CLI parity lands. -| DOCS-CONSOLE-23-015 | DONE (2025-10-27) | Docs Guild, Architecture Guild | CONSOLE-CORE-23-001, WEB-CONSOLE-23-001 | Produce `/docs/modules/ui/console-architecture.md` describing frontend packages, data flow diagrams, SSE design, performance budgets. | Architecture doc merged with diagrams + compliance checklist; reviewers approve. | -| DOCS-CONSOLE-23-016 | DONE (2025-10-28) | Docs Guild, Accessibility Guild | CONSOLE-QA-23-402, CONSOLE-FEAT-23-102 | Refresh `/docs/accessibility.md` with Console-specific keyboard flows, color tokens, testing tools, and compliance checklist updates. | Accessibility doc updated; audits referenced; checklist appended. | -> 2025-10-28: Added guide covering keyboard matrix, screen reader behaviour, colour/focus tokens, testing workflow, offline guidance, and compliance checklist. -| DOCS-CONSOLE-23-017 | DONE (2025-10-27) | Docs Guild, Console Guild | CONSOLE-FEAT-23-101..109 | Create `/docs/examples/ui-tours.md` providing triage, audit, policy rollout walkthroughs with annotated screenshots and GIFs. | UI tours doc merged; capture instructions + asset placeholders committed; compliance checklist appended. | -| DOCS-CONSOLE-23-018 | DONE (2025-10-27) | Docs Guild, Security Guild | DOCS-CONSOLE-23-012 | Execute console security compliance checklist and capture Security Guild sign-off in Sprint 23 log. | Checklist completed; findings addressed or tickets filed; sign-off noted in updates file. | -| DOCS-LNM-22-006 | DONE (2025-10-27) | Docs Guild, Architecture Guild | CONCELIER-LNM-21-001..005, EXCITITOR-LNM-21-001..005 | Refresh `/docs/modules/concelier/architecture.md` and `/docs/modules/excititor/architecture.md` describing observation/linkset pipelines and event contracts. | Architecture docs updated with observation/linkset flow + event tables; revisit once service implementations land. | -> Follow-up: align diagrams/examples after `CONCELIER-LNM-21` & `EXCITITOR-LNM-21` work merges (currently TODO). | DOCS-LNM-22-007 | TODO | Docs Guild, Observability Guild | CONCELIER-LNM-21-005, EXCITITOR-LNM-21-005, DEVOPS-LNM-22-002 | Publish `/docs/observability/aggregation.md` with metrics/traces/logs/SLOs. | Observability doc merged; dashboards referenced; checklist appended. | | DOCS-LNM-22-008 | TODO | Docs Guild, DevOps Guild | MERGE-LNM-21-001, CONCELIER-LNM-21-102 | Write `/docs/migration/no-merge.md` describing migration plan, backfill steps, rollback, feature flags. | Migration doc approved by stakeholders; checklist appended. | @@ -201,7 +145,6 @@ | DOCS-EXC-25-001 | TODO | Docs Guild, Governance Guild | WEB-EXC-25-001 | Author `/docs/governance/exceptions.md` covering lifecycle, scope patterns, examples, compliance checklist. | Doc merged; reviewers sign off; checklist included. | | DOCS-EXC-25-002 | TODO | Docs Guild, Authority Core | AUTH-EXC-25-001 | Publish `/docs/governance/approvals-and-routing.md` detailing roles, routing matrix, MFA rules, audit trails. | Doc merged; routing examples validated; checklist appended. | | DOCS-EXC-25-003 | TODO | Docs Guild, BE-Base Platform Guild | WEB-EXC-25-001..003 | Create `/docs/api/exceptions.md` with endpoints, payloads, errors, idempotency notes. | API doc aligned with OpenAPI; examples tested; checklist appended. | -| DOCS-EXC-25-004 | DONE (2025-10-27) | Docs Guild, Policy Guild | POLICY-ENGINE-70-001 | Document `/docs/policy/exception-effects.md` explaining evaluation order, conflicts, simulation. | Doc merged; tests cross-referenced; checklist appended. | | DOCS-EXC-25-005 | TODO | Docs Guild, UI Guild | UI-EXC-25-001..004 | Write `/docs/ui/exception-center.md` with UI walkthrough, badges, accessibility, shortcuts. | Doc merged with screenshots; accessibility checklist completed. | | DOCS-EXC-25-006 | TODO | Docs Guild, DevEx/CLI Guild | CLI-EXC-25-001..002 | Update `/docs/modules/cli/guides/exceptions.md` covering command usage and exit codes. | CLI doc updated; examples validated; checklist appended. | | DOCS-EXC-25-007 | TODO | Docs Guild, DevOps Guild | SCHED-WORKER-25-101, DEVOPS-GRAPH-24-003 | Publish `/docs/migration/exception-governance.md` describing cutover from legacy suppressions, notifications, rollback. | Migration doc approved; checklist included. | @@ -229,15 +172,6 @@ | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| DOCS-EXPORT-35-001 | DONE (2025-10-29) | Docs Guild | EXPORT-SVC-35-001..006 | Author `/docs/modules/export-center/overview.md` covering purpose, profiles, security, AOC alignment, surfaces, ending with imposed rule statement. | Doc merged with diagrams/examples; imposed rule line present; index updated. | -| DOCS-EXPORT-35-002 | DONE (2025-10-29) | Docs Guild | EXPORT-SVC-35-002..005 | Publish `/docs/modules/export-center/architecture.md` describing planner, adapters, manifests, signing, distribution flows, restating imposed rule. | Architecture doc merged; sequence diagrams included; rule statement appended. | -| DOCS-EXPORT-35-003 | DONE (2025-10-29) | Docs Guild | EXPORT-SVC-35-003..004 | Publish `/docs/modules/export-center/profiles.md` detailing schema fields, examples, compatibility, and imposed rule reminder. | Profiles doc merged; JSON schemas linked; imposed rule noted. | -| DOCS-EXPORT-36-004 | DONE (2025-10-29) | Docs Guild | EXPORT-SVC-36-001..004, WEB-EXPORT-36-001 | Publish `/docs/modules/export-center/api.md` covering endpoints, payloads, errors, and mention imposed rule. | API doc merged; examples validated; rule included. | -| DOCS-EXPORT-36-005 | DONE (2025-10-29) | Docs Guild | CLI-EXPORT-35-001, CLI-EXPORT-36-001 | Publish `/docs/modules/export-center/cli.md` with command reference, CI scripts, verification steps, restating imposed rule. | CLI doc merged; script snippets tested; rule appended. | -| DOCS-EXPORT-36-006 | DONE (2025-10-29) | Docs Guild | EXPORT-SVC-36-001, DEVOPS-EXPORT-36-001 | Publish `/docs/modules/export-center/trivy-adapter.md` covering field mappings, compatibility matrix, and imposed rule reminder. | Doc merged; mapping tables validated; rule included. | -| DOCS-EXPORT-37-001 | DONE (2025-10-29) | Docs Guild | EXPORT-SVC-37-001, DEVOPS-EXPORT-37-001 | Publish `/docs/modules/export-center/mirror-bundles.md` describing filesystem/OCI layouts, delta/encryption, import guide, ending with imposed rule. | Doc merged; diagrams provided; verification steps tested; rule stated. | -| DOCS-EXPORT-37-002 | DONE (2025-10-29) | Docs Guild | EXPORT-SVC-35-005, EXPORT-SVC-37-002 | Publish `/docs/modules/export-center/provenance-and-signing.md` detailing manifests, attestation flow, verification, reiterating imposed rule. | Doc merged; signature examples validated; rule appended. | -| DOCS-EXPORT-37-003 | DONE (2025-10-29) | Docs Guild | DEVOPS-EXPORT-37-001 | Publish `/docs/operations/export-runbook.md` covering failures, tuning, capacity planning, with imposed rule reminder. | Runbook merged; procedures validated; rule included. | | DOCS-EXPORT-37-004 | TODO | Docs Guild | AUTH-EXPORT-37-001, EXPORT-SVC-37-002 | Publish `/docs/security/export-hardening.md` outlining RBAC, tenancy, encryption, redaction, restating imposed rule. | Security doc merged; checklist updated; rule appended. | | DOCS-EXPORT-37-101 | TODO | Docs Guild, DevEx/CLI Guild | CLI-EXPORT-37-001 | Refresh CLI verification sections once `stella export verify` lands (flags, exit codes, samples). | `docs/modules/export-center/cli.md` & `docs/modules/export-center/provenance-and-signing.md` updated with final command syntax; examples tested; rule reminder retained. | | DOCS-EXPORT-37-102 | TODO | Docs Guild, DevOps Guild | DEVOPS-EXPORT-37-001 | Embed export dashboards/alerts references into provenance/runbook docs after Grafana work ships. | Docs updated with dashboard IDs/alert notes; update logged; rule reminder present. | @@ -353,8 +287,6 @@ | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| DOCS-NOTIFY-38-001 | DONE (2025-10-29) | Docs Guild, Notifications Service Guild | NOTIFY-SVC-38-001..004 | Publish `/docs/notifications/overview.md` and `/docs/notifications/architecture.md`, each ending with imposed rule reminder. | Docs merged; diagrams verified; imposed rule appended. | -| DOCS-NOTIFY-39-002 | DONE (2025-10-29) | Docs Guild, Notifications Service Guild | NOTIFY-SVC-39-001..004 | Publish `/docs/notifications/rules.md`, `/docs/notifications/templates.md`, `/docs/notifications/digests.md` with examples and imposed rule line. | Docs merged; examples validated; imposed rule appended. | | DOCS-NOTIFY-40-001 | TODO | Docs Guild, Security Guild | AUTH-NOTIFY-38-001, NOTIFY-SVC-40-001..004 | Publish `/docs/notifications/channels.md`, `/docs/notifications/escalations.md`, `/docs/notifications/api.md`, `/docs/operations/notifier-runbook.md`, `/docs/security/notifications-hardening.md`; each ends with imposed rule line. | Docs merged; accessibility checks passed; imposed rule appended. | ## CLI Parity & Task Packs @@ -363,7 +295,6 @@ |----|--------|----------|------------|-------------|---------------| | DOCS-CLI-41-001 | TODO | Docs Guild, DevEx/CLI Guild | CLI-CORE-41-001 | Publish `/docs/modules/cli/guides/overview.md`, `/docs/modules/cli/guides/configuration.md`, `/docs/modules/cli/guides/output-and-exit-codes.md` with imposed rule statements. | Docs merged; examples verified; imposed rule appended. | | DOCS-CLI-42-001 | TODO | Docs Guild | DOCS-CLI-41-001, CLI-PARITY-41-001 | Publish `/docs/modules/cli/guides/parity-matrix.md` and command guides under `/docs/modules/cli/guides/commands/*.md` (policy, sbom, vuln, vex, advisory, export, orchestrator, notify, aoc, auth). | Guides merged; parity automation documented; imposed rule appended. | -| DOCS-PACKS-43-001 | DONE (2025-10-27) | Docs Guild, Task Runner Guild | PACKS-REG-42-001, TASKRUN-42-001 | Publish `/docs/task-packs/spec.md`, `/docs/task-packs/authoring-guide.md`, `/docs/task-packs/registry.md`, `/docs/task-packs/runbook.md`, `/docs/security/pack-signing-and-rbac.md`, `/docs/operations/cli-release-and-packaging.md` with imposed rule statements. | Docs merged; tutorials validated; imposed rule appended; cross-links added. | ## Containerized Distribution (Epic 13) diff --git a/docs/aoc/guard-library.md b/docs/aoc/guard-library.md new file mode 100644 index 00000000..9993ddee --- /dev/null +++ b/docs/aoc/guard-library.md @@ -0,0 +1,111 @@ +# Aggregation-Only Guard Library Reference + +> **Packages:** `StellaOps.Aoc`, `StellaOps.Aoc.AspNetCore` +> **Related tasks:** `WEB-AOC-19-001`, `WEB-AOC-19-003`, `DEVOPS-AOC-19-001` +> **Audience:** Concelier/Excititor service owners, Platform guild, QA + +The Aggregation-Only Contract (AOC) guard library enforces the canonical ingestion +rules described in `docs/ingestion/aggregation-only-contract.md`. Service owners +should use the guard whenever raw advisory or VEX payloads are accepted so that +forbidden fields are rejected long before they reach MongoDB. + +## Packages + +### `StellaOps.Aoc` +- `IAocGuard` / `AocWriteGuard` — validate JSON payloads and emit `AocGuardResult`. +- `AocGuardOptions` — toggles for signature enforcement, tenant requirements, and required top-level fields. +- `AocViolation` / `AocViolationCode` — structured violations surfaced to callers. +- `ServiceCollectionExtensions.AddAocGuard()` — DI helper that registers the singleton guard. +- `AocGuardExtensions.ValidateOrThrow()` — throws `AocGuardException` when validation fails. + +### `StellaOps.Aoc.AspNetCore` +- `AocGuardEndpointFilter` — Minimal API endpoint filter that evaluates request payloads through the guard before invoking handlers. +- `AocHttpResults.Problem()` — Produces a RFC 7807 payload that includes violation codes, suitable for API responses. + +## Minimal API integration + +```csharp +using StellaOps.Aoc; +using StellaOps.Aoc.AspNetCore.Routing; +using StellaOps.Aoc.AspNetCore.Results; + +var builder = WebApplication.CreateBuilder(args); + +builder.Services.AddAocGuard(); +builder.Services.Configure(options => +{ + options.RequireSignatureMetadata = true; + options.RequireTenant = true; +}); + +var app = builder.Build(); + +app.MapPost("/ingest", async (IngestionRequest request, IAocGuard guard, ILogger logger) => + { + // additional application logic + return Results.Accepted(); + }) + .AddEndpointFilter(new AocGuardEndpointFilter( + request => new object?[] { request.Payload }, + serializerOptions: null, + guardOptions: null)) + .ProducesProblem(StatusCodes.Status400BadRequest) + .WithTags("AOC"); + +app.UseExceptionHandler(errorApp => +{ + errorApp.Run(async context => + { + var exceptionHandler = context.Features.Get(); + if (exceptionHandler?.Error is AocGuardException guardException) + { + var result = AocHttpResults.Problem(context, guardException); + await result.ExecuteAsync(context); + return; + } + + context.Response.StatusCode = StatusCodes.Status500InternalServerError; + }); +}); +``` + +Key points: +- Register the guard singleton before wiring repositories or worker services. +- Use `AocGuardEndpointFilter` to protect Minimal API endpoints. The `payloadSelector` + can yield multiple payloads (e.g. batch ingestion) and the filter will validate each one. +- Wrap guard exceptions with `AocHttpResults.Problem` to ensure clients receive machine-readables codes (`ERR_AOC_00x`). + +## Worker / repository usage + +Inject `IAocGuard` (or a module-specific wrapper such as `IVexRawWriteGuard`) anywhere documents +are persisted. Call `ValidateOrThrow` before writes to guarantee fail-fast behaviour, for example: + +```csharp +public sealed class AdvisoryRawRepository +{ + private readonly IAocGuard _guard; + + public AdvisoryRawRepository(IAocGuard guard) => _guard = guard; + + public Task WriteAsync(JsonDocument document, CancellationToken cancellationToken) + { + _guard.ValidateOrThrow(document.RootElement); + // proceed with storage logic + } +} +``` + +## Configuration tips + +- Adjust `AocGuardOptions.RequiredTopLevelFields` when staging new schema changes. All configured names are case-insensitive. +- Set `RequireSignatureMetadata = false` for legacy feeds that do not provide signature envelopes yet; track the waiver in the module backlog. +- Use module-specific wrappers (`AddConcelierAocGuards`, `AddExcititorAocGuards`) to combine guard registration with domain exceptions and metrics. + +## Testing guidance + +- Unit-test guard behaviour with fixture payloads (see `src/Aoc/__Tests`). +- Service-level tests should assert that ingestion endpoints return `ERR_AOC_*` codes via `AocHttpResults`. +- CI must run `stella aoc verify` once CLI support lands (`DEVOPS-AOC-19-002`). +- Roslyn analyzer enforcement (`WEB-AOC-19-003`) will ensure the guard is registered; keep services wired through the shared extensions to prepare for that gate. + +For questions or updates, coordinate with the BE‑Base Platform guild and reference `WEB-AOC-19-001`. diff --git a/docs/backlog/2025-10-cleanup.md b/docs/backlog/2025-10-cleanup.md index b98a2b29..e99149fe 100644 --- a/docs/backlog/2025-10-cleanup.md +++ b/docs/backlog/2025-10-cleanup.md @@ -6,7 +6,7 @@ This note captures the Sprint backlog hygiene pass applied on 26 October 2025. T - **Console replaces legacy Angular UI.** Sprint 13 UI tasks (`UI-SCANS-13-002`, `UI-VEX-13-003`, `UI-ADMIN-13-004`, `UI-SCHED-13-005`) are retired. Console Sprint 23 (`CONSOLE-CORE-23-001..005`, `CONSOLE-FEAT-23-101..109`, `CONSOLE-REL-23-301..303`) owns the experience. - **Policy CLI runtime verbs consolidated.** `CLI-RUNTIME-13-005` is superseded by `CLI-POLICY-20-002` and Policy Studio flows (`CLI-POLICY-27-00x`). - **Notifier supersedes legacy Notify.* modules.** All Sprint 15 `StellaOps.Notify.*` tasks are archived. Replacement work lives in Notifications Studio / Notifier Sprints 38–40 (`NOTIFY-SVC-38-00x`, `NOTIFY-SVC-39-00x`, `NOTIFY-SVC-40-00x`, plus `WEB/CLI-NOTIFY-3x-00x`). -- **Cartographer owns graph construction.** `SBOM-GRAPH-24-00{1..4}` tasks are deleted from SBOM Service; Cartographer backlog (`CARTO-GRAPH-21-001..009`) covers graph storage, overlays, and tiling. +- **Graph platform realigned.** Cartographer backlog items are archived; Graph Indexer + Graph API own graph storage, overlays, and explorer flows. Update open work to reference `GRAPH-*` tasks and the governance note in `docs/devops/contracts-and-rules.md`. - **Dedicated Vuln Explorer service.** Gateway/UI/CLI entries that attempted to inline Vuln Explorer logic (`WEB-GRAPH-24-003`, `UI-GRAPH-24-005`, `CLI-VULN-24-003`) now defer to Sprint 29 Vuln Explorer (`VULN-API-29-00x`, `CONSOLE-VULN-29-00x`, `CLI-VULN-29-00x`). - **AOC enforcement.** Ingestion-layer tasks attempting to compute derived severity/safe-fix metadata (`CONCELIER-VULN-29-003`, `EXCITITOR-VULN-29-003`) were removed; the Policy Engine overlay backlog (`POLICY-ENGINE-29-001..003`) is the canonical home. - **CI/Offline adjustments.** `DEVOPS-UI-13-006` and `DEVOPS-OFFLINE-18-003` moved under Console release tasks (`CONSOLE-QA-23-401`, `DEVOPS-CONSOLE-23-001`, `CONSOLE-REL-23-302`). diff --git a/docs/dev/cartographer-graph-handshake.md b/docs/dev/cartographer-graph-handshake.md index fcea36b9..d466b07f 100644 --- a/docs/dev/cartographer-graph-handshake.md +++ b/docs/dev/cartographer-graph-handshake.md @@ -1,5 +1,7 @@ # Cartographer Graph Handshake Plan +> **Archived (2025-10-30).** Cartographer has been retired in favour of the Graph Indexer + Graph API platform (see `docs/devops/contracts-and-rules.md`). Keep this document only for historical reference; new work must reference `GRAPH-*` tasks and the Graph module docs. + _Status: 2025-10-29_ ## Why this exists diff --git a/docs/devops/contracts-and-rules.md b/docs/devops/contracts-and-rules.md new file mode 100644 index 00000000..c425a73c --- /dev/null +++ b/docs/devops/contracts-and-rules.md @@ -0,0 +1,24 @@ +# DevOps Governance Rules Anchor (Sprint 33) + +> **Scope** · Exit deliverable for `DEVOPS-RULES-33-001` +> **Audience** · DevOps Guild, Platform leads, service owners +> **Related** · `ops/devops/TASKS.md`, `docs/backlog/2025-10-cleanup.md`, `docs/modules/platform/architecture-overview.md` + +This note consolidates the platform governance rules ratified on 30 October 2025. +Each rule captures intent, affected surfaces, enforcement actions, and references to the +source-of-truth backlogs so that subsequent sprints do not re‑introduce conflicting work. + +| Rule | Intent & Rationale | Enforcement & Ownership | Follow-ups | +|------|--------------------|-------------------------|------------| +| **Gateway is a proxy only; Policy Engine owns overlays/simulations.** | Keep Gateway thin and deterministic: it authenticates, authorises, and forwards requests. All overlay composition, simulation, and policy evaluation stays inside Policy Engine so we avoid duplicated logic and time-of-check drift. | *Owners:* BE‑Base Platform Guild + Policy Engine Guild.
*Enforcement:* Gateway PR reviews block embedded overlay code, new endpoints require `Policy Engine` contracts, CI parity checks compare Gateway ↔ Policy overlay schemas. | - Update open tasks referencing “gateway overlay” work to point at `POLICY-ENGINE-20-00x`.
- Close or rewrite backlog items `WEB-POLICY-20-00x` that attempted to compute overlays in Gateway. | +| **AOC ingestion is canonical-only; no merges at ingest.** | Concelier/Excititor persist upstream truth plus provenance. Derived severity, merges, or dedupe belong to downstream Policy workflows. This keeps ingestion auditable and replayable. | *Owners:* Concelier & Excititor guilds, DevOps Guild for CI pipelines.
*Enforcement:* `StellaOps.Aoc` guard library, Mongo validators, Roslyn analyzer backlog (`WEB-AOC-19-003`), CI job `stella aoc verify`. | - Ensure ingestion tasks reference the guard library (`StellaOps.Aoc`).
- Retire legacy tasks that still mention merge-at-ingest (see backlog cleanup note). | +| **Single graph platform: Graph Indexer + Graph API (Cartographer retired).** | Replace the historical Cartographer service with the Graph Indexer + Graph API pairing so graph storage, overlays, and explorer flows share one platform. | *Owners:* Graph Platform Guild, Scheduler Guild, DevOps Guild.
*Enforcement:* New graph work lands in `docs/modules/graph/**` and `src/Graph/**`. Gateway/UI/CLI tickets reference the Graph API endpoints only. | - Archive Cartographer handshake docs and mark Cartographer backlog items as historical.
- Update Scheduler/SBOM/Console tickets to depend on `GRAPH-*` IDs instead of `CARTO-*`. | + +## Tracking & documentation + +- ✅ Rules recorded in `docs/implplan/SPRINTS.md` (Sprint 33) and `ops/devops/TASKS.md`. +- ✅ Repository-wide references to “Cartographer as active platform” updated (see backlog note amendment and doc banner). +- ✅ Changelog entry (`docs/updates/2025-10-30-devops-governance.md`) captures reviewer acknowledgement. + +Future adjustments to these rules must update this file and reference `DEVOPS-RULES-33-001` +when proposing changes so the DevOps Guild can track history. diff --git a/docs/evaluate/checklist.md b/docs/evaluate/checklist.md new file mode 100644 index 00000000..e6745580 --- /dev/null +++ b/docs/evaluate/checklist.md @@ -0,0 +1,38 @@ +# Evaluation Checklist – 30-Day Adoption Plan + +## Day 0–1: Kick the Tires + +- [ ] Follow the [Quickstart](../quickstart.md) to run the first scan and confirm quota headers (`X-Stella-Quota-Remaining`). +- [ ] Capture the deterministic replay bundle (`stella replay export`) to verify SRM evidence. +- [ ] Log into the Console, review the explain trace for the latest scan, and test policy waiver creation. + +## Day 2–7: Prove Fit + +- [ ] Import the [Offline Update Kit](../24_OFFLINE_KIT.md) and confirm feeds refresh with no Internet access. +- [ ] Apply a sovereign CryptoProfile matching your regulatory environment (FIPS, eIDAS, GOST, SM). +- [ ] Run policy simulations with your SBOMs using `stella policy simulate --input `; log explain outcomes for review. +- [ ] Validate attestation workflows by exporting DSSE bundles and replaying them on a secondary host. + +## Day 8–14: Integrate + +- [ ] Wire the CLI into CI/CD to gate images using exit codes and `X-Stella-Quota-Remaining` telemetry. +- [ ] Configure `StellaOps.Notify` with at least one channel (email/webhook) and confirm digest delivery. +- [ ] Map existing advisory/VEX sources to Concelier connectors; note any feeds requiring custom plug-ins. +- [ ] Review `StellaOps.Policy.Engine` audit logs to ensure waiver ownership and expiry meet governance needs. + +## Day 15–30: Harden & Measure + +- [ ] Follow the [Security Hardening Guide](../17_SECURITY_HARDENING_GUIDE.md) to rotate keys and enable mTLS across modules. +- [ ] Enable observability pipelines (metrics + OpenTelemetry) to capture scan throughput and policy outcomes. +- [ ] Run performance checks against the [Performance Workbook](../12_PERFORMANCE_WORKBOOK.md) targets; note P95 latencies. +- [ ] Document operational runbooks (install, upgrade, rollback) referencing [Release Engineering Playbook](../13_RELEASE_ENGINEERING_PLAYBOOK.md). + +## Decision Gates + +| Question | Evidence to collect | Source | +|----------|--------------------|--------| +| Can we operate fully offline? | Offline kit import logs, quota JWT validation without Internet | Quickstart, Offline Kit guide | +| Are findings explainable and reproducible? | SRM replay results, policy explain traces | Key features, Policy Engine UI | +| Does it meet regional compliance? | CryptoProfile application, Attestor/Rekor mirror configuration | Sovereign crypto docs, Attestor guide | + +**Next step:** once the checklist is green, plan production rollout with module-specific architecture docs under `docs/modules/`. diff --git a/docs/events/README.md b/docs/events/README.md index 0583df40..953180e5 100644 --- a/docs/events/README.md +++ b/docs/events/README.md @@ -35,7 +35,7 @@ Additive payload changes (new optional fields) can stay within the same version. | `payload` | `object` | Event-specific body; schemas embed the canonical report and DSSE envelope. | | `attributes` | `object` | Optional metadata bag (`string` keys/values) for downstream correlation. | -For Scanner orchestrator events, `links` include console and API deep links (`ui`, `report`, and `policy`) plus an optional `attestation` URL when a DSSE envelope is present. See [`orchestrator-scanner-events.md`](orchestrator-scanner-events.md) for details. +For Scanner orchestrator events, `links` include console and API deep links (`report.ui`, `report.api`, etc.) plus optional attestation references when a DSSE envelope is present. See [`orchestrator-scanner-events.md`](orchestrator-scanner-events.md) for details. ### Legacy Redis envelope | Field | Type | Notes | diff --git a/docs/events/orchestrator-scanner-events.md b/docs/events/orchestrator-scanner-events.md index 992fffb9..4c59fa56 100644 --- a/docs/events/orchestrator-scanner-events.md +++ b/docs/events/orchestrator-scanner-events.md @@ -35,10 +35,12 @@ Emitted once a signed report is persisted and attested. Payload highlights: - `reportId` / `scanId` — identifiers for the persisted report and originating scan. Until Scan IDs are surfaced by the API, `scanId` mirrors `reportId` so downstream correlators can stabilise on a single key. - **Attributes:** `reportId`, `policyRevisionId`, `policyDigest`, `verdict` — pre-sorted for deterministic routing. - **Links:** - - `ui` → `/ui/reports/{reportId}` on the current host. - - `report` → `{apiBasePath}/{reportsSegment}/{reportId}` (defaults to `/api/v1/reports/{reportId}`). - - `policy` → `{apiBasePath}/{policySegment}/revisions/{revisionId}` when a revision is present. - - `attestation` → `/ui/attestations/{reportId}` when a DSSE envelope is included. + - `report.ui` → `/ui/reports/{reportId}` on the current host. + - `report.api` → `{apiBasePath}/{reportsSegment}/{reportId}` (defaults to `/api/v1/reports/{reportId}`). + - `policy.ui` → `/ui/policy/revisions/{revisionId}` when a revision is present. + - `policy.api` → `{apiBasePath}/{policySegment}/revisions/{revisionId}` when a revision is present. + - `attestation.ui` → `/ui/attestations/{reportId}` when a DSSE envelope is included. + - `attestation.api` → `{apiBasePath}/{reportsSegment}/{reportId}/attestation` when a DSSE envelope is included. - `imageDigest` — OCI image digest associated with the analysis. - `generatedAt` — report generation timestamp (ISO-8601 UTC). - `verdict` — `pass`, `warn`, or `fail` after policy evaluation. @@ -59,7 +61,7 @@ Emitted after scan execution finishes (success or policy failure). Payload highl - `reportId` / `scanId` / `imageDigest` — identifiers mirroring the report-ready event. As with the report-ready payload, `scanId` currently mirrors `reportId` as a temporary shim. - **Attributes:** `reportId`, `policyRevisionId`, `policyDigest`, `verdict`. -- **Links:** same as above (`ui`, `report`, `policy`) with `attestation` populated when DSSE metadata exists. +- **Links:** same as above (`report.*`, `policy.*`) with `attestation.*` populated when DSSE metadata exists. - `verdict`, `summary`, `delta`, `policy` — same semantics as above. - `findings` — array of surfaced findings with `id`, `severity`, optional `cve`, `purl`, and `reachability`. - `links`, `dsse`, `report` — same structure as §2.1 (allows Notifier to reuse signatures). diff --git a/docs/events/samples/scanner.event.report.ready@1.sample.json b/docs/events/samples/scanner.event.report.ready@1.sample.json index 978c0322..a7ef9dda 100644 --- a/docs/events/samples/scanner.event.report.ready@1.sample.json +++ b/docs/events/samples/scanner.event.report.ready@1.sample.json @@ -45,12 +45,20 @@ "digest": "digest-123", "revisionId": "rev-42" }, - "links": { - "ui": "https://scanner.example/ui/reports/report-abc", - "report": "https://scanner.example/api/v1/reports/report-abc", - "policy": "https://scanner.example/api/v1/policy/revisions/rev-42", - "attestation": "https://scanner.example/ui/attestations/report-abc" - }, + "links": { + "report": { + "ui": "https://scanner.example/ui/reports/report-abc", + "api": "https://scanner.example/api/v1/reports/report-abc" + }, + "policy": { + "ui": "https://scanner.example/ui/policy/revisions/rev-42", + "api": "https://scanner.example/api/v1/policy/revisions/rev-42" + }, + "attestation": { + "ui": "https://scanner.example/ui/attestations/report-abc", + "api": "https://scanner.example/api/v1/reports/report-abc/attestation" + } + }, "dsse": { "payloadType": "application/vnd.stellaops.report+json", "payload": "eyJyZXBvcnRJZCI6InJlcG9ydC1hYmMiLCJpbWFnZURpZ2VzdCI6InNoYTI1NjpmZWVkZmFjZSIsImdlbmVyYXRlZEF0IjoiMjAyNS0xMC0xOVQxMjozNDo1NiswMDowMCIsInZlcmRpY3QiOiJibG9ja2VkIiwicG9saWN5Ijp7InJldmlzaW9uSWQiOiJyZXYtNDIiLCJkaWdlc3QiOiJkaWdlc3QtMTIzIn0sInN1bW1hcnkiOnsidG90YWwiOjEsImJsb2NrZWQiOjEsIndhcm5lZCI6MCwiaWdub3JlZCI6MCwicXVpZXRlZCI6MH0sInZlcmRpY3RzIjpbeyJmaW5kaW5nSWQiOiJmaW5kaW5nLTEiLCJzdGF0dXMiOiJCbG9ja2VkIiwic2NvcmUiOjQ3LjUsInNvdXJjZVRydXN0IjoiTlZEIiwicmVhY2hhYmlsaXR5IjoicnVudGltZSJ9XSwiaXNzdWVzIjpbXX0=", diff --git a/docs/events/samples/scanner.event.scan.completed@1.sample.json b/docs/events/samples/scanner.event.scan.completed@1.sample.json index b6096ec4..8559a9ba 100644 --- a/docs/events/samples/scanner.event.scan.completed@1.sample.json +++ b/docs/events/samples/scanner.event.scan.completed@1.sample.json @@ -51,12 +51,20 @@ "reachability": "runtime" } ], - "links": { - "ui": "https://scanner.example/ui/reports/report-abc", - "report": "https://scanner.example/api/v1/reports/report-abc", - "policy": "https://scanner.example/api/v1/policy/revisions/rev-42", - "attestation": "https://scanner.example/ui/attestations/report-abc" - }, + "links": { + "report": { + "ui": "https://scanner.example/ui/reports/report-abc", + "api": "https://scanner.example/api/v1/reports/report-abc" + }, + "policy": { + "ui": "https://scanner.example/ui/policy/revisions/rev-42", + "api": "https://scanner.example/api/v1/policy/revisions/rev-42" + }, + "attestation": { + "ui": "https://scanner.example/ui/attestations/report-abc", + "api": "https://scanner.example/api/v1/reports/report-abc/attestation" + } + }, "dsse": { "payloadType": "application/vnd.stellaops.report+json", "payload": "eyJyZXBvcnRJZCI6InJlcG9ydC1hYmMiLCJpbWFnZURpZ2VzdCI6InNoYTI1NjpmZWVkZmFjZSIsImdlbmVyYXRlZEF0IjoiMjAyNS0xMC0xOVQxMjozNDo1NiswMDowMCIsInZlcmRpY3QiOiJibG9ja2VkIiwicG9saWN5Ijp7InJldmlzaW9uSWQiOiJyZXYtNDIiLCJkaWdlc3QiOiJkaWdlc3QtMTIzIn0sInN1bW1hcnkiOnsidG90YWwiOjEsImJsb2NrZWQiOjEsIndhcm5lZCI6MCwiaWdub3JlZCI6MCwicXVpZXRlZCI6MH0sInZlcmRpY3RzIjpbeyJmaW5kaW5nSWQiOiJmaW5kaW5nLTEiLCJzdGF0dXMiOiJCbG9ja2VkIiwic2NvcmUiOjQ3LjUsInNvdXJjZVRydXN0IjoiTlZEIiwicmVhY2hhYmlsaXR5IjoicnVudGltZSJ9XSwiaXNzdWVzIjpbXX0=", diff --git a/docs/events/scanner.event.report.ready@1.json b/docs/events/scanner.event.report.ready@1.json index 611df5d7..2372a682 100644 --- a/docs/events/scanner.event.report.ready@1.json +++ b/docs/events/scanner.event.report.ready@1.json @@ -122,16 +122,15 @@ "type": "object", "description": "Policy revision metadata surfaced alongside the report." }, - "links": { - "type": "object", - "additionalProperties": false, - "properties": { - "ui": {"type": "string", "format": "uri"}, - "report": {"type": "string", "format": "uri"}, - "policy": {"type": "string", "format": "uri"}, - "attestation": {"type": "string", "format": "uri"} - } - }, + "links": { + "type": "object", + "additionalProperties": false, + "properties": { + "report": {"$ref": "#/definitions/linkTarget"}, + "policy": {"$ref": "#/definitions/linkTarget"}, + "attestation": {"$ref": "#/definitions/linkTarget"} + } + }, "dsse": { "type": "object", "additionalProperties": false, @@ -161,4 +160,14 @@ } } } -} + "definitions": { + "linkTarget": { + "type": "object", + "additionalProperties": false, + "properties": { + "ui": {"type": "string", "format": "uri"}, + "api": {"type": "string", "format": "uri"} + } + } + } +} diff --git a/docs/events/scanner.event.scan.completed@1.json b/docs/events/scanner.event.scan.completed@1.json index 2cda8a0d..a50dfdfd 100644 --- a/docs/events/scanner.event.scan.completed@1.json +++ b/docs/events/scanner.event.scan.completed@1.json @@ -132,16 +132,15 @@ } } }, - "links": { - "type": "object", - "additionalProperties": false, - "properties": { - "ui": {"type": "string", "format": "uri"}, - "report": {"type": "string", "format": "uri"}, - "policy": {"type": "string", "format": "uri"}, - "attestation": {"type": "string", "format": "uri"} - } - }, + "links": { + "type": "object", + "additionalProperties": false, + "properties": { + "report": {"$ref": "#/definitions/linkTarget"}, + "policy": {"$ref": "#/definitions/linkTarget"}, + "attestation": {"$ref": "#/definitions/linkTarget"} + } + }, "dsse": { "type": "object", "additionalProperties": false, @@ -171,4 +170,14 @@ } } } -} + "definitions": { + "linkTarget": { + "type": "object", + "additionalProperties": false, + "properties": { + "ui": {"type": "string", "format": "uri"}, + "api": {"type": "string", "format": "uri"} + } + } + } +} diff --git a/docs/examples/ui-tours.md b/docs/examples/ui-tours.md index 725f6b93..8bbf891f 100644 --- a/docs/examples/ui-tours.md +++ b/docs/examples/ui-tours.md @@ -40,14 +40,10 @@ These tours stitch together the primary Console workspaces so trainers can deliv 8. **Wrap with CLI:** Pop terminal and run `stella findings explain --policy --finding --format markdown` to show reproducibility. ### 2.3 Capture checklist -- `docs/assets/ui/tours/triage-step-01.png` — dashboard ticker highlighting new criticals. - ![Tour A – dashboard criticals](../assets/ui/tours/triage-step-01.png) -- `docs/assets/ui/tours/triage-step-03.png` — filter tray with severity/time window applied. - ![Tour A – filter tray](../assets/ui/tours/triage-step-03.png) -- `docs/assets/ui/tours/triage-step-04.png` — explain drawer evidence tab. - ![Tour A – explain drawer evidence](../assets/ui/tours/triage-step-04.png) -- `docs/assets/ui/tours/triage-flow.gif` — 20 s screen recording of steps 1–5 with annotations. - ![Tour A – walkthrough GIF](../assets/ui/tours/triage-flow.gif) +- `docs/assets/ui/tours/triage-step-01.png` — dashboard ticker highlighting new criticals. *(capture pending)* +- `docs/assets/ui/tours/triage-step-03.png` — filter tray with severity/time window applied. *(capture pending)* +- `docs/assets/ui/tours/triage-step-04.png` — explain drawer evidence tab. *(capture pending)* +- `docs/assets/ui/tours/triage-flow.gif` — 20 s screen recording of steps 1–5 with annotations. *(capture pending)* ### 2.4 Talking points & callouts - Call out Aggregation-Only boundaries: findings reference Concelier/Excititor provenance, UI stays read-only. @@ -78,12 +74,9 @@ These tours stitch together the primary Console workspaces so trainers can deliv 8. **CLI parity close-out:** Run `stella downloads manifest --channel stable` to mirror UI manifest retrieval. Confirm digests match. ### 3.3 Capture checklist -- `docs/assets/ui/tours/audit-step-02.png` — manifest verification banner (green). - ![Tour B – manifest verification](../assets/ui/tours/audit-step-02.png) -- `docs/assets/ui/tours/audit-step-05.png` — exports tab showing evidence bundle ready. - ![Tour B – exports tab](../assets/ui/tours/audit-step-05.png) -- `docs/assets/ui/tours/audit-flow.gif` — 25 s capture from manifest view through export download. - ![Tour B – walkthrough GIF](../assets/ui/tours/audit-flow.gif) +- `docs/assets/ui/tours/audit-step-02.png` — manifest verification banner (green). *(capture pending)* +- `docs/assets/ui/tours/audit-step-05.png` — exports tab showing evidence bundle ready. *(capture pending)* +- `docs/assets/ui/tours/audit-flow.gif` — 25 s capture from manifest view through export download. *(capture pending)* ### 3.4 Talking points & callouts - Stress deterministic manifests and Cosign signatures; reference deployment doc for TLS/CSP alignment. @@ -114,12 +107,9 @@ These tours stitch together the primary Console workspaces so trainers can deliv 8. **Publish CLI parity:** Execute `stella policy promote --policy --revision --run-mode full` to reinforce reproducibility. ### 4.3 Capture checklist -- `docs/assets/ui/tours/policy-step-02.png` — editor checklist with lint/simulation statuses. - ![Tour C – editor checklist](../assets/ui/tours/policy-step-02.png) -- `docs/assets/ui/tours/policy-step-04.png` — simulation diff comparing Active vs Staged. - ![Tour C – simulation diff](../assets/ui/tours/policy-step-04.png) -- `docs/assets/ui/tours/policy-flow.gif` — 30 s clip from draft view through promotion confirmation. - ![Tour C – walkthrough GIF](../assets/ui/tours/policy-flow.gif) +- `docs/assets/ui/tours/policy-step-02.png` — editor checklist with lint/simulation statuses. *(capture pending)* +- `docs/assets/ui/tours/policy-step-04.png` — simulation diff comparing Active vs Staged. *(capture pending)* +- `docs/assets/ui/tours/policy-flow.gif` — 30 s clip from draft view through promotion confirmation. *(capture pending)* ### 4.4 Talking points & callouts - Stress governance: approvals logged with correlation IDs, fresh-auth enforced. diff --git a/docs/faq/policy-faq.md b/docs/faq/policy-faq.md index 06b6995d..8cd84e1c 100644 --- a/docs/faq/policy-faq.md +++ b/docs/faq/policy-faq.md @@ -1,6 +1,6 @@ # Policy Engine FAQ -Answers to questions that Support, Ops, and Policy Guild teams receive most frequently. Pair this FAQ with the [Policy Lifecycle](../policy/lifecycle.md), [Runs](../policy/runs.md), and [CLI guide](../cli/policy.md) for deeper explanations. +Answers to questions that Support, Ops, and Policy Guild teams receive most frequently. Pair this FAQ with the [Policy Lifecycle](../policy/lifecycle.md), [Runs](../policy/runs.md), and [CLI guide](../modules/cli/guides/policy.md) for deeper explanations. --- diff --git a/docs/high-level-architecture.md b/docs/high-level-architecture.md new file mode 100644 index 00000000..91257642 --- /dev/null +++ b/docs/high-level-architecture.md @@ -0,0 +1,49 @@ +# High-Level Architecture – 10-Minute Tour + +``` +Build → Sign → Store → Scan → Policy → Attest → Notify/Export +``` + +## 1. Guiding Principles + +- **SBOM-first everything:** scanners prefer CycloneDX/SPDX inputs and only unpack images when SBOMs are absent. +- **Restart-time plug-ins:** analyzers, exporters, and connectors are loaded at startup, keeping runtime surfaces predictable. +- **Sovereign posture:** all services tolerate zero outbound traffic; Offline Update Kits mirror feeds and trust roots. + +## 2. System Map + +| Tier | Services | Key responsibilities | +|------|----------|----------------------| +| **Edge / Identity** | `StellaOps.Authority` | Issues short-lived OpToks (DPoP + mTLS), exposes OIDC device-code + auth-code flows, rotates JWKS. | +| **Scan & attest** | `StellaOps.Scanner` (API + Worker), `StellaOps.Signer`, `StellaOps.Attestor` | Accept SBOMs/images, drive analyzers, produce DSSE/SRM bundles, optionally log to Rekor mirror. | +| **Evidence graph** | `StellaOps.Concelier`, `StellaOps.Excititor`, `StellaOps.Policy.Engine` | Ingest advisories/VEX, correlate linksets, run lattice policy and VEX-first decisioning. | +| **Experience** | `StellaOps.UI`, `StellaOps.Cli`, `StellaOps.Notify`, `StellaOps.ExportCenter` | Surface findings, automate policy workflows, deliver notifications, package offline mirrors. | +| **Data plane** | MongoDB, Redis, RustFS/object storage, NATS/Redis Streams | Deterministic storage, counters, queue orchestration, Delta SBOM cache. | + +## 3. Request Lifecycle + +1. **Evidence enters** via Concelier and Excititor connectors (Aggregation-Only Contract). +2. **SBOM arrives** from CLI/CI, Scanner deduplicates layers and enqueues work. +3. **Analyzer bundle** runs inside Worker, streams SRM events, stores SBOM fragments in content-addressed cache. +4. **Policy Engine** merges advisories, VEX, and SBOM inventory, applies lattice logic, emits explain trace. +5. **Signer + Attestor** wrap results into DSSE, optionally record to Rekor, and hand proof bundles to Export Center. +6. **UI/CLI** surface findings, quotas, and replay manifests; Notify pushes channel-specific digests. + +## 4. Extension Points + +- **Scanner analyzers** (`plugins/scanner/**`): ship restart-time plug-ins with deterministic manifests. +- **Concelier connectors** (`src/Concelier/__Libraries/**`): fetch advisories, adhere to Aggregation-Only Contract. +- **Policy packs**: upload YAML/Rego bundles with fixtures; simulation endpoints test impacts before promotion. +- **Crypto profiles**: import trust-root packs to align with regional signature mandates. + +## 5. Sovereign & Offline Considerations + +- **Offline Update Kit** carries vulnerability feeds, container images (x86-64 + arm64), Cosign signatures, and detatched JWS manifests. +- **Transparency mirrors**: Attestor caches Rekor proofs; mirrors can be deployed on-prem for DSSE verification. +- **Quota enforcement** uses Redis counters with local JWT validation, so no central service is required. + +## 6. Where to Learn More + +- Deep dive per module in `docs/modules//architecture.md`. +- Study strategic themes in [moat.md](moat.md). +- Review API and CLI contracts in [09_API_CLI_REFERENCE.md](09_API_CLI_REFERENCE.md). diff --git a/docs/implplan/SPRINTS.md b/docs/implplan/SPRINTS.md index 1e89c0c4..1604f101 100644 --- a/docs/implplan/SPRINTS.md +++ b/docs/implplan/SPRINTS.md @@ -492,7 +492,7 @@ This file describe implementation of Stella Ops (docs/README.md). Implementation | Sprint 33 | Orchestrator Dashboard | docs/TASKS.md | TODO | Docs Guild | DOCS-ORCH-33-001 | Author `/docs/orchestrator/api.md` with endpoints, WebSocket events, error codes, and imposed rule reminder. | | Sprint 33 | Orchestrator Dashboard | docs/TASKS.md | TODO | Docs Guild | DOCS-ORCH-33-002 | Author `/docs/orchestrator/console.md` covering screens, accessibility, and live updates. | | Sprint 33 | Orchestrator Dashboard | docs/TASKS.md | TODO | Docs Guild | DOCS-ORCH-33-003 | Author `/docs/orchestrator/cli.md` with command reference, examples, and exit codes. | -| Sprint 33 | Governance & Rules | ops/devops/TASKS.md | DOING (2025-10-26) | DevOps Guild, Platform Leads | DEVOPS-RULES-33-001 | Contracts & Rules anchor (gateway proxy-only; Policy Engine overlays/simulations; AOC ingestion canonicalization; Graph Indexer + Graph API as sole platform). | +| Sprint 33 | Governance & Rules | ops/devops/TASKS.md | REVIEW (2025-10-30) | DevOps Guild, Platform Leads | DEVOPS-RULES-33-001 | Contracts & Rules anchor (gateway proxy-only; Policy Engine overlays/simulations; AOC ingestion canonicalization; Graph Indexer + Graph API as sole platform). | | Sprint 33 | Orchestrator Dashboard | ops/devops/TASKS.md | TODO | DevOps Guild | DEVOPS-ORCH-33-001 | Publish Grafana dashboards for rate-limit/backpressure/error clustering and configure alert rules with runbooks. | | Sprint 33 | Orchestrator Dashboard | src/Authority/StellaOps.Authority/TASKS.md | TODO | Authority Core & Security Guild | AUTH-ORCH-33-001 | Add `Orch.Operator` role, control action scopes, and enforce reason/ticket field capture. | | Sprint 33 | Orchestrator Dashboard | src/Concelier/__Libraries/StellaOps.Concelier.Core/TASKS.md | TODO | Concelier Core Guild | CONCELIER-ORCH-33-001 | Wire orchestrator control hooks (pause, throttle, retry) into Concelier workers with safe checkpoints. | diff --git a/docs/ingestion/aggregation-only-contract.md b/docs/ingestion/aggregation-only-contract.md index 10c1578f..eee8575a 100644 --- a/docs/ingestion/aggregation-only-contract.md +++ b/docs/ingestion/aggregation-only-contract.md @@ -7,7 +7,7 @@ - Defines the canonical behaviour for `advisory_raw` and `vex_raw` collections and the linkset hints they may emit. - Applies to every ingestion runtime (`StellaOps.Concelier.*`, `StellaOps.Excititor.*`), the Authority scopes that guard them, and the DevOps/QA surfaces that verify compliance. - Complements the high-level architecture in [Concelier](../modules/concelier/architecture.md) and Authority enforcement documented in [Authority Architecture](../modules/authority/architecture.md). -- Paired guidance: see the guard-rail checkpoints in [AOC Guardrails](../aoc/aoc-guardrails.md) and CLI usage that will land in `/docs/modules/cli/guides/` as part of Sprint 19 follow-up. +- Paired guidance: see the guard-rail checkpoints in [AOC Guardrails](../aoc/aoc-guardrails.md), the implementation reference in [AOC Guard Library](../aoc/guard-library.md), and CLI usage that will land in `/docs/modules/cli/guides/` as part of Sprint 19 follow-up. ## 2. Philosophy and Goals diff --git a/docs/key-features.md b/docs/key-features.md new file mode 100644 index 00000000..92c78376 --- /dev/null +++ b/docs/key-features.md @@ -0,0 +1,33 @@ +# Key Features – Capability Cards + +Each card is a fast read pairing the headline capability with the evidence that backs it and why it matters day to day. + +## 1. Delta SBOM Engine +- **What it is:** Layer-aware ingestion keeps the SBOM catalog content-addressed; rescans only fetch new layers and update dependency/vulnerability cartographs. +- **Evidence:** Deterministic Replay Manifest (SRM) captures the exact analyzer inputs/outputs per layer. +- **Why it matters:** Warm scans drop below one second, so CI/CD pipelines stay fast even under the free-tier quota. + +## 2. Lattice Policy + OpenVEX +- **What it is:** Policy engine merges SBOM, advisories, VEX, and waivers through lattice logic that prioritises exploitability. +- **Evidence:** OpenVEX is treated as first-class input; the policy UI renders explain traces, while custom rule packs let teams automate muting, expirations, and non-VEX alert logic. +- **Why it matters:** Teams can distinguish exploitable risk from noise, tune the experience beyond VEX statements, and prove why a deployment was blocked or allowed. + +## 3. Sovereign Crypto Profiles +- **What it is:** Bring-your-own trust bundles that switch signing algorithms (FIPS, eIDAS, GOST, SM) without code changes. +- **Evidence:** Crypto profiles travel with Offline Update Kits and post-quantum trust packs, keeping signatures verifiable in regulated sectors. +- **Why it matters:** You meet regional crypto requirements while keeping provenance attestations consistent across tenants. + +## 4. Deterministic Replay & Evidence Bundles +- **What it is:** Every scan produces a DSSE + SRM bundle that can be replayed with `stella replay`. +- **Evidence:** Replay manifests capture analyzer versions, lattice state, and attestations in content-addressed storage for audit trails. +- **Why it matters:** Auditors and incident responders can re-run a historical scan and trust the findings were not tampered with. + +## 5. Transparent Quotas & Offline Operations +- **What it is:** Redis-backed counters surface `{{ quota_token }}` scans/day via headers, UI banners, and `/quota` API; Offline Update Kits mirror feeds. +- **Evidence:** Quota tokens verify locally using bundled public keys, and Offline Update Kits include mirrored advisories, SBOM feeds, and VEX sources. +- **Why it matters:** You stay within predictable limits, avoid surprise throttling, and operate entirely offline when needed. + +### Explore Further +- Walk the first deployment in [quickstart.md](quickstart.md). +- Dive into architectural flows in [high-level-architecture.md](high-level-architecture.md). +- Need detailed matrices? The legacy [feature matrix](04_FEATURE_MATRIX.md) and [vision](03_VISION.md) remain available for deep dives. diff --git a/docs/modules/authority/operations/backup-restore.md b/docs/modules/authority/operations/backup-restore.md index 4890f749..4e201e5e 100644 --- a/docs/modules/authority/operations/backup-restore.md +++ b/docs/modules/authority/operations/backup-restore.md @@ -80,12 +80,12 @@ docker compose up -d curl -fsS http://localhost:8080/health ``` -6. **Validate JWKS and tokens:** call `/jwks` and issue a short-lived token via the CLI to confirm key material matches expectations. If the restored environment requires a fresh signing key, follow the rotation SOP in [`docs/11_AUTHORITY.md`](../11_AUTHORITY.md) using `ops/authority/key-rotation.sh` to invoke `/internal/signing/rotate`. +6. **Validate JWKS and tokens:** call `/jwks` and issue a short-lived token via the CLI to confirm key material matches expectations. If the restored environment requires a fresh signing key, follow the rotation SOP in [`docs/11_AUTHORITY.md`](../../../11_AUTHORITY.md) using `ops/authority/key-rotation.sh` to invoke `/internal/signing/rotate`. ## Disaster Recovery Notes - **Air-gapped replication:** replicate archives via the Offline Update Kit transport channels; never attach USB devices without scanning. - **Retention:** maintain 30 daily snapshots + 12 monthly archival copies. Rotate encryption keys annually. -- **Key compromise:** if signing keys are suspected compromised, restore from the latest clean backup, rotate via OPS3 (see `ops/authority/key-rotation.sh` and `docs/11_AUTHORITY.md`), and publish a revocation notice. +- **Key compromise:** if signing keys are suspected compromised, restore from the latest clean backup, rotate via OPS3 (see `ops/authority/key-rotation.sh` and [`docs/11_AUTHORITY.md`](../../../11_AUTHORITY.md)), and publish a revocation notice. - **Mongo version:** keep dump/restore images pinned to the deployment version (compose uses `mongo:7`). Driver 3.5.0 requires MongoDB **4.2+**—clusters still on 4.0 must be upgraded before restore, and future driver releases will drop 4.0 entirely. citeturn1open1 ## Verification Checklist diff --git a/docs/modules/cli/guides/cli-reference.md b/docs/modules/cli/guides/cli-reference.md index c1515685..0bcfeded 100644 --- a/docs/modules/cli/guides/cli-reference.md +++ b/docs/modules/cli/guides/cli-reference.md @@ -3,7 +3,7 @@ > **Audience:** DevEx engineers, operators, and CI authors integrating the `stella` CLI with Aggregation-Only Contract (AOC) workflows. > **Scope:** Command synopsis, options, exit codes, and offline considerations for `stella sources ingest --dry-run` and `stella aoc verify` as introduced in Sprint 19. -Both commands are designed to enforce the AOC guardrails documented in the [aggregation-only reference](../../ingestion/aggregation-only-contract.md) and the [architecture overview](../../platform/architecture-overview.md). They consume Authority-issued tokens with tenant scopes and never mutate ingestion stores. +Both commands are designed to enforce the AOC guardrails documented in the [aggregation-only reference](../../../ingestion/aggregation-only-contract.md) and the [architecture overview](../architecture.md). They consume Authority-issued tokens with tenant scopes and never mutate ingestion stores. --- @@ -285,9 +285,9 @@ Additional notes: ## 5 · Related references -- [Aggregation-Only Contract reference](../../ingestion/aggregation-only-contract.md) +- [Aggregation-Only Contract reference](../../../ingestion/aggregation-only-contract.md) - [Architecture overview](../../platform/architecture-overview.md) -- [Console AOC dashboard](../../ui/console.md) +- [Console AOC dashboard](../../../ui/console.md) - [Authority scopes](../../authority/architecture.md) --- diff --git a/docs/modules/platform/architecture-overview.md b/docs/modules/platform/architecture-overview.md index 568de1c8..8f851edb 100644 --- a/docs/modules/platform/architecture-overview.md +++ b/docs/modules/platform/architecture-overview.md @@ -147,7 +147,7 @@ sequenceDiagram - [Aggregation-Only Contract reference](../../ingestion/aggregation-only-contract.md) - [Concelier architecture](../concelier/architecture.md) - [Excititor architecture](../excititor/architecture.md) -- [Policy Engine architecture](policy-engine.md) +- [Policy Engine architecture](../policy/architecture.md) - [Authority service](../authority/architecture.md) - [Observability standards (upcoming)](../../observability/policy.md) – interim reference for telemetry naming. diff --git a/docs/modules/telemetry/operations/collector.md b/docs/modules/telemetry/operations/collector.md index 588115f6..3f64e4a6 100644 --- a/docs/modules/telemetry/operations/collector.md +++ b/docs/modules/telemetry/operations/collector.md @@ -39,7 +39,7 @@ docker compose -f docker-compose.telemetry-storage.yaml up -d python ../../ops/devops/telemetry/smoke_otel_collector.py --host localhost ``` -The smoke test posts sample traces, metrics, and logs and verifies that the collector increments the `otelcol_receiver_accepted_*` counters exposed via the Prometheus exporter. The storage overlay gives you a local Prometheus/Tempo/Loki stack to confirm end-to-end wiring. The same client certificate can be used by local services to weave traces together. See [`Telemetry Storage Deployment`](telemetry-storage.md) for the storage configuration guidelines used in staging/production. +The smoke test posts sample traces, metrics, and logs and verifies that the collector increments the `otelcol_receiver_accepted_*` counters exposed via the Prometheus exporter. The storage overlay gives you a local Prometheus/Tempo/Loki stack to confirm end-to-end wiring. The same client certificate can be used by local services to weave traces together. See [`Telemetry Storage Deployment`](storage.md) for the storage configuration guidelines used in staging/production. --- diff --git a/docs/modules/telemetry/operations/storage.md b/docs/modules/telemetry/operations/storage.md index f2a3ca6a..e0efd244 100644 --- a/docs/modules/telemetry/operations/storage.md +++ b/docs/modules/telemetry/operations/storage.md @@ -27,6 +27,7 @@ cd deploy/compose docker compose -f docker-compose.telemetry.yaml up -d docker compose -f docker-compose.telemetry-storage.yaml up -d python ../../ops/devops/telemetry/smoke_otel_collector.py --host localhost +python ../../ops/devops/telemetry/validate_storage_stack.py ``` Configuration files live in `deploy/telemetry/storage/`. Adjust the overrides before shipping to staging/production. @@ -160,6 +161,7 @@ Provision the following secrets/configs (names can be overridden via Helm values - [ ] Prometheus scrape succeeds (`curl -sk --cert client.crt --key client.key https://collector:9464`). - [ ] Tempo and Loki report tenant activity (`/api/status`). - [ ] Retention policy tested by uploading sample data and verifying expiry. +- [ ] `python ops/devops/telemetry/validate_storage_stack.py` passes before committing updated configs. - [ ] Alerts wired into SLO evaluator (DEVOPS-OBS-51-001). - [ ] Component rule packs imported (e.g. `docs/modules/scheduler/operations/worker-prometheus-rules.yaml`). diff --git a/docs/notifications/overview.md b/docs/notifications/overview.md index 94a946a4..a10fc838 100644 --- a/docs/notifications/overview.md +++ b/docs/notifications/overview.md @@ -55,7 +55,7 @@ The Notify WebService fronts worker state with REST APIs used by the UI and CLI. | Step | Goal | Reference | |------|------|-----------| | 1 | Deploy Notify WebService + Worker with Mongo and Redis | [`modules/notify/architecture.md`](../modules/notify/architecture.md#1-runtime-shape--projects) | -| 2 | Register OAuth clients/scopes in Authority | [`etc/authority.yaml.sample`](../etc/authority.yaml.sample) | +| 2 | Register OAuth clients/scopes in Authority | [`etc/authority.yaml.sample`](../../etc/authority.yaml.sample) | | 3 | Install channel plug-ins and capture secret references | [`plugins/notify`](../../plugins) | | 4 | Create a tenant rule and test preview | [`POST /channels/{id}/test`](../modules/notify/architecture.md#8-external-apis-webservice) | | 5 | Inspect deliveries and digests | `/api/v1/notify/deliveries`, `/api/v1/notify/digests` | diff --git a/docs/observability/observability.md b/docs/observability/observability.md index 8084f267..6dbe866b 100644 --- a/docs/observability/observability.md +++ b/docs/observability/observability.md @@ -120,10 +120,10 @@ Update `docs/assets/dashboards/` with screenshots when Grafana capture pipeline - [Aggregation-Only Contract reference](../ingestion/aggregation-only-contract.md) - [Architecture overview](../modules/platform/architecture-overview.md) - [Console AOC dashboard](../ui/console.md) -- [CLI AOC commands](../cli/cli-reference.md) +- [CLI AOC commands](../modules/cli/guides/cli-reference.md) - [Concelier architecture](../modules/concelier/architecture.md) - [Excititor architecture](../modules/excititor/architecture.md) -- [Scheduler Worker observability guide](../ops/scheduler-worker-operations.md) +- [Scheduler Worker observability guide](../modules/scheduler/operations/worker.md) --- diff --git a/docs/overview.md b/docs/overview.md new file mode 100644 index 00000000..2654dff0 --- /dev/null +++ b/docs/overview.md @@ -0,0 +1,39 @@ +# Stella Ops – 2‑Minute Overview + +## The Problem We Solve + +- **Supply-chain attacks exploded 742 % in three years;** regulated teams still need to scan hundreds of containers a day while disconnected from the public Internet. +- **Existing scanners trade freedom for SaaS:** no offline feeds, hidden quotas, noisy results that lack exploitability context. +- **Audit fatigue is real:** Policy decisions are opaque, replaying scans is guesswork, and trust hinges on external transparency logs you do not control. + +## The Promise + +Stella Ops delivers **deterministic, sovereign container security** that works the same online or fully air-gapped: + +1. **Deterministic replay manifests** (SRM) prove every scan result, so auditors can rerun evidence and see the exact same outcome. +2. **Lattice policy engine + OpenVEX** keeps findings explainable; exploitability, attestation, and waivers merge into one verdict. +3. **Sovereign crypto profiles** let you anchor signatures to eIDAS, FIPS, GOST, or SM roots, mirror your feeds, and keep Sigstore-compatible transparency logs offline. + +## Core Capability Clusters + +| Cluster | What you get | Why it matters | +|---------|--------------|----------------| +| **SBOM-first scanning** | Delta-layer SBOM cache, sub‑5 s warm scans, Trivy/CycloneDX/SPDX ingestion + dependency cartographing | Speeds repeat scans 10× and keeps SBOMs the source of truth | +| **Explainable policy** | OpenVEX + lattice logic, policy engine for custom rule packs, waiver expirations | Reduces alert fatigue, supports alert muting beyond VEX, and shows why a finding blocks deploy | +| **Attestation & provenance** | DSSE bundles, optional Rekor mirror, DSSE → CLI/UI exports | Lets you prove integrity without relying on external services | +| **Offline operations** | Offline Update Kit bundles, mirrored feeds, quota tokens verified locally | Works for sovereign clouds, SCIFs, and heavily regulated sectors | +| **Governance & observability** | Structured audit trails, quota transparency, per-tenant metrics | Keeps compliance teams and operators in sync without extra tooling | + +## Who Benefits + +| Persona | Outcome in week one | +|---------|--------------------| +| **Security engineering** | Deterministic replay + explain traces | cuts review time, keeps waivers honest | +| **Platform / SRE** | Fast scans, local registry, no Internet dependency | fits pipelines and air-gapped staging | +| **Compliance & risk** | Signed SBOMs, provable quotas, legal/attestation docs | supports audits without custom tooling | + +## Where to Go Next + +- Ready to pull the containers? Head to [quickstart.md](quickstart.md). +- Want the capability detail? Browse the five cards in [key-features.md](key-features.md). +- Need to evaluate fit and build a rollout plan? Grab the [evaluation checklist](evaluate/checklist.md). diff --git a/docs/quickstart.md b/docs/quickstart.md new file mode 100644 index 00000000..f0505b6d --- /dev/null +++ b/docs/quickstart.md @@ -0,0 +1,93 @@ +# Quickstart – First Scan in Five Minutes + +> **Status:** public α image ships late 2025 (`registry.stella-ops.org/stella-ops/stella-ops:0.1.0-alpha`). Commands below are ready the moment the tag lands. + +## 0. Prerequisites (1 min) + +| Requirement | Minimum | Notes | +|-------------|---------|-------| +| OS | Ubuntu 22.04 LTS / Alma 9 | x86‑64 or arm64 | +| Docker | Engine 25 + Compose v2 | `docker -v` | +| Resources | 2 vCPU / 2 GiB RAM / 10 GiB SSD | Fits developer laptops | +| TLS trust | Built-in self-signed or your own certs | Replace `/certs` before production | + +Keep Redis and MongoDB bundled unless you already operate managed instances. + +## 1. Download the signed bundles (1 min) + +```bash +curl -LO https://get.stella-ops.org/docker-compose.infrastructure.yml +curl -LO https://get.stella-ops.org/docker-compose.infrastructure.yml.sig +curl -LO https://get.stella-ops.org/docker-compose.stella-ops.yml +curl -LO https://get.stella-ops.org/docker-compose.stella-ops.yml.sig + +cosign verify-blob \ + --key https://stella-ops.org/keys/cosign.pub \ + --signature docker-compose.infrastructure.yml.sig \ + docker-compose.infrastructure.yml + +cosign verify-blob \ + --key https://stella-ops.org/keys/cosign.pub \ + --signature docker-compose.stella-ops.yml.sig \ + docker-compose.stella-ops.yml +``` + +*Air-gapped?* The [Offline Update Kit](24_OFFLINE_KIT.md) ships these files plus feeds and plug-ins. + +## 2. Configure `.env` (1 min) + +Create `.env` with the essentials: + +```dotenv +STELLA_OPS_COMPANY_NAME="Acme Corp" +STELLA_OPS_DEFAULT_ADMIN_USERNAME="admin" +STELLA_OPS_DEFAULT_ADMIN_PASSWORD="change-me!" +MONGO_INITDB_ROOT_USERNAME=stella_admin +MONGO_INITDB_ROOT_PASSWORD=$(openssl rand -base64 18) +MONGO_URL=mongodb +REDIS_PASSWORD=$(openssl rand -base64 18) +REDIS_URL=redis +``` + +Use existing Redis/Mongo endpoints by setting `MONGO_URL` and `REDIS_URL`. Keep credentials scoped to Stella Ops; Redis counters enforce the transparent quota (`{{ quota_token }}` scans/day). + +## 3. Launch services (1 min) + +```bash +docker compose --env-file .env -f docker-compose.infrastructure.yml up -d +docker compose --env-file .env -f docker-compose.stella-ops.yml up -d +``` + +- `StellaOps.Authority` issues short-lived OpToks for CLI/UI. +- `StellaOps.Scanner` hosts `/scan`, queues work to Workers. +- `StellaOps.Policy.Engine` and `StellaOps.Concelier` start with seeded policies, feeds sync in the background. + +## 4. Run your first scan (1 min) + +```bash +stella auth login --device-code +stella scan image \ + --image registry.stella-ops.org/demo/juice-shop:latest \ + --sbom-type cyclonedx-json +``` + +- Expect `<5 s` warm scans once the Delta SBOM cache is primed. +- CLI exits non-zero if lattice policy blocks the image; use `stella policy explain --last` for context. +- Headers `X-Stella-Quota-Remaining` and the UI banner keep quota usage transparent. + +## 5. Verify & explore (1 min) + +- Check the Console (`https://localhost:8443`) to view findings, VEX evidence, and deterministic replay manifests. +- Export the DSSE bundle: `stella export run --format dsse`. +- Capture evidence for audit: `stella attest bundle --output demo.dsse.json`. + +### Sovereign mode in one click + +- Import the Offline Update Kit (`stella offline-kit import ./stella-ouk-2025-alpha.tar.gz`) to replace every external feed. +- Apply a CryptoProfile (`stella authority crypto apply ./profiles/fips.yaml`) to swap signing algorithms without rebuilding. + +### Next steps + +- Harden the deployment with [`17_SECURITY_HARDENING_GUIDE.md`](17_SECURITY_HARDENING_GUIDE.md). +- Explore feature highlights in [`key-features.md`](key-features.md). +- Plan the rollout using the [evaluation checklist](evaluate/checklist.md). diff --git a/docs/security/authority-scopes.md b/docs/security/authority-scopes.md index c3b065b7..3bb3315d 100644 --- a/docs/security/authority-scopes.md +++ b/docs/security/authority-scopes.md @@ -241,7 +241,7 @@ clients: - [Concelier architecture](../modules/concelier/architecture.md) - [Excititor architecture](../modules/excititor/architecture.md) - [Policy governance](policy-governance.md) -- [Authority key rotation playbook](../ops/authority-key-rotation.md) +- [Authority key rotation playbook](../modules/authority/operations/key-rotation.md) --- diff --git a/docs/technical/README.md b/docs/technical/README.md new file mode 100644 index 00000000..6c9bdc78 --- /dev/null +++ b/docs/technical/README.md @@ -0,0 +1,17 @@ +# Technical Documentation Index + +> Use this hub when you need the full implementation detail: architecture blueprints, data contracts, developer guides, and operations playbooks. Each section below links to the canonical sources already living in `docs/**`. + +## Sections +- [Strategy & Core Specs](strategy/README.md) +- [Platform Architecture & Module Dossiers](architecture/README.md) + - [Component map](architecture/component-map.md) +- [Interfaces, Contracts & Schemas](interfaces/README.md) +- [Development Guides & Tooling](development/README.md) +- [Operations, Deployment & Offline](operations/README.md) +- [Observability, Notifications & Telemetry](observability/README.md) +- [Security, Risk & Governance](security/README.md) +- [Process, Coordination & Change Logs](process/README.md) + +--- +Need a doc that is missing here? Raise an entry in `docs/TASKS.md` so the index stays complete. diff --git a/docs/technical/architecture/README.md b/docs/technical/architecture/README.md new file mode 100644 index 00000000..f8241668 --- /dev/null +++ b/docs/technical/architecture/README.md @@ -0,0 +1,44 @@ +# Platform Architecture & Module Dossiers + +Use this index to locate architecture narratives, boundaries, and implementation plans for every Stella Ops component. + +## Core Views +- [../high-level-architecture.md](../../high-level-architecture.md) – 10-minute overview of the end-to-end flow. +- [../07_HIGH_LEVEL_ARCHITECTURE.md](../../07_HIGH_LEVEL_ARCHITECTURE.md) – exhaustive reference (data flows, trust boundaries, operational traits). +- [../40_ARCHITECTURE_OVERVIEW.md](../../40_ARCHITECTURE_OVERVIEW.md) – design principles applied across modules. +- [../scanner-core-contracts.md](../../scanner-core-contracts.md) – canonical DTOs shared by Scanner services and consumers. +- Legacy service dossier: [../11_AUTHORITY.md](../../11_AUTHORITY.md) – Authority overview before module split. +- UI documentation set: [../../ui/](../../ui/) (navigation, policies, findings, runs, tours). +- Component map: [component-map.md](component-map.md) – quick descriptions of every `src/` module and how they interact. + +## Module Catalogue +Each module directory bundles an ownership charter (`AGENTS.md`), current work (`TASKS.md`), architecture dossier, and implementation plan. Operations guides live under `operations/` where applicable. + +| Module | Architecture | Implementation Plan | Operations / Extras | +|--------|--------------|---------------------|---------------------| +| Authority | [architecture.md](../../modules/authority/architecture.md) | [implementation_plan.md](../../modules/authority/implementation_plan.md) | [operations](../../modules/authority/operations/) | +| Advisory AI | [architecture.md](../../modules/advisory-ai/architecture.md) | [implementation_plan.md](../../modules/advisory-ai/implementation_plan.md) | — | +| Attestor | [architecture.md](../../modules/attestor/architecture.md) | [implementation_plan.md](../../modules/attestor/implementation_plan.md) | — | +| CLI | [architecture.md](../../modules/cli/architecture.md) | [implementation_plan.md](../../modules/cli/implementation_plan.md) | [operations/release-and-packaging.md](../../modules/cli/operations/release-and-packaging.md) | +| CI Recipes | [architecture.md](../../modules/ci/architecture.md) | [implementation_plan.md](../../modules/ci/implementation_plan.md) | [recipes.md](../../modules/ci/recipes.md) | +| Concelier | [architecture.md](../../modules/concelier/architecture.md) | [implementation_plan.md](../../modules/concelier/implementation_plan.md) | [operations/](../../modules/concelier/operations/) | +| DevOps / Release | [architecture.md](../../modules/devops/architecture.md) | [implementation_plan.md](../../modules/devops/implementation_plan.md) | [runbooks](../../modules/devops/runbooks/) | +| Excititor | [architecture.md](../../modules/excititor/architecture.md) | [implementation_plan.md](../../modules/excititor/implementation_plan.md) | [mirrors.md](../../modules/excititor/mirrors.md) | +| Export Center | [architecture.md](../../modules/export-center/architecture.md) | [implementation_plan.md](../../modules/export-center/implementation_plan.md) | [operations/runbook.md](../../modules/export-center/operations/runbook.md) | +| Graph | [architecture.md](../../modules/graph/architecture.md) | [implementation_plan.md](../../modules/graph/implementation_plan.md) | — | +| Notify | [architecture.md](../../modules/notify/architecture.md) | [implementation_plan.md](../../modules/notify/implementation_plan.md) | — | +| Orchestrator | [architecture.md](../../modules/orchestrator/architecture.md) | [implementation_plan.md](../../modules/orchestrator/implementation_plan.md) | — | +| Platform | [architecture-overview.md](../../modules/platform/architecture-overview.md) + [architecture.md](../../modules/platform/architecture.md) | [implementation_plan.md](../../modules/platform/implementation_plan.md) | — | +| Policy Engine | [architecture.md](../../modules/policy/architecture.md) | [implementation_plan.md](../../modules/policy/implementation_plan.md) | — | +| Registry Token Service | [architecture.md](../../modules/registry/architecture.md) | [implementation_plan.md](../../modules/registry/implementation_plan.md) | [operations/token-service.md](../../modules/registry/operations/token-service.md) | +| Scanner | [architecture.md](../../modules/scanner/architecture.md) | [implementation_plan.md](../../modules/scanner/implementation_plan.md) | [operations/](../../modules/scanner/operations/) | +| Scheduler | [architecture.md](../../modules/scheduler/architecture.md) | [implementation_plan.md](../../modules/scheduler/implementation_plan.md) | [operations/](../../modules/scheduler/operations/) | +| Signer | [architecture.md](../../modules/signer/architecture.md) | [implementation_plan.md](../../modules/signer/implementation_plan.md) | — | +| Telemetry Stack | [architecture.md](../../modules/telemetry/architecture.md) | [implementation_plan.md](../../modules/telemetry/implementation_plan.md) | [operations/collector.md](../../modules/telemetry/operations/collector.md), [operations/storage.md](../../modules/telemetry/operations/storage.md) | +| UI / Console | [architecture.md](../../modules/ui/architecture.md), [console-architecture.md](../../modules/ui/console-architecture.md) | [implementation_plan.md](../../modules/ui/implementation_plan.md) | — | +| Vuln Explorer | [architecture.md](../../modules/vuln-explorer/architecture.md) | [implementation_plan.md](../../modules/vuln-explorer/implementation_plan.md) | — | +| VEX Lens | [architecture.md](../../modules/vex-lens/architecture.md) | [implementation_plan.md](../../modules/vex-lens/implementation_plan.md) | — | +| Vexer | [architecture.md](../../modules/vexer/architecture.md) | [implementation_plan.md](../../modules/vexer/implementation_plan.md) | [scoring.md](../../modules/vexer/scoring.md) | +| Zastava | [architecture.md](../../modules/zastava/architecture.md) | [implementation_plan.md](../../modules/zastava/implementation_plan.md) | — | + +> **Tip:** Every module directory also exposes `README.md`, `AGENTS.md`, and `TASKS.md` for roles, current backlog, and ownership responsibilities. diff --git a/docs/technical/architecture/component-map.md b/docs/technical/architecture/component-map.md new file mode 100644 index 00000000..0ff1e54c --- /dev/null +++ b/docs/technical/architecture/component-map.md @@ -0,0 +1,77 @@ +# Platform Component Map + +Concise descriptions of every top-level component under `src/`, summarising the role documented across Stella Ops technical guides and how each module interacts with the rest of the platform. Use this as a quick orientation map before diving into the module-specific dossiers listed in [architecture/README.md](README.md). + +## Advisory & Evidence Services +- **AdvisoryAI** — Experimental intelligence helpers that summarise and prioritise advisory data for humans. Ingests canonical observations from Concelier/Excititor, adds explainable insights, and feeds UI/CLI and Policy workflows. See `docs/modules/advisory-ai/architecture.md`. +- **Concelier** — Canonical advisory ingestion engine enforcing the Aggregation-Only Contract (AOC). Produces immutable observations/linksets consumed by Policy Engine, Graph, Scheduler, and Export Center. Docs in `docs/modules/concelier/architecture.md` and `docs/ingestion/aggregation-only-contract.md`. +- **Excititor** — VEX statement normaliser applying AOC guardrails. Supplies VEX observations to Policy Engine, VEX Lens, Scheduler, and UI. Reference `docs/modules/excititor/architecture.md` and `docs/vex/aggregation.md`. +- **VexLens** — Provides focused exploration of VEX evidence, conflict analysis, and waiver insights for UI/CLI. Backed by Excititor and Policy Engine (`docs/modules/vex-lens/architecture.md`). +- **EvidenceLocker** — Long-term store for signed evidence bundles (DSSE, SRM, policy waivers). Integrates with Attestor, Export Center, Policy, and replay tooling (`docs/forensics/evidence-locker.md`). +- **ExportCenter** — Packages reproducible evidence bundles and mirror artefacts for online/offline distribution. Pulls from Concelier, Excititor, Policy, Scanner, Attestor, and Registry (`docs/modules/export-center/architecture.md`). +- **Mirror** — Feed and artefact mirroring services supporting Offline Update Kits, registry mirrors, and air-gapped updates (`docs/modules/devops/architecture.md`, `docs/airgap/`). + +## Scanning, SBOM & Risk +- **Scanner** — Deterministic scanning with API + worker pair. Generates SBOM fragments, emits SRM/DSSE-ready reports, hands results to Signer/Attestor, and surfaces status to Scheduler/CLI/UI (`docs/modules/scanner/architecture.md`). +- **SbomService** — SBOM inventory store and delta cache leveraged by Scanner, Policy Engine, Cartographer, and Export Center (`docs/modules/scanner/architecture.md`, SBOM sections). +- **RiskEngine** — Consolidates Policy verdicts, runtime signals, and graph overlays into prioritised risk views (`docs/modules/policy/architecture.md`, `docs/modules/graph/architecture.md`). +- **Findings** — Materialises effective findings from Policy Engine outputs and evidence. Feeds UI, CLI, Notify, and Governance dashboards (`docs/modules/policy/architecture.md`, findings sections). +- **Cartographer** — Builds identity graphs from SBOM/advisory data for Graph Explorer and RiskEngine (`docs/modules/graph/architecture.md`). +- **Graph** — Graph API + indexer, exposing relationship queries to UI/CLI/Scheduler (`docs/modules/graph/architecture.md`). +- **VulnExplorer** — Explorer for vulnerabilities that combines Concelier data, graph overlays, and Policy results for UI/CLI consumption (`docs/modules/vuln-explorer/architecture.md`). + +## Policy & Governance +- **Policy** — Policy Engine core libraries and services executing lattice logic across SBOM, advisory, and VEX evidence. Emits explain traces, drives Findings, Notifier, and Export Center (`docs/modules/policy/architecture.md`). +- **Policy Studio / TaskRunner / PacksRegistry** — Authoring, automation, and reusable template services that orchestrate policy and operational workflows (`docs/task-packs/`, `docs/modules/cli/`, `docs/modules/ui/`). +- **Governance components** (Authority scopes, Policy governance, Console policy UI) are covered in `docs/security/policy-governance.md` and `docs/modules/ui/policies.md`. + +## Identity, Signing & Provenance +- **Authority** — Identity provider issuing short-lived OpToks, enforcing scopes/tenancy, and powering every module’s authentication story (`docs/11_AUTHORITY.md`, `docs/modules/authority/architecture.md`). +- **Signer** — DSSE signing backend supporting keyless/keyful modes with Authority-managed trust roots (`docs/modules/signer/architecture.md`). +- **Attestor** — Manages proof bundles, optional Rekor mirror, and distribution to consumers (`docs/modules/attestor/architecture.md`). +- **Provenance** — Utilities and services for DSSE/SLSA provenance verification, consumed by Export Center, EvidenceLocker, and Replay (`docs/modules/export-center/provenance-and-signing.md`). +- **IssuerDirectory** — Directory of trust issuers/KMS bindings used by Authority, Signer, Attestor, Export Center, and AirGap cryptographic profiles (`docs/modules/authority/architecture.md`, trust sections). + +## Scheduling, Orchestration & Automation +- **Scheduler** — Detects advisory/VEX deltas and orchestrates deterministic rescan runs toward Scanner and Policy Engine (`docs/modules/scheduler/architecture.md`). +- **Orchestrator** — Central coordination service dispatching jobs (scans, exports, policy runs) to modules, working closely with Scheduler, CLI, and UI (`docs/modules/orchestrator/architecture.md`). +- **TaskRunner** — Executes automation packs sourced from PacksRegistry, integrating with Orchestrator, CLI, Notify, and Authority (`docs/task-packs/runbook.md`). +- **Signals** — Ingests runtime posture signals and feeds Policy/Notifier workflows (`docs/modules/zastava/architecture.md`, signals sections). +- **TimelineIndexer** — Builds timelines of evidence/events for forensics and audit tooling (`docs/forensics/timeline.md`). + +## Notification & UI +- **Notifier** — New notifications studio with rule engine, digesting, and channel plug-ins (`docs/notifications/overview.md`). +- **Notify** — Legacy notification service referenced in backlog/cleanup docs; still handles existing deployments (`docs/modules/notify/architecture.md`). +- **UI** — Angular console surfacing scans, policy authoring, VEX evidence, runtime posture, and admin flows. Talks to Web gateway, Authority, Policy, Concelier, Scheduler, Notify, etc. (`docs/modules/ui/architecture.md`). +- **DevPortal** — Developer onboarding portal consuming Api definitions, CLI samples, and Authority auth flows (`docs/modules/devops/architecture.md`, dev portal sections). + +## Runtime & Registry +- **Registry** — Anonymous registry/token service hosting platform images and Offline Kit artefacts (`docs/modules/registry/architecture.md`). +- **Zastava** — Runtime observer/admission controller ensuring signed images, SBOM availability, and policy verdict enforcement in live clusters (`docs/modules/zastava/architecture.md`). +- **Signals** (shared above) plus runtime components integrate tightly with Zastava and Policy Engine. +- **Bench** — Performance benchmarking toolset validating platform SLAs (`docs/12_PERFORMANCE_WORKBOOK.md`). + +## Offline, Telemetry & Infrastructure +- **AirGap** — Bundles Offline Update Kits, enforces sealed-mode operations, and distributes trust roots/feeds (`docs/10_OFFLINE_KIT.md`, `docs/airgap/`). +- **Telemetry** — OpenTelemetry collector/storage deployment tooling, observability integrations, and offline metrics packages (`docs/modules/telemetry/architecture.md`, `docs/observability/`). +- **Mirror** and **ExportCenter** (above) complement AirGap by keeping offline mirrors in sync. +- **Tools** — Collection of utility programs (fixture generators, smoke tests, migration scripts) supporting all modules (`docs/dev/fixtures.md`, module-specific tooling sections). + +## CLI, SDK, Web Gateway +- **Cli** — Native command-line interface orchestrating scans, policy operations, offline workflows, and evidence replay (`docs/modules/cli/architecture.md`). +- **Sdk** — Shared SDK packages for third-party integration (C#, TS, etc.), wrapping Authority auth and API definitions (`docs/api/`). +- **Web** — API gateway/BFF exposing module APIs to UI/CLI and external clients, performing auth & route orchestration (`docs/modules/platform/architecture-overview.md`, gateway sections). + +## Remaining Shared Libraries +- **Api**, **Sdk**, **__Libraries** — Core shared contracts and helper libraries referenced throughout modules (configuration, messaging, federation). Each module dossier highlights its shared dependencies. +- **Aoc** library (mentioned above) is reused by ingestion components and verification tooling to enforce the Aggregation-Only Contract. + +## How It All Connects +High-level flows (see `docs/high-level-architecture.md` for diagrams): +1. **Ingest** — Concelier and Excititor use AOC to ingest advisories/VEX; Scheduler observes deltas. +2. **Scan & Evaluate** — Scanner generates SBOM evidence and hands to Signer/Attestor; Policy Engine merges SBOM, advisory, VEX, runtime signals; RiskEngine prioritises. +3. **Store & Export** — EvidenceLocker and Export Center package results; Registry serves artefacts; AirGap bundles offline editions. +4. **Observe & Notify** — Telemetry captures metrics/traces/logs; Notifier/Notify deliver alerts; UI/CLI/Web expose operations; TimelineIndexer builds audit trails. +5. **Govern & Secure** — Authority, IssuerDirectory, Signer, and Attestor maintain trust; Policy governance and console experiences let teams manage waivers and approvals. + +Refer back to module-specific documentation for APIs, configuration, schema details, and operational runbooks. This component map will stay updated alongside module architecture changes—log updates in `docs/updates/` whenever new modules are introduced or deprecated. diff --git a/docs/technical/development/README.md b/docs/technical/development/README.md new file mode 100644 index 00000000..18fd94cf --- /dev/null +++ b/docs/technical/development/README.md @@ -0,0 +1,33 @@ +# Development Guides & Tooling + +Resources for contributors building features, plug-ins, connectors, and tests. + +## Engineering Standards & Quality +- [../18_CODING_STANDARDS.md](../../18_CODING_STANDARDS.md) – language guidelines, project layout, review expectations. +- [../19_TEST_SUITE_OVERVIEW.md](../../19_TEST_SUITE_OVERVIEW.md) – unit, integration, golden, and determinism test strategy. +- [../12_PERFORMANCE_WORKBOOK.md](../../12_PERFORMANCE_WORKBOOK.md) – benchmark targets and reference rigs. +- [../cli-vs-ui-parity.md](../../cli-vs-ui-parity.md) – CLI vs Console feature parity tracking. +- [../scanner-core-contracts.md](../../scanner-core-contracts.md) – DTO fixtures consumed by tests. + +## Plug-ins, Connectors & Extensions +- [../10_PLUGIN_SDK_GUIDE.md](../../10_PLUGIN_SDK_GUIDE.md) – plug-in lifecycle, manifests, packaging. +- [../10_CONCELIER_CLI_QUICKSTART.md](../../10_CONCELIER_CLI_QUICKSTART.md) – local Concelier + CLI workflow for advisory ingestion. +- Developer guides under [../dev/](../../dev/): + - Connector playbooks (`30_EXCITITOR_CONNECTOR_GUIDE.md`, `30_VEXER_CONNECTOR_GUIDE.md`, `concelier-connector-research-20251011.md`, `kisa_connector_notes.md`). + - Authority and DPoP guidance (`31_AUTHORITY_PLUGIN_DEVELOPER_GUIDE.md`, `authority-dpop-mtls-plan.md`, `authority-plugin-di-coordination.md`, `authority-rate-limit-tuning-outline.md`, `32_AUTH_CLIENT_GUIDE.md`). + - Analyzer and cache configuration (`SCANNER_CACHE_CONFIGURATION.md`, `java-analyzer-observation-plan.md`, `EXCITITOR_STATEMENT_BACKFILL.md`). + - Normalisation & merge references (`aoc-normalization-removal-notes.md`, `merge_semver_playbook.md`, `normalized-rule-recipes.md`, `normalized_versions_rollout.md`). + - Operational templates and fixtures (`templates/`, `fixtures.md`). + - Mongo/Cartographer details (`mongo_indices.md`, `cartographer-graph-handshake.md`). + +## CLI, SDKs & Automation +- [../09_API_CLI_REFERENCE.md](../../09_API_CLI_REFERENCE.md) – authoritative CLI commands and flags (use for scripting). +- [../api/sdk-openapi-program.md](../../api/sdk-openapi-program.md) – guidance for downstream SDK generation. +- [../policy/gateway.md](../../policy/gateway.md) & [../policy/dsl.md](../../policy/dsl.md) – foundations for automating policy programs. + +## Scaffolding & Examples +- [../examples/policies/README.md](../../examples/policies/README.md) – sample policy bundles. +- [../examples/ui-tours.md](../../examples/ui-tours.md) and [../assets/ui/tours/README.md](../../assets/ui/tours/README.md) – console tour authoring guides. +- [../task-packs/](../../task-packs/) – reusable task templates for sprints. +- [../faq/policy-faq.md](../../faq/policy-faq.md) – policy author FAQ. +- [../faq/](../../faq/) – additional Q&A sets useful during development. diff --git a/docs/technical/interfaces/README.md b/docs/technical/interfaces/README.md new file mode 100644 index 00000000..e3fb88dd --- /dev/null +++ b/docs/technical/interfaces/README.md @@ -0,0 +1,48 @@ +# Interfaces, Contracts & Schemas + +Specifications covering APIs, data contracts, event envelopes, and enforcement models. + +## External & Internal APIs +- [../09_API_CLI_REFERENCE.md](../../09_API_CLI_REFERENCE.md) – canonical REST and CLI surface (scan, policy, auth, health). +- [../api/policy.md](../../api/policy.md) – Policy Engine REST endpoints. +- Module APIs: see relevant module architecture docs (e.g., [../../modules/export-center/api.md](../../modules/export-center/api.md)). + +## Policy & Decisioning +- [../policy/overview.md](../../policy/overview.md) – Policy Engine fundamentals. +- [../policy/dsl.md](../../policy/dsl.md) – `stella-dsl@1` grammar. +- [../policy/lifecycle.md](../../policy/lifecycle.md) – creation, promotion, approval flows. +- [../policy/runs.md](../../policy/runs.md) – execution orchestrations. +- [../policy/exception-effects.md](../../policy/exception-effects.md) – waiver semantics. +- [../policy/gateway.md](../../policy/gateway.md) – gateway service contract. +- [../60_POLICY_TEMPLATES.md](../../60_POLICY_TEMPLATES.md) – YAML/Rego samples. + +## Data Schemas & Storage Contracts +- [../11_DATA_SCHEMAS.md](../../11_DATA_SCHEMAS.md) – MongoDB/Redis/document shapes. +- JSON schemas under [../schemas/](../../schemas/) – policy diff, explain trace, run request, run status, preview sample, report sample. +- [../../modules/scanner/architecture.md](../../modules/scanner/architecture.md) – SBOM cache and scan job contracts. +- [../../scanner-core-contracts.md](../../scanner-core-contracts.md) – shared scanner DTOs. + +## Events & Messaging +- [../events/README.md](../../events/README.md) – event catalogue (`scanner.scan.completed@1`, `scheduler.rescan.delta@1`, etc.). +- Payload schemas in [../events/*.json](../../events/) and samples in [../events/samples/](../../events/samples/). +- [../observability/policy.md](../../observability/policy.md) and [../observability/ui-telemetry.md](../../observability/ui-telemetry.md) – telemetry event guidance. + +## Ingestion & Evidence Contracts +- [../ingestion/aggregation-only-contract.md](../../ingestion/aggregation-only-contract.md) – Aggregation-Only Contract reference. +- [../aoc/aoc-guardrails.md](../../aoc/aoc-guardrails.md) – guardrails checklist. +- [../advisories/aggregation.md](../../advisories/aggregation.md) – advisory observation schema. +- [../vex/aggregation.md](../../vex/aggregation.md) – VEX observation schema. +- [../../modules/concelier/operations/connectors/](../../modules/concelier/operations/connectors/) – connector-specific payload notes. + +## Identity, Quota & Licence Enforcement +- [../license-jwt-quota.md](../../license-jwt-quota.md) – offline quota token design. +- [../30_QUOTA_ENFORCEMENT_FLOW1.md](../../30_QUOTA_ENFORCEMENT_FLOW1.md) – enforcement sequence diagram. +- [../33_333_QUOTA_OVERVIEW.md](../../33_333_QUOTA_OVERVIEW.md) – free tier policy. +- [../30_QUOTA_ENFORCEMENT_FLOW1.md](../../30_QUOTA_ENFORCEMENT_FLOW1.md) and [../33_333_QUOTA_OVERVIEW.md](../../33_333_QUOTA_OVERVIEW.md) – pair with [../29_LEGAL_FAQ_QUOTA.md](../../29_LEGAL_FAQ_QUOTA.md) for legal framing. +- [../../modules/authority/architecture.md](../../modules/authority/architecture.md) – OpTok issuance & validation contracts. +- [../../modules/registry/architecture.md](../../modules/registry/architecture.md) – token service scope and audit requirements. + +## Transparency & Attestation +- [../../modules/attestor/architecture.md](../../modules/attestor/architecture.md) – DSSE/Rekor bundle contracts. +- [../../modules/signer/architecture.md](../../modules/signer/architecture.md) – signing workflow contracts. +- [../../modules/export-center/provenance-and-signing.md](../../modules/export-center/provenance-and-signing.md) – export bundle evidence artefacts. diff --git a/docs/technical/observability/README.md b/docs/technical/observability/README.md new file mode 100644 index 00000000..30dd07b0 --- /dev/null +++ b/docs/technical/observability/README.md @@ -0,0 +1,29 @@ +# Observability, Notifications & Telemetry + +Guides for capturing metrics, logs, traces, and delivering notifications. + +## Observability Stack +- [../observability/observability.md](../../observability/observability.md) – AOC observability overview. +- [../observability/policy.md](../../observability/policy.md) – policy-specific telemetry guidance. +- [../observability/ui-telemetry.md](../../observability/ui-telemetry.md) – UI instrumentation and SSE tracing. +- Telemetry module docs: [../../modules/telemetry/architecture.md](../../modules/telemetry/architecture.md), [../../modules/telemetry/implementation_plan.md](../../modules/telemetry/implementation_plan.md), [../../modules/telemetry/operations/collector.md](../../modules/telemetry/operations/collector.md), [../../modules/telemetry/operations/storage.md](../../modules/telemetry/operations/storage.md). +- Authority / Scanner dashboards: see respective module `operations/*.json` and Grafana runbooks. + +## Events & Streaming +- [../events/README.md](../../events/README.md) – canonical event definitions. +- Payload schemas (JSON): [../events/scanner.scan.completed@1.json](../../events/scanner.scan.completed@1.json), [../events/scanner.report.ready@1.json](../../events/scanner.report.ready@1.json), [../events/scheduler.rescan.delta@1.json](../../events/scheduler.rescan.delta@1.json), [../events/attestor.logged@1.json](../../events/attestor.logged@1.json), etc. +- [../events/samples/](../../events/samples/) – sample payloads with validation workflow. +- [../../modules/export-center/provenance-and-signing.md](../../modules/export-center/provenance-and-signing.md) – provenance event integration. + +## Notifications Studio +- [../notifications/overview.md](../../notifications/overview.md) – architecture and channels. +- [../notifications/rules.md](../../notifications/rules.md) – rule authoring. +- [../notifications/templates.md](../../notifications/templates.md) – template management. +- [../notifications/digests.md](../../notifications/digests.md) – digest scheduling. +- [../../modules/notify/architecture.md](../../modules/notify/architecture.md) & [../../modules/notify/implementation_plan.md](../../modules/notify/implementation_plan.md) – implementation detail. + +## Metrics & Dashboards +- Scanner analyzers dashboard: [../../modules/scanner/operations/analyzers-grafana-dashboard.json](../../modules/scanner/operations/analyzers-grafana-dashboard.json). +- Scheduler worker dashboards & alert rules: [../../modules/scheduler/operations/worker-grafana-dashboard.json](../../modules/scheduler/operations/worker-grafana-dashboard.json), [../../modules/scheduler/operations/worker-prometheus-rules.yaml](../../modules/scheduler/operations/worker-prometheus-rules.yaml). +- Authority monitoring: [../../modules/authority/operations/monitoring.md](../../modules/authority/operations/monitoring.md). +- DevOps observability tasks: see [../../modules/devops/architecture.md](../../modules/devops/architecture.md) and runbooks. diff --git a/docs/technical/operations/README.md b/docs/technical/operations/README.md new file mode 100644 index 00000000..f9fd48c2 --- /dev/null +++ b/docs/technical/operations/README.md @@ -0,0 +1,47 @@ +# Operations, Deployment & Offline + +Deployment, runtime operations, and air-gap playbooks for running Stella Ops in production. + +## Install & Upgrade +- [../21_INSTALL_GUIDE.md](../../21_INSTALL_GUIDE.md) – canonical install guide (Docker, air-gap considerations). +- [../install/docker.md](../../install/docker.md) – Docker install recipes. +- [../deploy/containers.md](../../deploy/containers.md) – container deployment guidance for AOC environments. +- [../deploy/console.md](../../deploy/console.md) – console deployment specifics. +- [../13_RELEASE_ENGINEERING_PLAYBOOK.md](../../13_RELEASE_ENGINEERING_PLAYBOOK.md) – release automation, signing, reproducibility. +- [../artifacts/bom-index/README.md](../../artifacts/bom-index/README.md) – BOM index artifact layout for Offline Kit exports. + +## Offline & Sovereign Operations +- [../quickstart.md](../../quickstart.md) – 5-minute path to first scan (useful for smoke testing installs). +- [../10_OFFLINE_KIT.md](../../10_OFFLINE_KIT.md) & [../24_OFFLINE_KIT.md](../../24_OFFLINE_KIT.md) – bundle contents, import/export workflow. +- [../airgap/airgap-mode.md](../../airgap/airgap-mode.md) – configuration for sealed environments. +- [../license-jwt-quota.md](../../license-jwt-quota.md) – offline quota token lifecycle. +- [../10_CONCELIER_CLI_QUICKSTART.md](../../10_CONCELIER_CLI_QUICKSTART.md) – workstation ingest/export workflow (operators). + +## Hardening & Governance +- [../17_SECURITY_HARDENING_GUIDE.md](../../17_SECURITY_HARDENING_GUIDE.md) – platform hardening checklist. +- [../accessibility.md](../../accessibility.md) – accessibility checklist for console deployments. +- [../security/console-security.md](../../security/console-security.md) – console-specific controls. +- [../security/authority-scopes.md](../../security/authority-scopes.md) – Authority scope model. +- [../security/rate-limits.md](../../security/rate-limits.md) – throttling policy reference. +- [../security/policy-governance.md](../../security/policy-governance.md) – policy governance guardrails. +- [../security/audit-events.md](../../security/audit-events.md) – audit event catalogue. +- [../security/revocation-bundle.md](../../security/revocation-bundle.md) & [../security/revocation-bundle-example.json](../../security/revocation-bundle-example.json) – revocation workflow. +- [../security/password-hashing.md](../../security/password-hashing.md) – credential storage details. + +## Module Runbooks & Ops Guides +- Module operations directories under [../../modules/](../../modules/) (Authority backups/monitoring, Concelier connectors, Scanner analyzers, Scheduler worker dashboards, Export Center runbook, DevOps launch readiness, Telemetry collector/storage, etc.). +- [../runtime/SCANNER_RUNTIME_READINESS.md](../../runtime/SCANNER_RUNTIME_READINESS.md) – runtime readiness checklist. +- Notifications Studio operations: see [../notifications/architecture.md](../../notifications/architecture.md), [../notifications/overview.md](../../notifications/overview.md), [../notifications/rules.md](../../notifications/rules.md), [../notifications/templates.md](../../notifications/templates.md), [../notifications/digests.md](../../notifications/digests.md). +- Additional notification flows: [../notifications/pack-approvals-integration.md](../../notifications/pack-approvals-integration.md). +- Observability operations: [../observability/observability.md](../../observability/observability.md), [../observability/ui-telemetry.md](../../observability/ui-telemetry.md). + +## DevOps & Release Automation +- [../devops/policy-schema-export.md](../../devops/policy-schema-export.md) – policy schema export automation. +- [../modules/devops/runbooks/launch-readiness.md](../../modules/devops/runbooks/launch-readiness.md), [../modules/devops/runbooks/launch-cutover.md](../../modules/devops/runbooks/launch-cutover.md), [../modules/devops/runbooks/deployment-upgrade.md](../../modules/devops/runbooks/deployment-upgrade.md), [../modules/devops/runbooks/nuget-preview-bootstrap.md](../../modules/devops/runbooks/nuget-preview-bootstrap.md). +- [../modules/registry/operations/token-service.md](../../modules/registry/operations/token-service.md) – registry token runbook. +- [../modules/concelier/operations/mirror.md](../../modules/concelier/operations/mirror.md) – mirror operations. +- [../modules/concelier/operations/connectors/](../../modules/concelier/operations/connectors/) – connector-specific procedures (ACSC, CCCS, CERT-Bund, etc.). +- [../modules/authority/operations/](../../modules/authority/operations/) – key rotation, monitoring, backup/restore. +- [../modules/scanner/operations/](../../modules/scanner/operations/) – analyzer management, entrypoint guides, RustFS migration. +- [../modules/scheduler/operations/](../../modules/scheduler/operations/) – worker dashboards, Prometheus rules. +- [../modules/telemetry/operations/](../../modules/telemetry/operations/) – collector/storage deployment. diff --git a/docs/technical/process/README.md b/docs/technical/process/README.md new file mode 100644 index 00000000..0994bd47 --- /dev/null +++ b/docs/technical/process/README.md @@ -0,0 +1,25 @@ +# Process, Coordination & Change Logs + +Use these artefacts to understand team ownership, active workstreams, and historical updates. + +## Ownership & Roles +- [../AGENTS.md](../../AGENTS.md) – global agent/role definitions. +- Module ownership: each directory under [../modules/](../../modules/) includes `AGENTS.md`, `TASKS.md`, and `README.md` describing responsibilities. + +## Work Tracking +- [../TASKS.md](../../TASKS.md) – Docs Guild task board. +- Sprint plans and historical boards: [../implplan/SPRINTS.md](../../implplan/SPRINTS.md), [../implplan/SPRINTS_PRIOR_20251028.md](../../implplan/SPRINTS_PRIOR_20251028.md), [../implplan/SPRINTS_PRIOR_20251027.md](../../implplan/SPRINTS_PRIOR_20251027.md), [../implplan/SPRINTS_PRIOR_20251025.md](../../implplan/SPRINTS_PRIOR_20251025.md), [../implplan/SPRINTS_PRIOR_20251021.md](../../implplan/SPRINTS_PRIOR_20251021.md), [../implplan/SPRINTS_PRIOR_20251019.md](../../implplan/SPRINTS_PRIOR_20251019.md). +- Execution plan: [../implplan/EXECPLAN.md](../../implplan/EXECPLAN.md). +- Backlog hygiene and consolidation notes: [../backlog/](../../backlog/). +- Task packs and reusable templates: [../task-packs/](../../task-packs/). + +## Communication & Updates +- Architecture decision records: [../adr/index.md](../../adr/index.md) (template in [../adr/0000-template.md](../../adr/0000-template.md)). +- RFCs in flight: [../rfcs/authority-plugin-ldap.md](../../rfcs/authority-plugin-ldap.md). +- Release notes & updates: [../updates/](../../updates/). +- Frequently asked questions: [../faq/](../../faq/). +- Examples and golden data: [../examples/](../../examples/), [../events/samples/](../../events/samples/). + +## Supporting References +- Risk & governance: [../risk/risk-profiles.md](../../risk/risk-profiles.md), [../security/policy-governance.md](../../security/policy-governance.md). +- Observability/process integration: [../events/orchestrator-scanner-events.md](../../events/orchestrator-scanner-events.md), [../events/README.md](../../events/README.md). diff --git a/docs/technical/security/README.md b/docs/technical/security/README.md new file mode 100644 index 00000000..8758fe22 --- /dev/null +++ b/docs/technical/security/README.md @@ -0,0 +1,35 @@ +# Security, Risk & Governance + +Authoritative sources for threat models, governance, compliance, and security operations. + +## Policies & Governance +- [../13_SECURITY_POLICY.md](../../13_SECURITY_POLICY.md) – responsible disclosure, support windows. +- [../11_GOVERNANCE.md](../../11_GOVERNANCE.md) – project governance charter. +- [../12_CODE_OF_CONDUCT.md](../../12_CODE_OF_CONDUCT.md) – community expectations. +- [../17_SECURITY_HARDENING_GUIDE.md](../../17_SECURITY_HARDENING_GUIDE.md) – deployment hardening steps. +- [../security/policy-governance.md](../../security/policy-governance.md) – policy governance specifics. +- [../29_LEGAL_FAQ_QUOTA.md](../../29_LEGAL_FAQ_QUOTA.md) – legal interpretation of quota. +- [../33_333_QUOTA_OVERVIEW.md](../../33_333_QUOTA_OVERVIEW.md) – quota policy reference. +- [../risk/risk-profiles.md](../../risk/risk-profiles.md) – organisational risk personas. + +## Threat Models & Security Architecture +- [../security/authority-threat-model.md](../../security/authority-threat-model.md) – Authority service threat analysis. +- [../security/authority-scopes.md](../../security/authority-scopes.md) – scope model. +- [../security/console-security.md](../../security/console-security.md) – Console posture guidance. +- [../security/pack-signing-and-rbac.md](../../security/pack-signing-and-rbac.md) – pack signing, RBAC guardrails. +- [../security/policy-governance.md](../../security/policy-governance.md) – policy governance controls. +- [../security/rate-limits.md](../../security/rate-limits.md) – rate limiting behaviour. +- [../security/password-hashing.md](../../security/password-hashing.md) – credential storage. + +## Audit, Revocation & Compliance +- [../security/audit-events.md](../../security/audit-events.md) – audit event taxonomy. +- [../security/revocation-bundle.md](../../security/revocation-bundle.md) & [../security/revocation-bundle-example.json](../../security/revocation-bundle-example.json) – revocation process. +- [../license-jwt-quota.md](../../license-jwt-quota.md) – licence/quota enforcement controls. +- [../30_QUOTA_ENFORCEMENT_FLOW1.md](../../30_QUOTA_ENFORCEMENT_FLOW1.md) – quota enforcement sequence. +- [../10_OFFLINE_KIT.md](../../10_OFFLINE_KIT.md) & [../24_OFFLINE_KIT.md](../../24_OFFLINE_KIT.md) – tamper-evident offline artefacts. +- [../security/](../../security/) – browse for additional deep dives (audit, scopes, rate limits). + +## Supporting Material +- Module operations security notes: [../../modules/authority/operations/key-rotation.md](../../modules/authority/operations/key-rotation.md), [../../modules/concelier/operations/authority-audit-runbook.md](../../modules/concelier/operations/authority-audit-runbook.md), [../../modules/zastava/README.md](../../modules/zastava/README.md) (runtime enforcement). +- [../observability/policy.md](../../observability/policy.md) – security-relevant telemetry for policy. +- [../updates/2025-10-27-console-security-signoff.md](../../updates/2025-10-27-console-security-signoff.md) & [../updates/2025-10-31-console-security-refresh.md](../../updates/2025-10-31-console-security-refresh.md) – recent security sign-offs. diff --git a/docs/technical/strategy/README.md b/docs/technical/strategy/README.md new file mode 100644 index 00000000..e249da5e --- /dev/null +++ b/docs/technical/strategy/README.md @@ -0,0 +1,22 @@ +# Strategy & Core Specifications + +Foundational references that describe Stella Ops’ goals, scope, and differentiators. + +- [../03_VISION.md](../../03_VISION.md) – north-star, KPIs, quarterly themes. +- [../04_FEATURE_MATRIX.md](../../04_FEATURE_MATRIX.md) – capability matrix by tier (free, community, commercial). +- [../05_SYSTEM_REQUIREMENTS_SPEC.md](../../05_SYSTEM_REQUIREMENTS_SPEC.md) – functional and non-functional requirements for the `v0.1.0-alpha` release (quota, scanning, policy, SLAs). +- [../40_ARCHITECTURE_OVERVIEW.md](../../40_ARCHITECTURE_OVERVIEW.md) – guiding principles and platform-level design rationale. +- [../moat.md](../../moat.md) – differentiating workstreams (deterministic replay, lattice policy, sovereign crypto readiness, attestation graph). +- [../05_ROADMAP.md](../../05_ROADMAP.md) – legacy pointer to the public web roadmap (kept for historical links). +- [../33_333_QUOTA_OVERVIEW.md](../../33_333_QUOTA_OVERVIEW.md) – free tier policy framing. +- [../29_LEGAL_FAQ_QUOTA.md](../../29_LEGAL_FAQ_QUOTA.md) – legal interpretation of quota enforcement under AGPL-3.0. +- [../13_SECURITY_POLICY.md](../../13_SECURITY_POLICY.md) – responsible disclosure support window and release line commitments. +- [../14_GLOSSARY_OF_TERMS.md](../../14_GLOSSARY_OF_TERMS.md) – canonical vocabulary used across documentation. +- [../15_UI_GUIDE.md](../../15_UI_GUIDE.md) – UX overview for stakeholders evaluating the console. +- [../23_FAQ_MATRIX.md](../../23_FAQ_MATRIX.md) – stakeholder FAQ. + +## Related Concepts +- [../33_333_QUOTA_OVERVIEW.md](../../33_333_QUOTA_OVERVIEW.md) and [../30_QUOTA_ENFORCEMENT_FLOW1.md](../../30_QUOTA_ENFORCEMENT_FLOW1.md) align business policy with enforcement diagrams. +- [../license-jwt-quota.md](../../license-jwt-quota.md) – offline licensing narrative for quota tokens. +- [../moat.md](../../moat.md) – includes procurement-grade trust statement blueprint. +- [../10_OFFLINE_KIT.md](../../10_OFFLINE_KIT.md) & [../24_OFFLINE_KIT.md](../../24_OFFLINE_KIT.md) – strategic offline story (also referenced in Operations). diff --git a/docs/ui/admin.md b/docs/ui/admin.md index 47d80fb6..88ab798e 100644 --- a/docs/ui/admin.md +++ b/docs/ui/admin.md @@ -138,10 +138,10 @@ Audit entries appear for every user/token change. CLI parity: `stella auth token ## 11. Screenshot coordination -- Placeholders: - - `![Admin tenants placeholder](../assets/ui/admin/tenants-placeholder.png)` - - `![Admin roles placeholder](../assets/ui/admin/roles-placeholder.png)` - - `![Admin tokens placeholder](../assets/ui/admin/tokens-placeholder.png)` +- Placeholders (captures pending upload): + - `docs/assets/ui/admin/tenants-placeholder.png` + - `docs/assets/ui/admin/roles-placeholder.png` + - `docs/assets/ui/admin/tokens-placeholder.png` - Capture real screenshots with Authority Guild once Sprint 23 UI is final (tracked in `#console-screenshots`, 2025-10-26 entry). Provide both light and dark theme variants. --- @@ -171,4 +171,3 @@ Audit entries appear for every user/token change. CLI parity: `stella auth token --- *Last updated: 2025-10-26 (Sprint 23).* - diff --git a/docs/ui/advisories-and-vex.md b/docs/ui/advisories-and-vex.md index aa58daa6..f500ea38 100644 --- a/docs/ui/advisories-and-vex.md +++ b/docs/ui/advisories-and-vex.md @@ -1,199 +1,198 @@ -# StellaOps Console - Advisories and VEX - -> **Audience:** Console UX team, Concelier and Excititor guilds, support and compliance engineers. -> **Scope:** Advisory aggregation UX, VEX consensus display, conflict indicators, raw document viewer, provenance banners, CLI parity, and Aggregation-Only Contract (AOC) guardrails for Sprint 23. - -The Advisories and VEX surfaces expose Concelier and Excititor outputs without mutating the underlying data. Operators can review upstream statements, check consensus summaries, inspect conflicts, and hand off evidence to downstream tooling while staying within the Aggregation-Only Contract. - ---- - -## 1. Access and prerequisites - -- **Routes:** - - `/console/advisories` (advisory list and detail) - - `/console/vex` (VEX consensus and raw claim explorer) -- **Scopes:** `advisory.read` and `vex.read` (base access), `advisory.verify` / `vex.verify` for verification actions, `downloads.read` for evidence exports. -- **Feature flags:** `advisoryExplorer.enabled`, `vexExplorer.enabled`, `aggregation.conflictIndicators`. -- **Dependencies:** Concelier WebService (aggregation API + delta metrics), Excititor WebService (consensus API + conflict feeds), Policy Engine explain hints (optional link-outs), Authority tenant enforcement. -- **Offline behaviour:** Uses Offline Kit snapshots when gateway is in sealed mode; verify buttons queue until connectivity resumes. - ---- - -## 2. Layout overview - -``` -+---------------------------------------------------------------------+ -| Header: Tenant badge - global filters - status ticker - actions | -+---------------------------------------------------------------------+ -| Left rail: Saved views - provider filters - verification queue | -+---------------------------------------------------------------------+ -| Main split pane | -| - Advisories tab (grid + detail drawer) | -| - VEX tab (consensus table + claim drawer) | -| Tabs remember last active view per tenant. | -+---------------------------------------------------------------------+ -``` - -The header reuses console-wide context chips (`Tenant`, `Severity`, `Source`, `Time`) and the status ticker that streams Concelier and Excititor deltas. - ---- - -## 3. Advisory aggregation view - -| Element | Description | -|---------|-------------| -| **Grid columns** | Vulnerability key (CVE/GHSA/vendor), Title, Source set, Last merged, Severity badge, KEV flag, Affected product count, Merge hash. | -| **Source chips** | Show contributing providers (NVD, Red Hat, Debian, vendor PSIRT). Hover reveals precedence order and timestamps. | -| **Severity** | Displays the highest severity declared by any source; tooltip lists per-source severities and vectors. | -| **KEV / Exploit status** | Badge highlights known exploited status from Concelier enrichment; links to KEV reference. | -| **Merge hash** | Deterministic hash from Concelier `merge_event`. Clicking copies hash and opens provenance banner. | -| **Filters** | Vulnerability identifier search, provider multi-select, severity picker, KEV toggle, affected product range slider, time window. | -| **List actions** | `Open detail`, `Copy CLI` (`stella advisory show ...`), `Compare sources`, `Queue verify`. | - -The grid virtualises up to 15,000 advisories per tenant. Beyond that, the UI engages server-side pagination with cursor hints supplied by Concelier. - ---- - -## 4. Advisory detail drawer - -Sections within the drawer: - -1. **Summary cards** (title, published/modified timestamps, advisory merge hash, total sources, exploited flag). -2. **Sources timeline** listing each contributing document with signature status, fetched timestamps, precedence rank, and quick links to raw view. -3. **Affected products** table (product key, introduced/fixed, range semantics, distro qualifiers, notes). Column toggles allow switching between SemVer and distro notation. -4. **Conflict indicators** show when sources disagree on fixed versions, severity, or affected sets. Each conflict row links to an explainer panel that describes the winning value, losing sources, and precedence rule. -5. **References** collapsible list (patches, advisories, exploits). -6. **Raw JSON** viewer (read-only) using canonical Concelier payload. Users can copy JSON or download via `GET /console/advisories/raw/{id}`. -7. **CLI parity** card with commands: - - `stella advisory show --tenant --vuln ` - - `stella advisory sources --tenant --vuln ` - - `stella advisory export --tenant --vuln --format cdx-json` - -Provenance banner at the top indicates whether all sources are signed, partially signed, or unsigned, referencing AOC guardrails. Unsigned sources trigger a warning and link to the verification checklist. - ---- - -## 5. VEX explorer - -| Feature | Description | -|---------|-------------| -| **Consensus table** | Rows keyed by `(vulnId, productKey)` with rollup status (affected, not affected, fixed, under investigation), confidence score, provider count, and last evaluation timestamp. | -| **Status badges** | Colour-coded (red affected, green not affected, blue fixed, amber under investigation). Tooltips show justification and policy revision used. | -| **Provider breakdown** | Hover or expand to see source list with accepted/ignored flag, status, justification code, signature state, weight. | -| **Filters** | Product search (PURL), status filter, provider filter, justification codes, confidence threshold slider. | -| **Saved views** | Prebuilt presets: `Vendor consensus`, `Distro overrides`, `Conflicts`, `Pending investigation`. | - ---- - -## 6. VEX detail drawer - -Tabs within the drawer: - -- **Consensus summary**: Restates rollup status, policy revision, confidence benchmarks, and referencing runs. -- **Claims list**: Every raw claim from Excititor with provenance, signature result, justification, supersedes chain, evidence snippets. Claims are grouped by provider tier (vendor, distro, ecosystem, CERT). -- **Conflict explainers**: For conflicting claims, shows why a claim was ignored (weight, stale timestamp, failing justification gate). Includes inline diff between competing claims. -- **Events**: Timeline of claim arrivals and consensus evaluations with correlation IDs, accessible for debugging. -- **Raw JSON**: Canonical `VexClaim` or `VexConsensus` payloads with copy/download. CLI parity callouts: - - `stella vex consensus show --tenant --vuln --product ` - - `stella vex claims show --tenant --vuln --provider ` - ---- - -## 7. Raw viewers and provenance - -- Raw viewers display canonical payloads with syntax highlighting and copy-as-JSON support. -- Provenance banner presents: source URI, document digest, signature status, fetch timestamps, collector version. -- Users can open raw documents in a modal that includes: - - `sha256` digest with copy button - - Signature verification summary (passing keys, missing signatures, errors) - - `Download DSSE bundle` button when the document is attested - - `Open in logs` link that copies search query (`correlationId=...`) for log aggregation tools. - -All raw views are read-only to maintain Aggregation-Only guarantees. - ---- - -## 8. Conflict indicators and aggregation-not-merge UX - -- Concelier retains every source; the UI surfaces conflicts rather than merging them. -- Conflict badges appear in grids and detail views when sources disagree on affected ranges, fixed versions, severity, or exploit flags. -- Clicking a badge opens the conflict explainer panel (powered by Concelier merge metadata) that lists winning/losing sources, ranks, and reasoning (e.g., "Vendor PSIRT overrides ecosystem advisory"). -- Excititor conflicts highlight discarded claims with reasons (stale, failing justification, low weight). Operators can override weights downstream via Policy Engine if needed. -- UI copy explicitly reminds users that policy decisions happen elsewhere; these views show aggregated facts only. - ---- - -## 9. Verification workflows - -- **Run verify** buttons call Concelier or Excititor verification endpoints (`POST /console/advisories/verify`, `POST /console/vex/verify`) scoped by tenant and source filters. -- Verification results appear as banners summarising documents checked, signatures verified, and guard violations. -- Failed verifications show actionable error IDs (`ERR_AOC_00x`), matching CLI output. -- Verification history accessible via the status ticker dropdown; entries include operator, scope, and correlation IDs. - ---- - -## 10. Exports and automation - -- Advisory tab exposes export actions: `Download normalized advisory`, `Download affected products CSV`, `Download source bundle` (raw documents packaged with manifest). -- VEX tab supports exports for consensus snapshots, raw claims, and provider deltas. -- Export manifests include merge hash or consensus digest, tenant ID, timestamp, and signature state. -- CLI parity snippets accompany each export (e.g., `stella advisory export`, `stella vex export`). -- Automation: copy buttons for webhook subscription (`/downloads/hooks/subscribe`) and ORAS push commands when using remote registries. - ---- - -## 11. Observability and SSE updates - -- Status ticker shows ingest lag (`advisory_delta_minutes`, `vex_delta_minutes`), last merge event hash, and verification queue depth. -- Advisory and VEX grids refresh via SSE channels; updates animate row badges (new source, conflict resolved). -- Metrics surfaced in drawers: ingestion age, signature pass rate, consensus evaluation duration. -- Errors display correlation IDs linking to Concelier/Excititor logs. - ---- - -## 12. Offline and air-gap behaviour - -- When offline, list views display snapshot badge, staleness timer, and disable real-time verification. -- Raw downloads reference local snapshot directories and include checksum instructions. -- Exports queue locally; UI offers `Copy to removable media` instructions. -- CLI parity switches to offline commands (`--offline`, `--snapshot`). -- Tenant picker hides tenants not present in the snapshot to avoid partial data views. - ---- - -## 13. Screenshot coordination - -- Placeholders: - - `![Advisory grid placeholder](../assets/ui/advisories/grid-placeholder.png)` - - `![VEX consensus placeholder](../assets/ui/advisories/vex-placeholder.png)` -- Coordinate with Console Guild to capture updated screenshots (dark and light themes) once Sprint 23 build candidate is tagged. Tracking in Slack channel `#console-screenshots` (entry 2025-10-26). - ---- - -## 14. References - -- `/docs/ui/console-overview.md` - shell, filters, tenant model. -- `/docs/ui/navigation.md` - command palette, deep-link schema. -- `/docs/ingestion/aggregation-only-contract.md` - AOC guardrails. -- `/docs/architecture/CONCELIER.md` - merge rules, provenance. -- `/docs/architecture/EXCITITOR.md` - VEX consensus model. -- `/docs/security/console-security.md` - scopes, DPoP, CSP. -- `/docs/cli-vs-ui-parity.md` - CLI equivalence matrix. - ---- - -## 15. Compliance checklist - -- [ ] Advisory grid columns, filters, and merge hash behaviour documented. -- [ ] VEX consensus view covers status badges, provider breakdown, and filters. -- [ ] Raw viewer and provenance banners explained with AOC alignment. -- [ ] Conflict indicators and explainers tied to aggregation-not-merge rules. -- [ ] Verification workflow and CLI parity documented. -- [ ] Offline behaviour and automation paths captured. -- [ ] Screenshot placeholders and coordination notes recorded. -- [ ] References validated. - ---- - -*Last updated: 2025-10-26 (Sprint 23).* - +# StellaOps Console - Advisories and VEX + +> **Audience:** Console UX team, Concelier and Excititor guilds, support and compliance engineers. +> **Scope:** Advisory aggregation UX, VEX consensus display, conflict indicators, raw document viewer, provenance banners, CLI parity, and Aggregation-Only Contract (AOC) guardrails for Sprint 23. + +The Advisories and VEX surfaces expose Concelier and Excititor outputs without mutating the underlying data. Operators can review upstream statements, check consensus summaries, inspect conflicts, and hand off evidence to downstream tooling while staying within the Aggregation-Only Contract. + +--- + +## 1. Access and prerequisites + +- **Routes:** + - `/console/advisories` (advisory list and detail) + - `/console/vex` (VEX consensus and raw claim explorer) +- **Scopes:** `advisory.read` and `vex.read` (base access), `advisory.verify` / `vex.verify` for verification actions, `downloads.read` for evidence exports. +- **Feature flags:** `advisoryExplorer.enabled`, `vexExplorer.enabled`, `aggregation.conflictIndicators`. +- **Dependencies:** Concelier WebService (aggregation API + delta metrics), Excititor WebService (consensus API + conflict feeds), Policy Engine explain hints (optional link-outs), Authority tenant enforcement. +- **Offline behaviour:** Uses Offline Kit snapshots when gateway is in sealed mode; verify buttons queue until connectivity resumes. + +--- + +## 2. Layout overview + +``` ++---------------------------------------------------------------------+ +| Header: Tenant badge - global filters - status ticker - actions | ++---------------------------------------------------------------------+ +| Left rail: Saved views - provider filters - verification queue | ++---------------------------------------------------------------------+ +| Main split pane | +| - Advisories tab (grid + detail drawer) | +| - VEX tab (consensus table + claim drawer) | +| Tabs remember last active view per tenant. | ++---------------------------------------------------------------------+ +``` + +The header reuses console-wide context chips (`Tenant`, `Severity`, `Source`, `Time`) and the status ticker that streams Concelier and Excititor deltas. + +--- + +## 3. Advisory aggregation view + +| Element | Description | +|---------|-------------| +| **Grid columns** | Vulnerability key (CVE/GHSA/vendor), Title, Source set, Last merged, Severity badge, KEV flag, Affected product count, Merge hash. | +| **Source chips** | Show contributing providers (NVD, Red Hat, Debian, vendor PSIRT). Hover reveals precedence order and timestamps. | +| **Severity** | Displays the highest severity declared by any source; tooltip lists per-source severities and vectors. | +| **KEV / Exploit status** | Badge highlights known exploited status from Concelier enrichment; links to KEV reference. | +| **Merge hash** | Deterministic hash from Concelier `merge_event`. Clicking copies hash and opens provenance banner. | +| **Filters** | Vulnerability identifier search, provider multi-select, severity picker, KEV toggle, affected product range slider, time window. | +| **List actions** | `Open detail`, `Copy CLI` (`stella advisory show ...`), `Compare sources`, `Queue verify`. | + +The grid virtualises up to 15,000 advisories per tenant. Beyond that, the UI engages server-side pagination with cursor hints supplied by Concelier. + +--- + +## 4. Advisory detail drawer + +Sections within the drawer: + +1. **Summary cards** (title, published/modified timestamps, advisory merge hash, total sources, exploited flag). +2. **Sources timeline** listing each contributing document with signature status, fetched timestamps, precedence rank, and quick links to raw view. +3. **Affected products** table (product key, introduced/fixed, range semantics, distro qualifiers, notes). Column toggles allow switching between SemVer and distro notation. +4. **Conflict indicators** show when sources disagree on fixed versions, severity, or affected sets. Each conflict row links to an explainer panel that describes the winning value, losing sources, and precedence rule. +5. **References** collapsible list (patches, advisories, exploits). +6. **Raw JSON** viewer (read-only) using canonical Concelier payload. Users can copy JSON or download via `GET /console/advisories/raw/{id}`. +7. **CLI parity** card with commands: + - `stella advisory show --tenant --vuln ` + - `stella advisory sources --tenant --vuln ` + - `stella advisory export --tenant --vuln --format cdx-json` + +Provenance banner at the top indicates whether all sources are signed, partially signed, or unsigned, referencing AOC guardrails. Unsigned sources trigger a warning and link to the verification checklist. + +--- + +## 5. VEX explorer + +| Feature | Description | +|---------|-------------| +| **Consensus table** | Rows keyed by `(vulnId, productKey)` with rollup status (affected, not affected, fixed, under investigation), confidence score, provider count, and last evaluation timestamp. | +| **Status badges** | Colour-coded (red affected, green not affected, blue fixed, amber under investigation). Tooltips show justification and policy revision used. | +| **Provider breakdown** | Hover or expand to see source list with accepted/ignored flag, status, justification code, signature state, weight. | +| **Filters** | Product search (PURL), status filter, provider filter, justification codes, confidence threshold slider. | +| **Saved views** | Prebuilt presets: `Vendor consensus`, `Distro overrides`, `Conflicts`, `Pending investigation`. | + +--- + +## 6. VEX detail drawer + +Tabs within the drawer: + +- **Consensus summary**: Restates rollup status, policy revision, confidence benchmarks, and referencing runs. +- **Claims list**: Every raw claim from Excititor with provenance, signature result, justification, supersedes chain, evidence snippets. Claims are grouped by provider tier (vendor, distro, ecosystem, CERT). +- **Conflict explainers**: For conflicting claims, shows why a claim was ignored (weight, stale timestamp, failing justification gate). Includes inline diff between competing claims. +- **Events**: Timeline of claim arrivals and consensus evaluations with correlation IDs, accessible for debugging. +- **Raw JSON**: Canonical `VexClaim` or `VexConsensus` payloads with copy/download. CLI parity callouts: + - `stella vex consensus show --tenant --vuln --product ` + - `stella vex claims show --tenant --vuln --provider ` + +--- + +## 7. Raw viewers and provenance + +- Raw viewers display canonical payloads with syntax highlighting and copy-as-JSON support. +- Provenance banner presents: source URI, document digest, signature status, fetch timestamps, collector version. +- Users can open raw documents in a modal that includes: + - `sha256` digest with copy button + - Signature verification summary (passing keys, missing signatures, errors) + - `Download DSSE bundle` button when the document is attested + - `Open in logs` link that copies search query (`correlationId=...`) for log aggregation tools. + +All raw views are read-only to maintain Aggregation-Only guarantees. + +--- + +## 8. Conflict indicators and aggregation-not-merge UX + +- Concelier retains every source; the UI surfaces conflicts rather than merging them. +- Conflict badges appear in grids and detail views when sources disagree on affected ranges, fixed versions, severity, or exploit flags. +- Clicking a badge opens the conflict explainer panel (powered by Concelier merge metadata) that lists winning/losing sources, ranks, and reasoning (e.g., "Vendor PSIRT overrides ecosystem advisory"). +- Excititor conflicts highlight discarded claims with reasons (stale, failing justification, low weight). Operators can override weights downstream via Policy Engine if needed. +- UI copy explicitly reminds users that policy decisions happen elsewhere; these views show aggregated facts only. + +--- + +## 9. Verification workflows + +- **Run verify** buttons call Concelier or Excititor verification endpoints (`POST /console/advisories/verify`, `POST /console/vex/verify`) scoped by tenant and source filters. +- Verification results appear as banners summarising documents checked, signatures verified, and guard violations. +- Failed verifications show actionable error IDs (`ERR_AOC_00x`), matching CLI output. +- Verification history accessible via the status ticker dropdown; entries include operator, scope, and correlation IDs. + +--- + +## 10. Exports and automation + +- Advisory tab exposes export actions: `Download normalized advisory`, `Download affected products CSV`, `Download source bundle` (raw documents packaged with manifest). +- VEX tab supports exports for consensus snapshots, raw claims, and provider deltas. +- Export manifests include merge hash or consensus digest, tenant ID, timestamp, and signature state. +- CLI parity snippets accompany each export (e.g., `stella advisory export`, `stella vex export`). +- Automation: copy buttons for webhook subscription (`/downloads/hooks/subscribe`) and ORAS push commands when using remote registries. + +--- + +## 11. Observability and SSE updates + +- Status ticker shows ingest lag (`advisory_delta_minutes`, `vex_delta_minutes`), last merge event hash, and verification queue depth. +- Advisory and VEX grids refresh via SSE channels; updates animate row badges (new source, conflict resolved). +- Metrics surfaced in drawers: ingestion age, signature pass rate, consensus evaluation duration. +- Errors display correlation IDs linking to Concelier/Excititor logs. + +--- + +## 12. Offline and air-gap behaviour + +- When offline, list views display snapshot badge, staleness timer, and disable real-time verification. +- Raw downloads reference local snapshot directories and include checksum instructions. +- Exports queue locally; UI offers `Copy to removable media` instructions. +- CLI parity switches to offline commands (`--offline`, `--snapshot`). +- Tenant picker hides tenants not present in the snapshot to avoid partial data views. + +--- + +## 13. Screenshot coordination + +- Placeholders (captures pending upload): + - `docs/assets/ui/advisories/grid-placeholder.png` + - `docs/assets/ui/advisories/vex-placeholder.png` +- Coordinate with Console Guild to capture updated screenshots (dark and light themes) once Sprint 23 build candidate is tagged. Tracking in Slack channel `#console-screenshots` (entry 2025-10-26). + +--- + +## 14. References + +- `/docs/ui/console-overview.md` - shell, filters, tenant model. +- `/docs/ui/navigation.md` - command palette, deep-link schema. +- `/docs/ingestion/aggregation-only-contract.md` - AOC guardrails. +- `/docs/architecture/CONCELIER.md` - merge rules, provenance. +- `/docs/architecture/EXCITITOR.md` - VEX consensus model. +- `/docs/security/console-security.md` - scopes, DPoP, CSP. +- `/docs/cli-vs-ui-parity.md` - CLI equivalence matrix. + +--- + +## 15. Compliance checklist + +- [ ] Advisory grid columns, filters, and merge hash behaviour documented. +- [ ] VEX consensus view covers status badges, provider breakdown, and filters. +- [ ] Raw viewer and provenance banners explained with AOC alignment. +- [ ] Conflict indicators and explainers tied to aggregation-not-merge rules. +- [ ] Verification workflow and CLI parity documented. +- [ ] Offline behaviour and automation paths captured. +- [ ] Screenshot placeholders and coordination notes recorded. +- [ ] References validated. + +--- + +*Last updated: 2025-10-26 (Sprint 23).* diff --git a/docs/ui/console-overview.md b/docs/ui/console-overview.md index 0e2fed7c..bf87706c 100644 --- a/docs/ui/console-overview.md +++ b/docs/ui/console-overview.md @@ -1,130 +1,130 @@ -# StellaOps Console – Overview - -> **Audience:** Console product leads, Docs Guild writers, backend/API partners. -> **Scope:** Information architecture, tenant scoping, global filters, and Aggregation‑Only Contract (AOC) alignment for the unified StellaOps Console that lands with Sprint 23. - -The StellaOps Console is the single entry point for operators to explore SBOMs, advisories, policies, runs, and administrative surfaces. This overview explains how the console is organised, how users move between tenants, and how shared filters keep data views consistent across modules while respecting AOC boundaries. - ---- - -## 1 · Mission & Principles - -- **Deterministic navigation.** Every route is stable and deep-link friendly. URLs carry enough context (tenant, filter tokens, view modes) to let operators resume work without reapplying filters. -- **Tenant isolation first.** Any cross-tenant action requires fresh authority, and cross-tenant comparisons are made explicit so users never accidentally mix data sets. -- **Aggregation-not-merge UX.** Console surfaces advisory and VEX rollups exactly as produced by Concelier and Excititor—no client-side re-weighting or mutation. -- **Offline parity.** Every view has an offline equivalent powered by Offline Kit bundles or cached data, and exposes the staleness budget prominently. - ---- - -## 2 · Information Architecture - -### 2.1 Primary navigation - -``` -Console Root - ├─ Dashboard # KPIs, alerts, feed age, queue depth - ├─ Findings # Aggregated vulns + explanations (Policy Engine) - ├─ SBOM Explorer # Catalog, component graph, overlays - ├─ Advisories & VEX # Concelier / Excititor aggregation outputs - ├─ Runs # Scheduler runs, scan evidence, retry controls - ├─ Policies # Editor, simulations, approvals - ├─ Downloads # Signed artifacts, Offline Kit parity - ├─ Admin # Tenants, roles, tokens, integrations - └─ Help & Tours # Contextual docs, guided walkthroughs -``` - -Routes lazy-load feature shells so the UI can grow without increasing first-paint cost. Each feature owns its sub-navigation and exposes a `KeyboardShortcuts` modal describing the available accelerators. - -### 2.2 Shared surfaces - -| Surface | Purpose | Notes | -|---------|---------|-------| -| **Top bar** | Shows active tenant, environment badge (prod/non-prod), offline status pill, user menu, notifications inbox, and the command palette trigger (`⌘/Ctrl K`). | Offline status turns amber when data staleness exceeds configured thresholds. | -| **Global filter tray** | Expands from the right edge (`Shift F`). Hosts universal filters (tenant, time window, tags, severity) that apply across compatible routes. | Filter tray remembers per-tenant presets; stored in IndexedDB (non-sensitive). | -| **Context chips** | Display active global filters underneath page titles, with one-click removal (`⌫`). | Chips include the origin (e.g., `Tenant: west-prod`). | -| **Status ticker** | SSE-driven strip that surfaces Concelier/Excititor ingestion deltas, scheduler lag, and attestor queue depth. | Pulls from `/console/status` proxy (see WEB-CONSOLE-23-002). | - ---- - -## 3 · Tenant Model - -| Aspect | Detail | -|--------|--------| -| **Tenant sources** | The console obtains the tenant list and metadata from Authority `/v1/tenants` after login. Tenant descriptors include display name, slug, environment tag, and RBAC hints (role mask). | -| **Selection workflow** | First visit prompts for a default tenant. Afterwards, the tenant picker (`⌘/Ctrl T`) switches context without full reload, issuing `Authorization` refresh with the new tenant scope. | -| **Token handling** | Each tenant change generates a short-lived, DPoP-bound access token (`aud=console`, `tenant=`). Tokens live in memory; metadata persists in `sessionStorage` for reload continuity. | -| **Cross-tenant comparisons** | Side-by-side dashboards (Dashboard, Findings, SBOM Explorer) allow multi-tenant comparison only via explicit *"Add tenant"* control. Requests issue parallel API calls with separate tokens; results render in split panes labelled per tenant. | -| **Fresh-auth gated actions** | Admin and policy approvals call `Authority /fresh-auth` before executing. UI enforces a 5-minute window; afterwards, actions remain visible but disabled pending re-auth. | -| **Audit trail** | Tenant switches emit structured logs (`action=ui.tenant.switch`, `tenantId`, `subject`, `previousTenant`) and appear in Authority audit exports. | - -### 3.1 Offline operation - -In offline or sealed environments, the tenant picker only lists tenants bundled within the Offline Kit snapshot. Switching tenants prompts an "offline snapshot" banner showing the snapshot timestamp. Actions that require round-trips to Authority (fresh-auth, token rotation) show guidance to perform the step on an online bastion and import credentials later. - ---- - -## 4 · Global Filters & Context Tokens - -| Filter | Applies To | Source & Behaviour | -|--------|------------|--------------------| -| **Tenant** | All modules | Primary isolation control. Stored in URL (`?tenant=`) and via `x-tenant-id` header injected by the web proxy. Changes invalidate cached data stores. | -| **Time window** | Dashboard, Findings, Advisories & VEX, Runs | Options: `24 h`, `7 d`, `30 d`, custom ISO range. Default aligns with Compliance/Authority reporting window. Shared via query param `since=`/`until=`. | -| **Severity / Impact** | Findings, Advisories & VEX, SBOM Explorer overlays | Multi-select (Critical/High/Medium/Low/Informational, plus `Exploited` tag). Values map to Policy Engine impact buckets and Concelier KEV flags. | -| **Component tags** | SBOM Explorer, Findings | Tags drawn from SBOM metadata (`component.tags[]`). Includes search-as-you-type with scoped suggestions (package type, supplier, license). | -| **Source providers** | Advisories & VEX | Filter by provider IDs (e.g., NVD, GHSA, vendor VEX). Tied to Aggregation-Only provenance; filtering never alters base precedence. | -| **Run status** | Runs, Dashboard | States: `queued`, `running`, `completed`, `failed`, `cancelled`. Pulled from Scheduler SSE stream; default shows non-terminal states. | -| **Policy view** | Findings, Policies | Toggles between Active policy, Staged policy, and Simulation snapshots. Selecting Simulation requires prior simulation run; console links to create one if absent. | - -Filters emit deterministic tokens placed in the URL hash for copy/paste parity with CLI commands (see `/docs/cli-vs-ui-parity.md`). The console warns when a filter combination has no effect on the current view and offers to reset to defaults. - -### 4.1 Presets & Saved Views - -Users can save a set of global filters as named presets (stored per tenant). Presets show up in the command palette and the dashboard landing cards for quick access (`⌘/Ctrl 1..9`). - ---- - -## 5 · Aggregation-Only Alignment - -- **Read-only aggregation.** Pages that list advisories or VEX claims consume the canonical aggregation endpoints (`/console/advisories`, `/console/vex`). They never merge or reconcile records client-side. Instead, they highlight the source lineage and precedence as supplied by Concelier and Excititor. -- **Consistency indicators.** Each aggregated item displays source badges, precedence order, and a "last merge event hash" so operators can cross-reference Concelier logs. When a source is missing or stale, the UI surfaces a provenance banner linking to the raw document. -- **AOC guardrails.** Workflow actions (e.g., "request verify", "download evidence bundle") route through Concelier WebService guard endpoints that enforce Aggregation-Only rules. UI strings reinforce that policy decisions happen in Policy Engine, not here. -- **Audit alignment.** Any cross-navigation from aggregated data into findings or policies preserves the underlying IDs so analysts can track how aggregated data influences policy verdicts without altering the data itself. -- **CLI parity.** Inline callouts copy the equivalent `stella` CLI commands, ensuring console users can recreate the exact aggregation query offline. - ---- - -## 6 · Performance & Telemetry Anchors - -- Initial boot target: **< 2.5 s** `LargestContentfulPaint` on 4 vCPU air-gapped runner with cached assets. -- Route budget: each feature shell must keep first interaction (hydrated data + filters) under **1.5 s** once tokens resolve. -- Telemetry: console emits metrics via the `/console/telemetry` batch endpoint—`ui_route_render_seconds`, `ui_filter_apply_total`, `ui_tenant_switch_total`, `ui_offline_banner_seconds`. Logs carry correlation IDs matching backend responses for unified tracing. -- Lighthouse CI runs in the console pipeline (see `DEVOPS-CONSOLE-23-001`) and asserts budgets above; failing runs gate releases. - ---- - -## 7 · References - -- `/docs/architecture/console.md` – component-level diagrams (pending Sprint 23 task). -- `/docs/ui/navigation.md` – detailed routes, breadcrumbs, keyboard shortcuts. -- `/docs/ui/downloads.md` – downloads manifest, parity workflows, offline guidance. -- `/docs/ui/sbom-explorer.md` – SBOM-specific flows and overlays. -- `/docs/ui/advisories-and-vex.md` – aggregation UX details. -- `/docs/ui/findings.md` – explain drawer and filter matrix. -- `/docs/security/console-security.md` – OIDC, scopes, CSP, evidence handling. -- `/docs/cli-vs-ui-parity.md` – CLI equivalents and regression automation. - ---- - -## 8 · Compliance Checklist - -- [ ] Tenant picker enforces Authority-issued scopes and logs `ui.tenant.switch`. -- [ ] Global filters update URLs/query tokens for deterministic deep links. -- [ ] Aggregation views show provenance badges and merge hash indicators. -- [ ] CLI parity callouts aligned with `stella` commands for equivalent queries. -- [ ] Offline banner tested with Offline Kit snapshot import and documented staleness thresholds. -- [ ] Accessibility audit covers global filter tray, tenant picker, and keyboard shortcuts (WCAG 2.2 AA). -- [ ] Telemetry and Lighthouse budgets tracked in console CI (`DEVOPS-CONSOLE-23-001`). - ---- - -*Last updated: 2025-10-26 (Sprint 23).* +# StellaOps Console – Overview + +> **Audience:** Console product leads, Docs Guild writers, backend/API partners. +> **Scope:** Information architecture, tenant scoping, global filters, and Aggregation‑Only Contract (AOC) alignment for the unified StellaOps Console that lands with Sprint 23. + +The StellaOps Console is the single entry point for operators to explore SBOMs, advisories, policies, runs, and administrative surfaces. This overview explains how the console is organised, how users move between tenants, and how shared filters keep data views consistent across modules while respecting AOC boundaries. + +--- + +## 1 · Mission & Principles + +- **Deterministic navigation.** Every route is stable and deep-link friendly. URLs carry enough context (tenant, filter tokens, view modes) to let operators resume work without reapplying filters. +- **Tenant isolation first.** Any cross-tenant action requires fresh authority, and cross-tenant comparisons are made explicit so users never accidentally mix data sets. +- **Aggregation-not-merge UX.** Console surfaces advisory and VEX rollups exactly as produced by Concelier and Excititor—no client-side re-weighting or mutation. +- **Offline parity.** Every view has an offline equivalent powered by Offline Kit bundles or cached data, and exposes the staleness budget prominently. + +--- + +## 2 · Information Architecture + +### 2.1 Primary navigation + +``` +Console Root + ├─ Dashboard # KPIs, alerts, feed age, queue depth + ├─ Findings # Aggregated vulns + explanations (Policy Engine) + ├─ SBOM Explorer # Catalog, component graph, overlays + ├─ Advisories & VEX # Concelier / Excititor aggregation outputs + ├─ Runs # Scheduler runs, scan evidence, retry controls + ├─ Policies # Editor, simulations, approvals + ├─ Downloads # Signed artifacts, Offline Kit parity + ├─ Admin # Tenants, roles, tokens, integrations + └─ Help & Tours # Contextual docs, guided walkthroughs +``` + +Routes lazy-load feature shells so the UI can grow without increasing first-paint cost. Each feature owns its sub-navigation and exposes a `KeyboardShortcuts` modal describing the available accelerators. + +### 2.2 Shared surfaces + +| Surface | Purpose | Notes | +|---------|---------|-------| +| **Top bar** | Shows active tenant, environment badge (prod/non-prod), offline status pill, user menu, notifications inbox, and the command palette trigger (`⌘/Ctrl K`). | Offline status turns amber when data staleness exceeds configured thresholds. | +| **Global filter tray** | Expands from the right edge (`Shift F`). Hosts universal filters (tenant, time window, tags, severity) that apply across compatible routes. | Filter tray remembers per-tenant presets; stored in IndexedDB (non-sensitive). | +| **Context chips** | Display active global filters underneath page titles, with one-click removal (`⌫`). | Chips include the origin (e.g., `Tenant: west-prod`). | +| **Status ticker** | SSE-driven strip that surfaces Concelier/Excititor ingestion deltas, scheduler lag, and attestor queue depth. | Pulls from `/console/status` proxy (see WEB-CONSOLE-23-002). | + +--- + +## 3 · Tenant Model + +| Aspect | Detail | +|--------|--------| +| **Tenant sources** | The console obtains the tenant list and metadata from Authority `/v1/tenants` after login. Tenant descriptors include display name, slug, environment tag, and RBAC hints (role mask). | +| **Selection workflow** | First visit prompts for a default tenant. Afterwards, the tenant picker (`⌘/Ctrl T`) switches context without full reload, issuing `Authorization` refresh with the new tenant scope. | +| **Token handling** | Each tenant change generates a short-lived, DPoP-bound access token (`aud=console`, `tenant=`). Tokens live in memory; metadata persists in `sessionStorage` for reload continuity. | +| **Cross-tenant comparisons** | Side-by-side dashboards (Dashboard, Findings, SBOM Explorer) allow multi-tenant comparison only via explicit *"Add tenant"* control. Requests issue parallel API calls with separate tokens; results render in split panes labelled per tenant. | +| **Fresh-auth gated actions** | Admin and policy approvals call `Authority /fresh-auth` before executing. UI enforces a 5-minute window; afterwards, actions remain visible but disabled pending re-auth. | +| **Audit trail** | Tenant switches emit structured logs (`action=ui.tenant.switch`, `tenantId`, `subject`, `previousTenant`) and appear in Authority audit exports. | + +### 3.1 Offline operation + +In offline or sealed environments, the tenant picker only lists tenants bundled within the Offline Kit snapshot. Switching tenants prompts an "offline snapshot" banner showing the snapshot timestamp. Actions that require round-trips to Authority (fresh-auth, token rotation) show guidance to perform the step on an online bastion and import credentials later. + +--- + +## 4 · Global Filters & Context Tokens + +| Filter | Applies To | Source & Behaviour | +|--------|------------|--------------------| +| **Tenant** | All modules | Primary isolation control. Stored in URL (`?tenant=`) and via `x-tenant-id` header injected by the web proxy. Changes invalidate cached data stores. | +| **Time window** | Dashboard, Findings, Advisories & VEX, Runs | Options: `24 h`, `7 d`, `30 d`, custom ISO range. Default aligns with Compliance/Authority reporting window. Shared via query param `since=`/`until=`. | +| **Severity / Impact** | Findings, Advisories & VEX, SBOM Explorer overlays | Multi-select (Critical/High/Medium/Low/Informational, plus `Exploited` tag). Values map to Policy Engine impact buckets and Concelier KEV flags. | +| **Component tags** | SBOM Explorer, Findings | Tags drawn from SBOM metadata (`component.tags[]`). Includes search-as-you-type with scoped suggestions (package type, supplier, license). | +| **Source providers** | Advisories & VEX | Filter by provider IDs (e.g., NVD, GHSA, vendor VEX). Tied to Aggregation-Only provenance; filtering never alters base precedence. | +| **Run status** | Runs, Dashboard | States: `queued`, `running`, `completed`, `failed`, `cancelled`. Pulled from Scheduler SSE stream; default shows non-terminal states. | +| **Policy view** | Findings, Policies | Toggles between Active policy, Staged policy, and Simulation snapshots. Selecting Simulation requires prior simulation run; console links to create one if absent. | + +Filters emit deterministic tokens placed in the URL hash for copy/paste parity with CLI commands (see `/docs/cli-vs-ui-parity.md`). The console warns when a filter combination has no effect on the current view and offers to reset to defaults. + +### 4.1 Presets & Saved Views + +Users can save a set of global filters as named presets (stored per tenant). Presets show up in the command palette and the dashboard landing cards for quick access (`⌘/Ctrl 1..9`). + +--- + +## 5 · Aggregation-Only Alignment + +- **Read-only aggregation.** Pages that list advisories or VEX claims consume the canonical aggregation endpoints (`/console/advisories`, `/console/vex`). They never merge or reconcile records client-side. Instead, they highlight the source lineage and precedence as supplied by Concelier and Excititor. +- **Consistency indicators.** Each aggregated item displays source badges, precedence order, and a "last merge event hash" so operators can cross-reference Concelier logs. When a source is missing or stale, the UI surfaces a provenance banner linking to the raw document. +- **AOC guardrails.** Workflow actions (e.g., "request verify", "download evidence bundle") route through Concelier WebService guard endpoints that enforce Aggregation-Only rules. UI strings reinforce that policy decisions happen in Policy Engine, not here. +- **Audit alignment.** Any cross-navigation from aggregated data into findings or policies preserves the underlying IDs so analysts can track how aggregated data influences policy verdicts without altering the data itself. +- **CLI parity.** Inline callouts copy the equivalent `stella` CLI commands, ensuring console users can recreate the exact aggregation query offline. + +--- + +## 6 · Performance & Telemetry Anchors + +- Initial boot target: **< 2.5 s** `LargestContentfulPaint` on 4 vCPU air-gapped runner with cached assets. +- Route budget: each feature shell must keep first interaction (hydrated data + filters) under **1.5 s** once tokens resolve. +- Telemetry: console emits metrics via the `/console/telemetry` batch endpoint—`ui_route_render_seconds`, `ui_filter_apply_total`, `ui_tenant_switch_total`, `ui_offline_banner_seconds`. Logs carry correlation IDs matching backend responses for unified tracing. +- Lighthouse CI runs in the console pipeline (see `DEVOPS-CONSOLE-23-001`) and asserts budgets above; failing runs gate releases. + +--- + +## 7 · References + +- `/docs/architecture/console.md` – component-level diagrams (pending Sprint 23 task). +- `/docs/ui/navigation.md` – detailed routes, breadcrumbs, keyboard shortcuts. +- `/docs/ui/downloads.md` – downloads manifest, parity workflows, offline guidance. +- `/docs/ui/sbom-explorer.md` – SBOM-specific flows and overlays. +- `/docs/ui/advisories-and-vex.md` – aggregation UX details. +- `/docs/ui/findings.md` – explain drawer and filter matrix. +- `/docs/security/console-security.md` – OIDC, scopes, CSP, evidence handling. +- `/docs/cli-vs-ui-parity.md` – CLI equivalents and regression automation. + +--- + +## 8 · Compliance Checklist + +- [ ] Tenant picker enforces Authority-issued scopes and logs `ui.tenant.switch`. +- [ ] Global filters update URLs/query tokens for deterministic deep links. +- [ ] Aggregation views show provenance badges and merge hash indicators. +- [ ] CLI parity callouts aligned with `stella` commands for equivalent queries. +- [ ] Offline banner tested with Offline Kit snapshot import and documented staleness thresholds. +- [ ] Accessibility audit covers global filter tray, tenant picker, and keyboard shortcuts (WCAG 2.2 AA). +- [ ] Telemetry and Lighthouse budgets tracked in console CI (`DEVOPS-CONSOLE-23-001`). + +--- + +*Last updated: 2025-10-26 (Sprint 23).* diff --git a/docs/ui/downloads.md b/docs/ui/downloads.md index f741c56a..f7cd1c4e 100644 --- a/docs/ui/downloads.md +++ b/docs/ui/downloads.md @@ -1,212 +1,212 @@ -# StellaOps Console - Downloads Manager - -> **Audience:** DevOps guild, Console engineers, enablement writers, and operators who promote releases or maintain offline mirrors. -> **Scope:** `/console/downloads` workspace covering artifact catalog, signed manifest plumbing, export status handling, CLI parity, automation hooks, and offline guidance (Sprint 23). - -The Downloads workspace centralises every artefact required to deploy or validate StellaOps in connected and air-gapped environments. It keeps Console operators aligned with release engineering by surfacing the signed downloads manifest, live export jobs, parity checks against Offline Kit bundles, and automation hooks that mirror the CLI experience. - ---- - -## 1 - Access and prerequisites - -- **Route:** `/console/downloads` (list) with detail drawer `/console/downloads/:artifactId`. -- **Scopes:** `downloads.read` (baseline) and `downloads.manage` for cancelling or expiring stale exports. Evidence bundles inherit the originating scope (`runs.read`, `findings.read`, etc.). -- **Dependencies:** Web gateway `/console/downloads` API (WEB-CONSOLE-23-005), DevOps manifest pipeline (`deploy/downloads/manifest.json`), Offline Kit metadata (`manifest/offline-manifest.json`), and export orchestrator `/console/exports`. -- **Feature flags:** `downloads.workspace.enabled`, `downloads.exportQueue`, `downloads.offlineParity`. -- **Tenancy:** Artefacts are tenant-agnostic except evidence bundles, which are tagged with originating tenant and require matching Authority scopes. - ---- - -## 2 - Workspace layout - -``` -+---------------------------------------------------------------+ -| Header: Snapshot timestamp - Manifest signature status | -+---------------------------------------------------------------+ -| Cards: Latest release - Offline kit parity - Export queue | -+---------------------------------------------------------------+ -| Tabs: Artefacts | Exports | Offline Kits | Webhooks | -+---------------------------------------------------------------+ -| Filter bar: Channel - Kind - Architecture - Scope tags | -+---------------------------------------------------------------+ -| Table (virtualised): Artifact | Channel | Digest | Status | -| Detail drawer: Metadata | Commands | Provenance | History | -+---------------------------------------------------------------+ -``` - -- **Snapshot banner:** shows `manifest.version`, `generatedAt`, and cosign verification state. If verification fails, the banner turns red and links to troubleshooting guidance. -- **Quick actions:** Copy manifest URL, download attestation bundle, trigger parity check, open CLI parity doc (`/docs/cli-vs-ui-parity.md`). -- **Filters:** allow narrowing by channel (`edge`, `stable`, `airgap`), artefact kind (`container.image`, `helm.chart`, `compose.bundle`, `offline.bundle`, `export.bundle`), architecture (`linux/amd64`, `linux/arm64`), and scope tags (`console`, `scheduler`, `authority`). - ---- - -## 3 - Artefact catalogue - -| Category | Artefacts surfaced | Source | Notes | -|----------|-------------------|--------|-------| -| **Core containers** | `stellaops/web-ui`, `stellaops/web`, `stellaops/concelier`, `stellaops/excititor`, `stellaops/scanner-*`, `stellaops/authority`, `stellaops/attestor`, `stellaops/scheduler-*` | `deploy/downloads/manifest.json` (`artifacts[].kind = "container.image"`) | Digest-only pulls with copy-to-clipboard `docker pull` and `oras copy` commands; badges show arch availability. | -| **Helm charts** | `deploy/helm/stellaops-*.tgz` plus values files | Manifest entries where `kind = "helm.chart"` | Commands reference `helm repo add` (online) and `helm install --values` (offline). UI links to values matrix in `/docs/install/helm-prod.md` when available. | -| **Compose bundles** | `deploy/compose/docker-compose.*.yaml`, `.env` seeds | `kind = "compose.bundle"` | Inline diff viewer highlights digest changes vs previous snapshot; `docker compose pull` command copies digest pins. | -| **Offline kit** | `stella-ops-offline-kit--.tar.gz` + signatures and manifest | Offline Kit metadata (`manifest/offline-manifest.json`) merged into downloads view | Drawer shows bundle size, signed manifest digest, cosign verification command (mirrors `/docs/24_OFFLINE_KIT.md`). | -| **Evidence exports** | Completed jobs from `/console/exports` (findings delta, policy explain, run evidence) | Export orchestrator job queue | Entries expire after retention window; UI exposes `stella runs export` and `stella findings export` parity buttons. | -| **Webhooks & parity** | `/downloads/hooks/subscribe` configs, CI parity reports | Manifest extras (`kind = "webhook.config"`, `kind = "parity.report"`) | Operators can download webhook payload templates and review the latest CLI parity check report generated by docs CI. | - ---- - -## 4 - Manifest structure - -The DevOps pipeline publishes a deterministic manifest at `deploy/downloads/manifest.json`, signed with the release Cosign key (`DOWNLOADS-CONSOLE-23-001`). The Console fetches it on workspace load and caches it with `If-None-Match` headers to avoid redundant pulls. The manifest schema: - -- **`version`** - monotonically increasing integer tied to pipeline run. -- **`generatedAt`** - ISO-8601 UTC timestamp. -- **`signature`** - URL to detached Cosign signature (`manifest.json.sig`). -- **`artifacts[]`** - ordered list keyed by `id`. - -Each artefact contains: - -| Field | Description | -|-------|-------------| -| `id` | Stable identifier (`::`). | -| `kind` | One of `container.image`, `helm.chart`, `compose.bundle`, `offline.bundle`, `export.bundle`, `webhook.config`, `parity.report`. | -| `channel` | `edge`, `stable`, or `airgap`. | -| `version` | Semantic or calendar version (for containers, matches release manifest). | -| `architectures` | Array of supported platforms (empty for arch-agnostic artefacts). | -| `digest` | SHA-256 for immutable artefacts; Compose bundles include file hash. | -| `sizeBytes` | File size (optional for export bundles that stream). | -| `downloadUrl` | HTTPS endpoint (registry, object store, or mirror). | -| `signatureUrl` | Detached signature (Cosign, DSSE, or attestation) if available. | -| `sbomUrl` | Optional SBOM pointer (CycloneDX JSON). | -| `attestationUrl` | Optional in-toto/SLSA attestation. | -| `docs` | Array of documentation links (e.g., `/docs/install/docker.md`). | -| `tags` | Free-form tags (e.g., `["console","ui","offline"]`). | - -### 4.1 Example excerpt - -```json -{ - "version": 42, - "generatedAt": "2025-10-27T04:00:00Z", - "signature": "https://downloads.stella-ops.org/manifest/manifest.json.sig", - "artifacts": [ - { - "id": "container.image:web-ui:2025.10.0-edge", - "kind": "container.image", - "channel": "edge", - "version": "2025.10.0-edge", - "architectures": ["linux/amd64", "linux/arm64"], - "digest": "sha256:38b225fa7767a5b94ebae4dae8696044126aac429415e93de514d5dd95748dcf", - "sizeBytes": 187563210, - "downloadUrl": "https://registry.stella-ops.org/v2/stellaops/web-ui/manifests/sha256:38b225fa7767a5b94ebae4dae8696044126aac429415e93de514d5dd95748dcf", - "signatureUrl": "https://downloads.stella-ops.org/signatures/web-ui-2025.10.0-edge.cosign.sig", - "sbomUrl": "https://downloads.stella-ops.org/sbom/web-ui-2025.10.0-edge.cdx.json", - "attestationUrl": "https://downloads.stella-ops.org/attestations/web-ui-2025.10.0-edge.intoto.jsonl", - "docs": ["/docs/install/docker.md", "/docs/security/console-security.md"], - "tags": ["console", "ui"] - }, - { - "id": "offline.bundle:ouk:2025.10.0-edge", - "kind": "offline.bundle", - "channel": "edge", - "version": "2025.10.0-edge", - "digest": "sha256:4f7d2f7a8d0cf4b5f3af689f6c74cd213f4c1b3a1d76d24f6f9f3d9075e51f90", - "downloadUrl": "https://downloads.stella-ops.org/offline/stella-ops-offline-kit-2025.10.0-edge.tar.gz", - "signatureUrl": "https://downloads.stella-ops.org/offline/stella-ops-offline-kit-2025.10.0-edge.tar.gz.sig", - "sbomUrl": "https://downloads.stella-ops.org/offline/offline-manifest-2025.10.0-edge.json", - "docs": ["/docs/24_OFFLINE_KIT.md"], - "tags": ["offline", "airgap"] - } - ] -} -``` - -Console caches the manifest hash and surfaces differences when a new version lands, helping operators confirm digests drift only when expected. - ---- - -## 5 - Download workflows and statuses - -| Status | Applies to | Behaviour | -|--------|------------|-----------| -| **Ready** | Immutable artefacts (images, Helm/Compose bundles, offline kit) | Commands available immediately. Digest, size, and last verification timestamp display in the table. | -| **Pending export** | Async exports queued via `/console/exports` | Shows job owner, scope, and estimated completion time. UI polls every 15 s and updates progress bar. | -| **Processing** | Long-running export (evidence bundle, large SBOM) | Drawer shows current stage (`collecting`, `compressing`, `signing`). Operators can cancel if they own the request and hold `downloads.manage`. | -| **Delivered** | Completed export within retention window | Provides download links, resume token, and parity snippet for CLI. | -| **Expired** | Export past retention or manually expired | Row grays out; clicking opens housekeeping guidance with CLI command to regenerate (`stella runs export --run `). | - -Exports inherit retention defaults defined in policy (`downloads.retentionDays`, min 3, max 30). Operators can override per tenant if they have the appropriate scope. - ---- - -## 6 - CLI parity and copy-to-clipboard - -- **Digest pulls:** Each container entry exposes `docker pull @` and `oras copy @ --to-dir ./downloads` buttons. Commands include architecture hints for multi-platform images. -- **Helm/Compose:** Buttons output `helm pull` / `helm install` with the manifest URL and `docker compose --env-file` commands referencing the downloaded bundle. -- **Offline kit:** Copy buttons produce the full verification sequence: - -```bash -curl -LO https://downloads.stella-ops.org/offline/stella-ops-offline-kit-2025.10.0-edge.tar.gz -curl -LO https://downloads.stella-ops.org/offline/stella-ops-offline-kit-2025.10.0-edge.tar.gz.sig -cosign verify-blob \ - --key https://stella-ops.org/keys/cosign.pub \ - --signature stella-ops-offline-kit-2025.10.0-edge.tar.gz.sig \ - stella-ops-offline-kit-2025.10.0-edge.tar.gz -``` - -- **Exports:** Drawer lists CLI equivalents (for example, `stella findings export --run `). When the CLI supports resume tokens, the command includes `--resume-token` from the manifest entry. -- **Automation:** Webhook tab copies `curl` snippets to subscribe to `/downloads/hooks/subscribe?topic=` and includes payload schema for integration tests. - -Parity buttons write commands to the clipboard and display a toast confirming scope hints (for example, `Requires downloads.read + tenant scope`). Accessibility shortcuts (`Shift+D`) trigger the primary copy action for keyboard users. - ---- - -## 7 - Offline and air-gap workflow - -- **Manifest sync:** Offline users download `manifest/offline-manifest.json` plus detached JWS and import it via `stella offline kit import`. Console highlights if the offline manifest predates the online manifest by more than 7 days. -- **Artefact staging:** The workspace enumerates removable media instructions (export to `./staging//`) and warns when artefacts exceed configured media size thresholds. -- **Mirrors:** Buttons copy `oras copy` commands that mirror images to an internal registry (`registry..internal`). Operators can toggle `--insecure-policy` if the destination uses custom trust roots. -- **Parity checks:** `downloads.offlineParity` flag surfaces the latest parity report verifying that Offline Kit contents match the downloads manifest digests. If diff detected, UI raises a banner linking to remediation steps. -- **Audit logging:** Every download command triggered from the UI emits `ui.download.commandCopied` with artifact ID, digest, and tenant. Logs feed the evidence locker so air-gap imports can demonstrate provenance. - ---- - -## 8 - Observability and quotas - -| Signal | Source | Description | -|--------|--------|-------------| -| `ui_download_manifest_refresh_seconds` | Console metrics | Measures time to fetch and verify manifest. Targets < 3 s. | -| `ui_download_export_queue_depth` | `/console/downloads` API | Number of pending exports (per tenant). Surfaces as card and Grafana panel. | -| `ui_download_command_copied_total` | Console logs | Count of copy actions by artifact type, used to gauge CLI parity adoption. | -| `downloads.export.duration` | Export orchestrator | Duration histograms for bundle generation; alerts if P95 > 60 s. | -| `downloads.quota.remaining` | Authority quota service | Anonymous users limited to 33 exports/day, verified users 333/day. Banner turns amber at 90 % usage as per platform policy. | - -Telemetry entries include correlation IDs that match backend manifest refresh logs and export job records to keep troubleshooting deterministic. - ---- - -## 9 - References - -- `/docs/ui/console-overview.md` - primary shell, tenant controls, SSE ticker. -- `/docs/ui/navigation.md` - route ownership and keyboard shortcuts. -- `/docs/ui/sbom-explorer.md` - export flows feeding the downloads queue. -- `/docs/ui/runs.md` - evidence bundle integration. -- `/docs/24_OFFLINE_KIT.md` - offline kit packaging and verification. -- `/docs/security/console-security.md` - scopes, CSP, and download token handling. -- `/docs/cli-vs-ui-parity.md` - CLI equivalence checks (pending). -- `deploy/releases/*.yaml` - source of container digests mirrored into the manifest. - ---- - -## 10 - Compliance checklist - -- [ ] Manifest schema documented (fields, signature, caching) and sample kept current. -- [ ] Artefact categories mapped to manifest entries and parity workflows. -- [ ] Download statuses, retention, and cancellation rules explained. -- [ ] CLI copy-to-clipboard commands mirror console actions with scope hints. -- [ ] Offline/air-gap parity workflow, mirror commands, and audit logging captured. -- [ ] Observability metrics and quota signalling documented. -- [ ] References cross-linked to adjacent docs (navigation, exports, offline kit). -- [ ] Accessibility shortcuts and copy-to-clipboard behaviour noted with compliance reminder. - ---- - -*Last updated: 2025-10-27 (Sprint 23).* +# StellaOps Console - Downloads Manager + +> **Audience:** DevOps guild, Console engineers, enablement writers, and operators who promote releases or maintain offline mirrors. +> **Scope:** `/console/downloads` workspace covering artifact catalog, signed manifest plumbing, export status handling, CLI parity, automation hooks, and offline guidance (Sprint 23). + +The Downloads workspace centralises every artefact required to deploy or validate StellaOps in connected and air-gapped environments. It keeps Console operators aligned with release engineering by surfacing the signed downloads manifest, live export jobs, parity checks against Offline Kit bundles, and automation hooks that mirror the CLI experience. + +--- + +## 1 - Access and prerequisites + +- **Route:** `/console/downloads` (list) with detail drawer `/console/downloads/:artifactId`. +- **Scopes:** `downloads.read` (baseline) and `downloads.manage` for cancelling or expiring stale exports. Evidence bundles inherit the originating scope (`runs.read`, `findings.read`, etc.). +- **Dependencies:** Web gateway `/console/downloads` API (WEB-CONSOLE-23-005), DevOps manifest pipeline (`deploy/downloads/manifest.json`), Offline Kit metadata (`manifest/offline-manifest.json`), and export orchestrator `/console/exports`. +- **Feature flags:** `downloads.workspace.enabled`, `downloads.exportQueue`, `downloads.offlineParity`. +- **Tenancy:** Artefacts are tenant-agnostic except evidence bundles, which are tagged with originating tenant and require matching Authority scopes. + +--- + +## 2 - Workspace layout + +``` ++---------------------------------------------------------------+ +| Header: Snapshot timestamp - Manifest signature status | ++---------------------------------------------------------------+ +| Cards: Latest release - Offline kit parity - Export queue | ++---------------------------------------------------------------+ +| Tabs: Artefacts | Exports | Offline Kits | Webhooks | ++---------------------------------------------------------------+ +| Filter bar: Channel - Kind - Architecture - Scope tags | ++---------------------------------------------------------------+ +| Table (virtualised): Artifact | Channel | Digest | Status | +| Detail drawer: Metadata | Commands | Provenance | History | ++---------------------------------------------------------------+ +``` + +- **Snapshot banner:** shows `manifest.version`, `generatedAt`, and cosign verification state. If verification fails, the banner turns red and links to troubleshooting guidance. +- **Quick actions:** Copy manifest URL, download attestation bundle, trigger parity check, open CLI parity doc (`/docs/cli-vs-ui-parity.md`). +- **Filters:** allow narrowing by channel (`edge`, `stable`, `airgap`), artefact kind (`container.image`, `helm.chart`, `compose.bundle`, `offline.bundle`, `export.bundle`), architecture (`linux/amd64`, `linux/arm64`), and scope tags (`console`, `scheduler`, `authority`). + +--- + +## 3 - Artefact catalogue + +| Category | Artefacts surfaced | Source | Notes | +|----------|-------------------|--------|-------| +| **Core containers** | `stellaops/web-ui`, `stellaops/web`, `stellaops/concelier`, `stellaops/excititor`, `stellaops/scanner-*`, `stellaops/authority`, `stellaops/attestor`, `stellaops/scheduler-*` | `deploy/downloads/manifest.json` (`artifacts[].kind = "container.image"`) | Digest-only pulls with copy-to-clipboard `docker pull` and `oras copy` commands; badges show arch availability. | +| **Helm charts** | `deploy/helm/stellaops-*.tgz` plus values files | Manifest entries where `kind = "helm.chart"` | Commands reference `helm repo add` (online) and `helm install --values` (offline). UI links to values matrix in `/docs/install/helm-prod.md` when available. | +| **Compose bundles** | `deploy/compose/docker-compose.*.yaml`, `.env` seeds | `kind = "compose.bundle"` | Inline diff viewer highlights digest changes vs previous snapshot; `docker compose pull` command copies digest pins. | +| **Offline kit** | `stella-ops-offline-kit--.tar.gz` + signatures and manifest | Offline Kit metadata (`manifest/offline-manifest.json`) merged into downloads view | Drawer shows bundle size, signed manifest digest, cosign verification command (mirrors `/docs/24_OFFLINE_KIT.md`). | +| **Evidence exports** | Completed jobs from `/console/exports` (findings delta, policy explain, run evidence) | Export orchestrator job queue | Entries expire after retention window; UI exposes `stella runs export` and `stella findings export` parity buttons. | +| **Webhooks & parity** | `/downloads/hooks/subscribe` configs, CI parity reports | Manifest extras (`kind = "webhook.config"`, `kind = "parity.report"`) | Operators can download webhook payload templates and review the latest CLI parity check report generated by docs CI. | + +--- + +## 4 - Manifest structure + +The DevOps pipeline publishes a deterministic manifest at `deploy/downloads/manifest.json`, signed with the release Cosign key (`DOWNLOADS-CONSOLE-23-001`). The Console fetches it on workspace load and caches it with `If-None-Match` headers to avoid redundant pulls. The manifest schema: + +- **`version`** - monotonically increasing integer tied to pipeline run. +- **`generatedAt`** - ISO-8601 UTC timestamp. +- **`signature`** - URL to detached Cosign signature (`manifest.json.sig`). +- **`artifacts[]`** - ordered list keyed by `id`. + +Each artefact contains: + +| Field | Description | +|-------|-------------| +| `id` | Stable identifier (`::`). | +| `kind` | One of `container.image`, `helm.chart`, `compose.bundle`, `offline.bundle`, `export.bundle`, `webhook.config`, `parity.report`. | +| `channel` | `edge`, `stable`, or `airgap`. | +| `version` | Semantic or calendar version (for containers, matches release manifest). | +| `architectures` | Array of supported platforms (empty for arch-agnostic artefacts). | +| `digest` | SHA-256 for immutable artefacts; Compose bundles include file hash. | +| `sizeBytes` | File size (optional for export bundles that stream). | +| `downloadUrl` | HTTPS endpoint (registry, object store, or mirror). | +| `signatureUrl` | Detached signature (Cosign, DSSE, or attestation) if available. | +| `sbomUrl` | Optional SBOM pointer (CycloneDX JSON). | +| `attestationUrl` | Optional in-toto/SLSA attestation. | +| `docs` | Array of documentation links (e.g., `/docs/install/docker.md`). | +| `tags` | Free-form tags (e.g., `["console","ui","offline"]`). | + +### 4.1 Example excerpt + +```json +{ + "version": 42, + "generatedAt": "2025-10-27T04:00:00Z", + "signature": "https://downloads.stella-ops.org/manifest/manifest.json.sig", + "artifacts": [ + { + "id": "container.image:web-ui:2025.10.0-edge", + "kind": "container.image", + "channel": "edge", + "version": "2025.10.0-edge", + "architectures": ["linux/amd64", "linux/arm64"], + "digest": "sha256:38b225fa7767a5b94ebae4dae8696044126aac429415e93de514d5dd95748dcf", + "sizeBytes": 187563210, + "downloadUrl": "https://registry.stella-ops.org/v2/stellaops/web-ui/manifests/sha256:38b225fa7767a5b94ebae4dae8696044126aac429415e93de514d5dd95748dcf", + "signatureUrl": "https://downloads.stella-ops.org/signatures/web-ui-2025.10.0-edge.cosign.sig", + "sbomUrl": "https://downloads.stella-ops.org/sbom/web-ui-2025.10.0-edge.cdx.json", + "attestationUrl": "https://downloads.stella-ops.org/attestations/web-ui-2025.10.0-edge.intoto.jsonl", + "docs": ["/docs/install/docker.md", "/docs/security/console-security.md"], + "tags": ["console", "ui"] + }, + { + "id": "offline.bundle:ouk:2025.10.0-edge", + "kind": "offline.bundle", + "channel": "edge", + "version": "2025.10.0-edge", + "digest": "sha256:4f7d2f7a8d0cf4b5f3af689f6c74cd213f4c1b3a1d76d24f6f9f3d9075e51f90", + "downloadUrl": "https://downloads.stella-ops.org/offline/stella-ops-offline-kit-2025.10.0-edge.tar.gz", + "signatureUrl": "https://downloads.stella-ops.org/offline/stella-ops-offline-kit-2025.10.0-edge.tar.gz.sig", + "sbomUrl": "https://downloads.stella-ops.org/offline/offline-manifest-2025.10.0-edge.json", + "docs": ["/docs/24_OFFLINE_KIT.md"], + "tags": ["offline", "airgap"] + } + ] +} +``` + +Console caches the manifest hash and surfaces differences when a new version lands, helping operators confirm digests drift only when expected. + +--- + +## 5 - Download workflows and statuses + +| Status | Applies to | Behaviour | +|--------|------------|-----------| +| **Ready** | Immutable artefacts (images, Helm/Compose bundles, offline kit) | Commands available immediately. Digest, size, and last verification timestamp display in the table. | +| **Pending export** | Async exports queued via `/console/exports` | Shows job owner, scope, and estimated completion time. UI polls every 15 s and updates progress bar. | +| **Processing** | Long-running export (evidence bundle, large SBOM) | Drawer shows current stage (`collecting`, `compressing`, `signing`). Operators can cancel if they own the request and hold `downloads.manage`. | +| **Delivered** | Completed export within retention window | Provides download links, resume token, and parity snippet for CLI. | +| **Expired** | Export past retention or manually expired | Row grays out; clicking opens housekeeping guidance with CLI command to regenerate (`stella runs export --run `). | + +Exports inherit retention defaults defined in policy (`downloads.retentionDays`, min 3, max 30). Operators can override per tenant if they have the appropriate scope. + +--- + +## 6 - CLI parity and copy-to-clipboard + +- **Digest pulls:** Each container entry exposes `docker pull @` and `oras copy @ --to-dir ./downloads` buttons. Commands include architecture hints for multi-platform images. +- **Helm/Compose:** Buttons output `helm pull` / `helm install` with the manifest URL and `docker compose --env-file` commands referencing the downloaded bundle. +- **Offline kit:** Copy buttons produce the full verification sequence: + +```bash +curl -LO https://downloads.stella-ops.org/offline/stella-ops-offline-kit-2025.10.0-edge.tar.gz +curl -LO https://downloads.stella-ops.org/offline/stella-ops-offline-kit-2025.10.0-edge.tar.gz.sig +cosign verify-blob \ + --key https://stella-ops.org/keys/cosign.pub \ + --signature stella-ops-offline-kit-2025.10.0-edge.tar.gz.sig \ + stella-ops-offline-kit-2025.10.0-edge.tar.gz +``` + +- **Exports:** Drawer lists CLI equivalents (for example, `stella findings export --run `). When the CLI supports resume tokens, the command includes `--resume-token` from the manifest entry. +- **Automation:** Webhook tab copies `curl` snippets to subscribe to `/downloads/hooks/subscribe?topic=` and includes payload schema for integration tests. + +Parity buttons write commands to the clipboard and display a toast confirming scope hints (for example, `Requires downloads.read + tenant scope`). Accessibility shortcuts (`Shift+D`) trigger the primary copy action for keyboard users. + +--- + +## 7 - Offline and air-gap workflow + +- **Manifest sync:** Offline users download `manifest/offline-manifest.json` plus detached JWS and import it via `stella offline kit import`. Console highlights if the offline manifest predates the online manifest by more than 7 days. +- **Artefact staging:** The workspace enumerates removable media instructions (export to `./staging//`) and warns when artefacts exceed configured media size thresholds. +- **Mirrors:** Buttons copy `oras copy` commands that mirror images to an internal registry (`registry..internal`). Operators can toggle `--insecure-policy` if the destination uses custom trust roots. +- **Parity checks:** `downloads.offlineParity` flag surfaces the latest parity report verifying that Offline Kit contents match the downloads manifest digests. If diff detected, UI raises a banner linking to remediation steps. +- **Audit logging:** Every download command triggered from the UI emits `ui.download.commandCopied` with artifact ID, digest, and tenant. Logs feed the evidence locker so air-gap imports can demonstrate provenance. + +--- + +## 8 - Observability and quotas + +| Signal | Source | Description | +|--------|--------|-------------| +| `ui_download_manifest_refresh_seconds` | Console metrics | Measures time to fetch and verify manifest. Targets < 3 s. | +| `ui_download_export_queue_depth` | `/console/downloads` API | Number of pending exports (per tenant). Surfaces as card and Grafana panel. | +| `ui_download_command_copied_total` | Console logs | Count of copy actions by artifact type, used to gauge CLI parity adoption. | +| `downloads.export.duration` | Export orchestrator | Duration histograms for bundle generation; alerts if P95 > 60 s. | +| `downloads.quota.remaining` | Authority quota service | Anonymous users limited to 33 exports/day, verified users 333/day. Banner turns amber at 90 % usage as per platform policy. | + +Telemetry entries include correlation IDs that match backend manifest refresh logs and export job records to keep troubleshooting deterministic. + +--- + +## 9 - References + +- `/docs/ui/console-overview.md` - primary shell, tenant controls, SSE ticker. +- `/docs/ui/navigation.md` - route ownership and keyboard shortcuts. +- `/docs/ui/sbom-explorer.md` - export flows feeding the downloads queue. +- `/docs/ui/runs.md` - evidence bundle integration. +- `/docs/24_OFFLINE_KIT.md` - offline kit packaging and verification. +- `/docs/security/console-security.md` - scopes, CSP, and download token handling. +- `/docs/cli-vs-ui-parity.md` - CLI equivalence checks (pending). +- `deploy/releases/*.yaml` - source of container digests mirrored into the manifest. + +--- + +## 10 - Compliance checklist + +- [ ] Manifest schema documented (fields, signature, caching) and sample kept current. +- [ ] Artefact categories mapped to manifest entries and parity workflows. +- [ ] Download statuses, retention, and cancellation rules explained. +- [ ] CLI copy-to-clipboard commands mirror console actions with scope hints. +- [ ] Offline/air-gap parity workflow, mirror commands, and audit logging captured. +- [ ] Observability metrics and quota signalling documented. +- [ ] References cross-linked to adjacent docs (navigation, exports, offline kit). +- [ ] Accessibility shortcuts and copy-to-clipboard behaviour noted with compliance reminder. + +--- + +*Last updated: 2025-10-27 (Sprint 23).* diff --git a/docs/ui/findings.md b/docs/ui/findings.md index 5dd6c22d..4fb7dab9 100644 --- a/docs/ui/findings.md +++ b/docs/ui/findings.md @@ -143,9 +143,9 @@ Explain drawer includes copy-to-clipboard buttons for rule chain and evidence JS ## 10. Screenshot coordination -- Placeholders: - - `![Findings grid placeholder](../assets/ui/findings/grid-placeholder.png)` - - `![Explain drawer placeholder](../assets/ui/findings/explain-placeholder.png)` +- Placeholders (captures pending upload): + - `docs/assets/ui/findings/grid-placeholder.png` + - `docs/assets/ui/findings/explain-placeholder.png` - Coordinate with Console Guild (Slack `#console-screenshots`, entry 2025-10-26) to capture updated light and dark theme shots before release. --- @@ -176,4 +176,3 @@ Explain drawer includes copy-to-clipboard buttons for rule chain and evidence JS --- *Last updated: 2025-10-26 (Sprint 23).* - diff --git a/docs/ui/navigation.md b/docs/ui/navigation.md index 6bd6b473..066558ab 100644 --- a/docs/ui/navigation.md +++ b/docs/ui/navigation.md @@ -1,163 +1,163 @@ -# StellaOps Console - Navigation - -> **Audience:** Console UX writers, UI engineers, QA, and enablement teams. -> **Scope:** Primary route map, layout conventions, keyboard shortcuts, deep-link patterns, and tenant context switching for the StellaOps Console (Sprint 23). - -The navigation framework keeps Console workflows predictable across tenants and deployment modes. This guide explains how the global shell, feature routes, and context tokens cooperate so operators can jump between findings, SBOMs, advisories, policies, and runs without losing scope. - ---- - -## 1. Information Architecture - -### 1.1 Primary routes - -| Route pattern | Module owner | Purpose | Required scopes (minimum) | Core services | -|---------------|--------------|---------|---------------------------|---------------| -| `/console/dashboard` | Web gateway | Landing KPIs, feed age, queue depth, alerts | `ui.read` | Web, Scheduler WebService, Concelier WebService, Excititor WebService | -| `/console/findings` | Policy Engine | Aggregated findings, explain drawer, export | `findings.read` | Policy Engine, Concelier WebService, SBOM Service | -| `/console/sbom` | SBOM Service | Catalog view, component graph, overlays | `sbom.read` | SBOM Service, Policy Engine (overlays) | -| `/console/advisories` | Concelier | Advisory aggregation with provenance banners | `advisory.read` | Concelier WebService | -| `/console/vex` | Excititor | VEX aggregation, consensus, conflicts | `vex.read` | Excititor WebService | -| `/console/runs` | Scheduler | Run list, live progress, evidence downloads | `runs.read` | Scheduler WebService, Policy Engine, Scanner WebService | -| `/console/policies` | Policy Engine | Editor, simulations, approvals | `policy.read` (read) / `policy.write` (edit) | Policy Engine, Authority | -| `/console/downloads` | DevOps | Signed artifacts, Offline Kit parity checklist | `downloads.read` | DevOps manifest API, Offline Kit | -| `/console/admin` | Authority | Tenants, roles, tokens, integrations | `ui.admin` (plus scoped `authority:*`) | Authority | -| `/console/help` | Docs Guild | Guides, tours, release notes | `ui.read` | Docs static assets | - -### 1.2 Secondary navigation elements - -- **Left rail:** highlights the active top-level route, exposes quick metrics, and shows pinned saved views. Keyboard focus cycles through rail entries with `Tab`/`Shift+Tab`. -- **Breadcrumb bar:** renders `Home / Module / Detail` format. Detail crumbs include IDs and titles for shareable context (for example, `Findings / High Severity / CVE-2025-1234`). -- **Action shelf:** right-aligned controls for context actions (export, verify, retry). Buttons disable automatically if the current subject lacks the requisite scope. - ---- - -## 2. Command Palette and Search - -- **Trigger:** `Ctrl/Cmd + K`. Palette opens in place, keeps focus, and announces results via ARIA live region. -- **Capabilities:** jump to routes, saved views, tenants, recent entities (findings, SBOMs, advisories), and command actions (for example, "Start verification", "Open explain drawer"). -- **Result tokens:** palette entries carry metadata (`type`, `tenant`, `filters`). Selecting an item updates the URL and applies stored filters without a full reload. -- **Offline fallback:** in sealed/offline mode, palette restricts actions to cached routes and saved views; remote-only items show a grayed-out badge. - ---- - -## 3. Global Filters and Context Chips - -| Control | Shortcut | Persistence | Notes | -|---------|----------|-------------|-------| -| **Tenant picker** | `Ctrl/Cmd + T` | SessionStorage + URL `tenant` query | Issues fresh Authority token, invalidates caches, emits `ui.tenant.switch` log. | -| **Filter tray** | `Shift + F` | IndexedDB (per tenant) + URL query (`since`, `severity`, `tags`, `source`, `status`, `policyView`) | Applies instantly to compatible routes; incompatible filters show a reset suggestion. | -| **Component search** | `/` when filters closed | URL `component` query | Context-aware; scopes results to current tenant and module. | -| **Time window** | `Ctrl/Cmd + Shift + 1-4` | URL `since`/`until`, palette preset | Mapped to preset windows: 24 h, 7 d, 30 d, custom. | - -Context chips appear beneath page titles summarising active filters (for example, `Tenant: west-prod`, `Severity: Critical+High`, `Time: Last 7 days`). Removing a chip updates the tray and URL atomically. - ---- - -## 4. Keyboard Shortcut Matrix - -| Scope | Shortcut (Mac / Windows) | Action | Notes | -|-------|--------------------------|--------|-------| -| Global | `Cmd+K / Ctrl+K` | Open command palette | Accessible from any route except modal dialogs. | -| Global | `Cmd+T / Ctrl+T` | Open tenant switcher | Requires `ui.read`. Confirm selection with `Enter`; `Esc` cancels without switching. | -| Global | `Shift+F` | Toggle global filter tray | Focus lands on first filter control. | -| Global | `Cmd+1-9 / Ctrl+1-9` | Load saved view preset | Each preset bound per tenant; non-assigned keys show tooltip. | -| Global | `?` | Show keyboard reference overlay | Overlay lists context-specific shortcuts; closes with `Esc`. | -| Findings module | `Cmd+/ / Ctrl+/` | Focus explain search | Works when explain drawer is open. | -| SBOM module | `Cmd+G / Ctrl+G` | Toggle graph overlays | Persists per session. | -| Advisories & VEX | `Cmd+Opt+F / Ctrl+Alt+F` | Focus provider filter | Highlights provider chip strip. | -| Runs module | `Cmd+R / Ctrl+R` | Refresh SSE snapshot | Schedules soft refresh (no hard reload). | -| Policies module | `Cmd+S / Ctrl+S` | Save draft (if edit rights) | Mirrors Policy Editor behaviour. | - -Shortcut handling follows WCAG 2.2 best practices: all accelerators are remappable via Settings -> Accessibility -> Keyboard shortcuts, and the overlay documents platform differences. - ---- - -## 5. Deep-Link Patterns - -### 5.1 URL schema - -Console URLs adopt the format: - -``` -/console/[/:id][/:tab]?tenant=&since=&severity=&view=&panel=&component= -``` - -- **`tenant`** is mandatory and matches Authority slugs (e.g., `acme-prod`). -- **`since` / `until`** use ISO-8601 timestamps (UTC). Preset ranges set only `since`; UI computes `until` on load. -- **`severity`** accepts comma-separated policy buckets (e.g., `critical,high,kev`). -- **`view`** stores module-specific state (e.g., `sbomView=usage`, `findingsPreset=threat-hunting`). -- **`panel`** selects drawers or tabs (`panel=explain`, `panel=timeline`). - -### 5.2 Copyable links - -- Share links from the action shelf or context chips; both copy canonical URLs with all active filters. -- CLI parity: inline callouts provide `stella` commands derived from the URL parameters to ensure console/CLI equivalence. -- Offline note: links copied in sealed mode include the snapshot ID (`snapshot=`) so recipients know which offline data set to load. - -### 5.3 Examples - -- **`since` / `until`** use ISO-8601 timestamps (UTC). Preset ranges set only `since`; UI computes `until` on load. -- **`severity`** accepts comma-separated policy buckets (e.g., `critical,high,kev`). -- **`view`** stores module-specific state (e.g., `sbomView=usage`, `findingsPreset=threat-hunting`). -- **`panel`** selects drawers or tabs (`panel=explain`, `panel=timeline`). -- **`component`** encodes package selection using percent-encoded PURL syntax. -- **`snapshot`** appears when copying links offline to reference Offline Kit build hash. -@@ -| Use case | Example URL | Description | -|----------|-------------|-------------| -| Findings triage | `/console/findings?v=table&severity=critical,high&tenant=west-prod&since=2025-10-20T00:00:00Z` | Opens the findings table limited to critical/high for west-prod, last 7 days. | -| SBOM component focus | `/console/sbom/sha256:abcd?tenant=west-prod&component=pkg:npm/react@18.3.0&view=usage` | Deep-links to a specific image digest and highlights an NPM package in Usage view. | -| Advisory explain | `/console/advisories?tenant=west-prod&source=nvd&panel=detail&documentId=CVE-2025-1234` | Opens advisory list filtered to NVD and expands CVE detail drawer. | -| Run monitor | `/console/runs/42?tenant=west-prod&panel=progress` | Focuses run ID 42 with progress drawer active (SSE stream attached). | - ---- - -## 6. Tenant Switching Lifecycle - -1. **Initiate:** User triggers `Ctrl/Cmd + T` or clicks the tenant badge. Switcher modal lists authorised tenants and recent selections. -2. **Preview:** Selecting a tenant shows summary (environment, last snapshot, role coverage). The modal flags tenants missing required scopes for the current route. -3. **Confirm:** On confirmation, the UI requests a new DPoP-bound access token from Authority (`aud=console`, `tenant=`). -4. **Invalidate caches:** Stores keyed by tenant purge automatically; modules emit `tenantChanged` events so in-flight SSE streams reconnect with new headers. -5. **Restore state:** Global filters reapply where valid. Incompatible filters (for example, a saved view unavailable in the new tenant) prompt users to pick a fallback. -6. **Audit and telemetry:** `ui.tenant.switch` log writes subject, from/to tenant, correlation ID. Metric `ui_tenant_switch_total` increments for observability dashboards. -7. **Offline behaviour:** If the target tenant is absent from the offline snapshot, switcher displays guidance to import updated Offline Kit data before proceeding. - ---- - -## 7. Breadcrumbs, Tabs, and Focus Management - -- Breadcrumb titles update synchronously with route data loads. When fragments change (for example, selecting a finding), the breadcrumb text updates without pushing a new history entry to keep back/forward predictable. -- Detail views rely on accessible tabs (`role="tablist"`) with keyboard support (`ArrowLeft/Right`). Tab selection updates the URL `tab` parameter for deep linking. -- Focus management: - - Route changes send focus to the primary heading (`h1`) using the live region announcer. - - Opening drawers or modals traps focus until closed; ESC returns focus to the triggering element. - - Keyboard-only navigation is validated via automated Playwright accessibility checks as part of `DEVOPS-CONSOLE-23-001`. - ---- - -## 8. References - -- `/docs/ui/console-overview.md` - structural overview, tenant model, global filters. -- `/docs/ui/sbom-explorer.md` - SBOM-specific navigation and graphs (pending). -- `/docs/ui/advisories-and-vex.md` - aggregation UX details (pending). -- `/docs/ui/findings.md` - findings filters and explain drawer (pending). -- `/docs/security/console-security.md` - Authority, scopes, CSP. -- `/docs/cli-vs-ui-parity.md` - CLI equivalence matrix. -- `/docs/accessibility.md` - keyboard remapping, WCAG validation checklists. - ---- - -## 9. Compliance Checklist - -- [ ] Route table matches Console build (paths, scopes, owners verified with Console Guild). -- [ ] Keyboard shortcut matrix reflects implemented accelerators and accessibility overlay. -- [ ] Deep-link examples tested for copy/share parity and CLI alignment. -- [ ] Tenant switching flow documents cache invalidation and audit logging. -- [ ] Filter tray, command palette, and presets cross-referenced with accessibility guidance. -- [ ] Offline/air-gap notes included for palette, tenant switcher, and deep-link metadata. -- [ ] Links to dependent docs (`/docs/ui/*`, `/docs/security/*`) validated. - ---- - -*Last updated: 2025-10-26 (Sprint 23).* +# StellaOps Console - Navigation + +> **Audience:** Console UX writers, UI engineers, QA, and enablement teams. +> **Scope:** Primary route map, layout conventions, keyboard shortcuts, deep-link patterns, and tenant context switching for the StellaOps Console (Sprint 23). + +The navigation framework keeps Console workflows predictable across tenants and deployment modes. This guide explains how the global shell, feature routes, and context tokens cooperate so operators can jump between findings, SBOMs, advisories, policies, and runs without losing scope. + +--- + +## 1. Information Architecture + +### 1.1 Primary routes + +| Route pattern | Module owner | Purpose | Required scopes (minimum) | Core services | +|---------------|--------------|---------|---------------------------|---------------| +| `/console/dashboard` | Web gateway | Landing KPIs, feed age, queue depth, alerts | `ui.read` | Web, Scheduler WebService, Concelier WebService, Excititor WebService | +| `/console/findings` | Policy Engine | Aggregated findings, explain drawer, export | `findings.read` | Policy Engine, Concelier WebService, SBOM Service | +| `/console/sbom` | SBOM Service | Catalog view, component graph, overlays | `sbom.read` | SBOM Service, Policy Engine (overlays) | +| `/console/advisories` | Concelier | Advisory aggregation with provenance banners | `advisory.read` | Concelier WebService | +| `/console/vex` | Excititor | VEX aggregation, consensus, conflicts | `vex.read` | Excititor WebService | +| `/console/runs` | Scheduler | Run list, live progress, evidence downloads | `runs.read` | Scheduler WebService, Policy Engine, Scanner WebService | +| `/console/policies` | Policy Engine | Editor, simulations, approvals | `policy.read` (read) / `policy.write` (edit) | Policy Engine, Authority | +| `/console/downloads` | DevOps | Signed artifacts, Offline Kit parity checklist | `downloads.read` | DevOps manifest API, Offline Kit | +| `/console/admin` | Authority | Tenants, roles, tokens, integrations | `ui.admin` (plus scoped `authority:*`) | Authority | +| `/console/help` | Docs Guild | Guides, tours, release notes | `ui.read` | Docs static assets | + +### 1.2 Secondary navigation elements + +- **Left rail:** highlights the active top-level route, exposes quick metrics, and shows pinned saved views. Keyboard focus cycles through rail entries with `Tab`/`Shift+Tab`. +- **Breadcrumb bar:** renders `Home / Module / Detail` format. Detail crumbs include IDs and titles for shareable context (for example, `Findings / High Severity / CVE-2025-1234`). +- **Action shelf:** right-aligned controls for context actions (export, verify, retry). Buttons disable automatically if the current subject lacks the requisite scope. + +--- + +## 2. Command Palette and Search + +- **Trigger:** `Ctrl/Cmd + K`. Palette opens in place, keeps focus, and announces results via ARIA live region. +- **Capabilities:** jump to routes, saved views, tenants, recent entities (findings, SBOMs, advisories), and command actions (for example, "Start verification", "Open explain drawer"). +- **Result tokens:** palette entries carry metadata (`type`, `tenant`, `filters`). Selecting an item updates the URL and applies stored filters without a full reload. +- **Offline fallback:** in sealed/offline mode, palette restricts actions to cached routes and saved views; remote-only items show a grayed-out badge. + +--- + +## 3. Global Filters and Context Chips + +| Control | Shortcut | Persistence | Notes | +|---------|----------|-------------|-------| +| **Tenant picker** | `Ctrl/Cmd + T` | SessionStorage + URL `tenant` query | Issues fresh Authority token, invalidates caches, emits `ui.tenant.switch` log. | +| **Filter tray** | `Shift + F` | IndexedDB (per tenant) + URL query (`since`, `severity`, `tags`, `source`, `status`, `policyView`) | Applies instantly to compatible routes; incompatible filters show a reset suggestion. | +| **Component search** | `/` when filters closed | URL `component` query | Context-aware; scopes results to current tenant and module. | +| **Time window** | `Ctrl/Cmd + Shift + 1-4` | URL `since`/`until`, palette preset | Mapped to preset windows: 24 h, 7 d, 30 d, custom. | + +Context chips appear beneath page titles summarising active filters (for example, `Tenant: west-prod`, `Severity: Critical+High`, `Time: Last 7 days`). Removing a chip updates the tray and URL atomically. + +--- + +## 4. Keyboard Shortcut Matrix + +| Scope | Shortcut (Mac / Windows) | Action | Notes | +|-------|--------------------------|--------|-------| +| Global | `Cmd+K / Ctrl+K` | Open command palette | Accessible from any route except modal dialogs. | +| Global | `Cmd+T / Ctrl+T` | Open tenant switcher | Requires `ui.read`. Confirm selection with `Enter`; `Esc` cancels without switching. | +| Global | `Shift+F` | Toggle global filter tray | Focus lands on first filter control. | +| Global | `Cmd+1-9 / Ctrl+1-9` | Load saved view preset | Each preset bound per tenant; non-assigned keys show tooltip. | +| Global | `?` | Show keyboard reference overlay | Overlay lists context-specific shortcuts; closes with `Esc`. | +| Findings module | `Cmd+/ / Ctrl+/` | Focus explain search | Works when explain drawer is open. | +| SBOM module | `Cmd+G / Ctrl+G` | Toggle graph overlays | Persists per session. | +| Advisories & VEX | `Cmd+Opt+F / Ctrl+Alt+F` | Focus provider filter | Highlights provider chip strip. | +| Runs module | `Cmd+R / Ctrl+R` | Refresh SSE snapshot | Schedules soft refresh (no hard reload). | +| Policies module | `Cmd+S / Ctrl+S` | Save draft (if edit rights) | Mirrors Policy Editor behaviour. | + +Shortcut handling follows WCAG 2.2 best practices: all accelerators are remappable via Settings -> Accessibility -> Keyboard shortcuts, and the overlay documents platform differences. + +--- + +## 5. Deep-Link Patterns + +### 5.1 URL schema + +Console URLs adopt the format: + +``` +/console/[/:id][/:tab]?tenant=&since=&severity=&view=&panel=&component= +``` + +- **`tenant`** is mandatory and matches Authority slugs (e.g., `acme-prod`). +- **`since` / `until`** use ISO-8601 timestamps (UTC). Preset ranges set only `since`; UI computes `until` on load. +- **`severity`** accepts comma-separated policy buckets (e.g., `critical,high,kev`). +- **`view`** stores module-specific state (e.g., `sbomView=usage`, `findingsPreset=threat-hunting`). +- **`panel`** selects drawers or tabs (`panel=explain`, `panel=timeline`). + +### 5.2 Copyable links + +- Share links from the action shelf or context chips; both copy canonical URLs with all active filters. +- CLI parity: inline callouts provide `stella` commands derived from the URL parameters to ensure console/CLI equivalence. +- Offline note: links copied in sealed mode include the snapshot ID (`snapshot=`) so recipients know which offline data set to load. + +### 5.3 Examples + +- **`since` / `until`** use ISO-8601 timestamps (UTC). Preset ranges set only `since`; UI computes `until` on load. +- **`severity`** accepts comma-separated policy buckets (e.g., `critical,high,kev`). +- **`view`** stores module-specific state (e.g., `sbomView=usage`, `findingsPreset=threat-hunting`). +- **`panel`** selects drawers or tabs (`panel=explain`, `panel=timeline`). +- **`component`** encodes package selection using percent-encoded PURL syntax. +- **`snapshot`** appears when copying links offline to reference Offline Kit build hash. +@@ +| Use case | Example URL | Description | +|----------|-------------|-------------| +| Findings triage | `/console/findings?v=table&severity=critical,high&tenant=west-prod&since=2025-10-20T00:00:00Z` | Opens the findings table limited to critical/high for west-prod, last 7 days. | +| SBOM component focus | `/console/sbom/sha256:abcd?tenant=west-prod&component=pkg:npm/react@18.3.0&view=usage` | Deep-links to a specific image digest and highlights an NPM package in Usage view. | +| Advisory explain | `/console/advisories?tenant=west-prod&source=nvd&panel=detail&documentId=CVE-2025-1234` | Opens advisory list filtered to NVD and expands CVE detail drawer. | +| Run monitor | `/console/runs/42?tenant=west-prod&panel=progress` | Focuses run ID 42 with progress drawer active (SSE stream attached). | + +--- + +## 6. Tenant Switching Lifecycle + +1. **Initiate:** User triggers `Ctrl/Cmd + T` or clicks the tenant badge. Switcher modal lists authorised tenants and recent selections. +2. **Preview:** Selecting a tenant shows summary (environment, last snapshot, role coverage). The modal flags tenants missing required scopes for the current route. +3. **Confirm:** On confirmation, the UI requests a new DPoP-bound access token from Authority (`aud=console`, `tenant=`). +4. **Invalidate caches:** Stores keyed by tenant purge automatically; modules emit `tenantChanged` events so in-flight SSE streams reconnect with new headers. +5. **Restore state:** Global filters reapply where valid. Incompatible filters (for example, a saved view unavailable in the new tenant) prompt users to pick a fallback. +6. **Audit and telemetry:** `ui.tenant.switch` log writes subject, from/to tenant, correlation ID. Metric `ui_tenant_switch_total` increments for observability dashboards. +7. **Offline behaviour:** If the target tenant is absent from the offline snapshot, switcher displays guidance to import updated Offline Kit data before proceeding. + +--- + +## 7. Breadcrumbs, Tabs, and Focus Management + +- Breadcrumb titles update synchronously with route data loads. When fragments change (for example, selecting a finding), the breadcrumb text updates without pushing a new history entry to keep back/forward predictable. +- Detail views rely on accessible tabs (`role="tablist"`) with keyboard support (`ArrowLeft/Right`). Tab selection updates the URL `tab` parameter for deep linking. +- Focus management: + - Route changes send focus to the primary heading (`h1`) using the live region announcer. + - Opening drawers or modals traps focus until closed; ESC returns focus to the triggering element. + - Keyboard-only navigation is validated via automated Playwright accessibility checks as part of `DEVOPS-CONSOLE-23-001`. + +--- + +## 8. References + +- `/docs/ui/console-overview.md` - structural overview, tenant model, global filters. +- `/docs/ui/sbom-explorer.md` - SBOM-specific navigation and graphs (pending). +- `/docs/ui/advisories-and-vex.md` - aggregation UX details (pending). +- `/docs/ui/findings.md` - findings filters and explain drawer (pending). +- `/docs/security/console-security.md` - Authority, scopes, CSP. +- `/docs/cli-vs-ui-parity.md` - CLI equivalence matrix. +- `/docs/accessibility.md` - keyboard remapping, WCAG validation checklists. + +--- + +## 9. Compliance Checklist + +- [ ] Route table matches Console build (paths, scopes, owners verified with Console Guild). +- [ ] Keyboard shortcut matrix reflects implemented accelerators and accessibility overlay. +- [ ] Deep-link examples tested for copy/share parity and CLI alignment. +- [ ] Tenant switching flow documents cache invalidation and audit logging. +- [ ] Filter tray, command palette, and presets cross-referenced with accessibility guidance. +- [ ] Offline/air-gap notes included for palette, tenant switcher, and deep-link metadata. +- [ ] Links to dependent docs (`/docs/ui/*`, `/docs/security/*`) validated. + +--- + +*Last updated: 2025-10-26 (Sprint 23).* diff --git a/docs/ui/policies.md b/docs/ui/policies.md index 878141b9..3e2c96ca 100644 --- a/docs/ui/policies.md +++ b/docs/ui/policies.md @@ -158,9 +158,9 @@ UI disables controls not allowed by current scope and surfaces tooltip with requ ## 12. Screenshot coordination - Placeholders: - - `![Policy list placeholder](../assets/ui/policies/list-placeholder.png)` - - `![Policy approval placeholder](../assets/ui/policies/approval-placeholder.png)` - - `![Simulation diff placeholder](../assets/ui/policies/simulation-placeholder.png)` + - `docs/assets/ui/policies/list-placeholder.png` (capture pending) + - `docs/assets/ui/policies/approval-placeholder.png` (capture pending) + - `docs/assets/ui/policies/simulation-placeholder.png` (capture pending) - Coordinate with Console Guild via `#console-screenshots` (entry 2025-10-26) to replace placeholders once UI captures are ready (light and dark themes). --- diff --git a/docs/ui/policy-editor.md b/docs/ui/policy-editor.md index 441d5ccd..9a14d451 100644 --- a/docs/ui/policy-editor.md +++ b/docs/ui/policy-editor.md @@ -46,7 +46,7 @@ The Policy Editor is the primary Console workspace for composing, simulating, an - *Explain Explorer* (optional drawer for findings) - **Right rail:** context cards for VEX providers, policy metadata, quick links to CLI/API docs. -> Placeholder screenshot: `![Policy editor workspace](../assets/policy-editor/workspace.png)` (add after UI team captures latest build). +> Placeholder screenshot: `docs/assets/policy-editor/workspace.png` (pending upload after UI team captures latest build). --- diff --git a/docs/ui/runs.md b/docs/ui/runs.md index f4c011fb..00d9a594 100644 --- a/docs/ui/runs.md +++ b/docs/ui/runs.md @@ -134,8 +134,8 @@ Sections: ## 11. Screenshot coordination - Placeholders: - - `![Runs dashboard placeholder](../assets/ui/runs/dashboard-placeholder.png)` - - `![Run detail placeholder](../assets/ui/runs/detail-placeholder.png)` + - `docs/assets/ui/runs/dashboard-placeholder.png` (capture pending) + - `docs/assets/ui/runs/detail-placeholder.png` (capture pending) - Coordinate with Scheduler Guild for updated screenshots after Sprint 23 UI stabilises (tracked in `#console-screenshots`, entry 2025-10-26). --- diff --git a/docs/ui/sbom-explorer.md b/docs/ui/sbom-explorer.md index 74f22473..5dffbf57 100644 --- a/docs/ui/sbom-explorer.md +++ b/docs/ui/sbom-explorer.md @@ -161,8 +161,8 @@ Saved views store combinations of these filters and expose command palette short ## 10. Screenshot coordination - Placeholder images: - - `![SBOM catalog view placeholder](../assets/ui/sbom/catalog-placeholder.png)` - - `![Overlay graph placeholder](../assets/ui/sbom/overlay-placeholder.png)` + - `docs/assets/ui/sbom/catalog-placeholder.png` (capture pending) + - `docs/assets/ui/sbom/overlay-placeholder.png` (capture pending) - Coordinate with Console Guild to capture updated screenshots (dark and light theme) once Sprint 23 UI stabilises. Track follow-up in Console Guild thread `#console-screenshots` dated 2025-10-26. --- diff --git a/docs/updates/2025-10-27-task-packs-docs.md b/docs/updates/2025-10-27-task-packs-docs.md index 22d6b275..f479a5b1 100644 --- a/docs/updates/2025-10-27-task-packs-docs.md +++ b/docs/updates/2025-10-27-task-packs-docs.md @@ -6,10 +6,10 @@ - `/docs/task-packs/registry.md` - `/docs/task-packs/runbook.md` - `/docs/security/pack-signing-and-rbac.md` - - `/docs/operations/cli-release-and-packaging.md` + - `/docs/modules/cli/operations/release-and-packaging.md` - Each doc includes imposed-rule reminder, compliance checklist, and cross-links to Task Runner, Packs Registry, CLI release tasks. - Created asset staging instructions at `docs/assets/ui/tours/README.md` (shared with CLI enablement). - Circulated spec + authoring guide links to Task Runner, Packs Registry, Authority, and DevOps guild channels for technical review (2025-10-27). Target follow-up review once CLI parity tasks (`CLI-PACKS-42-001`, `CLI-PACKS-43-001`) land; tentative sync held for 2025-11-03 (Docs Guild to confirm). - Sprint tracker `DOCS-PACKS-43-001` marked DOING→DONE; follow-up reviews scheduled with Task Runner and Security guilds. -Artifacts: [Spec](../task-packs/spec.md), [Authoring guide](../task-packs/authoring-guide.md), [Registry](../task-packs/registry.md), [Runbook](../task-packs/runbook.md), [Signing/RBAC](../security/pack-signing-and-rbac.md), [CLI release runbook](../operations/cli-release-and-packaging.md). +Artifacts: [Spec](../task-packs/spec.md), [Authoring guide](../task-packs/authoring-guide.md), [Registry](../task-packs/registry.md), [Runbook](../task-packs/runbook.md), [Signing/RBAC](../security/pack-signing-and-rbac.md), [CLI release runbook](../modules/cli/operations/release-and-packaging.md). diff --git a/docs/updates/2025-10-30-devops-governance.md b/docs/updates/2025-10-30-devops-governance.md new file mode 100644 index 00000000..62af9eaf --- /dev/null +++ b/docs/updates/2025-10-30-devops-governance.md @@ -0,0 +1,17 @@ +# 30 Oct 2025 — Governance rules anchor consolidated + +**What changed** + +- Published `docs/devops/contracts-and-rules.md` capturing the Sprint 33 governance rules: + 1. API Gateway remains a proxy; Policy Engine composes overlays/simulations. + 2. AOC ingestion persists upstream truth only (no merge/deduplicate logic). + 3. Graph platform standardised on Graph Indexer + Graph API (Cartographer retired). +- Updated backlog hygiene note (`docs/backlog/2025-10-cleanup.md`) and archived the Cartographer handshake plan to point at the new graph platform. +- Logged the rules in `ops/devops/TASKS.md` and `docs/implplan/SPRINTS.md`, removing duplicate references to Cartographer as an active service. + +**Reviewers / acknowledgements** + +- Platform Leads (DevOps + Graph) confirmed the retirement of Cartographer in favour of Graph Indexer + Graph API. +- Policy Engine Guild acknowledged the proxy-only Gateway posture and downstream overlay ownership. + +See `DEVOPS-RULES-33-001` for the owning task. diff --git a/etc/authority.yaml b/etc/authority.yaml index 31de7611..05c2cabe 100644 --- a/etc/authority.yaml +++ b/etc/authority.yaml @@ -17,12 +17,13 @@ storage: databaseName: "stellaops_authority" commandTimeout: "00:00:30" -signing: - enabled: true - activeKeyId: "authority-signing-dev" - keyPath: "../certificates/authority-signing-dev.pem" - algorithm: "ES256" - keySource: "file" +signing: + enabled: true + activeKeyId: "authority-signing-dev" + keyPath: "../certificates/authority-signing-dev.pem" + algorithm: "ES256" + keySource: "file" + jwksCacheLifetime: "00:05:00" bootstrap: enabled: false diff --git a/etc/authority.yaml.sample b/etc/authority.yaml.sample index 84513cc3..93a1caa3 100644 --- a/etc/authority.yaml.sample +++ b/etc/authority.yaml.sample @@ -24,17 +24,18 @@ storage: commandTimeout: "00:00:30" # Signing configuration for revocation bundles and JWKS. -signing: - enabled: true - activeKeyId: "authority-signing-2025-dev" - keyPath: "../certificates/authority-signing-2025-dev.pem" - algorithm: "ES256" - keySource: "file" - # provider: "default" - additionalKeys: - - keyId: "authority-signing-dev" - path: "../certificates/authority-signing-dev.pem" - source: "file" +signing: + enabled: true + activeKeyId: "authority-signing-2025-dev" + keyPath: "../certificates/authority-signing-2025-dev.pem" + algorithm: "ES256" + keySource: "file" + jwksCacheLifetime: "00:05:00" + # provider: "default" + additionalKeys: + - keyId: "authority-signing-dev" + path: "../certificates/authority-signing-dev.pem" + source: "file" # Rotation flow: # 1. Generate a new PEM under ./certificates (e.g. authority-signing-2026-dev.pem). # 2. Trigger the .gitea/workflows/authority-key-rotation.yml workflow (or run diff --git a/ops/authority/TASKS.completed.md b/ops/authority/TASKS.completed.md new file mode 100644 index 00000000..738f274b --- /dev/null +++ b/ops/authority/TASKS.completed.md @@ -0,0 +1,5 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +| OPS3.KEY-ROTATION | DONE (2025-10-12) | DevOps Crew, Authority Core | CORE10.JWKS | Implement key rotation tooling + pipeline hook once rotating JWKS lands. Document SOP and secret handling. | ✅ CLI/script rotates keys + updates JWKS; ✅ Pipeline job documented; ✅ docs/ops runbook updated. | diff --git a/ops/authority/TASKS.md b/ops/authority/TASKS.md index d6e6c161..4c86e45c 100644 --- a/ops/authority/TASKS.md +++ b/ops/authority/TASKS.md @@ -3,4 +3,3 @@ | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| OPS3.KEY-ROTATION | DONE (2025-10-12) | DevOps Crew, Authority Core | CORE10.JWKS | Implement key rotation tooling + pipeline hook once rotating JWKS lands. Document SOP and secret handling. | ✅ CLI/script rotates keys + updates JWKS; ✅ Pipeline job documented; ✅ docs/ops runbook updated. | diff --git a/ops/deployment/TASKS.completed.md b/ops/deployment/TASKS.completed.md new file mode 100644 index 00000000..851ecb88 --- /dev/null +++ b/ops/deployment/TASKS.completed.md @@ -0,0 +1,5 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| DEVOPS-OPS-14-003 | DONE (2025-10-26) | Deployment Guild | DEVOPS-REL-14-001 | Document and script upgrade/rollback flows, channel management, and compatibility matrices per architecture. | Helm/Compose guides updated with digest pinning, automated checks committed, rollback drill recorded. | diff --git a/ops/deployment/TASKS.md b/ops/deployment/TASKS.md index 60126ebc..d48e172b 100644 --- a/ops/deployment/TASKS.md +++ b/ops/deployment/TASKS.md @@ -1,8 +1,7 @@ -# Deployment Task Board - +# Deployment Task Board + | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| DEVOPS-OPS-14-003 | DONE (2025-10-26) | Deployment Guild | DEVOPS-REL-14-001 | Document and script upgrade/rollback flows, channel management, and compatibility matrices per architecture. | Helm/Compose guides updated with digest pinning, automated checks committed, rollback drill recorded. | | DOWNLOADS-CONSOLE-23-001 | TODO | Deployment Guild, DevOps Guild | DEVOPS-CONSOLE-23-002 | Maintain signed downloads manifest pipeline (images, Helm, offline bundles), publish JSON under `deploy/downloads/manifest.json`, and document sync cadence for Console + docs parity. | Pipeline generates signed manifest with checksums, automated PR updates manifest, docs updated with sync workflow, parity check in CI passes. | | DEPLOY-NOTIFY-38-001 | BLOCKED (2025-10-29) | Deployment Guild, DevOps Guild | NOTIFY-SVC-38-001..004 | Package notifier API/worker Helm overlays (email/chat/webhook), secrets templates, rollout guide. | Overlays committed; smoke deploy executed; rollback steps recorded; secrets templates provided. | | DEPLOY-POLICY-27-001 | TODO | Deployment Guild, Policy Registry Guild | REGISTRY-API-27-001, DEVOPS-POLICY-27-003 | Produce Helm/Compose overlays for Policy Registry + simulation workers, including Mongo migrations, object storage buckets, signing key secrets, and tenancy defaults. | Overlays committed with deterministic digests; install docs updated; smoke deploy validated in staging. | diff --git a/ops/devops/TASKS.completed.md b/ops/devops/TASKS.completed.md new file mode 100644 index 00000000..d62e5516 --- /dev/null +++ b/ops/devops/TASKS.completed.md @@ -0,0 +1,27 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| DEVOPS-HELM-09-001 | DONE | DevOps Guild | SCANNER-WEB-09-101 | Create Helm/Compose environment profiles (dev, staging, airgap) with deterministic digests. | Profiles committed under `deploy/`; docs updated; CI smoke deploy passes. | +| DEVOPS-SCANNER-09-204 | DONE (2025-10-21) | DevOps Guild, Scanner WebService Guild | SCANNER-EVENTS-15-201 | Surface `SCANNER__EVENTS__*` environment variables across docker-compose (dev/stage/airgap) and Helm values, defaulting to share the Redis queue DSN. | Compose/Helm configs ship enabled Redis event publishing with documented overrides; lint jobs updated; docs cross-link to new knobs. | +| DEVOPS-SCANNER-09-205 | DONE (2025-10-21) | DevOps Guild, Notify Guild | DEVOPS-SCANNER-09-204 | Add Notify smoke stage that tails the Redis stream and asserts `scanner.report.ready`/`scanner.scan.completed` reach Notify WebService in staging. | CI job reads Redis stream during scanner smoke deploy, confirms Notify ingestion via API, alerts on failure. | +| DEVOPS-PERF-10-001 | DONE | DevOps Guild | BENCH-SCANNER-10-001 | Add perf smoke job (SBOM compose <5 s target) to CI. | CI job runs sample build verifying <5 s; alerts configured. | +| DEVOPS-PERF-10-002 | DONE (2025-10-23) | DevOps Guild | BENCH-SCANNER-10-002 | Publish analyzer bench metrics to Grafana/perf workbook and alarm on ≥20 % regressions. | CI exports JSON for dashboards; Grafana panel wired; Ops on-call doc updated with alert hook. | +| DEVOPS-REL-14-001 | DONE (2025-10-26) | DevOps Guild | SIGNER-API-11-101, ATTESTOR-API-11-201 | Deterministic build/release pipeline with SBOM/provenance, signing, manifest generation. | CI pipeline produces signed images + SBOM/attestations, manifests published with verified hashes, docs updated. | +| DEVOPS-REL-14-004 | DONE (2025-10-26) | DevOps Guild, Scanner Guild | DEVOPS-REL-14-001, SCANNER-ANALYZERS-LANG-10-309P | Extend release/offline smoke jobs to exercise the Python analyzer plug-in (warm/cold scans, determinism, signature checks). | Release/Offline pipelines run Python analyzer smoke suite; alerts hooked; docs updated with new coverage matrix. | +| DEVOPS-REL-17-002 | DONE (2025-10-26) | DevOps Guild | DEVOPS-REL-14-001, SCANNER-EMIT-17-701 | Persist stripped-debug artifacts organised by GNU build-id and bundle them into release/offline kits with checksum manifests. | CI job writes `.debug` files under `artifacts/debug/.build-id/`, manifest + checksums published, offline kit includes cache, smoke job proves symbol lookup via build-id. | +| DEVOPS-MIRROR-08-001 | DONE (2025-10-19) | DevOps Guild | DEVOPS-REL-14-001 | Stand up managed mirror profiles for `*.stella-ops.org` (Concelier/Excititor), including Helm/Compose overlays, multi-tenant secrets, CDN caching, and sync documentation. | Infra overlays committed, CI smoke deploy hits mirror endpoints, runbooks published for downstream sync and quota management. | +| DEVOPS-POLICY-20-001 | DONE (2025-10-26) | DevOps Guild, Policy Guild | POLICY-ENGINE-20-001 | Integrate DSL linting in CI (parser/compile) to block invalid policies; add pipeline step compiling sample policies. | CI fails on syntax errors; lint logs surfaced; docs updated with pipeline instructions. | +| DEVOPS-POLICY-20-003 | DONE (2025-10-26) | DevOps Guild, QA Guild | DEVOPS-POLICY-20-001, POLICY-ENGINE-20-005 | Determinism CI: run Policy Engine twice with identical inputs and diff outputs to guard non-determinism. | CI job compares outputs, fails on differences, logs stored; documentation updated. | +| DEVOPS-POLICY-20-004 | DONE (2025-10-27) | DevOps Guild, Scheduler Guild, CLI Guild | SCHED-MODELS-20-001, CLI-POLICY-20-002 | Automate policy schema exports: generate JSON Schema from `PolicyRun*` DTOs during CI, publish artefacts, and emit change alerts for CLI consumers (Slack + changelog). | CI stage outputs versioned schema files, uploads artefacts, notifies #policy-engine channel on change; docs/CLI references updated. | +| DEVOPS-OBS-50-001 | DONE (2025-10-26) | DevOps Guild, Observability Guild | TELEMETRY-OBS-50-001 | Deliver default OpenTelemetry collector deployment (Compose/Helm manifests), OTLP ingestion endpoints, and secure pipeline (authN, mTLS, tenant partitioning). Provide smoke test verifying traces/logs/metrics ingestion. | Collector manifests committed; smoke test green; docs updated; imposed rule banner reminder noted. | +| DEVOPS-OBS-50-003 | DONE (2025-10-26) | DevOps Guild, Offline Kit Guild | DEVOPS-OBS-50-001 | Package telemetry stack configs for air-gapped installs (Offline Kit bundle, documented overrides, sample values) and automate checksum/signature generation. | Offline bundle includes collector+storage configs; checksums published; docs cross-linked; imposed rule annotation recorded. | +| DEVOPS-LAUNCH-18-100 | DONE (2025-10-26) | DevOps Guild | - | Finalise production environment footprint (clusters, secrets, network overlays) for full-platform go-live. | IaC/compose overlays committed, secrets placeholders documented, dry-run deploy succeeds in staging. | + +| DEVOPS-CONSOLE-23-002 | TODO | DevOps Guild, Console Guild | DEVOPS-CONSOLE-23-001, CONSOLE-REL-23-301 | Produce `stella-console` container build + Helm chart overlays with deterministic digests, SBOM/provenance artefacts, and offline bundle packaging scripts. | Container published to registry mirror, Helm values committed, SBOM/attestations generated, offline kit job passes smoke test, docs updated. | +| DEVOPS-LAUNCH-18-100 | DONE (2025-10-26) | DevOps Guild | - | Finalise production environment footprint (clusters, secrets, network overlays) for full-platform go-live. | IaC/compose overlays committed, secrets placeholders documented, dry-run deploy succeeds in staging. | +| DEVOPS-LAUNCH-18-900 | DONE (2025-10-26) | DevOps Guild, Module Leads | Wave 0 completion | Collect “full implementation” sign-off from module owners and consolidate launch readiness checklist. | Sign-off record stored under `docs/modules/devops/runbooks/launch-readiness.md`; outstanding gaps triaged; checklist approved. | +| DEVOPS-LAUNCH-18-001 | DONE (2025-10-26) | DevOps Guild | DEVOPS-LAUNCH-18-100, DEVOPS-LAUNCH-18-900 | Production launch cutover rehearsal and runbook publication. | `docs/modules/devops/runbooks/launch-cutover.md` drafted, rehearsal executed with rollback drill, approvals captured. | +| DEVOPS-NUGET-13-001 | DONE (2025-10-25) | DevOps Guild, Platform Leads | DEVOPS-REL-14-001 | Add .NET 10 preview feeds / local mirrors so `Microsoft.Extensions.*` 10.0 preview packages restore offline; refresh restore docs. | NuGet.config maps preview feeds (or local mirrored packages), `dotnet restore` succeeds for Excititor/Concelier solutions without ad-hoc feed edits, docs updated for offline bootstrap. | +| DEVOPS-NUGET-13-002 | DONE (2025-10-26) | DevOps Guild | DEVOPS-NUGET-13-001 | Ensure all solutions/projects prefer `local-nuget` before public sources and document restore order validation. | `NuGet.config` and solution-level configs resolve from `local-nuget` first; automated check verifies priority; docs updated for restore ordering. | +| DEVOPS-NUGET-13-003 | DONE (2025-10-26) | DevOps Guild, Platform Leads | DEVOPS-NUGET-13-002 | Sweep `Microsoft.*` NuGet dependencies pinned to 8.* and upgrade to latest .NET 10 equivalents (or .NET 9 when 10 unavailable), updating restore guidance. | Dependency audit shows no 8.* `Microsoft.*` packages remaining; CI builds green; changelog/doc sections capture upgrade rationale. | diff --git a/ops/devops/TASKS.md b/ops/devops/TASKS.md index 3fb7bfd2..10d615b3 100644 --- a/ops/devops/TASKS.md +++ b/ops/devops/TASKS.md @@ -4,15 +4,11 @@ | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| DEVOPS-RULES-33-001 | DOING (2025-10-26) | DevOps Guild, Platform Leads | — | Contracts & Rules anchor:
• Gateway proxies only; Policy Engine composes overlays/simulations.
• AOC ingestion cannot merge; only lossless canonicalization.
• One graph platform: Graph Indexer + Graph API. Cartographer retired. | Rules posted in SPRINTS/TASKS; duplicates cleaned per guidance; reviewers acknowledge in changelog. | +| DEVOPS-RULES-33-001 | REVIEW (2025-10-30) | DevOps Guild, Platform Leads | — | Contracts & Rules anchor:
• Gateway proxies only; Policy Engine composes overlays/simulations.
• AOC ingestion cannot merge; only lossless canonicalization.
• One graph platform: Graph Indexer + Graph API. Cartographer retired. | Rules posted in SPRINTS/TASKS; duplicates cleaned per guidance; reviewers acknowledge in changelog. | +> 2025-10-30: Published governance anchor (`docs/devops/contracts-and-rules.md`), archived Cartographer plan, and logged reviewer acknowledgement in `docs/updates/2025-10-30-devops-governance.md`. | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| DEVOPS-HELM-09-001 | DONE | DevOps Guild | SCANNER-WEB-09-101 | Create Helm/Compose environment profiles (dev, staging, airgap) with deterministic digests. | Profiles committed under `deploy/`; docs updated; CI smoke deploy passes. | -| DEVOPS-SCANNER-09-204 | DONE (2025-10-21) | DevOps Guild, Scanner WebService Guild | SCANNER-EVENTS-15-201 | Surface `SCANNER__EVENTS__*` environment variables across docker-compose (dev/stage/airgap) and Helm values, defaulting to share the Redis queue DSN. | Compose/Helm configs ship enabled Redis event publishing with documented overrides; lint jobs updated; docs cross-link to new knobs. | -| DEVOPS-SCANNER-09-205 | DONE (2025-10-21) | DevOps Guild, Notify Guild | DEVOPS-SCANNER-09-204 | Add Notify smoke stage that tails the Redis stream and asserts `scanner.report.ready`/`scanner.scan.completed` reach Notify WebService in staging. | CI job reads Redis stream during scanner smoke deploy, confirms Notify ingestion via API, alerts on failure. | -| DEVOPS-PERF-10-001 | DONE | DevOps Guild | BENCH-SCANNER-10-001 | Add perf smoke job (SBOM compose <5 s target) to CI. | CI job runs sample build verifying <5 s; alerts configured. | -| DEVOPS-PERF-10-002 | DONE (2025-10-23) | DevOps Guild | BENCH-SCANNER-10-002 | Publish analyzer bench metrics to Grafana/perf workbook and alarm on ≥20 % regressions. | CI exports JSON for dashboards; Grafana panel wired; Ops on-call doc updated with alert hook. | | DEVOPS-AOC-19-001 | BLOCKED (2025-10-26) | DevOps Guild, Platform Guild | WEB-AOC-19-003 | Integrate the AOC Roslyn analyzer and guard tests into CI, failing builds when ingestion projects attempt banned writes. | Analyzer runs in PR/CI pipelines, results surfaced in build summary, docs updated under `docs/modules/devops/runbooks/ci-aoc.md`. | > Docs hand-off (2025-10-26): see `docs/ingestion/aggregation-only-contract.md` §5, `docs/modules/platform/architecture-overview.md`, and `docs/modules/cli/guides/cli-reference.md` for guard + verifier expectations. | DEVOPS-AOC-19-002 | BLOCKED (2025-10-26) | DevOps Guild | CLI-AOC-19-002, CONCELIER-WEB-AOC-19-004, EXCITITOR-WEB-AOC-19-004 | Add pipeline stage executing `stella aoc verify --since` against seeded Mongo snapshots for Concelier + Excititor, publishing violation report artefacts. | Stage runs on main/nightly, fails on violations, artifacts retained, runbook documented. | @@ -20,10 +16,9 @@ | DEVOPS-AOC-19-003 | BLOCKED (2025-10-26) | DevOps Guild, QA Guild | CONCELIER-WEB-AOC-19-003, EXCITITOR-WEB-AOC-19-003 | Enforce unit test coverage thresholds for AOC guard suites and ensure coverage exported to dashboards. | Coverage report includes guard projects, threshold gate passes/fails as expected, dashboards refreshed with new metrics. | > Blocked: guard coverage suites and exporter hooks pending in Concelier/Excititor (CONCELIER-WEB-AOC-19-003, EXCITITOR-WEB-AOC-19-003). | DEVOPS-AOC-19-101 | TODO (2025-10-28) | DevOps Guild, Concelier Storage Guild | CONCELIER-STORE-AOC-19-002 | Draft supersedes backfill rollout (freeze window, dry-run steps, rollback) once advisory_raw idempotency index passes staging verification. | Runbook committed in `docs/deploy/containers.md` + Offline Kit notes, staging rehearsal scheduled with dependencies captured in SPRINTS. | -| DEVOPS-OBS-50-001 | DONE (2025-10-26) | DevOps Guild, Observability Guild | TELEMETRY-OBS-50-001 | Deliver default OpenTelemetry collector deployment (Compose/Helm manifests), OTLP ingestion endpoints, and secure pipeline (authN, mTLS, tenant partitioning). Provide smoke test verifying traces/logs/metrics ingestion. | Collector manifests committed; smoke test green; docs updated; imposed rule banner reminder noted. | | DEVOPS-OBS-50-002 | DOING (2025-10-26) | DevOps Guild, Security Guild | DEVOPS-OBS-50-001, TELEMETRY-OBS-51-002 | Stand up multi-tenant storage backends (Prometheus, Tempo/Jaeger, Loki) with retention policies, tenant isolation, and redaction guard rails. Integrate with Authority scopes for read paths. | Storage stack deployed with auth; retention configured; integration tests verify tenant isolation; runbook drafted. | > Coordination started with Observability Guild (2025-10-26) to schedule staging rollout and provision service accounts. Staging bootstrap commands and secret names documented in `docs/modules/telemetry/operations/storage.md`. -| DEVOPS-OBS-50-003 | DONE (2025-10-26) | DevOps Guild, Offline Kit Guild | DEVOPS-OBS-50-001 | Package telemetry stack configs for air-gapped installs (Offline Kit bundle, documented overrides, sample values) and automate checksum/signature generation. | Offline bundle includes collector+storage configs; checksums published; docs cross-linked; imposed rule annotation recorded. | +> 2025-10-30: Added static validator `ops/devops/telemetry/validate_storage_stack.py` and updated storage runbook to require it alongside TLS/tenant setup. | DEVOPS-OBS-51-001 | TODO | DevOps Guild, Observability Guild | WEB-OBS-51-001, DEVOPS-OBS-50-001 | Implement SLO evaluator service (burn rate calculators, webhook emitters), Grafana dashboards, and alert routing to Notifier. Provide Terraform/Helm automation. | Dashboards live; evaluator emits webhooks; alert runbook referenced; staging alert fired in test. | | DEVOPS-OBS-52-001 | TODO | DevOps Guild, Timeline Indexer Guild | TIMELINE-OBS-52-002 | Configure streaming pipeline (NATS/Redis/Kafka) with retention, partitioning, and backpressure tuning for timeline events; add CI validation of schema + rate caps. | Pipeline deployed; load test meets SLA; schema validation job passes; documentation updated. | | DEVOPS-OBS-53-001 | TODO | DevOps Guild, Evidence Locker Guild | EVID-OBS-53-001 | Provision object storage with WORM/retention options (S3 Object Lock / MinIO immutability), legal hold automation, and backup/restore scripts for evidence locker. | Storage configured with WORM; legal hold script documented; backup test performed; runbook updated. | @@ -40,29 +35,16 @@ | DEVOPS-AIRGAP-57-002 | TODO | DevOps Guild, Authority Guild | AUTH-OBS-50-001 | Configure sealed-mode CI tests that run services with sealed flag and ensure no egress occurs (iptables + mock DNS). | CI suite fails on attempted egress; reports remediation; documentation updated. | | DEVOPS-AIRGAP-58-001 | TODO | DevOps Guild, Notifications Guild | NOTIFY-AIRGAP-56-002 | Provide local SMTP/syslog container templates and health checks for sealed environments; integrate into Bootstrap Pack. | Templates deployed successfully; health checks in CI; docs updated. | | DEVOPS-AIRGAP-58-002 | TODO | DevOps Guild, Observability Guild | DEVOPS-AIRGAP-56-001, DEVOPS-OBS-51-001 | Ship sealed-mode observability stack (Prometheus/Grafana/Tempo/Loki) pre-configured with offline dashboards and no remote exporters. | Stack boots offline; dashboards available; verification script confirms zero egress. | -| DEVOPS-REL-14-001 | DONE (2025-10-26) | DevOps Guild | SIGNER-API-11-101, ATTESTOR-API-11-201 | Deterministic build/release pipeline with SBOM/provenance, signing, manifest generation. | CI pipeline produces signed images + SBOM/attestations, manifests published with verified hashes, docs updated. | -| DEVOPS-REL-14-004 | DONE (2025-10-26) | DevOps Guild, Scanner Guild | DEVOPS-REL-14-001, SCANNER-ANALYZERS-LANG-10-309P | Extend release/offline smoke jobs to exercise the Python analyzer plug-in (warm/cold scans, determinism, signature checks). | Release/Offline pipelines run Python analyzer smoke suite; alerts hooked; docs updated with new coverage matrix. | -| DEVOPS-REL-17-002 | DONE (2025-10-26) | DevOps Guild | DEVOPS-REL-14-001, SCANNER-EMIT-17-701 | Persist stripped-debug artifacts organised by GNU build-id and bundle them into release/offline kits with checksum manifests. | CI job writes `.debug` files under `artifacts/debug/.build-id/`, manifest + checksums published, offline kit includes cache, smoke job proves symbol lookup via build-id. | | DEVOPS-REL-17-004 | BLOCKED (2025-10-26) | DevOps Guild | DEVOPS-REL-17-002 | Ensure release workflow publishes `out/release/debug` (build-id tree + manifest) and fails when symbols are missing. | Release job emits debug artefacts, `mirror_debug_store.py` summary committed, warning cleared from build logs, docs updated. | -| DEVOPS-MIRROR-08-001 | DONE (2025-10-19) | DevOps Guild | DEVOPS-REL-14-001 | Stand up managed mirror profiles for `*.stella-ops.org` (Concelier/Excititor), including Helm/Compose overlays, multi-tenant secrets, CDN caching, and sync documentation. | Infra overlays committed, CI smoke deploy hits mirror endpoints, runbooks published for downstream sync and quota management. | > Note (2025-10-26, BLOCKED): IdentityModel.Tokens patched for logging 9.x, but release bundle still fails because Docker cannot stream multi-arch build context (`unix:///var/run/docker.sock` unavailable, EOF during copy). Retry once docker daemon/socket is healthy; until then `out/release/debug` cannot be generated. | DEVOPS-CONSOLE-23-001 | BLOCKED (2025-10-26) | DevOps Guild, Console Guild | CONSOLE-CORE-23-001 | Add console CI workflow (pnpm cache, lint, type-check, unit, Storybook a11y, Playwright, Lighthouse) with offline runners and artifact retention for screenshots/reports. | Workflow runs on PR & main, caches reduce install time, failing checks block merges, artifacts uploaded for triage, docs updated. | > Blocked: Console workspace and package scripts (CONSOLE-CORE-23-001..005) are not yet present; CI cannot execute pnpm/Playwright/Lighthouse until the Next.js app lands. | DEVOPS-CONSOLE-23-002 | TODO | DevOps Guild, Console Guild | DEVOPS-CONSOLE-23-001, CONSOLE-REL-23-301 | Produce `stella-console` container build + Helm chart overlays with deterministic digests, SBOM/provenance artefacts, and offline bundle packaging scripts. | Container published to registry mirror, Helm values committed, SBOM/attestations generated, offline kit job passes smoke test, docs updated. | -| DEVOPS-LAUNCH-18-100 | DONE (2025-10-26) | DevOps Guild | - | Finalise production environment footprint (clusters, secrets, network overlays) for full-platform go-live. | IaC/compose overlays committed, secrets placeholders documented, dry-run deploy succeeds in staging. | -| DEVOPS-LAUNCH-18-900 | DONE (2025-10-26) | DevOps Guild, Module Leads | Wave 0 completion | Collect “full implementation” sign-off from module owners and consolidate launch readiness checklist. | Sign-off record stored under `docs/modules/devops/runbooks/launch-readiness.md`; outstanding gaps triaged; checklist approved. | -| DEVOPS-LAUNCH-18-001 | DONE (2025-10-26) | DevOps Guild | DEVOPS-LAUNCH-18-100, DEVOPS-LAUNCH-18-900 | Production launch cutover rehearsal and runbook publication. | `docs/modules/devops/runbooks/launch-cutover.md` drafted, rehearsal executed with rollback drill, approvals captured. | -| DEVOPS-NUGET-13-001 | DONE (2025-10-25) | DevOps Guild, Platform Leads | DEVOPS-REL-14-001 | Add .NET 10 preview feeds / local mirrors so `Microsoft.Extensions.*` 10.0 preview packages restore offline; refresh restore docs. | NuGet.config maps preview feeds (or local mirrored packages), `dotnet restore` succeeds for Excititor/Concelier solutions without ad-hoc feed edits, docs updated for offline bootstrap. | -| DEVOPS-NUGET-13-002 | DONE (2025-10-26) | DevOps Guild | DEVOPS-NUGET-13-001 | Ensure all solutions/projects prefer `local-nuget` before public sources and document restore order validation. | `NuGet.config` and solution-level configs resolve from `local-nuget` first; automated check verifies priority; docs updated for restore ordering. | -| DEVOPS-NUGET-13-003 | DONE (2025-10-26) | DevOps Guild, Platform Leads | DEVOPS-NUGET-13-002 | Sweep `Microsoft.*` NuGet dependencies pinned to 8.* and upgrade to latest .NET 10 equivalents (or .NET 9 when 10 unavailable), updating restore guidance. | Dependency audit shows no 8.* `Microsoft.*` packages remaining; CI builds green; changelog/doc sections capture upgrade rationale. | ## Policy Engine v2 | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| DEVOPS-POLICY-20-001 | DONE (2025-10-26) | DevOps Guild, Policy Guild | POLICY-ENGINE-20-001 | Integrate DSL linting in CI (parser/compile) to block invalid policies; add pipeline step compiling sample policies. | CI fails on syntax errors; lint logs surfaced; docs updated with pipeline instructions. | -| DEVOPS-POLICY-20-003 | DONE (2025-10-26) | DevOps Guild, QA Guild | DEVOPS-POLICY-20-001, POLICY-ENGINE-20-005 | Determinism CI: run Policy Engine twice with identical inputs and diff outputs to guard non-determinism. | CI job compares outputs, fails on differences, logs stored; documentation updated. | -| DEVOPS-POLICY-20-004 | DONE (2025-10-27) | DevOps Guild, Scheduler Guild, CLI Guild | SCHED-MODELS-20-001, CLI-POLICY-20-002 | Automate policy schema exports: generate JSON Schema from `PolicyRun*` DTOs during CI, publish artefacts, and emit change alerts for CLI consumers (Slack + changelog). | CI stage outputs versioned schema files, uploads artefacts, notifies #policy-engine channel on change; docs/CLI references updated. | > 2025-10-27: `.gitea/workflows/build-test-deploy.yml` publishes the `policy-schema-exports` artefact under `artifacts/policy-schemas//` and posts Slack diffs via `POLICY_ENGINE_SCHEMA_WEBHOOK`; diff stored as `policy-schema-diff.patch`. ## Graph Explorer v1 diff --git a/ops/devops/telemetry/validate_storage_stack.py b/ops/devops/telemetry/validate_storage_stack.py new file mode 100644 index 00000000..0eba382a --- /dev/null +++ b/ops/devops/telemetry/validate_storage_stack.py @@ -0,0 +1,83 @@ +#!/usr/bin/env python3 +""" +Static validation for the telemetry storage stack configuration. + +Checks the Prometheus, Tempo, and Loki configuration snippets to ensure: +- mutual TLS is enabled end-to-end +- tenant override files are referenced +- multitenancy flags are set +- retention/limit defaults exist for __default__ tenant entries + +This script is intended to back `DEVOPS-OBS-50-002` and can run in CI +before publishing bundles or rolling out staging updates. +""" + +from __future__ import annotations + +import sys +from pathlib import Path + +REPO_ROOT = Path(__file__).resolve().parents[3] +PROMETHEUS_PATH = REPO_ROOT / "deploy/telemetry/storage/prometheus.yaml" +TEMPO_PATH = REPO_ROOT / "deploy/telemetry/storage/tempo.yaml" +LOKI_PATH = REPO_ROOT / "deploy/telemetry/storage/loki.yaml" +TEMPO_OVERRIDES_PATH = REPO_ROOT / "deploy/telemetry/storage/tenants/tempo-overrides.yaml" +LOKI_OVERRIDES_PATH = REPO_ROOT / "deploy/telemetry/storage/tenants/loki-overrides.yaml" + + +def read(path: Path) -> str: + if not path.exists(): + raise FileNotFoundError(f"Required configuration file missing: {path}") + return path.read_text(encoding="utf-8") + + +def assert_contains(haystack: str, needle: str, path: Path) -> None: + if needle not in haystack: + raise AssertionError(f"{path} is missing required snippet: {needle!r}") + + +def validate_prometheus() -> None: + content = read(PROMETHEUS_PATH) + assert_contains(content, "tls_config:", PROMETHEUS_PATH) + assert_contains(content, "ca_file:", PROMETHEUS_PATH) + assert_contains(content, "cert_file:", PROMETHEUS_PATH) + assert_contains(content, "key_file:", PROMETHEUS_PATH) + assert_contains(content, "authorization:", PROMETHEUS_PATH) + assert_contains(content, "credentials_file:", PROMETHEUS_PATH) + + +def validate_tempo() -> None: + content = read(TEMPO_PATH) + assert_contains(content, "multitenancy_enabled: true", TEMPO_PATH) + assert_contains(content, "require_client_cert: true", TEMPO_PATH) + assert_contains(content, "per_tenant_override_config", TEMPO_PATH) + overrides = read(TEMPO_OVERRIDES_PATH) + assert_contains(overrides, "__default__", TEMPO_OVERRIDES_PATH) + assert_contains(overrides, "traces_per_second_limit", TEMPO_OVERRIDES_PATH) + assert_contains(overrides, "max_bytes_per_trace", TEMPO_OVERRIDES_PATH) + + +def validate_loki() -> None: + content = read(LOKI_PATH) + assert_contains(content, "auth_enabled: true", LOKI_PATH) + assert_contains(content, "per_tenant_override_config", LOKI_PATH) + overrides = read(LOKI_OVERRIDES_PATH) + assert_contains(overrides, "__default__", LOKI_OVERRIDES_PATH) + assert_contains(overrides, "retention_period", LOKI_OVERRIDES_PATH) + + +def main() -> int: + try: + validate_prometheus() + validate_tempo() + validate_loki() + except (AssertionError, FileNotFoundError) as exc: + print(f"[❌] telemetry storage validation failed: {exc}", file=sys.stderr) + return 1 + + print("[✓] telemetry storage configuration meets multi-tenant guard rails.") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/ops/licensing/TASKS.completed.md b/ops/licensing/TASKS.completed.md new file mode 100644 index 00000000..dc172422 --- /dev/null +++ b/ops/licensing/TASKS.completed.md @@ -0,0 +1,5 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| DEVOPS-LIC-14-004 | DONE (2025-10-26) | Licensing Guild | AUTH-MTLS-11-002 | Implement registry token service tied to Authority (DPoP/mTLS), plan gating, revocation handling, and monitoring per architecture. | Token service issues scoped tokens, revocation tested, monitoring dashboards in place, docs updated. | diff --git a/ops/licensing/TASKS.md b/ops/licensing/TASKS.md index 08d7b4fb..b4529e55 100644 --- a/ops/licensing/TASKS.md +++ b/ops/licensing/TASKS.md @@ -1,5 +1,4 @@ -# Licensing Task Board - -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| DEVOPS-LIC-14-004 | DONE (2025-10-26) | Licensing Guild | AUTH-MTLS-11-002 | Implement registry token service tied to Authority (DPoP/mTLS), plan gating, revocation handling, and monitoring per architecture. | Token service issues scoped tokens, revocation tested, monitoring dashboards in place, docs updated. | +# Licensing Task Board + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| diff --git a/ops/offline-kit/TASKS.completed.md b/ops/offline-kit/TASKS.completed.md new file mode 100644 index 00000000..9f8ee808 --- /dev/null +++ b/ops/offline-kit/TASKS.completed.md @@ -0,0 +1,8 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| DEVOPS-OFFLINE-14-002 | DONE (2025-10-26) | Offline Kit Guild | DEVOPS-REL-14-001 | Build offline kit packaging workflow (artifact bundling, manifest generation, signature verification). | Offline tarball generated with manifest + checksums + signatures; `ops/offline-kit/run-python-analyzer-smoke.sh` invoked as part of packaging; `debug/.build-id` tree mirrored from release output; import script verifies integrity; docs updated. | +| DEVOPS-OFFLINE-18-004 | DONE (2025-10-22) | Offline Kit Guild, Scanner Guild | DEVOPS-OFFLINE-18-003, SCANNER-ANALYZERS-LANG-10-309G | Rebuild Offline Kit bundle with Go analyzer plug-in and updated manifest/signature set. | Kit tarball includes Go analyzer artifacts; manifest/signature refreshed; verification steps executed and logged; docs updated with new bundle version. | +| DEVOPS-OFFLINE-18-005 | DONE (2025-10-26) | Offline Kit Guild, Scanner Guild | DEVOPS-REL-14-004, SCANNER-ANALYZERS-LANG-10-309P | Repackage Offline Kit with Python analyzer plug-in artefacts and refreshed manifest/signature set. | Kit tarball includes Python analyzer DLL/PDB/manifest; signature + manifest updated; Offline Kit guide references Python coverage; smoke import validated. | +| DEVOPS-OFFLINE-17-003 | DONE (2025-10-26) | Offline Kit Guild, DevOps Guild | DEVOPS-REL-17-002 | Mirror release debug-store artefacts ( `.build-id/` tree and `debug-manifest.json`) into Offline Kit packaging and document import validation. | Offline kit archives `debug/.build-id/` with manifest/sha256, docs cover symbol lookup workflow, smoke job confirms build-id lookup succeeds on air-gapped install. | diff --git a/ops/offline-kit/TASKS.md b/ops/offline-kit/TASKS.md index f3c85a8e..025fb06f 100644 --- a/ops/offline-kit/TASKS.md +++ b/ops/offline-kit/TASKS.md @@ -1,14 +1,10 @@ -# Offline Kit Task Board - +# Offline Kit Task Board + | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| DEVOPS-OFFLINE-14-002 | DONE (2025-10-26) | Offline Kit Guild | DEVOPS-REL-14-001 | Build offline kit packaging workflow (artifact bundling, manifest generation, signature verification). | Offline tarball generated with manifest + checksums + signatures; `ops/offline-kit/run-python-analyzer-smoke.sh` invoked as part of packaging; `debug/.build-id` tree mirrored from release output; import script verifies integrity; docs updated. | -| DEVOPS-OFFLINE-18-004 | DONE (2025-10-22) | Offline Kit Guild, Scanner Guild | DEVOPS-OFFLINE-18-003, SCANNER-ANALYZERS-LANG-10-309G | Rebuild Offline Kit bundle with Go analyzer plug-in and updated manifest/signature set. | Kit tarball includes Go analyzer artifacts; manifest/signature refreshed; verification steps executed and logged; docs updated with new bundle version. | -| DEVOPS-OFFLINE-18-005 | DONE (2025-10-26) | Offline Kit Guild, Scanner Guild | DEVOPS-REL-14-004, SCANNER-ANALYZERS-LANG-10-309P | Repackage Offline Kit with Python analyzer plug-in artefacts and refreshed manifest/signature set. | Kit tarball includes Python analyzer DLL/PDB/manifest; signature + manifest updated; Offline Kit guide references Python coverage; smoke import validated. | | DEVOPS-OFFLINE-34-006 | TODO | Offline Kit Guild, Orchestrator Service Guild | ORCH-SVC-34-004, DEPLOY-ORCH-34-001 | Bundle orchestrator service container, worker SDK samples, Postgres snapshot, and dashboards into Offline Kit with manifest/signature updates. | Offline kit contains orchestrator assets; manifest/signature validated; docs updated with air-gapped install steps; smoke import executed. | | DEVOPS-OFFLINE-37-001 | TODO | Offline Kit Guild, Exporter Service Guild | EXPORT-SVC-37-001..004, DEPLOY-EXPORT-36-001 | Export Center offline bundles + verification tooling (mirror artefacts, verification CLI, manifest/signature refresh, air-gap import script). | Offline kit includes export bundles/tools; verification script passes; manifest/signature updated; docs detail import workflow. | | DEVOPS-OFFLINE-37-002 | TODO | Offline Kit Guild, Notifications Service Guild | NOTIFY-SVC-40-001..004, WEB-NOTIFY-40-001 | Notifier offline packs (sample configs, template/digest packs, dry-run harness) with integrity checks and operator docs. | Offline kit ships notifier assets with checksums; dry-run harness validated; docs outline sealed/connected install steps. | | CLI-PACKS-43-002 | TODO | Offline Kit Guild, Packs Registry Guild | PACKS-REG-42-001, DEPLOY-PACKS-43-001 | Bundle Task Pack samples, registry mirror seeds, Task Runner configs, and CLI binaries with checksums into Offline Kit. | Offline kit includes packs registry mirror, Task Runner configs, CLI binaries; manifest/signature updated; docs describe air-gapped execution. | | OFFLINE-CONTAINERS-46-001 | TODO | Offline Kit Guild, Deployment Guild | DEVOPS-CONTAINERS-46-001, DEPLOY-AIRGAP-46-001 | Include container air-gap bundle, verification docs, and mirrored registry instructions inside Offline Kit. | Offline kit ships bundle + how-to; verification steps validated; manifest/signature updated; imposed rule noted. | -| DEVOPS-OFFLINE-17-003 | DONE (2025-10-26) | Offline Kit Guild, DevOps Guild | DEVOPS-REL-17-002 | Mirror release debug-store artefacts ( `.build-id/` tree and `debug-manifest.json`) into Offline Kit packaging and document import validation. | Offline kit archives `debug/.build-id/` with manifest/sha256, docs cover symbol lookup workflow, smoke job confirms build-id lookup succeeds on air-gapped install. | | DEVOPS-OFFLINE-17-004 | BLOCKED (2025-10-26) | Offline Kit Guild, DevOps Guild | DEVOPS-REL-17-002 | Execute `mirror_debug_store.py` after the next release pipeline emits `out/release/debug`, verify manifest hashes, and archive `metadata/debug-store.json` with the kit. | Debug store mirrored post-release, manifest SHA validated, summary committed alongside Offline Kit bundle evidence. ⏳ Blocked until the release pipeline publishes the next `out/release/debug` tree; rerun the mirroring script as part of that pipeline. | diff --git a/samples/TASKS.completed.md b/samples/TASKS.completed.md new file mode 100644 index 00000000..a12d1c68 --- /dev/null +++ b/samples/TASKS.completed.md @@ -0,0 +1,8 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| SAMPLES-10-001 | DONE | Samples Guild, Scanner Team | SCANNER-EMIT-10-605 | Curate sample images (nginx, alpine+busybox, distroless+go, .NET AOT, python venv, npm monorepo) with expected SBOM/BOM-Index sidecars. | Samples committed under `samples/`; golden SBOM/BOM-Index files present; documented usage. | +| SAMPLES-13-004 | DONE (2025-10-23) | Samples Guild, Policy Guild | POLICY-CORE-09-006, UI-POLICY-13-007 | Add policy preview/report fixtures showing confidence bands and unknown-age tags. | Confidence sample (`samples/policy/policy-preview-unknown.json`) reviewed, documented usage in UI dev guide, ajv validation hook updated. | +| SAMPLES-POLICY-20-001 | DONE (2025-10-26) | Samples Guild, Policy Guild | POLICY-ENGINE-20-002, DOCS-POLICY-20-011 | Create sample policies (`baseline.pol`, `serverless.pol`, `internal-only.pol`) with annotated SBOM/advisory fixtures. | Samples stored under `samples/policy/`; README documents usage; tests validate deterministic outputs. | +| SAMPLES-POLICY-20-002 | DONE (2025-10-26) | Samples Guild, UI Guild | UI-POLICY-20-002 | Produce simulation diff fixtures (before/after JSON) for UI/CLI tests. | Fixtures committed with schema validation; referenced by UI+CLI tests; docs cross-link. | diff --git a/samples/TASKS.md b/samples/TASKS.md index 133408a5..a328e449 100644 --- a/samples/TASKS.md +++ b/samples/TASKS.md @@ -1,16 +1,12 @@ -# Samples Task Board - -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| SAMPLES-10-001 | DONE | Samples Guild, Scanner Team | SCANNER-EMIT-10-605 | Curate sample images (nginx, alpine+busybox, distroless+go, .NET AOT, python venv, npm monorepo) with expected SBOM/BOM-Index sidecars. | Samples committed under `samples/`; golden SBOM/BOM-Index files present; documented usage. | -| SAMPLES-13-004 | DONE (2025-10-23) | Samples Guild, Policy Guild | POLICY-CORE-09-006, UI-POLICY-13-007 | Add policy preview/report fixtures showing confidence bands and unknown-age tags. | Confidence sample (`samples/policy/policy-preview-unknown.json`) reviewed, documented usage in UI dev guide, ajv validation hook updated. | +# Samples Task Board + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| ## Policy Engine v2 | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| SAMPLES-POLICY-20-001 | DONE (2025-10-26) | Samples Guild, Policy Guild | POLICY-ENGINE-20-002, DOCS-POLICY-20-011 | Create sample policies (`baseline.pol`, `serverless.pol`, `internal-only.pol`) with annotated SBOM/advisory fixtures. | Samples stored under `samples/policy/`; README documents usage; tests validate deterministic outputs. | -| SAMPLES-POLICY-20-002 | DONE (2025-10-26) | Samples Guild, UI Guild | UI-POLICY-20-002 | Produce simulation diff fixtures (before/after JSON) for UI/CLI tests. | Fixtures committed with schema validation; referenced by UI+CLI tests; docs cross-link. | ## Graph Explorer v1 diff --git a/samples/api/reports/report-sample.dsse.json b/samples/api/reports/report-sample.dsse.json index b56bf2c5..28c8bb9e 100644 --- a/samples/api/reports/report-sample.dsse.json +++ b/samples/api/reports/report-sample.dsse.json @@ -1,58 +1,11 @@ -{ - "report": { - "reportId": "report-3def5f362aa475ef14b6", - "imageDigest": "sha256:deadbeef", - "generatedAt": "2025-10-19T08:28:09.3699267+00:00", - "verdict": "blocked", - "policy": { - "revisionId": "rev-1", - "digest": "27d2ec2b34feedc304fc564d252ecee1c8fa14ea581a5ff5c1ea8963313d5c8d" - }, - "summary": { - "total": 1, - "blocked": 1, - "warned": 0, - "ignored": 0, - "quieted": 1 - }, - "verdicts": [ - { - "findingId": "finding-1", - "status": "Blocked", - "ruleName": "Block Critical", - "ruleAction": "Block", - "score": 40.5, - "configVersion": "1.0", - "inputs": { - "reachabilityWeight": 0.45, - "baseScore": 40.5, - "severityWeight": 90, - "trustWeight": 1, - "trustWeight.NVD": 1, - "reachability.runtime": 0.45, - "unknownConfidence": 0.52, - "unknownAgeDays": 4 - }, - "quietedBy": "policy/quiet-critical-runtime", - "quiet": true, - "unknownConfidence": 0.52, - "confidenceBand": "medium", - "unknownAgeDays": 4, - "sourceTrust": "NVD", - "reachability": "runtime" - } - ], - "issues": [] - }, - "dsse": { - "payloadType": "application/vnd.stellaops.report+json", - "payload": "eyJyZXBvcnRJZCI6InJlcG9ydC0zZGVmNWYzNjJhYTQ3NWVmMTRiNiIsImltYWdlRGlnZXN0Ijoic2hhMjU2OmRlYWRiZWVmIiwiZ2VuZXJhdGVkQXQiOiIyMDI1LTEwLTE5VDA4OjI4OjA5LjM2OTkyNjcrMDA6MDAiLCJ2ZXJkaWN0IjoiYmxvY2tlZCIsInBvbGljeSI6eyJyZXZpc2lvbklkIjoicmV2LTEiLCJkaWdlc3QiOiIyN2QyZWMyYjM0ZmVlZGMzMDRmYzU2NGQyNTJlY2VlMWM4ZmExNGVhNTgxYTVmZjVjMWVhODk2MzMxM2Q1YzhkIn0sInN1bW1hcnkiOnsidG90YWwiOjEsImJsb2NrZWQiOjEsIndhcm5lZCI6MCwiaWdub3JlZCI6MCwicXVpZXRlZCI6MX0sInZlcmRpY3RzIjpbeyJmaW5kaW5nSWQiOiJmaW5kaW5nLTEiLCJzdGF0dXMiOiJCbG9ja2VkIiwicnVsZU5hbWUiOiJCbG9jayBDcml0aWNhbCIsInJ1bGVBY3Rpb24iOiJCbG9jayIsInNjb3JlIjo0MC41LCJjb25maWdWZXJzaW9uIjoiMS4wIiwiaW5wdXRzIjp7InJlYWNoYWJpbGl0eVdlaWdodCI6MC40NSwiYmFzZVNjb3JlIjo0MC41LCJzZXZlcml0eVdlaWdodCI6OTAsInRydXN0V2VpZ2h0IjoxLCJ0cnVzdFdlaWdodC5OVkQiOjEsInJlYWNoYWJpbGl0eS5ydW50aW1lIjowLjQ1LCJ1bmtub3duQ29uZmlkZW5jZSI6MC41MiwidW5rbm93bkFnZURheXMiOjR9LCJxdWlldGVkQnkiOiJwb2xpY3kvcXVpZXQtY3JpdGljYWwtcnVudGltZSIsInF1aWV0Ijp0cnVlLCJ1bmtub3duQ29uZmlkZW5jZSI6MC41MiwiY29uZmlkZW5jZUJhbmQiOiJtZWRpdW0iLCJ1bmtub3duQWdlRGF5cyI6NCwic291cmNlVHJ1c3QiOiJOVkQiLCJyZWFjaGFiaWxpdHkiOiJydW50aW1lIn1dLCJpc3N1ZXMiOltdfQ==", - "signatures": [ - { - "keyId": "scanner-report-signing", - "algorithm": "hs256", - "signature": "s3qnWeRsYs+QA/nO84Us8G2xjZcvphc2P7KnOdTVwQs=" - } - ] - } -} +{ + "payloadType": "application/vnd.stellaops.report+json", + "payload": "eyJyZXBvcnRJZCI6InJlcG9ydC1hYmMiLCJpbWFnZURpZ2VzdCI6InNoYTI1NjpmZWVkZmFjZSIsImdlbmVyYXRlZEF0IjoiMjAyNS0xMC0xOVQxMjozNDo1NiswMDowMCIsInZlcmRpY3QiOiJibG9ja2VkIiwicG9saWN5Ijp7InJldmlzaW9uSWQiOiJyZXYtNDIiLCJkaWdlc3QiOiJkaWdlc3QtMTIzIn0sInN1bW1hcnkiOnsidG90YWwiOjEsImJsb2NrZWQiOjEsIndhcm5lZCI6MCwiaWdub3JlZCI6MCwicXVpZXRlZCI6MH0sInZlcmRpY3RzIjpbeyJmaW5kaW5nSWQiOiJmaW5kaW5nLTEiLCJzdGF0dXMiOiJCbG9ja2VkIiwic2NvcmUiOjQ3LjUsInNvdXJjZVRydXN0IjoiTlZEIiwicmVhY2hhYmlsaXR5IjoicnVudGltZSJ9XSwiaXNzdWVzIjpbXX0=", + "signatures": [ + { + "keyId": "test-key", + "algorithm": "hs256", + "signature": "signature-value" + } + ] +} diff --git a/src/Aoc/__Libraries/StellaOps.Aoc.AspNetCore/Results/AocHttpResults.cs b/src/Aoc/__Libraries/StellaOps.Aoc.AspNetCore/Results/AocHttpResults.cs new file mode 100644 index 00000000..e297e2f2 --- /dev/null +++ b/src/Aoc/__Libraries/StellaOps.Aoc.AspNetCore/Results/AocHttpResults.cs @@ -0,0 +1,95 @@ +using System; +using System.Collections.Generic; +using System.Linq; +using Microsoft.AspNetCore.Http; +using HttpResults = Microsoft.AspNetCore.Http.Results; +using StellaOps.Aoc; + +namespace StellaOps.Aoc.AspNetCore.Results; + +/// +/// Helpers for emitting Aggregation-Only Contract error responses. +/// +public static class AocHttpResults +{ + private const string DefaultProblemType = "https://stella-ops.org/problems/aoc-violation"; + + /// + /// Converts an into a RFC 7807 problem response. + /// + /// The current HTTP context. + /// The guard exception. + /// Optional problem title. + /// Optional problem detail. + /// Optional problem type URI. + /// Optional HTTP status code override. + /// Optional extension payload merged with guard details. + /// An HTTP result representing the problem response. + public static IResult Problem( + HttpContext httpContext, + AocGuardException exception, + string? title = null, + string? detail = null, + string? type = null, + int? status = null, + IDictionary? extensions = null) + { + if (httpContext is null) + { + throw new ArgumentNullException(nameof(httpContext)); + } + + if (exception is null) + { + throw new ArgumentNullException(nameof(exception)); + } + + var primaryCode = exception.Result.Violations.IsDefaultOrEmpty + ? "ERR_AOC_000" + : exception.Result.Violations[0].ErrorCode; + + var violationPayload = exception.Result.Violations + .Select(v => new Dictionary(StringComparer.Ordinal) + { + ["code"] = v.ErrorCode, + ["path"] = v.Path, + ["message"] = v.Message + }) + .ToArray(); + + var extensionPayload = new Dictionary(StringComparer.Ordinal) + { + ["code"] = primaryCode, + ["violations"] = violationPayload + }; + + if (extensions is not null) + { + foreach (var kvp in extensions) + { + extensionPayload[kvp.Key] = kvp.Value; + } + } + + var statusCode = status ?? MapErrorCodeToStatus(primaryCode); + var problemType = type ?? DefaultProblemType; + var problemDetail = detail ?? $"AOC guard rejected the request with {primaryCode}."; + var problemTitle = title ?? "Aggregation-Only Contract violation"; + + return HttpResults.Problem( + statusCode: statusCode, + title: problemTitle, + detail: problemDetail, + type: problemType, + extensions: extensionPayload); + } + + private static int MapErrorCodeToStatus(string errorCode) => errorCode switch + { + "ERR_AOC_003" => StatusCodes.Status409Conflict, + "ERR_AOC_004" => StatusCodes.Status422UnprocessableEntity, + "ERR_AOC_005" => StatusCodes.Status422UnprocessableEntity, + "ERR_AOC_006" => StatusCodes.Status403Forbidden, + _ => StatusCodes.Status400BadRequest, + }; +} diff --git a/src/Aoc/__Libraries/StellaOps.Aoc.AspNetCore/Routing/AocGuardEndpointFilter.cs b/src/Aoc/__Libraries/StellaOps.Aoc.AspNetCore/Routing/AocGuardEndpointFilter.cs new file mode 100644 index 00000000..0d4a5527 --- /dev/null +++ b/src/Aoc/__Libraries/StellaOps.Aoc.AspNetCore/Routing/AocGuardEndpointFilter.cs @@ -0,0 +1,91 @@ +using System; +using System.Collections.Generic; +using System.Text.Json; +using System.Threading.Tasks; +using Microsoft.AspNetCore.Http; +using Microsoft.AspNetCore.Routing; +using Microsoft.Extensions.DependencyInjection; +using Microsoft.Extensions.Options; +using StellaOps.Aoc; + +namespace StellaOps.Aoc.AspNetCore.Routing; + +public sealed class AocGuardEndpointFilter : IEndpointFilter +{ + private readonly Func> _payloadSelector; + private readonly JsonSerializerOptions _serializerOptions; + private readonly AocGuardOptions? _guardOptions; + + public AocGuardEndpointFilter( + Func> payloadSelector, + JsonSerializerOptions? serializerOptions, + AocGuardOptions? guardOptions) + { + _payloadSelector = payloadSelector ?? throw new ArgumentNullException(nameof(payloadSelector)); + _serializerOptions = serializerOptions ?? new JsonSerializerOptions(JsonSerializerDefaults.Web); + _guardOptions = guardOptions; + } + + public async ValueTask InvokeAsync(EndpointFilterInvocationContext context, EndpointFilterDelegate next) + { + if (context is null) + { + throw new ArgumentNullException(nameof(context)); + } + + if (TryGetArgument(context, out var request)) + { + var payloads = _payloadSelector(request); + if (payloads is not null) + { + var guard = context.HttpContext.RequestServices.GetRequiredService(); + var options = ResolveOptions(context.HttpContext.RequestServices); + + foreach (var payload in payloads) + { + if (payload is null) + { + continue; + } + + JsonElement element = payload switch + { + JsonElement jsonElement => jsonElement, + JsonDocument jsonDocument => jsonDocument.RootElement, + _ => JsonSerializer.SerializeToElement(payload, _serializerOptions) + }; + + guard.ValidateOrThrow(element, options); + } + } + } + + return await next(context).ConfigureAwait(false); + } + + private AocGuardOptions ResolveOptions(IServiceProvider services) + { + if (_guardOptions is not null) + { + return _guardOptions; + } + + var options = services.GetService>(); + return options?.Value ?? AocGuardOptions.Default; + } + + private static bool TryGetArgument(EndpointFilterInvocationContext context, out TRequest argument) + { + for (var i = 0; i < context.Arguments.Count; i++) + { + if (context.Arguments[i] is TRequest typedArgument) + { + argument = typedArgument; + return true; + } + } + + argument = default!; + return false; + } +} diff --git a/src/Aoc/__Libraries/StellaOps.Aoc.AspNetCore/StellaOps.Aoc.AspNetCore.csproj b/src/Aoc/__Libraries/StellaOps.Aoc.AspNetCore/StellaOps.Aoc.AspNetCore.csproj new file mode 100644 index 00000000..cafaf803 --- /dev/null +++ b/src/Aoc/__Libraries/StellaOps.Aoc.AspNetCore/StellaOps.Aoc.AspNetCore.csproj @@ -0,0 +1,14 @@ + + + + net10.0 + enable + enable + + + + + + + + diff --git a/src/Aoc/__Tests/StellaOps.Aoc.AspNetCore.Tests/AocHttpResultsTests.cs b/src/Aoc/__Tests/StellaOps.Aoc.AspNetCore.Tests/AocHttpResultsTests.cs new file mode 100644 index 00000000..837a4823 --- /dev/null +++ b/src/Aoc/__Tests/StellaOps.Aoc.AspNetCore.Tests/AocHttpResultsTests.cs @@ -0,0 +1,49 @@ +using System.Collections.Immutable; +using System.IO; +using System.Text.Json; +using System.Threading.Tasks; +using Microsoft.AspNetCore.Http; +using Microsoft.Extensions.DependencyInjection; +using StellaOps.Aoc; +using StellaOps.Aoc.AspNetCore.Results; + +namespace StellaOps.Aoc.AspNetCore.Tests; + +public sealed class AocHttpResultsTests +{ + [Fact] + public async Task Problem_WritesProblemDetails_WithGuardViolations() + { + // Arrange + var violations = ImmutableArray.Create( + AocViolation.Create(AocViolationCode.MissingProvenance, "/upstream", "Missing upstream"), + AocViolation.Create(AocViolationCode.ForbiddenField, "/severity", "Forbidden")); + var result = AocGuardResult.FromViolations(violations); + var exception = new AocGuardException(result); + + var context = new DefaultHttpContext(); + context.Response.Body = new MemoryStream(); + var services = new ServiceCollection(); + services.AddLogging(); + services.AddProblemDetails(); + context.RequestServices = services.BuildServiceProvider(); + + // Act + var problem = AocHttpResults.Problem(context, exception); + await problem.ExecuteAsync(context); + context.Response.Body.Seek(0, SeekOrigin.Begin); + + using var document = await JsonDocument.ParseAsync(context.Response.Body, cancellationToken: TestContext.Current.CancellationToken); + var root = document.RootElement; + + // Assert + Assert.Equal(StatusCodes.Status422UnprocessableEntity, context.Response.StatusCode); + Assert.Equal("Aggregation-Only Contract violation", root.GetProperty("title").GetString()); + Assert.Equal("ERR_AOC_004", root.GetProperty("code").GetString()); + + var violationsJson = root.GetProperty("violations"); + Assert.Equal(2, violationsJson.GetArrayLength()); + Assert.Equal("ERR_AOC_004", violationsJson[0].GetProperty("code").GetString()); + Assert.Equal("/upstream", violationsJson[0].GetProperty("path").GetString()); + } +} diff --git a/src/Aoc/__Tests/StellaOps.Aoc.AspNetCore.Tests/StellaOps.Aoc.AspNetCore.Tests.csproj b/src/Aoc/__Tests/StellaOps.Aoc.AspNetCore.Tests/StellaOps.Aoc.AspNetCore.Tests.csproj new file mode 100644 index 00000000..c2711aa0 --- /dev/null +++ b/src/Aoc/__Tests/StellaOps.Aoc.AspNetCore.Tests/StellaOps.Aoc.AspNetCore.Tests.csproj @@ -0,0 +1,32 @@ + + + + + net10.0 + preview + enable + enable + true + false + false + + + + + + + + + + + + + + + + + + + + + diff --git a/src/Attestor/StellaOps.Attestor/TASKS.completed.md b/src/Attestor/StellaOps.Attestor/TASKS.completed.md new file mode 100644 index 00000000..289bbf43 --- /dev/null +++ b/src/Attestor/StellaOps.Attestor/TASKS.completed.md @@ -0,0 +1,7 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| ATTESTOR-API-11-201 | DONE (2025-10-19) | Attestor Guild | — | `/rekor/entries` submission pipeline with dedupe, proof acquisition, and persistence. | ✅ `POST /api/v1/rekor/entries` enforces mTLS + Authority OpTok, validates DSSE bundles, and handles dual-log preferences.
✅ Redis/Mongo idempotency returns existing UUID on duplicate `bundleSha256` without re-submitting to Rekor.
✅ Rekor driver fetches inclusion proofs (or schedules async fetch) and persists canonical entry/proof metadata.
✅ Optional archive path stores DSSE/proof bundles to MinIO/S3; integration tests cover success/pending/error flows. | +| ATTESTOR-VERIFY-11-202 | DONE (2025-10-19) | Attestor Guild | — | `/rekor/verify` + retrieval endpoints validating signatures and Merkle proofs. | ✅ `GET /api/v1/rekor/entries/{uuid}` surfaces cached entries with optional backend refresh and handles not-found/refresh flows.
✅ `POST /api/v1/rekor/verify` accepts UUID, bundle, or artifact hash inputs; verifies DSSE signatures, Merkle proofs, and checkpoint anchors.
✅ Verification output returns `{ok, uuid, index, logURL, checkedAt}` with failure diagnostics for invalid proofs.
✅ Unit/integration tests exercise cache hits, backend refresh, invalid bundle/proof scenarios, and checkpoint trust anchor enforcement. | +| ATTESTOR-OBS-11-203 | DONE (2025-10-19) | Attestor Guild | — | Telemetry, alerting, mTLS hardening, and archive workflow for Attestor. | ✅ Structured logs, metrics, and optional traces record submission latency, proof fetch outcomes, verification results, and Rekor error buckets with correlation IDs.
✅ mTLS enforcement hardened (peer allowlist, SAN checks, rate limiting) and documented; TLS settings audited for modern ciphers only.
✅ Alerting/dashboard pack covers error rates, proof backlog, Redis/Mongo health, and archive job failures; runbook updated.
✅ Archive workflow includes retention policy jobs, failure alerts, and periodic verification of stored bundles and proofs. | diff --git a/src/Attestor/StellaOps.Attestor/TASKS.md b/src/Attestor/StellaOps.Attestor/TASKS.md index 8ba43acc..58dbf5a2 100644 --- a/src/Attestor/StellaOps.Attestor/TASKS.md +++ b/src/Attestor/StellaOps.Attestor/TASKS.md @@ -1,11 +1,8 @@ -# Attestor Guild Task Board (UTC 2025-10-19) - -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| ATTESTOR-API-11-201 | DONE (2025-10-19) | Attestor Guild | — | `/rekor/entries` submission pipeline with dedupe, proof acquisition, and persistence. | ✅ `POST /api/v1/rekor/entries` enforces mTLS + Authority OpTok, validates DSSE bundles, and handles dual-log preferences.
✅ Redis/Mongo idempotency returns existing UUID on duplicate `bundleSha256` without re-submitting to Rekor.
✅ Rekor driver fetches inclusion proofs (or schedules async fetch) and persists canonical entry/proof metadata.
✅ Optional archive path stores DSSE/proof bundles to MinIO/S3; integration tests cover success/pending/error flows. | -| ATTESTOR-VERIFY-11-202 | DONE (2025-10-19) | Attestor Guild | — | `/rekor/verify` + retrieval endpoints validating signatures and Merkle proofs. | ✅ `GET /api/v1/rekor/entries/{uuid}` surfaces cached entries with optional backend refresh and handles not-found/refresh flows.
✅ `POST /api/v1/rekor/verify` accepts UUID, bundle, or artifact hash inputs; verifies DSSE signatures, Merkle proofs, and checkpoint anchors.
✅ Verification output returns `{ok, uuid, index, logURL, checkedAt}` with failure diagnostics for invalid proofs.
✅ Unit/integration tests exercise cache hits, backend refresh, invalid bundle/proof scenarios, and checkpoint trust anchor enforcement. | -| ATTESTOR-OBS-11-203 | DONE (2025-10-19) | Attestor Guild | — | Telemetry, alerting, mTLS hardening, and archive workflow for Attestor. | ✅ Structured logs, metrics, and optional traces record submission latency, proof fetch outcomes, verification results, and Rekor error buckets with correlation IDs.
✅ mTLS enforcement hardened (peer allowlist, SAN checks, rate limiting) and documented; TLS settings audited for modern ciphers only.
✅ Alerting/dashboard pack covers error rates, proof backlog, Redis/Mongo health, and archive job failures; runbook updated.
✅ Archive workflow includes retention policy jobs, failure alerts, and periodic verification of stored bundles and proofs. | - +# Attestor Guild Task Board (UTC 2025-10-19) + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| + > Remark (2025-10-19): Wave 0 prerequisites reviewed (none outstanding); ATTESTOR-API-11-201, ATTESTOR-VERIFY-11-202, and ATTESTOR-OBS-11-203 tracked as DOING per Wave 0A kickoff. > Remark (2025-10-19): Dual-log submissions, signature/proof verification, and observability hardening landed; attestor endpoints now rate-limited per client with correlation-ID logging and updated docs/tests. diff --git a/src/Authority/StellaOps.Api.OpenApi/authority/openapi.yaml b/src/Authority/StellaOps.Api.OpenApi/authority/openapi.yaml new file mode 100644 index 00000000..6557f13b --- /dev/null +++ b/src/Authority/StellaOps.Api.OpenApi/authority/openapi.yaml @@ -0,0 +1,689 @@ +openapi: 3.1.0 +info: + title: StellaOps Authority Authentication API + summary: Token issuance, introspection, revocation, and key discovery endpoints exposed by the Authority service. + description: | + The Authority service issues OAuth 2.1 access tokens for StellaOps components, enforcing tenant and scope + restrictions configured per client. This specification describes the authentication surface only; domain APIs + are documented by their owning services. + version: 0.1.0 +jsonSchemaDialect: https://json-schema.org/draft/2020-12/schema +servers: + - url: https://authority.stellaops.local + description: Example Authority deployment +tags: + - name: Authentication + description: OAuth 2.1 token exchange, introspection, and revocation flows. + - name: Keys + description: JSON Web Key Set discovery. +components: + securitySchemes: + ClientSecretBasic: + type: http + scheme: basic + description: HTTP Basic authentication with `client_id` and `client_secret`. + OAuthPassword: + type: oauth2 + description: Resource owner password exchange for Authority-managed identities. + flows: + password: + tokenUrl: /token + refreshUrl: /token + scopes: + advisory:ingest: Submit advisory ingestion payloads. + advisory:read: Read advisory ingestion data. + aoc:verify: Execute Aggregation-Only Contract verification workflows. + authority.audit.read: Read Authority audit logs. + authority.clients.manage: Manage Authority client registrations. + authority.users.manage: Manage Authority users. + authority:tenants.read: Read the Authority tenant catalog. + concelier.jobs.trigger: Trigger Concelier aggregation jobs. + concelier.merge: Manage Concelier merge operations. + effective:write: Write effective findings (Policy Engine service identity only). + email: Access email claim data. + exceptions:approve: Approve exception workflows. + findings:read: Read effective findings emitted by Policy Engine. + graph:export: Export graph artefacts. + graph:read: Read graph explorer data. + graph:simulate: Run graph what-if simulations. + graph:write: Enqueue or mutate graph build jobs. + offline_access: Request refresh tokens for offline access. + openid: Request OpenID Connect identity tokens. + orch:operate: Execute privileged Orchestrator control actions. + orch:read: Read Orchestrator job state. + policy:author: Author Policy Studio drafts and workspaces. + policy:activate: Activate policy revisions. + policy:approve: Approve or reject policy drafts. + policy:audit: Inspect Policy Studio audit history. + policy:edit: Edit policy definitions. + policy:operate: Operate Policy Studio promotions and runs. + policy:read: Read policy definitions and metadata. + policy:run: Trigger policy executions. + policy:submit: Submit policy drafts for review. + policy:review: Review Policy Studio drafts and leave feedback. + policy:simulate: Execute Policy Studio simulations. + policy:write: Create or update policy drafts. + profile: Access profile claim data. + signals:admin: Administer Signals ingestion and routing settings. + signals:read: Read Signals events and state. + signals:write: Publish Signals events or mutate state. + stellaops.bypass: Bypass trust boundary protections (restricted identities only). + ui.read: Read Console UX resources. + vex:ingest: Submit VEX ingestion payloads. + vex:read: Read VEX ingestion data. + vuln:read: Read vulnerability permalinks and overlays. + authorizationCode: + authorizationUrl: /authorize + tokenUrl: /token + refreshUrl: /token + scopes: + advisory:ingest: Submit advisory ingestion payloads. + advisory:read: Read advisory ingestion data. + aoc:verify: Execute Aggregation-Only Contract verification workflows. + authority.audit.read: Read Authority audit logs. + authority.clients.manage: Manage Authority client registrations. + authority.users.manage: Manage Authority users. + authority:tenants.read: Read the Authority tenant catalog. + concelier.jobs.trigger: Trigger Concelier aggregation jobs. + concelier.merge: Manage Concelier merge operations. + effective:write: Write effective findings (Policy Engine service identity only). + email: Access email claim data. + exceptions:approve: Approve exception workflows. + findings:read: Read effective findings emitted by Policy Engine. + graph:export: Export graph artefacts. + graph:read: Read graph explorer data. + graph:simulate: Run graph what-if simulations. + graph:write: Enqueue or mutate graph build jobs. + offline_access: Request refresh tokens for offline access. + openid: Request OpenID Connect identity tokens. + orch:operate: Execute privileged Orchestrator control actions. + orch:read: Read Orchestrator job state. + policy:author: Author Policy Studio drafts and workspaces. + policy:activate: Activate policy revisions. + policy:approve: Approve or reject policy drafts. + policy:audit: Inspect Policy Studio audit history. + policy:edit: Edit policy definitions. + policy:operate: Operate Policy Studio promotions and runs. + policy:read: Read policy definitions and metadata. + policy:run: Trigger policy executions. + policy:submit: Submit policy drafts for review. + policy:review: Review Policy Studio drafts and leave feedback. + policy:simulate: Execute Policy Studio simulations. + policy:write: Create or update policy drafts. + profile: Access profile claim data. + signals:admin: Administer Signals ingestion and routing settings. + signals:read: Read Signals events and state. + signals:write: Publish Signals events or mutate state. + stellaops.bypass: Bypass trust boundary protections (restricted identities only). + ui.read: Read Console UX resources. + vex:ingest: Submit VEX ingestion payloads. + vex:read: Read VEX ingestion data. + vuln:read: Read vulnerability permalinks and overlays. + OAuthClientCredentials: + type: oauth2 + description: Client credential exchange for machine-to-machine identities. + flows: + clientCredentials: + tokenUrl: /token + scopes: + advisory:ingest: Submit advisory ingestion payloads. + advisory:read: Read advisory ingestion data. + aoc:verify: Execute Aggregation-Only Contract verification workflows. + authority.audit.read: Read Authority audit logs. + authority.clients.manage: Manage Authority client registrations. + authority.users.manage: Manage Authority users. + authority:tenants.read: Read the Authority tenant catalog. + concelier.jobs.trigger: Trigger Concelier aggregation jobs. + concelier.merge: Manage Concelier merge operations. + effective:write: Write effective findings (Policy Engine service identity only). + email: Access email claim data. + exceptions:approve: Approve exception workflows. + findings:read: Read effective findings emitted by Policy Engine. + graph:export: Export graph artefacts. + graph:read: Read graph explorer data. + graph:simulate: Run graph what-if simulations. + graph:write: Enqueue or mutate graph build jobs. + offline_access: Request refresh tokens for offline access. + openid: Request OpenID Connect identity tokens. + orch:operate: Execute privileged Orchestrator control actions. + orch:read: Read Orchestrator job state. + policy:author: Author Policy Studio drafts and workspaces. + policy:activate: Activate policy revisions. + policy:approve: Approve or reject policy drafts. + policy:audit: Inspect Policy Studio audit history. + policy:edit: Edit policy definitions. + policy:operate: Operate Policy Studio promotions and runs. + policy:read: Read policy definitions and metadata. + policy:run: Trigger policy executions. + policy:submit: Submit policy drafts for review. + policy:review: Review Policy Studio drafts and leave feedback. + policy:simulate: Execute Policy Studio simulations. + policy:write: Create or update policy drafts. + profile: Access profile claim data. + signals:admin: Administer Signals ingestion and routing settings. + signals:read: Read Signals events and state. + signals:write: Publish Signals events or mutate state. + stellaops.bypass: Bypass trust boundary protections (restricted identities only). + ui.read: Read Console UX resources. + vex:ingest: Submit VEX ingestion payloads. + vex:read: Read VEX ingestion data. + vuln:read: Read vulnerability permalinks and overlays. + schemas: + TokenResponse: + type: object + description: OAuth 2.1 bearer token response. + properties: + access_token: + type: string + description: Access token encoded as JWT. + token_type: + type: string + description: Token type indicator. Always `Bearer`. + expires_in: + type: integer + description: Lifetime of the access token, in seconds. + minimum: 1 + refresh_token: + type: string + description: Refresh token issued when the grant allows offline access. + scope: + type: string + description: Space-delimited scopes granted in the response. + id_token: + type: string + description: ID token issued for authorization-code flows. + required: + - access_token + - token_type + - expires_in + OAuthErrorResponse: + type: object + description: RFC 6749 compliant error envelope. + properties: + error: + type: string + description: Machine-readable error code. + error_description: + type: string + description: Human-readable error description. + error_uri: + type: string + format: uri + description: Link to documentation about the error. + required: + - error + PasswordGrantRequest: + type: object + required: + - grant_type + - client_id + - username + - password + properties: + grant_type: + type: string + const: password + client_id: + type: string + description: Registered client identifier. May also be supplied via HTTP Basic auth. + client_secret: + type: string + description: Client secret. Required for confidential clients when not using HTTP Basic auth. + scope: + type: string + description: Space-delimited scopes being requested. + username: + type: string + description: Resource owner username. + password: + type: string + description: Resource owner password. + authority_provider: + type: string + description: Optional identity provider hint. Required when multiple password-capable providers are registered. + description: Form-encoded payload for password grant exchange. + ClientCredentialsGrantRequest: + type: object + required: + - grant_type + - client_id + properties: + grant_type: + type: string + const: client_credentials + client_id: + type: string + description: Registered client identifier. May also be supplied via HTTP Basic auth. + client_secret: + type: string + description: Client secret. Required for confidential clients when not using HTTP Basic auth. + scope: + type: string + description: Space-delimited scopes being requested. + authority_provider: + type: string + description: Optional identity provider hint for plugin-backed clients. + operator_reason: + type: string + description: Required when requesting `orch:operate`; explains the operator action. + maxLength: 256 + operator_ticket: + type: string + description: Required when requesting `orch:operate`; tracks the external change ticket or incident. + maxLength: 128 + description: Form-encoded payload for client credentials exchange. + RefreshTokenGrantRequest: + type: object + required: + - grant_type + - refresh_token + properties: + grant_type: + type: string + const: refresh_token + client_id: + type: string + description: Registered client identifier. May also be supplied via HTTP Basic auth. + client_secret: + type: string + description: Client secret. Required for confidential clients when not using HTTP Basic auth. + refresh_token: + type: string + description: Previously issued refresh token. + scope: + type: string + description: Optional scope list to narrow the requested access. + description: Form-encoded payload for refresh token exchange. + RevocationRequest: + type: object + required: + - token + properties: + token: + type: string + description: Token value or token identifier to revoke. + token_type_hint: + type: string + description: Optional token type hint (`access_token` or `refresh_token`). + description: Form-encoded payload for token revocation. + IntrospectionRequest: + type: object + required: + - token + properties: + token: + type: string + description: Token value whose state should be introspected. + token_type_hint: + type: string + description: Optional token type hint (`access_token` or `refresh_token`). + description: Form-encoded payload for token introspection. + IntrospectionResponse: + type: object + description: Active token descriptor compliant with RFC 7662. + properties: + active: + type: boolean + description: Indicates whether the token is currently active. + scope: + type: string + description: Space-delimited list of scopes granted to the token. + client_id: + type: string + description: Client identifier associated with the token. + sub: + type: string + description: Subject identifier when the token represents an end-user. + username: + type: string + description: Preferred username associated with the subject. + token_type: + type: string + description: Type of the token (e.g., `Bearer`). + exp: + type: integer + description: Expiration timestamp (seconds since UNIX epoch). + iat: + type: integer + description: Issued-at timestamp (seconds since UNIX epoch). + nbf: + type: integer + description: Not-before timestamp (seconds since UNIX epoch). + aud: + type: array + description: Audience values associated with the token. + items: + type: string + iss: + type: string + description: Issuer identifier. + jti: + type: string + description: JWT identifier corresponding to the token. + tenant: + type: string + description: Tenant associated with the token, when assigned. + confirmation: + type: object + description: Sender-constrained confirmation data (e.g., mTLS thumbprint, DPoP JWK thumbprint). + required: + - active + JwksDocument: + type: object + description: JSON Web Key Set published by the Authority. + properties: + keys: + type: array + items: + $ref: '#/components/schemas/Jwk' + required: + - keys + Jwk: + type: object + description: Public key material for token signature validation. + properties: + kid: + type: string + description: Key identifier. + kty: + type: string + description: Key type (e.g., `EC`, `RSA`). + use: + type: string + description: Intended key use (`sig`). + alg: + type: string + description: Signing algorithm (e.g., `ES384`). + crv: + type: string + description: Elliptic curve identifier when applicable. + x: + type: string + description: X coordinate for EC keys. + y: + type: string + description: Y coordinate for EC keys. + status: + type: string + description: Operational status metadata for the key (e.g., `active`, `retiring`). +paths: + /token: + post: + tags: + - Authentication + summary: Exchange credentials for tokens + description: | + Issues OAuth 2.1 bearer tokens for StellaOps clients. Supports password, client credentials, + authorization-code, device, and refresh token grants. Confidential clients must authenticate using + HTTP Basic auth or `client_secret` form fields. + security: + - ClientSecretBasic: [] + - {} + requestBody: + required: true + content: + application/x-www-form-urlencoded: + schema: + oneOf: + - $ref: '#/components/schemas/PasswordGrantRequest' + - $ref: '#/components/schemas/ClientCredentialsGrantRequest' + - $ref: '#/components/schemas/RefreshTokenGrantRequest' + encoding: + authority_provider: + style: form + explode: false + examples: + passwordGrant: + summary: Password grant for tenant-scoped ingestion bot + value: + grant_type: password + client_id: ingest-cli + client_secret: s3cr3t + username: ingest-bot + password: pa55w0rd! + scope: advisory:ingest vex:ingest + authority_provider: primary-directory + authorizationCode: + summary: Authorization code exchange for Console UI session + value: + grant_type: authorization_code + client_id: console-ui + code: 2Lba1WtwPLfZ2b0Z9uPrsQ + redirect_uri: https://console.stellaops.local/auth/callback + code_verifier: g3ZnL91QJ6i4zO_86oI4CDnZ7gS0bSeK + clientCredentials: + summary: Client credentials exchange for Policy Engine + value: + grant_type: client_credentials + client_id: policy-engine + client_secret: 9c39f602-2f2b-4f29 + scope: effective:write findings:read + operator_reason: Deploying policy change 1234 + operator_ticket: CHG-004211 + refreshToken: + summary: Refresh token rotation for console session + value: + grant_type: refresh_token + client_id: console-ui + refresh_token: 0.rg9pVlsGzXE8Q + responses: + '200': + description: Token exchange succeeded. + content: + application/json: + schema: + $ref: '#/components/schemas/TokenResponse' + examples: + passwordGrant: + summary: Password grant success response + value: + access_token: eyJhbGciOiJFUzM4NCIsInR5cCI6IkpXVCJ9... + token_type: Bearer + expires_in: 3600 + refresh_token: OxGdVtZJ-mk49cFd38uRUw + scope: advisory:ingest vex:ingest + clientCredentials: + summary: Client credentials success response + value: + access_token: eyJhbGciOiJFUzM4NCIsInR5cCI6IkpXVCJ9... + token_type: Bearer + expires_in: 900 + scope: effective:write findings:read + authorizationCode: + summary: Authorization code success response + value: + access_token: eyJhbGciOiJFUzM4NCIsInR5cCI6IkpXVCJ9... + token_type: Bearer + expires_in: 900 + refresh_token: VxKpc9Vj9QjYV6gLrhQHTw + scope: ui.read authority:tenants.read + id_token: eyJhbGciOiJFUzM4NCIsImtpZCI6ImNvbnNvbGUifQ... + '400': + description: Malformed request, unsupported grant type, or invalid credentials. + content: + application/json: + schema: + $ref: '#/components/schemas/OAuthErrorResponse' + examples: + invalidProvider: + summary: Unknown identity provider hint + value: + error: invalid_request + error_description: "Unknown identity provider 'legacy-directory'." + invalidScope: + summary: Scope not permitted for client + value: + error: invalid_scope + error_description: Scope 'effective:write' is not permitted for this client. + '401': + description: Client authentication failed. + content: + application/json: + schema: + $ref: '#/components/schemas/OAuthErrorResponse' + examples: + badClientSecret: + summary: Invalid client secret + value: + error: invalid_client + error_description: Client authentication failed. + /revoke: + post: + tags: + - Authentication + summary: Revoke an access or refresh token + security: + - ClientSecretBasic: [] + requestBody: + required: true + content: + application/x-www-form-urlencoded: + schema: + $ref: '#/components/schemas/RevocationRequest' + examples: + revokeRefreshToken: + summary: Revoke refresh token after logout + value: + token: 0.rg9pVlsGzXE8Q + token_type_hint: refresh_token + responses: + '200': + description: Token revoked or already invalid. The response body is intentionally blank. + '400': + description: Malformed request. + content: + application/json: + schema: + $ref: '#/components/schemas/OAuthErrorResponse' + examples: + missingToken: + summary: Token parameter omitted + value: + error: invalid_request + error_description: The revocation request is missing the token parameter. + '401': + description: Client authentication failed. + content: + application/json: + schema: + $ref: '#/components/schemas/OAuthErrorResponse' + examples: + badClientSecret: + summary: Invalid client credentials + value: + error: invalid_client + error_description: Client authentication failed. + /introspect: + post: + tags: + - Authentication + summary: Introspect token state + description: Returns the active status and claims for a given token. Requires a privileged client. + security: + - ClientSecretBasic: [] + requestBody: + required: true + content: + application/x-www-form-urlencoded: + schema: + $ref: '#/components/schemas/IntrospectionRequest' + examples: + introspectToken: + summary: Validate an access token issued to Orchestrator + value: + token: eyJhbGciOiJFUzM4NCIsInR5cCI6IkpXVCJ9... + token_type_hint: access_token + responses: + '200': + description: Token state evaluated. + content: + application/json: + schema: + $ref: '#/components/schemas/IntrospectionResponse' + examples: + activeToken: + summary: Active token response + value: + active: true + scope: orch:operate orch:read + client_id: orch-control + sub: operator-7f12 + username: ops.engineer@tenant.example + token_type: Bearer + exp: 1761628800 + iat: 1761625200 + nbf: 1761625200 + iss: https://authority.stellaops.local + aud: + - https://orch.stellaops.local + jti: 01J8KYRAMG7FWBPRRV5XG20T7S + tenant: tenant-alpha + confirmation: + mtls_thumbprint: 079871b8c9a0f2e6 + inactiveToken: + summary: Revoked token response + value: + active: false + '400': + description: Malformed request. + content: + application/json: + schema: + $ref: '#/components/schemas/OAuthErrorResponse' + examples: + missingToken: + summary: Token missing + value: + error: invalid_request + error_description: token parameter is required. + '401': + description: Client authentication failed or client lacks introspection permission. + content: + application/json: + schema: + $ref: '#/components/schemas/OAuthErrorResponse' + examples: + unauthorizedClient: + summary: Client not allowed to introspect tokens + value: + error: invalid_client + error_description: Client authentication failed. + /jwks: + get: + tags: + - Keys + summary: Retrieve signing keys + description: Returns the JSON Web Key Set used to validate Authority-issued tokens. + responses: + '200': + description: JWKS document. + headers: + Cache-Control: + schema: + type: string + description: Standard caching headers apply; keys rotate infrequently. + content: + application/json: + schema: + $ref: '#/components/schemas/JwksDocument' + examples: + ecKeySet: + summary: EC signing keys + value: + keys: + - kid: auth-tokens-es384-202510 + kty: EC + use: sig + alg: ES384 + crv: P-384 + x: 7UchU5R77LtChrJx6uWg9mYjFvV6RIpSgZPDIj7d1q0 + y: v98nHe8a7mGZ9Fn1t4Jp9PTJv1ma35QPmhUrE4pH7H0 + status: active + - kid: auth-tokens-es384-202409 + kty: EC + use: sig + alg: ES384 + crv: P-384 + x: hjdKc0r8jvVHJ7S9mP0y0mU9bqN7v5PxS21SwclTzfc + y: yk6J3pz4TUpymN4mG-6th3dYvJ5N1lQvDK0PLuFv3Pg + status: retiring diff --git a/src/Authority/StellaOps.Authority/StellaOps.Auth.Abstractions/StellaOpsClaimTypes.cs b/src/Authority/StellaOps.Authority/StellaOps.Auth.Abstractions/StellaOpsClaimTypes.cs index 3fbe42eb..c3098b33 100644 --- a/src/Authority/StellaOps.Authority/StellaOps.Auth.Abstractions/StellaOpsClaimTypes.cs +++ b/src/Authority/StellaOps.Authority/StellaOps.Auth.Abstractions/StellaOpsClaimTypes.cs @@ -51,12 +51,22 @@ public static class StellaOpsClaimTypes public const string Audience = "aud"; /// - /// Identity provider hint for downstream services. - /// - public const string IdentityProvider = "stellaops:idp"; - - /// - /// Session identifier claim (sid). - /// - public const string SessionId = "sid"; -} + /// Identity provider hint for downstream services. + /// + public const string IdentityProvider = "stellaops:idp"; + + /// + /// Operator reason supplied when issuing orchestrator control tokens. + /// + public const string OperatorReason = "stellaops:operator_reason"; + + /// + /// Operator ticket supplied when issuing orchestrator control tokens. + /// + public const string OperatorTicket = "stellaops:operator_ticket"; + + /// + /// Session identifier claim (sid). + /// + public const string SessionId = "sid"; +} diff --git a/src/Authority/StellaOps.Authority/StellaOps.Authority.Plugin.Standard/TASKS.completed.md b/src/Authority/StellaOps.Authority/StellaOps.Authority.Plugin.Standard/TASKS.completed.md new file mode 100644 index 00000000..615ef1f4 --- /dev/null +++ b/src/Authority/StellaOps.Authority/StellaOps.Authority.Plugin.Standard/TASKS.completed.md @@ -0,0 +1,8 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| PLG6.DOC | DONE (2025-10-11) | BE-Auth Plugin, Docs Guild | PLG1–PLG5 | Final polish + diagrams for plugin developer guide (AUTHPLUG-DOCS-01-001). | Docs team delivers copy-edit + exported diagrams; PR merged. | +| SEC1.PLG | DONE (2025-10-11) | Security Guild, BE-Auth Plugin | SEC1.A (StellaOps.Cryptography) | Swap Standard plugin hashing to Argon2id via `StellaOps.Cryptography` abstractions; keep PBKDF2 verification for legacy. | ✅ `StandardUserCredentialStore` uses `ICryptoProvider` to hash/check; ✅ Transparent rehash on success; ✅ Unit tests cover tamper + legacy rehash. | +| SEC1.OPT | DONE (2025-10-11) | Security Guild | SEC1.PLG | Expose password hashing knobs in `StandardPluginOptions` (`memoryKiB`, `iterations`, `parallelism`, `algorithm`) with validation. | ✅ Options bound from YAML; ✅ Invalid configs throw; ✅ Docs include tuning guidance. | +| SEC4.PLG | DONE (2025-10-12) | Security Guild | SEC4.A (revocation schema) | Provide plugin hooks so revoked users/clients write reasons for revocation bundle export. | ✅ Revocation exporter consumes plugin data; ✅ Tests cover revoked user/client output. | diff --git a/src/Authority/StellaOps.Authority/StellaOps.Authority.Plugin.Standard/TASKS.md b/src/Authority/StellaOps.Authority/StellaOps.Authority.Plugin.Standard/TASKS.md index 20b9b35f..0e07cb6b 100644 --- a/src/Authority/StellaOps.Authority/StellaOps.Authority.Plugin.Standard/TASKS.md +++ b/src/Authority/StellaOps.Authority/StellaOps.Authority.Plugin.Standard/TASKS.md @@ -1,20 +1,16 @@ -# Team 8 / Plugin Standard Backlog (UTC 2025-10-10) - -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| PLG6.DOC | DONE (2025-10-11) | BE-Auth Plugin, Docs Guild | PLG1–PLG5 | Final polish + diagrams for plugin developer guide (AUTHPLUG-DOCS-01-001). | Docs team delivers copy-edit + exported diagrams; PR merged. | -| SEC1.PLG | DONE (2025-10-11) | Security Guild, BE-Auth Plugin | SEC1.A (StellaOps.Cryptography) | Swap Standard plugin hashing to Argon2id via `StellaOps.Cryptography` abstractions; keep PBKDF2 verification for legacy. | ✅ `StandardUserCredentialStore` uses `ICryptoProvider` to hash/check; ✅ Transparent rehash on success; ✅ Unit tests cover tamper + legacy rehash. | -| SEC1.OPT | DONE (2025-10-11) | Security Guild | SEC1.PLG | Expose password hashing knobs in `StandardPluginOptions` (`memoryKiB`, `iterations`, `parallelism`, `algorithm`) with validation. | ✅ Options bound from YAML; ✅ Invalid configs throw; ✅ Docs include tuning guidance. | -| SEC2.PLG | BLOCKED (2025-10-21) | Security Guild, Storage Guild | SEC2.A (audit contract) | Emit audit events from password verification outcomes and persist via `IAuthorityLoginAttemptStore`.
⛔ Waiting on AUTH-DPOP-11-001 / AUTH-MTLS-11-002 / PLUGIN-DI-08-001 to stabilise Authority auth surfaces before final verification + publish. | ✅ Serilog events enriched with subject/client/IP/outcome; ✅ Mongo records written per attempt; ✅ Tests assert success/lockout/failure cases. | -| SEC3.PLG | BLOCKED (2025-10-21) | Security Guild, BE-Auth Plugin | CORE8, SEC3.A (rate limiter) | Ensure lockout responses and rate-limit metadata flow through plugin logs/events (include retry-after).
⛔ Pending AUTH-DPOP-11-001 / AUTH-MTLS-11-002 / PLUGIN-DI-08-001 so limiter telemetry contract matches final authority surface. | ✅ Audit record includes retry-after; ✅ Tests confirm lockout + limiter interplay. | -| SEC4.PLG | DONE (2025-10-12) | Security Guild | SEC4.A (revocation schema) | Provide plugin hooks so revoked users/clients write reasons for revocation bundle export. | ✅ Revocation exporter consumes plugin data; ✅ Tests cover revoked user/client output. | -| SEC5.PLG | BLOCKED (2025-10-21) | Security Guild | SEC5.A (threat model) | Address plugin-specific mitigations (bootstrap user handling, password policy docs) in threat model backlog.
⛔ Final documentation depends on AUTH-DPOP-11-001 / AUTH-MTLS-11-002 / PLUGIN-DI-08-001 outcomes. | ✅ Threat model lists plugin attack surfaces; ✅ Mitigation items filed. | -| PLG4-6.CAPABILITIES | BLOCKED (2025-10-12) | BE-Auth Plugin, Docs Guild | PLG1–PLG3 | Finalise capability metadata exposure, config validation, and developer guide updates; remaining action is Docs polish/diagram export. | ✅ Capability metadata + validation merged; ✅ Plugin guide updated with final copy & diagrams; ✅ Release notes mention new toggles.
⛔ Blocked awaiting Authority rate-limiter stream (CORE8/SEC3) to resume so doc updates reflect final limiter behaviour. | -| PLG7.RFC | REVIEW | BE-Auth Plugin, Security Guild | PLG4 | Socialize LDAP plugin RFC (`docs/rfcs/authority-plugin-ldap.md`) and capture guild feedback. | ✅ Guild review sign-off recorded; ✅ Follow-up issues filed in module boards. | -| PLG6.DIAGRAM | TODO | Docs Guild | PLG6.DOC | Export final sequence/component diagrams for the developer guide and add offline-friendly assets under `docs/assets/authority`. | ✅ Mermaid sources committed; ✅ Rendered SVG/PNG linked from Section 2 + Section 9; ✅ Docs build preview shared with Plugin + Docs guilds. | - -> Update statuses to DOING/DONE/BLOCKED as you make progress. Always run `dotnet test` for touched projects before marking DONE. - -> Remark (2025-10-13, PLG6.DOC/PLG6.DIAGRAM): Security Guild delivered `docs/security/rate-limits.md`; Docs team can lift Section 3 (tuning table + alerts) into the developer guide diagrams when rendering assets. - -> Check-in (2025-10-19): Wave 0A dependencies (AUTH-DPOP-11-001, AUTH-MTLS-11-002, PLUGIN-DI-08-001) still open, so SEC2/SEC3/SEC5 remain in progress without new scope until upstream limiter updates land. +# Team 8 / Plugin Standard Backlog (UTC 2025-10-10) + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| SEC2.PLG | BLOCKED (2025-10-21) | Security Guild, Storage Guild | SEC2.A (audit contract) | Emit audit events from password verification outcomes and persist via `IAuthorityLoginAttemptStore`.
⛔ Waiting on AUTH-DPOP-11-001 / AUTH-MTLS-11-002 / PLUGIN-DI-08-001 to stabilise Authority auth surfaces before final verification + publish. | ✅ Serilog events enriched with subject/client/IP/outcome; ✅ Mongo records written per attempt; ✅ Tests assert success/lockout/failure cases. | +| SEC3.PLG | BLOCKED (2025-10-21) | Security Guild, BE-Auth Plugin | CORE8, SEC3.A (rate limiter) | Ensure lockout responses and rate-limit metadata flow through plugin logs/events (include retry-after).
⛔ Pending AUTH-DPOP-11-001 / AUTH-MTLS-11-002 / PLUGIN-DI-08-001 so limiter telemetry contract matches final authority surface. | ✅ Audit record includes retry-after; ✅ Tests confirm lockout + limiter interplay. | +| SEC5.PLG | BLOCKED (2025-10-21) | Security Guild | SEC5.A (threat model) | Address plugin-specific mitigations (bootstrap user handling, password policy docs) in threat model backlog.
⛔ Final documentation depends on AUTH-DPOP-11-001 / AUTH-MTLS-11-002 / PLUGIN-DI-08-001 outcomes. | ✅ Threat model lists plugin attack surfaces; ✅ Mitigation items filed. | +| PLG4-6.CAPABILITIES | BLOCKED (2025-10-12) | BE-Auth Plugin, Docs Guild | PLG1–PLG3 | Finalise capability metadata exposure, config validation, and developer guide updates; remaining action is Docs polish/diagram export. | ✅ Capability metadata + validation merged; ✅ Plugin guide updated with final copy & diagrams; ✅ Release notes mention new toggles.
⛔ Blocked awaiting Authority rate-limiter stream (CORE8/SEC3) to resume so doc updates reflect final limiter behaviour. | +| PLG7.RFC | REVIEW | BE-Auth Plugin, Security Guild | PLG4 | Socialize LDAP plugin RFC (`docs/rfcs/authority-plugin-ldap.md`) and capture guild feedback. | ✅ Guild review sign-off recorded; ✅ Follow-up issues filed in module boards. | +| PLG6.DIAGRAM | TODO | Docs Guild | PLG6.DOC | Export final sequence/component diagrams for the developer guide and add offline-friendly assets under `docs/assets/authority`. | ✅ Mermaid sources committed; ✅ Rendered SVG/PNG linked from Section 2 + Section 9; ✅ Docs build preview shared with Plugin + Docs guilds. | + +> Update statuses to DOING/DONE/BLOCKED as you make progress. Always run `dotnet test` for touched projects before marking DONE. + +> Remark (2025-10-13, PLG6.DOC/PLG6.DIAGRAM): Security Guild delivered `docs/security/rate-limits.md`; Docs team can lift Section 3 (tuning table + alerts) into the developer guide diagrams when rendering assets. + +> Check-in (2025-10-19): Wave 0A dependencies (AUTH-DPOP-11-001, AUTH-MTLS-11-002, PLUGIN-DI-08-001) still open, so SEC2/SEC3/SEC5 remain in progress without new scope until upstream limiter updates land. diff --git a/src/Authority/StellaOps.Authority/StellaOps.Authority.Tests/OpenIddict/ClientCredentialsAndTokenHandlersTests.cs b/src/Authority/StellaOps.Authority/StellaOps.Authority.Tests/OpenIddict/ClientCredentialsAndTokenHandlersTests.cs index 087f069a..c64982d2 100644 --- a/src/Authority/StellaOps.Authority/StellaOps.Authority.Tests/OpenIddict/ClientCredentialsAndTokenHandlersTests.cs +++ b/src/Authority/StellaOps.Authority/StellaOps.Authority.Tests/OpenIddict/ClientCredentialsAndTokenHandlersTests.cs @@ -593,11 +593,18 @@ public class ClientCredentialsHandlersTests Assert.Equal(new[] { "orch:operate" }, grantedScopes); var tenant = Assert.IsType(context.Transaction.Properties[AuthorityOpenIddictConstants.ClientTenantProperty]); Assert.Equal("tenant-default", tenant); - var reason = Assert.IsType(context.Transaction.Properties[AuthorityOpenIddictConstants.OperatorReasonProperty]); - Assert.Equal("resume source after maintenance", reason); - var ticket = Assert.IsType(context.Transaction.Properties[AuthorityOpenIddictConstants.OperatorTicketProperty]); - Assert.Equal("INC-2045", ticket); - } + var reason = Assert.IsType(context.Transaction.Properties[AuthorityOpenIddictConstants.OperatorReasonProperty]); + Assert.Equal("resume source after maintenance", reason); + var ticket = Assert.IsType(context.Transaction.Properties[AuthorityOpenIddictConstants.OperatorTicketProperty]); + Assert.Equal("INC-2045", ticket); + var principal = Assert.NotNull(context.Principal); + Assert.Equal("resume source after maintenance", principal.FindFirstValue(StellaOpsClaimTypes.OperatorReason)); + Assert.Equal("INC-2045", principal.FindFirstValue(StellaOpsClaimTypes.OperatorTicket)); + var scopeClaim = principal.FindFirstValue(StellaOpsClaimTypes.Scope); + Assert.Contains("orch:operate", scopeClaim.Split(' ', StringSplitOptions.RemoveEmptyEntries)); + var scopeItems = principal.FindAll(StellaOpsClaimTypes.ScopeItem).Select(claim => claim.Value).ToArray(); + Assert.Contains("orch:operate", scopeItems); + } [Fact] public async Task ValidateClientCredentials_RejectsExportViewer_WhenTenantMissing() diff --git a/src/Authority/StellaOps.Authority/StellaOps.Authority.Tests/Signing/AuthorityJwksServiceTests.cs b/src/Authority/StellaOps.Authority/StellaOps.Authority.Tests/Signing/AuthorityJwksServiceTests.cs new file mode 100644 index 00000000..c3f48936 --- /dev/null +++ b/src/Authority/StellaOps.Authority/StellaOps.Authority.Tests/Signing/AuthorityJwksServiceTests.cs @@ -0,0 +1,251 @@ +using System; +using System.Collections.Generic; +using System.Linq; +using System.Security.Cryptography; +using System.Threading; +using System.Threading.Tasks; +using Microsoft.Extensions.Caching.Memory; +using Microsoft.Extensions.Logging.Abstractions; +using Microsoft.Extensions.Options; +using Microsoft.Extensions.Time.Testing; +using Microsoft.IdentityModel.Tokens; +using StellaOps.Authority.Signing; +using StellaOps.Configuration; +using StellaOps.Cryptography; +using Xunit; + +namespace StellaOps.Authority.Tests.Signing; + +public sealed class AuthorityJwksServiceTests +{ + [Fact] + public void Get_ReusesCachedResponse_UntilLifetimeExpires() + { + var options = CreateAuthorityOptions(); + options.Signing.JwksCacheLifetime = TimeSpan.FromSeconds(10); + + var provider = new TestCryptoProvider(); + var registry = new TestRegistry(provider); + using var cache = new MemoryCache(new MemoryCacheOptions()); + var clock = new FakeTimeProvider(DateTimeOffset.Parse("2025-10-30T12:00:00Z")); + var service = new AuthorityJwksService( + registry, + NullLogger.Instance, + cache, + clock, + Options.Create(options)); + + var first = service.Get(); + Assert.Single(first.Response.Keys); + Assert.Equal(1, provider.EnumerationCount); + + var second = service.Get(); + Assert.Single(second.Response.Keys); + Assert.Equal(first.ETag, second.ETag); + Assert.Equal(1, provider.EnumerationCount); + + clock.Advance(TimeSpan.FromSeconds(11)); + + var third = service.Get(); + Assert.Single(third.Response.Keys); + Assert.NotEqual(first.ETag, third.ETag); + Assert.Equal(2, provider.EnumerationCount); + } + + [Fact] + public void Invalidate_ForcesRebuildOnNextRequest() + { + var options = CreateAuthorityOptions(); + options.Signing.JwksCacheLifetime = TimeSpan.FromMinutes(5); + + var provider = new TestCryptoProvider(); + var registry = new TestRegistry(provider); + using var cache = new MemoryCache(new MemoryCacheOptions()); + var clock = new FakeTimeProvider(DateTimeOffset.Parse("2025-10-30T12:00:00Z")); + var service = new AuthorityJwksService( + registry, + NullLogger.Instance, + cache, + clock, + Options.Create(options)); + + var first = service.Get(); + Assert.Equal(1, provider.EnumerationCount); + + service.Invalidate(); + + provider.AddKey("key-2"); + var second = service.Get(); + Assert.Equal(2, provider.EnumerationCount); + Assert.Equal(2, second.Response.Keys.Count); + Assert.Contains(second.Response.Keys, key => key.Kid == "key-2"); + } + + private static StellaOpsAuthorityOptions CreateAuthorityOptions() + { + return new StellaOpsAuthorityOptions + { + Issuer = new Uri("https://authority.test"), + Storage = + { + ConnectionString = "mongodb://localhost/test" + }, + Signing = + { + Enabled = true, + ActiveKeyId = "key-1", + KeyPath = "key-1.pem", + Algorithm = SignatureAlgorithms.Es256, + KeySource = "file", + Provider = "test" + } + }; + } + + private sealed class TestRegistry : ICryptoProviderRegistry + { + private readonly IReadOnlyCollection providers; + + public TestRegistry(ICryptoProvider provider) + { + providers = new[] { provider }; + } + + public IReadOnlyCollection Providers => providers; + + public bool TryResolve(string preferredProvider, out ICryptoProvider provider) + { + provider = providers.First(); + return true; + } + + public ICryptoProvider ResolveOrThrow(CryptoCapability capability, string algorithmId) + => providers.First(); + + public CryptoSignerResolution ResolveSigner( + CryptoCapability capability, + string algorithmId, + CryptoKeyReference keyReference, + string? preferredProvider = null) + { + var provider = providers.First(); + return new CryptoSignerResolution(provider.GetSigner(algorithmId, keyReference), provider.Name); + } + } + + private sealed class TestCryptoProvider : ICryptoProvider + { + private readonly Dictionary keys = new(StringComparer.OrdinalIgnoreCase); + private int counter; + + public TestCryptoProvider() + { + AddKey("key-1"); + } + + public string Name => "test"; + + public int EnumerationCount => counter; + + public bool Supports(CryptoCapability capability, string algorithmId) => true; + + public IPasswordHasher GetPasswordHasher(string algorithmId) => throw new NotSupportedException(); + + public ICryptoSigner GetSigner(string algorithmId, CryptoKeyReference keyReference) + { + if (!keys.TryGetValue(keyReference.KeyId, out var key)) + { + throw new InvalidOperationException($"Unknown key {keyReference.KeyId}."); + } + + return new TestSigner(keyReference.KeyId, algorithmId, key.Parameters); + } + + public void UpsertSigningKey(CryptoSigningKey signingKey) + { + keys[signingKey.Reference.KeyId] = new TestKey(signingKey.Reference.KeyId, signingKey.PublicParameters); + } + + public bool RemoveSigningKey(string keyId) => keys.Remove(keyId); + + public IReadOnlyCollection GetSigningKeys() + { + counter++; + return keys.Values + .Select(static key => key.ToSigningKey()) + .ToArray(); + } + + public void AddKey(string keyId) + { + using var ecdsa = ECDsa.Create(ECCurve.NamedCurves.nistP256); + var parameters = ecdsa.ExportParameters(true); + keys[keyId] = new TestKey(keyId, parameters); + } + + private sealed class TestKey + { + public TestKey(string keyId, ECParameters parameters) + { + KeyId = keyId; + Parameters = parameters; + } + + public string KeyId { get; } + + public ECParameters Parameters { get; } + + public CryptoSigningKey ToSigningKey() + { + return new CryptoSigningKey( + new CryptoKeyReference(KeyId, "test"), + SignatureAlgorithms.Es256, + in Parameters, + DateTimeOffset.UtcNow, + metadata: new Dictionary(StringComparer.OrdinalIgnoreCase) + { + ["status"] = AuthoritySigningKeyStatus.Active + }); + } + } + } + + private sealed class TestSigner : ICryptoSigner + { + private readonly ECParameters parameters; + + public TestSigner(string keyId, string algorithmId, ECParameters parameters) + { + KeyId = keyId; + AlgorithmId = algorithmId; + this.parameters = parameters; + } + + public string KeyId { get; } + + public string AlgorithmId { get; } + + public ValueTask SignAsync(ReadOnlyMemory data, CancellationToken cancellationToken = default) + => ValueTask.FromResult(Array.Empty()); + + public ValueTask VerifyAsync(ReadOnlyMemory data, ReadOnlyMemory signature, CancellationToken cancellationToken = default) + => ValueTask.FromResult(false); + + public JsonWebKey ExportPublicJsonWebKey() + { + var x = parameters.Q.X is null ? Array.Empty() : parameters.Q.X; + var y = parameters.Q.Y is null ? Array.Empty() : parameters.Q.Y; + + return new JsonWebKey + { + Kid = KeyId, + Alg = AlgorithmId, + Kty = JsonWebAlgorithmsKeyTypes.EllipticCurve, + Use = JsonWebKeyUseNames.Sig, + Crv = JsonWebKeyECTypes.P256, + X = Base64UrlEncoder.Encode(x), + Y = Base64UrlEncoder.Encode(y) + }; + } + } +} diff --git a/src/Authority/StellaOps.Authority/StellaOps.Authority.Tests/Signing/AuthoritySigningKeyManagerTests.cs b/src/Authority/StellaOps.Authority/StellaOps.Authority.Tests/Signing/AuthoritySigningKeyManagerTests.cs index 44967cc3..9eaf9b42 100644 --- a/src/Authority/StellaOps.Authority/StellaOps.Authority.Tests/Signing/AuthoritySigningKeyManagerTests.cs +++ b/src/Authority/StellaOps.Authority/StellaOps.Authority.Tests/Signing/AuthoritySigningKeyManagerTests.cs @@ -48,8 +48,8 @@ public sealed class AuthoritySigningKeyManagerTests var manager = provider.GetRequiredService(); var jwksService = provider.GetRequiredService(); - var initial = jwksService.Build(); - var initialKey = Assert.Single(initial.Keys); + var initial = jwksService.Get(); + var initialKey = Assert.Single(initial.Response.Keys); Assert.Equal("key-1", initialKey.Kid); Assert.Equal(AuthoritySigningKeyStatus.Active, initialKey.Status); @@ -71,13 +71,13 @@ public sealed class AuthoritySigningKeyManagerTests Assert.Equal(key1Relative, additional.Path); Assert.Equal("file", additional.Source); - var afterRotation = jwksService.Build(); - Assert.Equal(2, afterRotation.Keys.Count); + var afterRotation = jwksService.Get(); + Assert.Equal(2, afterRotation.Response.Keys.Count); - var activeEntry = Assert.Single(afterRotation.Keys, key => key.Status == AuthoritySigningKeyStatus.Active); + var activeEntry = Assert.Single(afterRotation.Response.Keys, key => key.Status == AuthoritySigningKeyStatus.Active); Assert.Equal("key-2", activeEntry.Kid); - var retiredEntry = Assert.Single(afterRotation.Keys, key => key.Status == AuthoritySigningKeyStatus.Retired); + var retiredEntry = Assert.Single(afterRotation.Response.Keys, key => key.Status == AuthoritySigningKeyStatus.Retired); Assert.Equal("key-1", retiredEntry.Kid); } finally @@ -100,10 +100,12 @@ public sealed class AuthoritySigningKeyManagerTests services.AddSingleton(new TestHostEnvironment(basePath)); services.AddSingleton(options); services.AddSingleton>(Options.Create(options)); + services.AddSingleton(TimeProvider.System); + services.AddMemoryCache(); services.AddStellaOpsCrypto(); services.TryAddEnumerable(ServiceDescriptor.Singleton()); - services.AddSingleton(); services.AddSingleton(); + services.AddSingleton(); return services.BuildServiceProvider(); } diff --git a/src/Authority/StellaOps.Authority/StellaOps.Authority/OpenIddict/Handlers/ClientCredentialsHandlers.cs b/src/Authority/StellaOps.Authority/StellaOps.Authority/OpenIddict/Handlers/ClientCredentialsHandlers.cs index 08c59a53..2ace328c 100644 --- a/src/Authority/StellaOps.Authority/StellaOps.Authority/OpenIddict/Handlers/ClientCredentialsHandlers.cs +++ b/src/Authority/StellaOps.Authority/StellaOps.Authority/OpenIddict/Handlers/ClientCredentialsHandlers.cs @@ -903,6 +903,20 @@ internal sealed class HandleClientCredentialsHandler : IOpenIddictServerHandler< metadataAccessor.SetProject(project); activity?.SetTag("authority.project", project); + if (context.Transaction.Properties.TryGetValue(AuthorityOpenIddictConstants.OperatorReasonProperty, out var operatorReasonValue) && + operatorReasonValue is string operatorReasonValueString && + !string.IsNullOrWhiteSpace(operatorReasonValueString)) + { + identity.SetClaim(StellaOpsClaimTypes.OperatorReason, operatorReasonValueString); + } + + if (context.Transaction.Properties.TryGetValue(AuthorityOpenIddictConstants.OperatorTicketProperty, out var operatorTicketValue) && + operatorTicketValue is string operatorTicketValueString && + !string.IsNullOrWhiteSpace(operatorTicketValueString)) + { + identity.SetClaim(StellaOpsClaimTypes.OperatorTicket, operatorTicketValueString); + } + var (providerHandle, descriptor) = await ResolveProviderAsync(context, document).ConfigureAwait(false); if (context.IsRejected) { diff --git a/src/Authority/StellaOps.Authority/StellaOps.Authority/Program.cs b/src/Authority/StellaOps.Authority/StellaOps.Authority/Program.cs index 5a8442f9..15570eb7 100644 --- a/src/Authority/StellaOps.Authority/StellaOps.Authority/Program.cs +++ b/src/Authority/StellaOps.Authority/StellaOps.Authority/Program.cs @@ -4,15 +4,16 @@ using System.Globalization; using System.IO; using Microsoft.AspNetCore.Builder; using Microsoft.AspNetCore.Http; -using Microsoft.Extensions.Configuration; -using Microsoft.Extensions.DependencyInjection; -using Microsoft.Extensions.DependencyInjection.Extensions; -using Microsoft.Extensions.Hosting; -using Microsoft.Extensions.Logging; -using Microsoft.Extensions.Options; +using Microsoft.Extensions.Configuration; +using Microsoft.Extensions.DependencyInjection; +using Microsoft.Extensions.DependencyInjection.Extensions; +using Microsoft.Extensions.Hosting; +using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Options; using Microsoft.AspNetCore.RateLimiting; -using Microsoft.AspNetCore.Server.Kestrel.Https; -using Microsoft.Extensions.Logging.Abstractions; +using Microsoft.AspNetCore.Server.Kestrel.Https; +using Microsoft.Extensions.Logging.Abstractions; +using Microsoft.Net.Http.Headers; using OpenIddict.Abstractions; using OpenIddict.Server; using OpenIddict.Server.AspNetCore; @@ -111,11 +112,12 @@ if (issuerUri is null) } authorityOptions.Issuer = issuerUri; -builder.Services.AddSingleton(authorityOptions); -builder.Services.AddSingleton>(Options.Create(authorityOptions)); -builder.Services.AddHttpContextAccessor(); -builder.Services.TryAddSingleton(_ => TimeProvider.System); -builder.Services.TryAddSingleton(); +builder.Services.AddSingleton(authorityOptions); +builder.Services.AddSingleton>(Options.Create(authorityOptions)); +builder.Services.AddHttpContextAccessor(); +builder.Services.TryAddSingleton(_ => TimeProvider.System); +builder.Services.AddMemoryCache(); +builder.Services.TryAddSingleton(); builder.Services.TryAddSingleton(); builder.Services.AddSingleton(); @@ -1293,8 +1295,26 @@ app.MapPost("/permalinks/vuln", async ( .RequireAuthorization(policy => policy.RequireStellaOpsScopes(StellaOpsScopes.VulnRead)) .WithName("CreateVulnPermalink"); -app.MapGet("/jwks", (AuthorityJwksService jwksService) => Results.Ok(jwksService.Build())) - .WithName("JsonWebKeySet"); +app.MapGet("/jwks", (AuthorityJwksService jwksService, HttpContext context) => +{ + var result = jwksService.Get(); + + if (context.Request.Headers.TryGetValue(HeaderNames.IfNoneMatch, out var etagValues) && + etagValues.Contains(result.ETag, StringComparer.Ordinal)) + { + context.Response.Headers.CacheControl = result.CacheControl; + context.Response.Headers.ETag = result.ETag; + context.Response.Headers.Expires = result.ExpiresAt.ToString("R", CultureInfo.InvariantCulture); + return Results.StatusCode(StatusCodes.Status304NotModified); + } + + context.Response.Headers.CacheControl = result.CacheControl; + context.Response.Headers.ETag = result.ETag; + context.Response.Headers.Expires = result.ExpiresAt.ToString("R", CultureInfo.InvariantCulture); + + return Results.Json(result.Response); +}) + .WithName("JsonWebKeySet"); // Ensure signing key manager initialises key material on startup. app.Services.GetRequiredService(); diff --git a/src/Authority/StellaOps.Authority/StellaOps.Authority/Signing/AuthorityJwksService.cs b/src/Authority/StellaOps.Authority/StellaOps.Authority/Signing/AuthorityJwksService.cs index 91908a55..7ab2c5a1 100644 --- a/src/Authority/StellaOps.Authority/StellaOps.Authority/Signing/AuthorityJwksService.cs +++ b/src/Authority/StellaOps.Authority/StellaOps.Authority/Signing/AuthorityJwksService.cs @@ -1,23 +1,81 @@ using System; using System.Collections.Generic; +using System.Linq; +using System.Security.Cryptography; +using System.Text; +using System.Text.Json; using System.Text.Json.Serialization; +using Microsoft.Extensions.Caching.Memory; using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Options; +using StellaOps.Configuration; using StellaOps.Cryptography; namespace StellaOps.Authority.Signing; internal sealed class AuthorityJwksService { + private const string CacheKey = "authority:jwks:current"; + private static readonly JsonSerializerOptions SerializerOptions = new(JsonSerializerDefaults.Web) + { + PropertyNamingPolicy = JsonNamingPolicy.CamelCase, + DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull + }; + private readonly ICryptoProviderRegistry registry; private readonly ILogger logger; + private readonly IMemoryCache cache; + private readonly TimeProvider timeProvider; + private readonly StellaOpsAuthorityOptions authorityOptions; - public AuthorityJwksService(ICryptoProviderRegistry registry, ILogger logger) + public AuthorityJwksService( + ICryptoProviderRegistry registry, + ILogger logger, + IMemoryCache cache, + TimeProvider timeProvider, + IOptions authorityOptions) { this.registry = registry ?? throw new ArgumentNullException(nameof(registry)); this.logger = logger ?? throw new ArgumentNullException(nameof(logger)); + this.cache = cache ?? throw new ArgumentNullException(nameof(cache)); + this.timeProvider = timeProvider ?? throw new ArgumentNullException(nameof(timeProvider)); + if (authorityOptions is null) + { + throw new ArgumentNullException(nameof(authorityOptions)); + } + + this.authorityOptions = authorityOptions.Value ?? throw new ArgumentNullException(nameof(authorityOptions)); } - public AuthorityJwksResponse Build() => new(BuildKeys()); + public AuthorityJwksResult Get() + { + if (cache.TryGetValue(CacheKey, out AuthorityJwksCacheEntry? cached) && + cached is not null && + cached.ExpiresAt > timeProvider.GetUtcNow()) + { + return cached.Result; + } + + var response = new AuthorityJwksResponse(BuildKeys()); + var etag = ComputeEtag(response); + var signingOptions = authorityOptions.Signing; + var lifetime = signingOptions.JwksCacheLifetime > TimeSpan.Zero + ? signingOptions.JwksCacheLifetime + : TimeSpan.FromMinutes(5); + var expires = timeProvider.GetUtcNow().Add(lifetime); + var cacheControl = $"public, max-age={(int)lifetime.TotalSeconds}"; + + var result = new AuthorityJwksResult(response, etag, expires, cacheControl); + var entry = new AuthorityJwksCacheEntry(result, expires); + + cache.Set(CacheKey, entry, expires); + return result; + } + + public void Invalidate() + { + cache.Remove(CacheKey); + } private IReadOnlyCollection BuildKeys() { @@ -58,12 +116,28 @@ internal sealed class AuthorityJwksService } } + keys.Sort(static (left, right) => string.Compare(left.Kid, right.Kid, StringComparison.Ordinal)); return keys; } + + private static string ComputeEtag(AuthorityJwksResponse response) + { + var payload = JsonSerializer.Serialize(response, SerializerOptions); + var hash = SHA256.HashData(Encoding.UTF8.GetBytes(payload)); + return $"\"{Convert.ToHexString(hash)}\""; + } + + private sealed record AuthorityJwksCacheEntry(AuthorityJwksResult Result, DateTimeOffset ExpiresAt); } internal sealed record AuthorityJwksResponse([property: JsonPropertyName("keys")] IReadOnlyCollection Keys); +internal sealed record AuthorityJwksResult( + AuthorityJwksResponse Response, + string ETag, + DateTimeOffset ExpiresAt, + string CacheControl); + internal sealed class JwksKeyEntry { [JsonPropertyName("kty")] diff --git a/src/Authority/StellaOps.Authority/StellaOps.Authority/Signing/AuthoritySigningKeyManager.cs b/src/Authority/StellaOps.Authority/StellaOps.Authority/Signing/AuthoritySigningKeyManager.cs index 600cd8e0..f4fe6101 100644 --- a/src/Authority/StellaOps.Authority/StellaOps.Authority/Signing/AuthoritySigningKeyManager.cs +++ b/src/Authority/StellaOps.Authority/StellaOps.Authority/Signing/AuthoritySigningKeyManager.cs @@ -19,13 +19,15 @@ internal sealed class AuthoritySigningKeyManager private readonly ILogger logger; private RegisteredSigningKey? activeKey; private readonly Dictionary retiredKeys = new(StringComparer.OrdinalIgnoreCase); + private readonly AuthorityJwksService jwksService; public AuthoritySigningKeyManager( ICryptoProviderRegistry registry, IEnumerable keySources, IOptions authorityOptions, IHostEnvironment environment, - ILogger logger) + ILogger logger, + AuthorityJwksService jwksService) { this.registry = registry ?? throw new ArgumentNullException(nameof(registry)); if (keySources is null) @@ -42,6 +44,7 @@ internal sealed class AuthoritySigningKeyManager this.authorityOptions = authorityOptions?.Value ?? throw new ArgumentNullException(nameof(authorityOptions)); basePath = environment?.ContentRootPath ?? throw new ArgumentNullException(nameof(environment)); this.logger = logger ?? throw new ArgumentNullException(nameof(logger)); + this.jwksService = jwksService ?? throw new ArgumentNullException(nameof(jwksService)); LoadInitialKeys(); } @@ -122,6 +125,7 @@ internal sealed class AuthoritySigningKeyManager RemoveAdditionalOption(keyId); logger.LogInformation("Authority signing key rotated. Active key is now {KeyId} via provider {Provider}.", keyId, provider.Name); + jwksService.Invalidate(); return new SigningRotationResult( keyId, @@ -221,6 +225,8 @@ internal sealed class AuthoritySigningKeyManager logger.LogWarning(ex, "Failed to load retired signing key {KeyId}. It will be ignored for JWKS responses.", keyId); } } + + jwksService.Invalidate(); } private void RetireCurrentActive() diff --git a/src/Authority/StellaOps.Authority/StellaOps.Authority/Signing/KmsAuthoritySigningKeySource.cs b/src/Authority/StellaOps.Authority/StellaOps.Authority/Signing/KmsAuthoritySigningKeySource.cs index 5336e239..af47c954 100644 --- a/src/Authority/StellaOps.Authority/StellaOps.Authority/Signing/KmsAuthoritySigningKeySource.cs +++ b/src/Authority/StellaOps.Authority/StellaOps.Authority/Signing/KmsAuthoritySigningKeySource.cs @@ -1,3 +1,4 @@ +using System; using System.Collections.Generic; using StellaOps.Cryptography; using StellaOps.Cryptography.Kms; @@ -29,19 +30,18 @@ internal sealed class KmsAuthoritySigningKeySource : IAuthoritySigningKeySource throw new InvalidOperationException("KMS signing keys require signing.keyPath/location to specify the key identifier."); } - request.AdditionalMetadata?.TryGetValue(KmsMetadataKeys.Version, out var versionId); + string? versionId = null; + if (request.AdditionalMetadata is not null && + request.AdditionalMetadata.TryGetValue(KmsMetadataKeys.Version, out var metadataVersion)) + { + versionId = metadataVersion; + } + var material = _kmsClient.ExportAsync(keyId, versionId).GetAwaiter().GetResult(); - var parameters = new ECParameters - { - Curve = ECCurve.NamedCurves.nistP256, - D = material.D.ToArray(), - Q = new ECPoint - { - X = material.Qx.ToArray(), - Y = material.Qy.ToArray(), - }, - }; + var publicKey = new byte[material.Qx.Length + material.Qy.Length]; + Buffer.BlockCopy(material.Qx, 0, publicKey, 0, material.Qx.Length); + Buffer.BlockCopy(material.Qy, 0, publicKey, material.Qx.Length, material.Qy.Length); var metadata = new Dictionary(StringComparer.OrdinalIgnoreCase) { @@ -49,7 +49,14 @@ internal sealed class KmsAuthoritySigningKeySource : IAuthoritySigningKeySource }; var reference = new CryptoKeyReference(request.KeyId, request.Provider); - return new CryptoSigningKey(reference, material.Algorithm, in parameters, material.CreatedAt, request.ExpiresAt, metadata: metadata); + return new CryptoSigningKey( + reference, + material.Algorithm, + material.D, + material.CreatedAt, + request.ExpiresAt, + publicKey, + metadata: metadata); } internal static class KmsMetadataKeys diff --git a/src/Authority/StellaOps.Authority/TASKS.completed.md b/src/Authority/StellaOps.Authority/TASKS.completed.md new file mode 100644 index 00000000..90aff9b6 --- /dev/null +++ b/src/Authority/StellaOps.Authority/TASKS.completed.md @@ -0,0 +1,33 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| AUTH-AOC-19-001 | DONE (2025-10-26) | Authority Core & Security Guild | — | Introduce scopes `advisory:read`, `advisory:ingest`, `vex:read`, `vex:ingest`, `aoc:verify` with configuration binding, migrations, and offline kit defaults. | Scopes published in metadata/OpenAPI, configuration validates scope lists, tests cover token issuance + enforcement. | +| AUTH-AOC-19-002 | DONE (2025-10-27) | Authority Core & Security Guild | AUTH-AOC-19-001 | Propagate tenant claim + scope enforcement for ingestion identities; ensure cross-tenant writes/read blocked and audit logs capture tenant context. | Tenant claim injected into downstream services; forbidden cross-tenant access rejected; audit/log fixtures updated. | +| AUTH-AOC-22-001 | DONE (2025-10-29) | Authority Core Guild | AUTH-AOC-19-001 | Roll out new advisory/vex ingest/read scopes. | Legacy scopes rejected; metadata/docs/configs updated; integration tests cover advisory/vex scope enforcement for Link-Not-Merge APIs. | +| AUTH-POLICY-20-001 | DONE (2025-10-26) | Authority Core & Security Guild | AUTH-AOC-19-001 | Add scopes `policy:write`, `policy:submit`, `policy:approve`, `policy:run`, `findings:read`, `effective:write` with configuration binding and issuer policy updates. | Scopes available in metadata; token issuance validated; offline kit defaults updated; tests cover scope combinations. | +| AUTH-POLICY-20-002 | DONE (2025-10-26) | Authority Core & Security Guild | AUTH-POLICY-20-001, AUTH-AOC-19-002 | Enforce Policy Engine service identity with `effective:write` and ensure API gateway enforces scopes/tenant claims for new endpoints. | Gateway policies updated; unauthorized requests rejected in tests; audit logs capture scope usage. | +| AUTH-GRAPH-21-001 | DONE (2025-10-26) | Authority Core & Security Guild | AUTH-POLICY-20-001 | Define scopes `graph:write`, `graph:read`, `graph:export`, `graph:simulate`, update metadata/OpenAPI, and add OFFLINE kit defaults. | Scopes exposed via discovery docs; smoke tests ensure enforcement; offline kit updated. | +| AUTH-GRAPH-21-002 | DONE (2025-10-26) | Authority Core & Security Guild | AUTH-GRAPH-21-001, AUTH-AOC-19-002 | Wire gateway enforcement for new graph scopes, Cartographer service identity, and tenant propagation across graph APIs. | Gateway config updated; unauthorized access blocked in integration tests; audit logs include graph scope usage. | +| AUTH-GRAPH-21-003 | DONE (2025-10-26) | Authority Core & Docs Guild | AUTH-GRAPH-21-001 | Update security docs and samples describing graph access roles, least privilege guidance, and service identities. | Docs merged with compliance checklist; examples refreshed; release notes prepared. | +| AUTH-POLICY-23-001 | DONE (2025-10-29) | Authority Core & Security Guild | AUTH-POLICY-20-001 | Introduce fine-grained scopes `policy:read`, `policy:edit`, `policy:approve`, `policy:activate`, `policy:simulate`; update issuer templates and metadata. | Scopes exposed; integration tests confirm enforcement; offline kit updated. | +| AUTH-VULN-24-001 | DONE (2025-10-29) | Authority Core & Security Guild | AUTH-GRAPH-21-001 | Extend scopes to include `vuln:read` and signed permalinks with scoped claims for Vuln Explorer; update metadata. | Scopes published; permalinks validated; integration tests cover RBAC. | +| AUTH-ORCH-32-001 | DONE (2025-10-31) | Authority Core & Security Guild | — | Define `orch:read` scope, register `Orch.Viewer` role, update discovery metadata, and seed offline defaults. | Scope/role available in metadata; integration tests confirm read-only enforcement; offline kit updated. | +| AUTH-CONSOLE-23-001 | DONE (2025-10-29) | Authority Core & Security Guild | AUTH-POLICY-20-001 | Register StellaOps Console confidential client with OIDC PKCE support, short-lived ID/access tokens, `console:*` audience claims, and SPA-friendly refresh (token exchange endpoint). Publish discovery metadata + offline kit defaults. | Client registration committed; configuration templates updated; integration tests validate PKCE + scope issuance; security review recorded. | +| AUTH-POLICY-27-001 | DONE (2025-10-31) | Authority Core & Security Guild | AUTH-POLICY-20-001, AUTH-CONSOLE-23-001 | Define Policy Studio roles (`policy:author`, `policy:review`, `policy:approve`, `policy:operate`, `policy:audit`) with tenant-scoped claims, update issuer metadata, and seed offline kit defaults. | Scopes/roles exposed via discovery docs; tokens issued with correct claims; integration tests cover role combinations; docs updated. | +| AUTH-EXC-25-001 | DONE (2025-10-29) | Authority Core & Security Guild | AUTH-POLICY-23-001 | Introduce exception scopes (`exceptions:read`, `exceptions:write`, `exceptions:approve`) and approval routing configuration with MFA gating. | Scopes published in metadata; routing matrix validated; integration tests enforce scope + MFA rules. | +| AUTH-SIG-26-001 | DONE (2025-10-29) | Authority Core & Security Guild | AUTH-EXC-25-001 | Add `signals:read`, `signals:write`, `signals:admin` scopes, issue `SignalsUploader` role template, and enforce AOC for sensor identities. | Scopes exposed; configuration validated; integration tests ensure RBAC + AOC enforcement. | +| AUTH-EXPORT-35-001 | DONE (2025-10-28) | Authority Core & Security Guild | AUTH-AOC-19-001 | Introduce `Export.Viewer`, `Export.Operator`, `Export.Admin` scopes, configure issuer templates, and update discovery metadata/offline defaults. | Scopes available; metadata updated; tests ensure enforcement; offline kit defaults refreshed. | +| AUTH-EXPORT-37-001 | DONE (2025-10-28) | Authority Core & Security Guild | AUTH-EXPORT-35-001, WEB-EXPORT-37-001 | Enforce admin-only access for scheduling, retention, encryption key references, and verify endpoints with audit reason capture. | Admin scope required; audit logs include reason/ticket; integration tests cover denial cases; docs updated. | +| AUTH-TEN-47-001 | DONE (2025-10-30) | Authority Core & Security Guild | AUTH-AOC-19-001 | Align Authority with OIDC/JWT claims (tenants, projects, scopes), implement JWKS caching/rotation, publish scope grammar, and enforce required claims on tokens. | Tokens include tenant/project claims; JWKS cache validated; docs updated; imposed rule noted.
2025-10-30: Introduced in-memory JWKS cache with configurable `signing.jwksCacheLifetime`, emitted cache-control/ETag headers on `/jwks`, invalidated cache on rotations, and expanded docs to detail scope grammar + claim catalogue. | +| AUTH-OAS-61-001 | DONE (2025-10-28) | Authority Core & Security Guild, API Contracts Guild | OAS-61-001 | Document Authority authentication/token endpoints in OAS with scopes, examples, and error envelopes. | Spec complete with security schemes; lint passes. | +| AUTH-AOC-19-003 | DONE (2025-10-27) | Authority Core & Docs Guild | AUTH-AOC-19-001 | Update Authority docs and sample configs to describe new scopes, tenancy enforcement, and verify endpoints. | Docs and examples refreshed; release notes prepared; smoke tests confirm new scopes required. | +| AUTH-AOC-19-004 | DONE (2025-10-31) | Authority Core & Security Guild | AUTH-AOC-19-002 | Enforce AOC scope pairings: require `aoc:verify` alongside advisory/vex read scopes and for any `signals:*` requests; emit deterministic errors and telemetry. | Client/token issuance rejects missing pairings with structured errors; logs/metrics capture violations; tests and docs updated. | +| AUTH-POLICY-20-003 | DONE (2025-10-26) | Authority Core & Docs Guild | AUTH-POLICY-20-001 | Update Authority configuration/docs with policy scopes, service identities, and approval workflows; include compliance checklist. | Docs refreshed; samples updated; release notes prepared; doc lint passes. | +| AUTH-POLICY-23-004 | DONE (2025-10-27) | Authority Core & DevOps Guild | AUTH-POLICY-23-001 | Migrate default Authority client registrations/offline kit templates to the new policy scope set and provide migration guidance for existing tokens. | Updated configs committed (Authority, CLI, CI samples); migration note added to release docs; verification script confirms scopes. | +| AUTH-ORCH-33-001 | DONE (2025-10-30) | Authority Core & Security Guild | AUTH-ORCH-32-001 | Add `Orch.Operator` role/scopes for control actions, require reason/ticket attributes, and update issuer templates. | Operator tokens issued; action endpoints enforce scope + reason; audit logs capture operator info; docs refreshed.
2025-10-30: Operator token flow now stamps `stellaops:operator_reason`/`stellaops:operator_ticket` claims, enforces claim presence on issuance, extends audit records, and refreshes config/docs (`authority.yaml`, security guides, CLI prompts). | +| AUTH-CONSOLE-23-002 | DONE (2025-10-31) | Authority Core & Security Guild | AUTH-CONSOLE-23-001, AUTH-AOC-19-002 | Expose tenant catalog, user profile, and token introspection endpoints required by Console (fresh-auth prompts, scope checks); enforce tenant header requirements and audit logging with correlation IDs. | Endpoints ship with RBAC enforcement, audit logs include tenant+scope, integration tests cover unauthorized/tenant-mismatch scenarios. | +| AUTH-CONSOLE-23-003 | DONE (2025-10-31) | Authority Core & Docs Guild | AUTH-CONSOLE-23-001, AUTH-CONSOLE-23-002 | Update security docs/config samples for Console flows (PKCE, tenant badge, fresh-auth for admin actions, session inactivity timeouts) with compliance checklist. | Docs merged, config samples validated, release notes updated, ops runbook references new flows. | +| AUTH-CONSOLE-23-004 | DONE (2025-10-31) | Authority Core & Security Guild | AUTH-CONSOLE-23-003, DOCS-CONSOLE-23-012 | Validate console security guide assumptions (120 s OpTok TTL, 300 s fresh-auth window, scope bundles) against Authority implementation and update configs/audit fixtures if needed. | Confirmation recorded in sprint log; Authority config samples/tests updated when adjustments required; `/fresh-auth` behaviour documented in release notes. | +| AUTH-EXC-25-002 | DONE (2025-10-31) | Authority Core & Docs Guild | AUTH-EXC-25-001 | Update documentation/samples for exception roles, routing matrix, MFA requirements, and audit trail references. | Docs merged with compliance checklist; samples verified. | +| AUTH-OAS-61-002 | DONE (2025-10-28) | Authority Core & Security Guild | AUTH-OAS-61-001 | Implement `/.well-known/openapi` with scope metadata, supported grant types, and build version. | Endpoint deployed; contract tests cover discovery. | diff --git a/src/Authority/StellaOps.Authority/TASKS.md b/src/Authority/StellaOps.Authority/TASKS.md index 5aac58bc..48ac0554 100644 --- a/src/Authority/StellaOps.Authority/TASKS.md +++ b/src/Authority/StellaOps.Authority/TASKS.md @@ -1,69 +1,53 @@ # Authority Host Task Board — Epic 1: Aggregation-Only Contract | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| AUTH-AOC-19-001 | DONE (2025-10-26) | Authority Core & Security Guild | — | Introduce scopes `advisory:read`, `advisory:ingest`, `vex:read`, `vex:ingest`, `aoc:verify` with configuration binding, migrations, and offline kit defaults. | Scopes published in metadata/OpenAPI, configuration validates scope lists, tests cover token issuance + enforcement. | -| AUTH-AOC-19-002 | DONE (2025-10-27) | Authority Core & Security Guild | AUTH-AOC-19-001 | Propagate tenant claim + scope enforcement for ingestion identities; ensure cross-tenant writes/read blocked and audit logs capture tenant context. | Tenant claim injected into downstream services; forbidden cross-tenant access rejected; audit/log fixtures updated. | > 2025-10-26: Rate limiter metadata/audit records now include tenants, password grant scopes/tenants enforced, token persistence + tests updated. Docs refresh tracked via AUTH-AOC-19-003. > 2025-10-27: Client credential ingestion scopes now require tenant assignment; access token validation backfills tenants and rejects cross-tenant mismatches with tests. > 2025-10-27: `dotnet test` blocked — Concelier build fails (`AdvisoryObservationQueryService` returns `ImmutableHashSet`), preventing Authority test suite run; waiting on Concelier fix before rerun. -| AUTH-AOC-19-003 | DONE (2025-10-27) | Authority Core & Docs Guild | AUTH-AOC-19-001 | Update Authority docs and sample configs to describe new scopes, tenancy enforcement, and verify endpoints. | Docs and examples refreshed; release notes prepared; smoke tests confirm new scopes required. | > 2025-10-26: Docs updated (`docs/11_AUTHORITY.md`, Concelier audit runbook, `docs/security/authority-scopes.md`); sample config highlights tenant-aware clients. Release notes + smoke verification pending (blocked on Concelier/Excititor smoke updates). > 2025-10-27: Scope catalogue aligned with `advisory:ingest/advisory:read/vex:ingest/vex:read`, `aoc:verify` pairing documented, console/CLI references refreshed, and `etc/authority.yaml.sample` updated to require read scopes for verification clients. -| AUTH-AOC-19-004 | DONE (2025-10-31) | Authority Core & Security Guild | AUTH-AOC-19-002 | Enforce AOC scope pairings: require `aoc:verify` alongside advisory/vex read scopes and for any `signals:*` requests; emit deterministic errors and telemetry. | Client/token issuance rejects missing pairings with structured errors; logs/metrics capture violations; tests and docs updated. | > 2025-10-31: Client credentials and password grants now reject advisory/vex read or signals scopes without `aoc:verify`, enforce tenant assignment for `aoc:verify`, tag violations via `authority.aoc_scope_violation`, extend tests, and refresh scope catalogue docs/sample roles. ## Link-Not-Merge v1 | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| AUTH-AOC-22-001 | DONE (2025-10-29) | Authority Core Guild | AUTH-AOC-19-001 | Roll out new advisory/vex ingest/read scopes. | Legacy scopes rejected; metadata/docs/configs updated; integration tests cover advisory/vex scope enforcement for Link-Not-Merge APIs. | > 2025-10-29: Rejected legacy `concelier.merge` scope during client credential validation, removed it from known scope catalog, blocked discovery/issuance, added regression tests, and refreshed scope documentation. ## Policy Engine v2 | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| AUTH-POLICY-20-001 | DONE (2025-10-26) | Authority Core & Security Guild | AUTH-AOC-19-001 | Add scopes `policy:write`, `policy:submit`, `policy:approve`, `policy:run`, `findings:read`, `effective:write` with configuration binding and issuer policy updates. | Scopes available in metadata; token issuance validated; offline kit defaults updated; tests cover scope combinations. | -| AUTH-POLICY-20-002 | DONE (2025-10-26) | Authority Core & Security Guild | AUTH-POLICY-20-001, AUTH-AOC-19-002 | Enforce Policy Engine service identity with `effective:write` and ensure API gateway enforces scopes/tenant claims for new endpoints. | Gateway policies updated; unauthorized requests rejected in tests; audit logs capture scope usage. | > 2025-10-26: Restricted `effective:write` to Policy Engine service identities with tenant requirement, registered full scope set, and tightened resource server default scope enforcement (unit tests pass). -| AUTH-POLICY-20-003 | DONE (2025-10-26) | Authority Core & Docs Guild | AUTH-POLICY-20-001 | Update Authority configuration/docs with policy scopes, service identities, and approval workflows; include compliance checklist. | Docs refreshed; samples updated; release notes prepared; doc lint passes. | > 2025-10-26: Authority docs now detail policy scopes/service identity guardrails with checklist; `authority.yaml.sample` includes `properties.serviceIdentity` example. ## Graph Explorer v1 | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| AUTH-GRAPH-21-001 | DONE (2025-10-26) | Authority Core & Security Guild | AUTH-POLICY-20-001 | Define scopes `graph:write`, `graph:read`, `graph:export`, `graph:simulate`, update metadata/OpenAPI, and add OFFLINE kit defaults. | Scopes exposed via discovery docs; smoke tests ensure enforcement; offline kit updated. | -| AUTH-GRAPH-21-002 | DONE (2025-10-26) | Authority Core & Security Guild | AUTH-GRAPH-21-001, AUTH-AOC-19-002 | Wire gateway enforcement for new graph scopes, Cartographer service identity, and tenant propagation across graph APIs. | Gateway config updated; unauthorized access blocked in integration tests; audit logs include graph scope usage. | -| AUTH-GRAPH-21-003 | DONE (2025-10-26) | Authority Core & Docs Guild | AUTH-GRAPH-21-001 | Update security docs and samples describing graph access roles, least privilege guidance, and service identities. | Docs merged with compliance checklist; examples refreshed; release notes prepared. | ## Policy Engine + Editor v1 | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| AUTH-POLICY-23-001 | DONE (2025-10-29) | Authority Core & Security Guild | AUTH-POLICY-20-001 | Introduce fine-grained scopes `policy:read`, `policy:edit`, `policy:approve`, `policy:activate`, `policy:simulate`; update issuer templates and metadata. | Scopes exposed; integration tests confirm enforcement; offline kit updated. | | AUTH-POLICY-23-002 | BLOCKED (2025-10-29) | Authority Core & Security Guild | AUTH-POLICY-23-001 | Implement optional two-person rule for activation: require two distinct `policy:activate` approvals when configured; emit audit logs. | Activation endpoint enforces rule; audit logs contain approver IDs; tests cover 2-person path. | > Blocked: Policy Engine/Studio have not yet exposed activation workflow endpoints or approval payloads needed to enforce dual-control (`WEB-POLICY-23-002`, `POLICY-ENGINE-23-002`). Revisit once activation contract lands. | AUTH-POLICY-23-003 | BLOCKED (2025-10-29) | Authority Core & Docs Guild | AUTH-POLICY-23-001 | Update documentation and sample configs for policy roles, approval workflow, and signing requirements. | Docs updated with reviewer checklist; configuration examples validated. | > Blocked pending AUTH-POLICY-23-002 dual-approval implementation so docs can capture final activation behaviour. -| AUTH-POLICY-23-004 | DONE (2025-10-27) | Authority Core & DevOps Guild | AUTH-POLICY-23-001 | Migrate default Authority client registrations/offline kit templates to the new policy scope set and provide migration guidance for existing tokens. | Updated configs committed (Authority, CLI, CI samples); migration note added to release docs; verification script confirms scopes. | > 2025-10-27: Added `policy-cli` defaults to Authority config/secrets, refreshed CLI/CI documentation with the new scope bundle, recorded release migration guidance, and introduced `scripts/verify-policy-scopes.py` to guard against regressions. ## Graph & Vuln Explorer v1 | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| AUTH-VULN-24-001 | DONE (2025-10-29) | Authority Core & Security Guild | AUTH-GRAPH-21-001 | Extend scopes to include `vuln:read` and signed permalinks with scoped claims for Vuln Explorer; update metadata. | Scopes published; permalinks validated; integration tests cover RBAC. | > 2025-10-27: Paused work after exploratory spike (scope enforcement still outstanding); no functional changes merged. ## Orchestrator Dashboard | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| AUTH-ORCH-32-001 | DONE (2025-10-31) | Authority Core & Security Guild | — | Define `orch:read` scope, register `Orch.Viewer` role, update discovery metadata, and seed offline defaults. | Scope/role available in metadata; integration tests confirm read-only enforcement; offline kit updated. | > 2025-10-31: Picked up during Console/Orchestrator alignment; focusing on scope catalog + tenant enforcement first. > 2025-10-31: `orch:read` added to scope catalogue and Authority runtime, Console defaults include the scope, `Orch.Viewer` role documented, and client-credential tests enforce tenant requirements. -| AUTH-ORCH-33-001 | DOING (2025-10-27) | Authority Core & Security Guild | AUTH-ORCH-32-001 | Add `Orch.Operator` role/scopes for control actions, require reason/ticket attributes, and update issuer templates. | Operator tokens issued; action endpoints enforce scope + reason; audit logs capture operator info; docs refreshed. | > 2025-10-27: Added `orch:operate` scope, enforced `operator_reason`/`operator_ticket` on token issuance, updated Authority configs/docs, and captured audit metadata for control actions. > 2025-10-28: Policy gateway + scanner now pass the expanded token client signature (`null` metadata by default), test stubs capture the optional parameters, and Policy Gateway/Scanner suites are green after fixing the Concelier storage build break. > 2025-10-28: Authority password-grant tests now hit the new constructors but still need updates to drop obsolete `IOptions` arguments before the suite can pass. @@ -73,21 +57,16 @@ | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| AUTH-CONSOLE-23-001 | DONE (2025-10-29) | Authority Core & Security Guild | AUTH-POLICY-20-001 | Register StellaOps Console confidential client with OIDC PKCE support, short-lived ID/access tokens, `console:*` audience claims, and SPA-friendly refresh (token exchange endpoint). Publish discovery metadata + offline kit defaults. | Client registration committed; configuration templates updated; integration tests validate PKCE + scope issuance; security review recorded. | > 2025-10-29: Authorization code flow enabled with PKCE requirement, console client seeded in `authority.yaml.sample`, discovery docs updated, and console runbook guidance added. -| AUTH-CONSOLE-23-002 | DONE (2025-10-31) | Authority Core & Security Guild | AUTH-CONSOLE-23-001, AUTH-AOC-19-002 | Expose tenant catalog, user profile, and token introspection endpoints required by Console (fresh-auth prompts, scope checks); enforce tenant header requirements and audit logging with correlation IDs. | Endpoints ship with RBAC enforcement, audit logs include tenant+scope, integration tests cover unauthorized/tenant-mismatch scenarios. | > 2025-10-31: Added `/console/tenants`, `/console/profile`, `/console/token/introspect` endpoints with tenant header filter, scope enforcement (`ui.read`, `authority:tenants.read`), and structured audit events. Console test harness covers success/mismatch cases. -| AUTH-CONSOLE-23-003 | DONE (2025-10-31) | Authority Core & Docs Guild | AUTH-CONSOLE-23-001, AUTH-CONSOLE-23-002 | Update security docs/config samples for Console flows (PKCE, tenant badge, fresh-auth for admin actions, session inactivity timeouts) with compliance checklist. | Docs merged, config samples validated, release notes updated, ops runbook references new flows. | > 2025-10-28: `docs/security/console-security.md` drafted with PKCE + DPoP (120 s OpTok, 300 s fresh-auth) and scope table. Authority Core to confirm `/fresh-auth` semantics, token lifetimes, and scope bundles align before closing task. > 2025-10-31: Security guide expanded for `/console` endpoints & orchestrator scope, sample YAML annotated, ops runbook updated, and release note `docs/updates/2025-10-31-console-security-refresh.md` published. -| AUTH-CONSOLE-23-004 | DONE (2025-10-31) | Authority Core & Security Guild | AUTH-CONSOLE-23-003, DOCS-CONSOLE-23-012 | Validate console security guide assumptions (120 s OpTok TTL, 300 s fresh-auth window, scope bundles) against Authority implementation and update configs/audit fixtures if needed. | Confirmation recorded in sprint log; Authority config samples/tests updated when adjustments required; `/fresh-auth` behaviour documented in release notes. | > 2025-10-31: Default access-token lifetime reduced to 120 s, console tests updated with dual auth schemes, docs/config/ops notes refreshed, release note logged. ## Policy Studio (Sprint 27) | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| AUTH-POLICY-27-001 | DONE (2025-10-31) | Authority Core & Security Guild | AUTH-POLICY-20-001, AUTH-CONSOLE-23-001 | Define Policy Studio roles (`policy:author`, `policy:review`, `policy:approve`, `policy:operate`, `policy:audit`) with tenant-scoped claims, update issuer metadata, and seed offline kit defaults. | Scopes/roles exposed via discovery docs; tokens issued with correct claims; integration tests cover role combinations; docs updated. | > 2025-10-31: Added Policy Studio scope family (`policy:author/review/operate/audit`), updated OpenAPI + discovery headers, enforced tenant requirements in grant handlers, seeded new roles in Authority config/offline kit docs, and refreshed CLI/Console documentation + tests to validate the new catalogue. | AUTH-POLICY-27-002 | TODO | Authority Core & Security Guild | AUTH-POLICY-27-001, REGISTRY-API-27-007 | Provide attestation signing service bindings (OIDC token exchange, cosign integration) and enforce publish/promote scope checks, fresh-auth requirements, and audit logging. | Publish/promote requests require fresh auth + correct scopes; attestations signed with validated identity; audit logs enriched with digest + tenant; integration tests pass. | > Docs dependency: `DOCS-POLICY-27-009` awaiting signing guidance from this work. @@ -97,16 +76,13 @@ | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| AUTH-EXC-25-001 | DONE (2025-10-29) | Authority Core & Security Guild | AUTH-POLICY-23-001 | Introduce exception scopes (`exceptions:read`, `exceptions:write`, `exceptions:approve`) and approval routing configuration with MFA gating. | Scopes published in metadata; routing matrix validated; integration tests enforce scope + MFA rules. | > 2025-10-29: Added exception scopes + routing template options, enforced MFA requirement in password grant handlers, updated configuration samples. -| AUTH-EXC-25-002 | DONE (2025-10-31) | Authority Core & Docs Guild | AUTH-EXC-25-001 | Update documentation/samples for exception roles, routing matrix, MFA requirements, and audit trail references. | Docs merged with compliance checklist; samples verified. | > 2025-10-31: Authority scopes/routing docs updated (`docs/security/authority-scopes.md`, `docs/11_AUTHORITY.md`, `docs/policy/exception-effects.md`), monitoring guide covers new MFA audit events, and `etc/authority.yaml.sample` now demonstrates exception clients/templates. ## Reachability v1 | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| AUTH-SIG-26-001 | DONE (2025-10-29) | Authority Core & Security Guild | AUTH-EXC-25-001 | Add `signals:read`, `signals:write`, `signals:admin` scopes, issue `SignalsUploader` role template, and enforce AOC for sensor identities. | Scopes exposed; configuration validated; integration tests ensure RBAC + AOC enforcement. | > 2025-10-29: Signals scopes added with tenant + aoc:verify enforcement; sensors guided via SignalsUploader template; tests cover gating. ## Vulnerability Explorer (Sprint 29) @@ -127,8 +103,6 @@ ## Export Center | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| AUTH-EXPORT-35-001 | DONE (2025-10-28) | Authority Core & Security Guild | AUTH-AOC-19-001 | Introduce `Export.Viewer`, `Export.Operator`, `Export.Admin` scopes, configure issuer templates, and update discovery metadata/offline defaults. | Scopes available; metadata updated; tests ensure enforcement; offline kit defaults refreshed. | -| AUTH-EXPORT-37-001 | DONE (2025-10-28) | Authority Core & Security Guild | AUTH-EXPORT-35-001, WEB-EXPORT-37-001 | Enforce admin-only access for scheduling, retention, encryption key references, and verify endpoints with audit reason capture. | Admin scope required; audit logs include reason/ticket; integration tests cover denial cases; docs updated. | ## Notifications Studio | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | @@ -146,7 +120,6 @@ ## Authority-Backed Scopes & Tenancy (Epic 14) | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| AUTH-TEN-47-001 | DOING (2025-10-28) | Authority Core & Security Guild | AUTH-AOC-19-001 | Align Authority with OIDC/JWT claims (tenants, projects, scopes), implement JWKS caching/rotation, publish scope grammar, and enforce required claims on tokens. | Tokens include tenant/project claims; JWKS cache validated; docs updated; imposed rule noted. | > 2025-10-28: Tidied advisory raw idempotency migration to avoid LINQ-on-`BsonValue` (explicit array copy) while continuing duplicate guardrail validation; scoped scanner/policy token call sites updated to honor new metadata parameter. | AUTH-TEN-49-001 | TODO | Authority Core & Security Guild | AUTH-TEN-47-001 | Implement service accounts & delegation tokens (`act` chain), per-tenant quotas, audit stream of auth decisions, and revocation APIs. | Service tokens minted with scopes/TTL; delegation logged; quotas configurable; audit stream live; docs updated. | @@ -169,9 +142,7 @@ ## SDKs & OpenAPI (Epic 17) | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| AUTH-OAS-61-001 | DONE (2025-10-28) | Authority Core & Security Guild, API Contracts Guild | OAS-61-001 | Document Authority authentication/token endpoints in OAS with scopes, examples, and error envelopes. | Spec complete with security schemes; lint passes. | > 2025-10-28: Auth OpenAPI authored at `src/Api/StellaOps.Api.OpenApi/authority/openapi.yaml` covering `/token`, `/introspect`, `/revoke`, `/jwks`, scope catalog, and error envelopes; parsed via PyYAML sanity check and referenced in Epic 17 docs. -| AUTH-OAS-61-002 | DONE (2025-10-28) | Authority Core & Security Guild | AUTH-OAS-61-001 | Implement `/.well-known/openapi` with scope metadata, supported grant types, and build version. | Endpoint deployed; contract tests cover discovery. | > 2025-10-28: Added `/.well-known/openapi` endpoint wiring cached spec metadata, YAML/JSON negotiation, HTTP cache headers, and tests verifying ETag + Accept handling. Authority spec (`src/Api/StellaOps.Api.OpenApi/authority/openapi.yaml`) now includes grant/scope extensions. | AUTH-OAS-62-001 | TODO | Authority Core & Security Guild, SDK Generator Guild | AUTH-OAS-61-001, SDKGEN-63-001 | Provide SDK helpers for OAuth2/PAT flows, tenancy override header; add integration tests. | SDKs expose auth helpers; tests cover token issuance; docs updated. | | AUTH-OAS-63-001 | TODO | Authority Core & Security Guild, API Governance Guild | APIGOV-63-001 | Emit deprecation headers and notifications for legacy auth endpoints. | Headers emitted; notifications verified; migration guide published. | diff --git a/src/Bench/StellaOps.Bench/TASKS.completed.md b/src/Bench/StellaOps.Bench/TASKS.completed.md new file mode 100644 index 00000000..fa85dc6b --- /dev/null +++ b/src/Bench/StellaOps.Bench/TASKS.completed.md @@ -0,0 +1,10 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| BENCH-SCANNER-10-001 | DONE | Bench Guild, Scanner Team | SCANNER-ANALYZERS-LANG-10-303 | Analyzer microbench harness (node_modules, site-packages) + baseline CSV. | Harness committed under `src/Bench/StellaOps.Bench/Scanner.Analyzers`; baseline CSV recorded; CI job publishes results. | +| BENCH-SCANNER-10-002 | DONE (2025-10-21) | Bench Guild, Language Analyzer Guild | SCANNER-ANALYZERS-LANG-10-301..309 | Wire real language analyzers into bench harness & refresh baselines post-implementation. | Harness executes analyzer assemblies end-to-end; updated baseline committed; CI trend doc linked. | +| BENCH-NOTIFY-15-001 | DONE (2025-10-26) | Bench Guild, Notify Team | NOTIFY-ENGINE-15-301 | Notify dispatch throughput bench (vary rule density) with results CSV. | Bench executed; results stored; regression alert configured. | +| BENCH-POLICY-20-001 | DONE (2025-10-26) | Bench Guild, Policy Guild | POLICY-ENGINE-20-002, POLICY-ENGINE-20-006 | Build policy evaluation benchmark suite (100k components, 1M advisories) capturing latency, throughput, memory. | Bench harness committed; baseline metrics recorded; ties into CI dashboards. | +| BENCH-LNM-22-001 | DONE (2025-10-26) | Bench Guild, Concelier Guild | CONCELIER-LNM-21-002 | Create ingest benchmark simulating 500 advisory observations/sec, measuring correlator latency and Mongo throughput; publish baseline metrics. | Harness added; baseline stored; alerts wired for SLA breach. | +| BENCH-LNM-22-002 | DONE (2025-10-26) | Bench Guild, Excititor Guild | EXCITITOR-LNM-21-002 | Build VEX ingestion/correlation perf test focusing on alias/product matching and event emission latency. | Benchmark executed; metrics captured; CI integration established. | diff --git a/src/Bench/StellaOps.Bench/TASKS.md b/src/Bench/StellaOps.Bench/TASKS.md index 85109a35..93c2beea 100644 --- a/src/Bench/StellaOps.Bench/TASKS.md +++ b/src/Bench/StellaOps.Bench/TASKS.md @@ -2,16 +2,12 @@ | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| BENCH-SCANNER-10-001 | DONE | Bench Guild, Scanner Team | SCANNER-ANALYZERS-LANG-10-303 | Analyzer microbench harness (node_modules, site-packages) + baseline CSV. | Harness committed under `src/Bench/StellaOps.Bench/Scanner.Analyzers`; baseline CSV recorded; CI job publishes results. | -| BENCH-SCANNER-10-002 | DONE (2025-10-21) | Bench Guild, Language Analyzer Guild | SCANNER-ANALYZERS-LANG-10-301..309 | Wire real language analyzers into bench harness & refresh baselines post-implementation. | Harness executes analyzer assemblies end-to-end; updated baseline committed; CI trend doc linked. | | BENCH-IMPACT-16-001 | TODO | Bench Guild, Scheduler Team | SCHED-IMPACT-16-301 | ImpactIndex throughput bench (resolve 10k productKeys) + RAM profile. | Benchmark script ready; baseline metrics recorded; alert thresholds defined. | -| BENCH-NOTIFY-15-001 | DONE (2025-10-26) | Bench Guild, Notify Team | NOTIFY-ENGINE-15-301 | Notify dispatch throughput bench (vary rule density) with results CSV. | Bench executed; results stored; regression alert configured. | ## Policy Engine v2 | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| BENCH-POLICY-20-001 | DONE (2025-10-26) | Bench Guild, Policy Guild | POLICY-ENGINE-20-002, POLICY-ENGINE-20-006 | Build policy evaluation benchmark suite (100k components, 1M advisories) capturing latency, throughput, memory. | Bench harness committed; baseline metrics recorded; ties into CI dashboards. | > 2025-10-26: Added `StellaOps.Bench.PolicyEngine` harness, synthetic dataset generator, baseline + Prom/JSON outputs; default thresholds cover latency/throughput/allocation. | BENCH-POLICY-20-002 | TODO | Bench Guild, Policy Guild, Scheduler Guild | BENCH-POLICY-20-001, SCHED-WORKER-20-302 | Add incremental run benchmark measuring delta evaluation vs full; capture SLA compliance. | Incremental bench executed; results stored; regression alerts configured. | > 2025-10-29: Scheduler delta targeting landed (see SCHED-WORKER-20-302 notes); incremental bench can proceed once Policy Engine change streams feed metadata. @@ -29,8 +25,6 @@ | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| BENCH-LNM-22-001 | DONE (2025-10-26) | Bench Guild, Concelier Guild | CONCELIER-LNM-21-002 | Create ingest benchmark simulating 500 advisory observations/sec, measuring correlator latency and Mongo throughput; publish baseline metrics. | Harness added; baseline stored; alerts wired for SLA breach. | -| BENCH-LNM-22-002 | DONE (2025-10-26) | Bench Guild, Excititor Guild | EXCITITOR-LNM-21-002 | Build VEX ingestion/correlation perf test focusing on alias/product matching and event emission latency. | Benchmark executed; metrics captured; CI integration established. | ## Graph & Vuln Explorer v1 diff --git a/src/Cartographer/StellaOps.Cartographer/TASKS.completed.md b/src/Cartographer/StellaOps.Cartographer/TASKS.completed.md new file mode 100644 index 00000000..fa14fefd --- /dev/null +++ b/src/Cartographer/StellaOps.Cartographer/TASKS.completed.md @@ -0,0 +1,5 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| CARTO-GRAPH-21-010 | DONE (2025-10-27) | Cartographer Guild | AUTH-GRAPH-21-001 | Replace hard-coded `graph:*` scope strings in Cartographer services/clients with `StellaOpsScopes` constants; document new dependency. | All scope checks reference `StellaOpsScopes`; documentation updated; unit tests adjusted if needed. | diff --git a/src/Cartographer/StellaOps.Cartographer/TASKS.md b/src/Cartographer/StellaOps.Cartographer/TASKS.md index 3e3f0301..9832edad 100644 --- a/src/Cartographer/StellaOps.Cartographer/TASKS.md +++ b/src/Cartographer/StellaOps.Cartographer/TASKS.md @@ -1,6 +1,5 @@ -# Cartographer Task Board — Epic 3: Graph Explorer v1 -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| CARTO-GRAPH-21-010 | DONE (2025-10-27) | Cartographer Guild | AUTH-GRAPH-21-001 | Replace hard-coded `graph:*` scope strings in Cartographer services/clients with `StellaOpsScopes` constants; document new dependency. | All scope checks reference `StellaOpsScopes`; documentation updated; unit tests adjusted if needed. | - -> 2025-10-26 — Note: awaiting Cartographer service bootstrap. Keep this task open until Cartographer routes exist so we can swap to `StellaOpsScopes` immediately. +# Cartographer Task Board — Epic 3: Graph Explorer v1 +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| + +> 2025-10-26 — Note: awaiting Cartographer service bootstrap. Keep this task open until Cartographer routes exist so we can swap to `StellaOpsScopes` immediately. diff --git a/src/Cli/StellaOps.Cli/TASKS.completed.md b/src/Cli/StellaOps.Cli/TASKS.completed.md new file mode 100644 index 00000000..95b2fa37 --- /dev/null +++ b/src/Cli/StellaOps.Cli/TASKS.completed.md @@ -0,0 +1,9 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| CLI-AOC-19-001 | DONE (2025-10-27) | DevEx/CLI Guild | CONCELIER-WEB-AOC-19-001, EXCITITOR-WEB-AOC-19-001 | Implement `stella sources ingest --dry-run` printing would-write payloads with forbidden field scan results and guard status. | Command displays diff-safe JSON, highlights forbidden fields, exits non-zero on guard violation, and has unit tests. | +| CLI-POLICY-20-002 | DONE (2025-10-27) | DevEx/CLI Guild | CLI-POLICY-20-001, WEB-POLICY-20-001, WEB-POLICY-20-002 | Implement `stella policy simulate` with SBOM/env arguments and diff output (table/JSON), handling exit codes for `ERR_POL_*`. | Simulation outputs deterministic diffs; JSON schema documented; tests validate exit codes + piping of env variables. | +| CLI-AOC-19-002 | DONE (2025-10-27) | DevEx/CLI Guild | CLI-AOC-19-001 | Add `stella aoc verify` command supporting `--since`/`--limit`, mapping `ERR_AOC_00x` to exit codes, with JSON/table output. | Command integrates with both services, exit codes documented, regression tests green. | +| CLI-AOC-19-003 | DONE (2025-10-27) | Docs/CLI Guild | CLI-AOC-19-001, CLI-AOC-19-002 | Update CLI reference and quickstart docs to cover new commands, exit codes, and offline verification workflows. | Docs updated; examples recorded; release notes mention new commands. | +| CLI-POLICY-20-003 | DONE (2025-10-30) | DevEx/CLI Guild, Docs Guild | CLI-POLICY-20-002, WEB-POLICY-20-003, DOCS-POLICY-20-006 | Extend `stella findings ls|get` commands for policy-filtered retrieval with pagination, severity filters, and explain output. | Commands stream paginated results; explain view renders rationale entries; docs/help updated; end-to-end tests cover filters. | diff --git a/src/Cli/StellaOps.Cli/TASKS.md b/src/Cli/StellaOps.Cli/TASKS.md index 9d68fcc3..3f9e8cd6 100644 --- a/src/Cli/StellaOps.Cli/TASKS.md +++ b/src/Cli/StellaOps.Cli/TASKS.md @@ -1,16 +1,13 @@ # CLI Task Board — Epic 1: Aggregation-Only Contract | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| CLI-AOC-19-001 | DONE (2025-10-27) | DevEx/CLI Guild | CONCELIER-WEB-AOC-19-001, EXCITITOR-WEB-AOC-19-001 | Implement `stella sources ingest --dry-run` printing would-write payloads with forbidden field scan results and guard status. | Command displays diff-safe JSON, highlights forbidden fields, exits non-zero on guard violation, and has unit tests. | > Docs ready (2025-10-26): Reference behaviour/spec in `docs/modules/cli/guides/cli-reference.md` §2 and AOC reference §5. > 2025-10-27: CLI command scaffolded with backend client call, JSON/table output, gzip/base64 normalisation, and exit-code mapping. Awaiting Concelier dry-run endpoint + integration tests once backend lands. > 2025-10-27: Progress paused before adding CLI unit tests; blocked on extending `StubBackendClient` + fixtures for `ExecuteAocIngestDryRunAsync` coverage. > 2025-10-27: Added stubbed ingest responses + unit tests covering success/violation paths, output writing, and exit-code mapping. -| CLI-AOC-19-002 | DONE (2025-10-27) | DevEx/CLI Guild | CLI-AOC-19-001 | Add `stella aoc verify` command supporting `--since`/`--limit`, mapping `ERR_AOC_00x` to exit codes, with JSON/table output. | Command integrates with both services, exit codes documented, regression tests green. | > Docs ready (2025-10-26): CLI guide §3 covers options/exit codes; deployment doc `docs/deploy/containers.md` describes required verifier user. > 2025-10-27: CLI wiring in progress; backend client/command surface being added with table/JSON output. > 2025-10-27: Added JSON/table Spectre output, integration tests for exit-code handling, CLI metrics, and updated quickstart/architecture docs to cover guard workflows. -| CLI-AOC-19-003 | DONE (2025-10-27) | Docs/CLI Guild | CLI-AOC-19-001, CLI-AOC-19-002 | Update CLI reference and quickstart docs to cover new commands, exit codes, and offline verification workflows. | Docs updated; examples recorded; release notes mention new commands. | > Docs note (2025-10-26): `docs/modules/cli/guides/cli-reference.md` now describes both commands, exit codes, and offline usage—sync help text once implementation lands. > 2025-10-27: CLI reference now reflects final summary fields/JSON schema, quickstart includes verification/dry-run workflows, and API reference tables list both `sources ingest --dry-run` and `aoc verify`. @@ -19,11 +16,9 @@ | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| | CLI-POLICY-20-001 | TODO | DevEx/CLI Guild | WEB-POLICY-20-001 | Add `stella policy new|edit|submit|approve` commands with local editor integration, version pinning, and approval workflow wiring. | Commands round-trip policy drafts with temp files; approval requires correct scopes; unit tests cover happy/error paths. | -| CLI-POLICY-20-002 | DONE (2025-10-27) | DevEx/CLI Guild | CLI-POLICY-20-001, WEB-POLICY-20-001, WEB-POLICY-20-002 | Implement `stella policy simulate` with SBOM/env arguments and diff output (table/JSON), handling exit codes for `ERR_POL_*`. | Simulation outputs deterministic diffs; JSON schema documented; tests validate exit codes + piping of env variables. | > 2025-10-26: Scheduler Models expose canonical run/diff schemas (`src/Scheduler/__Libraries/StellaOps.Scheduler.Models/docs/SCHED-MODELS-20-001-POLICY-RUNS.md`). Schema exporter lives at `scripts/export-policy-schemas.sh`; wire schema validation once DevOps publishes artifacts (see DEVOPS-POLICY-20-004). > 2025-10-27: DevOps pipeline now publishes `policy-schema-exports` artefacts per commit (see `.gitea/workflows/build-test-deploy.yml`); Slack `#policy-engine` alerts trigger on schema diffs. Pull the JSON from the CI artifact instead of committing local copies. > 2025-10-27: CLI command supports table/JSON output, environment parsing, `--fail-on-diff`, and maps `ERR_POL_*` to exit codes; tested in `StellaOps.Cli.Tests` against stubbed backend. -| CLI-POLICY-20-003 | DONE (2025-10-30) | DevEx/CLI Guild, Docs Guild | CLI-POLICY-20-002, WEB-POLICY-20-003, DOCS-POLICY-20-006 | Extend `stella findings ls|get` commands for policy-filtered retrieval with pagination, severity filters, and explain output. | Commands stream paginated results; explain view renders rationale entries; docs/help updated; end-to-end tests cover filters. | > 2025-10-27: Work paused after stubbing backend parsing helpers; command wiring/tests still pending. Resume by finishing backend query serialization + CLI output paths. > 2025-10-30: Resuming implementation; wiring backend query DTOs, CLI handlers, and tests for paginated policy-filtered findings. > 2025-10-30: Implemented backend client + CLI command surface for policy findings list/get/explain, added telemetry, interactive/json output, file writes, and unit tests covering filters + explain traces. diff --git a/src/Concelier/StellaOps.Concelier.WebService/Program.cs b/src/Concelier/StellaOps.Concelier.WebService/Program.cs index a7047b66..09ed085d 100644 --- a/src/Concelier/StellaOps.Concelier.WebService/Program.cs +++ b/src/Concelier/StellaOps.Concelier.WebService/Program.cs @@ -39,6 +39,7 @@ using StellaOps.Auth.Abstractions; using StellaOps.Auth.Client; using StellaOps.Auth.ServerIntegration; using StellaOps.Aoc; +using StellaOps.Aoc.AspNetCore.Results; using StellaOps.Concelier.WebService.Contracts; using StellaOps.Concelier.Core.Aoc; using StellaOps.Concelier.Core.Raw; @@ -869,39 +870,9 @@ static DateTimeOffset? ParseDateTime(string? value) IResult MapAocGuardException(HttpContext context, ConcelierAocGuardException exception) { - var violations = exception.Violations - .Select(v => new - { - code = v.ErrorCode, - path = v.Path, - message = v.Message - }) - .ToArray(); - - var status = MapErrorCodeToStatus(exception.PrimaryErrorCode); - var extensions = new Dictionary(StringComparer.Ordinal) - { - ["code"] = exception.PrimaryErrorCode, - ["violations"] = violations - }; - - return Problem( - context, - "Aggregation-Only Contract violation", - status, - ProblemTypes.Validation, - $"AOC guard rejected the request with {exception.PrimaryErrorCode}.", - extensions); + var guardException = new AocGuardException(exception.Result); + return AocHttpResults.Problem(context, guardException); } - -static int MapErrorCodeToStatus(string errorCode) => errorCode switch -{ - "ERR_AOC_003" => StatusCodes.Status409Conflict, - "ERR_AOC_004" => StatusCodes.Status422UnprocessableEntity, - "ERR_AOC_005" => StatusCodes.Status422UnprocessableEntity, - "ERR_AOC_006" => StatusCodes.Status403Forbidden, - _ => StatusCodes.Status400BadRequest, -}; static KeyValuePair[] BuildJobMetricTags(string jobKind, string trigger, string outcome) => new[] diff --git a/src/Concelier/StellaOps.Concelier.WebService/StellaOps.Concelier.WebService.csproj b/src/Concelier/StellaOps.Concelier.WebService/StellaOps.Concelier.WebService.csproj index 6e538da5..d2d1b0d3 100644 --- a/src/Concelier/StellaOps.Concelier.WebService/StellaOps.Concelier.WebService.csproj +++ b/src/Concelier/StellaOps.Concelier.WebService/StellaOps.Concelier.WebService.csproj @@ -34,5 +34,6 @@ + - \ No newline at end of file + diff --git a/src/Concelier/StellaOps.Concelier.WebService/TASKS.completed.md b/src/Concelier/StellaOps.Concelier.WebService/TASKS.completed.md new file mode 100644 index 00000000..76c090b1 --- /dev/null +++ b/src/Concelier/StellaOps.Concelier.WebService/TASKS.completed.md @@ -0,0 +1,5 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Notes | +|---|---|---|---|---| +| CONCELIER-WEB-AOC-19-001 `Raw ingestion endpoints` | DONE (2025-10-28) | Concelier WebService Guild | CONCELIER-CORE-AOC-19-001, CONCELIER-STORE-AOC-19-001 | Implement `POST /ingest/advisory`, `GET /advisories/raw*`, and `POST /aoc/verify` minimal API endpoints. Enforce new Authority scopes, inject tenant claims, and surface `AOCWriteGuard` to repository calls. | diff --git a/src/Concelier/StellaOps.Concelier.WebService/TASKS.md b/src/Concelier/StellaOps.Concelier.WebService/TASKS.md index 1f7d07ad..551e9762 100644 --- a/src/Concelier/StellaOps.Concelier.WebService/TASKS.md +++ b/src/Concelier/StellaOps.Concelier.WebService/TASKS.md @@ -1,95 +1,94 @@ -# TASKS — Epic 1: Aggregation-Only Contract -> **AOC Reminder:** service links and exposes raw data only—no precedence, severity, or hint computation inside Concelier APIs. -| ID | Status | Owner(s) | Depends on | Notes | -|---|---|---|---|---| -| CONCELIER-WEB-AOC-19-001 `Raw ingestion endpoints` | DONE (2025-10-28) | Concelier WebService Guild | CONCELIER-CORE-AOC-19-001, CONCELIER-STORE-AOC-19-001 | Implement `POST /ingest/advisory`, `GET /advisories/raw*`, and `POST /aoc/verify` minimal API endpoints. Enforce new Authority scopes, inject tenant claims, and surface `AOCWriteGuard` to repository calls. | -> Docs alignment (2025-10-26): Endpoint expectations + scope requirements detailed in `docs/ingestion/aggregation-only-contract.md` and `docs/security/authority-scopes.md`. -> 2025-10-28: Added coverage for pagination, tenancy enforcement, and ingestion/verification metrics; verified guard handling paths end-to-end. -| CONCELIER-WEB-AOC-19-002 `AOC observability` | TODO | Concelier WebService Guild, Observability Guild | CONCELIER-WEB-AOC-19-001 | Emit `ingestion_write_total`, `aoc_violation_total`, latency histograms, and tracing spans (`ingest.fetch/transform/write`, `aoc.guard`). Wire structured logging to include tenant, source vendor, upstream id, and content hash. | -> Docs alignment (2025-10-26): Metrics/traces/log schema in `docs/observability/observability.md`. -| CONCELIER-WEB-AOC-19-003 `Schema/guard unit tests` | TODO | QA Guild | CONCELIER-WEB-AOC-19-001 | Add unit tests covering schema validation failures, forbidden field rejections (`ERR_AOC_001/002/006/007`), idempotent upserts, and supersedes chains using deterministic fixtures. | -> Docs alignment (2025-10-26): Guard rules + error codes documented in AOC reference §5 and CLI guide. -| CONCELIER-WEB-AOC-19-004 `End-to-end ingest verification` | TODO | Concelier WebService Guild, QA Guild | CONCELIER-WEB-AOC-19-003, CONCELIER-CORE-AOC-19-002 | Create integration tests ingesting large advisory batches (cold/warm) validating linkset enrichment, metrics emission, and reproducible outputs. Capture load-test scripts + doc notes for Offline Kit dry runs. | -> Docs alignment (2025-10-26): Offline verification workflow referenced in `docs/deploy/containers.md` §5. - -## Policy Engine v2 - -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| CONCELIER-POLICY-20-001 `Policy selection endpoints` | TODO | Concelier WebService Guild | WEB-POLICY-20-001, CONCELIER-CORE-AOC-19-004 | Add batch advisory lookup APIs (`/policy/select/advisories`, `/policy/select/vex`) optimized for PURL/ID lists with pagination, tenant scoping, and explain metadata. | - -## StellaOps Console (Sprint 23) - -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| CONCELIER-CONSOLE-23-001 `Advisory aggregation views` | TODO | Concelier WebService Guild, BE-Base Platform Guild | CONCELIER-LNM-21-201, CONCELIER-LNM-21-202 | Expose `/console/advisories` endpoints returning aggregation groups (per linkset) with source chips, severity summaries, and provenance metadata for Console list + dashboard cards. Support filters by source, ecosystem, published/modified window, tenant enforcement. | -| CONCELIER-CONSOLE-23-002 `Dashboard deltas API` | TODO | Concelier WebService Guild | CONCELIER-CONSOLE-23-001, CONCELIER-LNM-21-203 | Provide aggregated advisory delta counts (new, modified, conflicting) for Console dashboard + live status ticker; emit structured events for queue lag metrics. Ensure deterministic counts across repeated queries. | -| CONCELIER-CONSOLE-23-003 `Search fan-out helpers` | TODO | Concelier WebService Guild | CONCELIER-CONSOLE-23-001 | Deliver fast lookup endpoints for CVE/GHSA/purl search (linksets, observations) returning evidence fragments for Console global search; implement caching + scope guards. | - -## Graph Explorer v1 - -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| - -## Link-Not-Merge v1 - -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| CONCELIER-LNM-21-201 `Observation APIs` | TODO | Concelier WebService Guild, BE-Base Platform Guild | CONCELIER-LNM-21-001 | Add REST endpoints for advisory observations (`GET /advisories/observations`) with filters (alias, purl, source), pagination, and tenancy enforcement. | -| CONCELIER-LNM-21-202 `Linkset APIs` | TODO | Concelier WebService Guild | CONCELIER-LNM-21-002, CONCELIER-LNM-21-003 | Implement linkset read/export endpoints (`/advisories/linksets/{id}`, `/advisories/by-purl/{purl}`, `/advisories/linksets/{id}/export`, `/evidence`) with correlation/conflict payloads and `ERR_AGG_*` mapping. | -| CONCELIER-LNM-21-203 `Ingest events` | TODO | Concelier WebService Guild, Platform Events Guild | CONCELIER-LNM-21-005 | Publish NATS/Redis events for new observations/linksets and ensure idempotent consumer contracts; document event schemas. | - -## Graph & Vuln Explorer v1 - -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| CONCELIER-GRAPH-24-101 `Advisory summary API` | TODO | Concelier WebService Guild | CONCELIER-GRAPH-24-001 | Expose `/advisories/summary` returning raw linkset/observation metadata for overlay services; no derived severity or fix hints. | -| CONCELIER-GRAPH-28-102 `Evidence batch API` | TODO | Concelier WebService Guild | CONCELIER-LNM-21-201 | Add batch fetch for advisory observations/linksets keyed by component sets to feed Graph overlay tooltips efficiently. | - -## VEX Lens (Sprint 30) - -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| CONCELIER-VEXLENS-30-001 `Advisory rationale bridges` | TODO | Concelier WebService Guild, VEX Lens Guild | CONCELIER-VULN-29-001, VEXLENS-30-005 | Guarantee advisory key consistency and cross-links for consensus rationale; Label: VEX-Lens. | - -## Vulnerability Explorer (Sprint 29) - -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| CONCELIER-VULN-29-001 `Advisory key canonicalization` | TODO | Concelier WebService Guild, Data Integrity Guild | CONCELIER-LNM-21-001 | Canonicalize (lossless) advisory identifiers (CVE/GHSA/vendor) into `advisory_key`, persist `links[]`, expose raw payload snapshots for Explorer evidence tabs; AOC-compliant: no merge, no derived fields, no suppression. Include migration/backfill scripts. | -| CONCELIER-VULN-29-002 `Evidence retrieval API` | TODO | Concelier WebService Guild | CONCELIER-VULN-29-001, VULN-API-29-003 | Provide `/vuln/evidence/advisories/{advisory_key}` returning raw advisory docs with provenance, filtering by tenant and source. | -| CONCELIER-VULN-29-004 `Observability enhancements` | TODO | Concelier WebService Guild, Observability Guild | CONCELIER-VULN-29-001 | Instrument metrics/logs for advisory normalization (key collisions, withdrawn flags), emit events consumed by Vuln Explorer resolver. | - -## Advisory AI (Sprint 31) - -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| CONCELIER-AIAI-31-001 `Paragraph anchors` | TODO | Concelier WebService Guild | CONCELIER-VULN-29-001 | Expose advisory chunk API returning paragraph anchors, section metadata, and token-safe text for Advisory AI retrieval. | -| CONCELIER-AIAI-31-002 `Structured fields` | TODO | Concelier WebService Guild | CONCELIER-AIAI-31-001 | Ensure normalized advisories expose workaround/fix/CVSS fields via API; add caching for summary queries. | -| CONCELIER-AIAI-31-003 `Advisory AI telemetry` | TODO | Concelier WebService Guild, Observability Guild | CONCELIER-AIAI-31-001 | Emit metrics/logs for chunk requests, cache hits, and guardrail blocks triggered by advisory payloads. | - -## Observability & Forensics (Epic 15) -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| CONCELIER-WEB-OBS-50-001 `Telemetry adoption` | TODO | Concelier WebService Guild | TELEMETRY-OBS-50-001, CONCELIER-OBS-50-001 | Adopt telemetry core in web service host, ensure ingest + read endpoints emit trace/log fields (`tenant_id`, `route`, `decision_effect`), and add correlation IDs to responses. | -| CONCELIER-WEB-OBS-51-001 `Observability APIs` | TODO | Concelier WebService Guild | CONCELIER-WEB-OBS-50-001, WEB-OBS-51-001 | Surface ingest health metrics, queue depth, and SLO status via `/obs/concelier/health` endpoint for Console widgets, with caching and tenant partitioning. | -| CONCELIER-WEB-OBS-52-001 `Timeline streaming` | TODO | Concelier WebService Guild | CONCELIER-WEB-OBS-50-001, TIMELINE-OBS-52-003 | Provide SSE stream `/obs/concelier/timeline` bridging to Timeline Indexer with paging tokens, guardrails, and audit logging. | -| CONCELIER-WEB-OBS-53-001 `Evidence locker integration` | TODO | Concelier WebService Guild, Evidence Locker Guild | CONCELIER-OBS-53-001, EVID-OBS-53-003 | Add `/evidence/advisories/*` routes invoking evidence locker snapshots, verifying tenant scopes (`evidence:read`), and returning signed manifest metadata. | -| CONCELIER-WEB-OBS-54-001 `Attestation exposure` | TODO | Concelier WebService Guild | CONCELIER-OBS-54-001, PROV-OBS-54-001 | Provide `/attestations/advisories/*` read APIs surfacing DSSE status, verification summary, and provenance chain for Console/CLI. | -| CONCELIER-WEB-OBS-55-001 `Incident mode toggles` | TODO | Concelier WebService Guild, DevOps Guild | CONCELIER-OBS-55-001, WEB-OBS-55-001 | Implement incident mode toggle endpoints, propagate to orchestrator/locker, and document cooldown/backoff semantics. | - -## Air-Gapped Mode (Epic 16) -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| CONCELIER-WEB-AIRGAP-56-001 `Mirror import APIs` | TODO | Concelier WebService Guild | AIRGAP-IMP-58-001, CONCELIER-AIRGAP-56-001 | Extend ingestion endpoints to register mirror bundle sources, expose bundle catalog queries, and block external feed URLs in sealed mode. | -| CONCELIER-WEB-AIRGAP-56-002 `Airgap status surfaces` | TODO | Concelier WebService Guild | CONCELIER-AIRGAP-57-002, AIRGAP-CTL-56-002 | Add staleness metadata and bundle provenance to advisory APIs (`/advisories/observations`, `/advisories/linksets`). | -| CONCELIER-WEB-AIRGAP-57-001 `Error remediation` | TODO | Concelier WebService Guild, AirGap Policy Guild | AIRGAP-POL-56-001 | Map sealed-mode violations to `AIRGAP_EGRESS_BLOCKED` responses with user guidance. | -| CONCELIER-WEB-AIRGAP-58-001 `Import timeline emission` | TODO | Concelier WebService Guild, AirGap Importer Guild | CONCELIER-WEB-AIRGAP-56-001, TIMELINE-OBS-53-001 | Emit timeline events for bundle ingestion operations with bundle ID, scope, and actor metadata. | - -## SDKs & OpenAPI (Epic 17) -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| CONCELIER-WEB-OAS-61-001 `/.well-known/openapi` | TODO | Concelier WebService Guild | OAS-61-001 | Implement discovery endpoint emitting Concelier spec with version metadata and ETag. | -| CONCELIER-WEB-OAS-61-002 `Error envelope migration` | TODO | Concelier WebService Guild | APIGOV-61-001 | Ensure all API responses use standardized error envelope; update controllers/tests. | -| CONCELIER-WEB-OAS-62-001 `Examples expansion` | TODO | Concelier WebService Guild | CONCELIER-OAS-61-002 | Add curated examples for advisory observations/linksets/conflicts; integrate into dev portal. | -| CONCELIER-WEB-OAS-63-001 `Deprecation headers` | TODO | Concelier WebService Guild, API Governance Guild | APIGOV-63-001 | Add Sunset/Deprecation headers for retiring endpoints and update documentation/notifications. | +# TASKS — Epic 1: Aggregation-Only Contract +> **AOC Reminder:** service links and exposes raw data only—no precedence, severity, or hint computation inside Concelier APIs. +| ID | Status | Owner(s) | Depends on | Notes | +|---|---|---|---|---| +> Docs alignment (2025-10-26): Endpoint expectations + scope requirements detailed in `docs/ingestion/aggregation-only-contract.md` and `docs/security/authority-scopes.md`. +> 2025-10-28: Added coverage for pagination, tenancy enforcement, and ingestion/verification metrics; verified guard handling paths end-to-end. +| CONCELIER-WEB-AOC-19-002 `AOC observability` | TODO | Concelier WebService Guild, Observability Guild | CONCELIER-WEB-AOC-19-001 | Emit `ingestion_write_total`, `aoc_violation_total`, latency histograms, and tracing spans (`ingest.fetch/transform/write`, `aoc.guard`). Wire structured logging to include tenant, source vendor, upstream id, and content hash. | +> Docs alignment (2025-10-26): Metrics/traces/log schema in `docs/observability/observability.md`. +| CONCELIER-WEB-AOC-19-003 `Schema/guard unit tests` | TODO | QA Guild | CONCELIER-WEB-AOC-19-001 | Add unit tests covering schema validation failures, forbidden field rejections (`ERR_AOC_001/002/006/007`), idempotent upserts, and supersedes chains using deterministic fixtures. | +> Docs alignment (2025-10-26): Guard rules + error codes documented in AOC reference §5 and CLI guide. +| CONCELIER-WEB-AOC-19-004 `End-to-end ingest verification` | TODO | Concelier WebService Guild, QA Guild | CONCELIER-WEB-AOC-19-003, CONCELIER-CORE-AOC-19-002 | Create integration tests ingesting large advisory batches (cold/warm) validating linkset enrichment, metrics emission, and reproducible outputs. Capture load-test scripts + doc notes for Offline Kit dry runs. | +> Docs alignment (2025-10-26): Offline verification workflow referenced in `docs/deploy/containers.md` §5. + +## Policy Engine v2 + +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| CONCELIER-POLICY-20-001 `Policy selection endpoints` | TODO | Concelier WebService Guild | WEB-POLICY-20-001, CONCELIER-CORE-AOC-19-004 | Add batch advisory lookup APIs (`/policy/select/advisories`, `/policy/select/vex`) optimized for PURL/ID lists with pagination, tenant scoping, and explain metadata. | + +## StellaOps Console (Sprint 23) + +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| CONCELIER-CONSOLE-23-001 `Advisory aggregation views` | TODO | Concelier WebService Guild, BE-Base Platform Guild | CONCELIER-LNM-21-201, CONCELIER-LNM-21-202 | Expose `/console/advisories` endpoints returning aggregation groups (per linkset) with source chips, severity summaries, and provenance metadata for Console list + dashboard cards. Support filters by source, ecosystem, published/modified window, tenant enforcement. | +| CONCELIER-CONSOLE-23-002 `Dashboard deltas API` | TODO | Concelier WebService Guild | CONCELIER-CONSOLE-23-001, CONCELIER-LNM-21-203 | Provide aggregated advisory delta counts (new, modified, conflicting) for Console dashboard + live status ticker; emit structured events for queue lag metrics. Ensure deterministic counts across repeated queries. | +| CONCELIER-CONSOLE-23-003 `Search fan-out helpers` | TODO | Concelier WebService Guild | CONCELIER-CONSOLE-23-001 | Deliver fast lookup endpoints for CVE/GHSA/purl search (linksets, observations) returning evidence fragments for Console global search; implement caching + scope guards. | + +## Graph Explorer v1 + +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| + +## Link-Not-Merge v1 + +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| CONCELIER-LNM-21-201 `Observation APIs` | TODO | Concelier WebService Guild, BE-Base Platform Guild | CONCELIER-LNM-21-001 | Add REST endpoints for advisory observations (`GET /advisories/observations`) with filters (alias, purl, source), pagination, and tenancy enforcement. | +| CONCELIER-LNM-21-202 `Linkset APIs` | TODO | Concelier WebService Guild | CONCELIER-LNM-21-002, CONCELIER-LNM-21-003 | Implement linkset read/export endpoints (`/advisories/linksets/{id}`, `/advisories/by-purl/{purl}`, `/advisories/linksets/{id}/export`, `/evidence`) with correlation/conflict payloads and `ERR_AGG_*` mapping. | +| CONCELIER-LNM-21-203 `Ingest events` | TODO | Concelier WebService Guild, Platform Events Guild | CONCELIER-LNM-21-005 | Publish NATS/Redis events for new observations/linksets and ensure idempotent consumer contracts; document event schemas. | + +## Graph & Vuln Explorer v1 + +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| CONCELIER-GRAPH-24-101 `Advisory summary API` | TODO | Concelier WebService Guild | CONCELIER-GRAPH-24-001 | Expose `/advisories/summary` returning raw linkset/observation metadata for overlay services; no derived severity or fix hints. | +| CONCELIER-GRAPH-28-102 `Evidence batch API` | TODO | Concelier WebService Guild | CONCELIER-LNM-21-201 | Add batch fetch for advisory observations/linksets keyed by component sets to feed Graph overlay tooltips efficiently. | + +## VEX Lens (Sprint 30) + +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| CONCELIER-VEXLENS-30-001 `Advisory rationale bridges` | TODO | Concelier WebService Guild, VEX Lens Guild | CONCELIER-VULN-29-001, VEXLENS-30-005 | Guarantee advisory key consistency and cross-links for consensus rationale; Label: VEX-Lens. | + +## Vulnerability Explorer (Sprint 29) + +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| CONCELIER-VULN-29-001 `Advisory key canonicalization` | TODO | Concelier WebService Guild, Data Integrity Guild | CONCELIER-LNM-21-001 | Canonicalize (lossless) advisory identifiers (CVE/GHSA/vendor) into `advisory_key`, persist `links[]`, expose raw payload snapshots for Explorer evidence tabs; AOC-compliant: no merge, no derived fields, no suppression. Include migration/backfill scripts. | +| CONCELIER-VULN-29-002 `Evidence retrieval API` | TODO | Concelier WebService Guild | CONCELIER-VULN-29-001, VULN-API-29-003 | Provide `/vuln/evidence/advisories/{advisory_key}` returning raw advisory docs with provenance, filtering by tenant and source. | +| CONCELIER-VULN-29-004 `Observability enhancements` | TODO | Concelier WebService Guild, Observability Guild | CONCELIER-VULN-29-001 | Instrument metrics/logs for advisory normalization (key collisions, withdrawn flags), emit events consumed by Vuln Explorer resolver. | + +## Advisory AI (Sprint 31) + +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| CONCELIER-AIAI-31-001 `Paragraph anchors` | TODO | Concelier WebService Guild | CONCELIER-VULN-29-001 | Expose advisory chunk API returning paragraph anchors, section metadata, and token-safe text for Advisory AI retrieval. | +| CONCELIER-AIAI-31-002 `Structured fields` | TODO | Concelier WebService Guild | CONCELIER-AIAI-31-001 | Ensure normalized advisories expose workaround/fix/CVSS fields via API; add caching for summary queries. | +| CONCELIER-AIAI-31-003 `Advisory AI telemetry` | TODO | Concelier WebService Guild, Observability Guild | CONCELIER-AIAI-31-001 | Emit metrics/logs for chunk requests, cache hits, and guardrail blocks triggered by advisory payloads. | + +## Observability & Forensics (Epic 15) +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| CONCELIER-WEB-OBS-50-001 `Telemetry adoption` | TODO | Concelier WebService Guild | TELEMETRY-OBS-50-001, CONCELIER-OBS-50-001 | Adopt telemetry core in web service host, ensure ingest + read endpoints emit trace/log fields (`tenant_id`, `route`, `decision_effect`), and add correlation IDs to responses. | +| CONCELIER-WEB-OBS-51-001 `Observability APIs` | TODO | Concelier WebService Guild | CONCELIER-WEB-OBS-50-001, WEB-OBS-51-001 | Surface ingest health metrics, queue depth, and SLO status via `/obs/concelier/health` endpoint for Console widgets, with caching and tenant partitioning. | +| CONCELIER-WEB-OBS-52-001 `Timeline streaming` | TODO | Concelier WebService Guild | CONCELIER-WEB-OBS-50-001, TIMELINE-OBS-52-003 | Provide SSE stream `/obs/concelier/timeline` bridging to Timeline Indexer with paging tokens, guardrails, and audit logging. | +| CONCELIER-WEB-OBS-53-001 `Evidence locker integration` | TODO | Concelier WebService Guild, Evidence Locker Guild | CONCELIER-OBS-53-001, EVID-OBS-53-003 | Add `/evidence/advisories/*` routes invoking evidence locker snapshots, verifying tenant scopes (`evidence:read`), and returning signed manifest metadata. | +| CONCELIER-WEB-OBS-54-001 `Attestation exposure` | TODO | Concelier WebService Guild | CONCELIER-OBS-54-001, PROV-OBS-54-001 | Provide `/attestations/advisories/*` read APIs surfacing DSSE status, verification summary, and provenance chain for Console/CLI. | +| CONCELIER-WEB-OBS-55-001 `Incident mode toggles` | TODO | Concelier WebService Guild, DevOps Guild | CONCELIER-OBS-55-001, WEB-OBS-55-001 | Implement incident mode toggle endpoints, propagate to orchestrator/locker, and document cooldown/backoff semantics. | + +## Air-Gapped Mode (Epic 16) +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| CONCELIER-WEB-AIRGAP-56-001 `Mirror import APIs` | TODO | Concelier WebService Guild | AIRGAP-IMP-58-001, CONCELIER-AIRGAP-56-001 | Extend ingestion endpoints to register mirror bundle sources, expose bundle catalog queries, and block external feed URLs in sealed mode. | +| CONCELIER-WEB-AIRGAP-56-002 `Airgap status surfaces` | TODO | Concelier WebService Guild | CONCELIER-AIRGAP-57-002, AIRGAP-CTL-56-002 | Add staleness metadata and bundle provenance to advisory APIs (`/advisories/observations`, `/advisories/linksets`). | +| CONCELIER-WEB-AIRGAP-57-001 `Error remediation` | TODO | Concelier WebService Guild, AirGap Policy Guild | AIRGAP-POL-56-001 | Map sealed-mode violations to `AIRGAP_EGRESS_BLOCKED` responses with user guidance. | +| CONCELIER-WEB-AIRGAP-58-001 `Import timeline emission` | TODO | Concelier WebService Guild, AirGap Importer Guild | CONCELIER-WEB-AIRGAP-56-001, TIMELINE-OBS-53-001 | Emit timeline events for bundle ingestion operations with bundle ID, scope, and actor metadata. | + +## SDKs & OpenAPI (Epic 17) +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| CONCELIER-WEB-OAS-61-001 `/.well-known/openapi` | TODO | Concelier WebService Guild | OAS-61-001 | Implement discovery endpoint emitting Concelier spec with version metadata and ETag. | +| CONCELIER-WEB-OAS-61-002 `Error envelope migration` | TODO | Concelier WebService Guild | APIGOV-61-001 | Ensure all API responses use standardized error envelope; update controllers/tests. | +| CONCELIER-WEB-OAS-62-001 `Examples expansion` | TODO | Concelier WebService Guild | CONCELIER-OAS-61-002 | Add curated examples for advisory observations/linksets/conflicts; integrate into dev portal. | +| CONCELIER-WEB-OAS-63-001 `Deprecation headers` | TODO | Concelier WebService Guild, API Governance Guild | APIGOV-63-001 | Add Sunset/Deprecation headers for retiring endpoints and update documentation/notifications. | diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Acsc/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Acsc/TASKS.completed.md new file mode 100644 index 00000000..b7a2307f --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Acsc/TASKS.completed.md @@ -0,0 +1,18 @@ +# Completed Tasks + +|FEEDCONN-ACSC-02-001 Source discovery & feed contract|BE-Conn-ACSC|Research|**DONE (2025-10-11)** – Catalogued feed slugs `/acsc/view-all-content/{alerts,advisories,news,publications,threats}/rss`; every endpoint currently negotiates HTTP/2 then aborts with `INTERNAL_ERROR` (curl exit 92) and hanging >600 s when forcing `--http1.1`. Documented traces + mitigations in `docs/concelier-connector-research-20251011.md` and opened `FEEDCONN-SHARED-HTTP2-001` for shared handler tweaks (force `RequestVersionOrLower`, jittered retries, relay option).| + +|FEEDCONN-ACSC-02-002 Fetch pipeline & cursor persistence|BE-Conn-ACSC|Source.Common, Storage.Mongo|**DONE (2025-10-12)** – HTTP client now pins `HttpRequestMessage.VersionPolicy = RequestVersionOrLower`, forces `AutomaticDecompression = GZip | Deflate`, and sends `User-Agent: StellaOps/Concelier (+https://stella-ops.org)` via `AddAcscConnector`. Fetch pipeline implemented in `AcscConnector` with relay-aware fallback (`AcscProbeJob` seeds preference), deterministic cursor updates (`preferredEndpoint`, published timestamp per feed), and metadata-deduped documents. Unit tests `AcscConnectorFetchTests` + `AcscHttpClientConfigurationTests` cover direct/relay flows and client wiring.| + +|FEEDCONN-ACSC-02-003 Parser & DTO sanitiser|BE-Conn-ACSC|Source.Common|**DONE (2025-10-12)** – Added `AcscFeedParser` to sanitise RSS payloads, collapse multi-paragraph summaries, dedupe references, and surface `serialNumber`/`advisoryType` fields as structured metadata + alias candidates. `ParseAsync` now materialises `acsc.feed.v1` DTOs, promotes documents to `pending-map`, and advances cursor state. Covered by `AcscConnectorParseTests`.| + +|FEEDCONN-ACSC-02-004 Canonical mapper + range primitives|BE-Conn-ACSC|Models|**DONE (2025-10-12)** – Introduced `AcscMapper` and wired `MapAsync` to emit canonical advisories with normalized aliases, source-tagged references, and optional vendor `affectedPackages` derived from “Systems/Products affected” fields. Documents transition to `mapped`, advisories persist via `IAdvisoryStore`, and metrics/logging capture mapped counts. `AcscConnectorParseTests` exercise fetch→parse→map flow.| + +|FEEDCONN-ACSC-02-005 Deterministic fixtures & regression tests|QA|Testing|**DONE (2025-10-12)** – `AcscConnectorParseTests` now snapshots fetch→parse→map output via `Acsc/Fixtures/acsc-advisories.snapshot.json`; set `UPDATE_ACSC_FIXTURES=1` to regenerate. Tests assert DTO status transitions, advisory persistence, and state cleanup.| + +|FEEDCONN-ACSC-02-006 Diagnostics & documentation|DevEx|Docs|**DONE (2025-10-12)** – Added module README describing configuration, job schedules, metrics (including new `acsc.map.success` counter), relay behaviour, and fixture workflow. Diagnostics updated to count map successes alongside existing fetch/parse metrics.| + +|FEEDCONN-ACSC-02-007 Feed retention & pagination validation|BE-Conn-ACSC|Research|**DONE (2025-10-11)** – Relay sampling shows retention ≥ July 2025; need to re-run once direct HTTP/2 path is stable to see if feed caps at ~50 items and whether `?page=` exists. Pending action tracked in shared HTTP downgrade task.| + +|FEEDCONN-ACSC-02-008 HTTP client compatibility plan|BE-Conn-ACSC|Source.Common|**DONE (2025-10-11)** – Reproduced Akamai resets, drafted downgrade plan (two-stage HTTP/2 retry + relay fallback), and filed `FEEDCONN-SHARED-HTTP2-001`; module README TODO will host the per-environment knob matrix.| + diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Acsc/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Acsc/TASKS.md index cacd3a5b..2265b2fc 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Acsc/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Acsc/TASKS.md @@ -1,11 +1,3 @@ -# TASKS -| Task | Owner(s) | Depends on | Notes | -|---|---|---|---| -|FEEDCONN-ACSC-02-001 Source discovery & feed contract|BE-Conn-ACSC|Research|**DONE (2025-10-11)** – Catalogued feed slugs `/acsc/view-all-content/{alerts,advisories,news,publications,threats}/rss`; every endpoint currently negotiates HTTP/2 then aborts with `INTERNAL_ERROR` (curl exit 92) and hanging >600 s when forcing `--http1.1`. Documented traces + mitigations in `docs/concelier-connector-research-20251011.md` and opened `FEEDCONN-SHARED-HTTP2-001` for shared handler tweaks (force `RequestVersionOrLower`, jittered retries, relay option).| -|FEEDCONN-ACSC-02-002 Fetch pipeline & cursor persistence|BE-Conn-ACSC|Source.Common, Storage.Mongo|**DONE (2025-10-12)** – HTTP client now pins `HttpRequestMessage.VersionPolicy = RequestVersionOrLower`, forces `AutomaticDecompression = GZip | Deflate`, and sends `User-Agent: StellaOps/Concelier (+https://stella-ops.org)` via `AddAcscConnector`. Fetch pipeline implemented in `AcscConnector` with relay-aware fallback (`AcscProbeJob` seeds preference), deterministic cursor updates (`preferredEndpoint`, published timestamp per feed), and metadata-deduped documents. Unit tests `AcscConnectorFetchTests` + `AcscHttpClientConfigurationTests` cover direct/relay flows and client wiring.| -|FEEDCONN-ACSC-02-003 Parser & DTO sanitiser|BE-Conn-ACSC|Source.Common|**DONE (2025-10-12)** – Added `AcscFeedParser` to sanitise RSS payloads, collapse multi-paragraph summaries, dedupe references, and surface `serialNumber`/`advisoryType` fields as structured metadata + alias candidates. `ParseAsync` now materialises `acsc.feed.v1` DTOs, promotes documents to `pending-map`, and advances cursor state. Covered by `AcscConnectorParseTests`.| -|FEEDCONN-ACSC-02-004 Canonical mapper + range primitives|BE-Conn-ACSC|Models|**DONE (2025-10-12)** – Introduced `AcscMapper` and wired `MapAsync` to emit canonical advisories with normalized aliases, source-tagged references, and optional vendor `affectedPackages` derived from “Systems/Products affected” fields. Documents transition to `mapped`, advisories persist via `IAdvisoryStore`, and metrics/logging capture mapped counts. `AcscConnectorParseTests` exercise fetch→parse→map flow.| -|FEEDCONN-ACSC-02-005 Deterministic fixtures & regression tests|QA|Testing|**DONE (2025-10-12)** – `AcscConnectorParseTests` now snapshots fetch→parse→map output via `Acsc/Fixtures/acsc-advisories.snapshot.json`; set `UPDATE_ACSC_FIXTURES=1` to regenerate. Tests assert DTO status transitions, advisory persistence, and state cleanup.| -|FEEDCONN-ACSC-02-006 Diagnostics & documentation|DevEx|Docs|**DONE (2025-10-12)** – Added module README describing configuration, job schedules, metrics (including new `acsc.map.success` counter), relay behaviour, and fixture workflow. Diagnostics updated to count map successes alongside existing fetch/parse metrics.| -|FEEDCONN-ACSC-02-007 Feed retention & pagination validation|BE-Conn-ACSC|Research|**DONE (2025-10-11)** – Relay sampling shows retention ≥ July 2025; need to re-run once direct HTTP/2 path is stable to see if feed caps at ~50 items and whether `?page=` exists. Pending action tracked in shared HTTP downgrade task.| -|FEEDCONN-ACSC-02-008 HTTP client compatibility plan|BE-Conn-ACSC|Source.Common|**DONE (2025-10-11)** – Reproduced Akamai resets, drafted downgrade plan (two-stage HTTP/2 retry + relay fallback), and filed `FEEDCONN-SHARED-HTTP2-001`; module README TODO will host the per-environment knob matrix.| +# TASKS +| Task | Owner(s) | Depends on | Notes | +|---|---|---|---| diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Cccs/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Cccs/TASKS.completed.md new file mode 100644 index 00000000..4908d0fb --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Cccs/TASKS.completed.md @@ -0,0 +1,18 @@ +# Completed Tasks + +|FEEDCONN-CCCS-02-001 Catalogue official CCCS advisory feeds|BE-Conn-CCCS|Research|**DONE (2025-10-11)** – Resolved RSS→Atom redirects (`/api/cccs/rss/v1/get?...` → `/api/cccs/atom/v1/get?...`), confirmed feed caps at 50 entries with inline HTML bodies, no `Last-Modified`/`ETag`, and `updated` timestamps in UTC. Findings and packet captures parked in `docs/concelier-connector-research-20251011.md`; retention sweep follow-up tracked in 02-007.| + +|FEEDCONN-CCCS-02-002 Implement fetch & source state handling|BE-Conn-CCCS|Source.Common, Storage.Mongo|**DONE (2025-10-14)** – `CccsConnector.FetchAsync` now hydrates feeds via `CccsFeedClient`, persists per-entry JSON payloads with SHA256 dedupe and cursor state, throttles requests, and records taxonomy + language metadata in document state.| + +|FEEDCONN-CCCS-02-003 DTO/parser implementation|BE-Conn-CCCS|Source.Common|**DONE (2025-10-14)** – Added `CccsHtmlParser` to sanitize Atom body HTML, extract serial/date/product bullets, collapse whitespace, and emit normalized reference URLs; `ParseAsync` now persists DTO records under schema `cccs.dto.v1`.| + +|FEEDCONN-CCCS-02-004 Canonical mapping & range primitives|BE-Conn-CCCS|Models|**DONE (2025-10-14)** – `CccsMapper` now materializes canonical advisories (aliases from serial/source/CVEs, references incl. canonical URL, vendor package records) with provenance masks; `MapAsync` stores results in `AdvisoryStore`.| + +|FEEDCONN-CCCS-02-005 Deterministic fixtures & tests|QA|Testing|**DONE (2025-10-14)** – Added English/French fixtures plus parser + connector end-to-end tests (`StellaOps.Concelier.Connector.Cccs.Tests`). Canned HTTP handler + Mongo fixture enables fetch→parse→map regression; fixtures refresh via `UPDATE_CCCS_FIXTURES=1`.| + +|FEEDCONN-CCCS-02-006 Observability & documentation|DevEx|Docs|**DONE (2025-10-15)** – Added `CccsDiagnostics` meter (fetch/parse/map counters), enriched connector logs with document counts, and published `docs/modules/concelier/operations/connectors/cccs.md` covering config, telemetry, and sanitiser guidance.| + +|FEEDCONN-CCCS-02-007 Historical advisory harvesting plan|BE-Conn-CCCS|Research|**DONE (2025-10-15)** – Measured `/api/cccs/threats/v1/get` inventory (~5.1k rows/lang; earliest 2018-06-08), documented backfill workflow + language split strategy, and linked the runbook for Offline Kit execution.| + +|FEEDCONN-CCCS-02-008 Raw DOM parsing refinement|BE-Conn-CCCS|Source.Common|**DONE (2025-10-15)** – Parser now walks unsanitised DOM (heading + nested list coverage), sanitizer keeps ``/`section` nodes, and regression fixtures/tests assert EN/FR list handling + preserved HTML structure.| + diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Cccs/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Cccs/TASKS.md index fc95c11e..93ec1ccc 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Cccs/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Cccs/TASKS.md @@ -1,12 +1,4 @@ # TASKS | Task | Owner(s) | Depends on | Notes | |---|---|---|---| -|FEEDCONN-CCCS-02-001 Catalogue official CCCS advisory feeds|BE-Conn-CCCS|Research|**DONE (2025-10-11)** – Resolved RSS→Atom redirects (`/api/cccs/rss/v1/get?...` → `/api/cccs/atom/v1/get?...`), confirmed feed caps at 50 entries with inline HTML bodies, no `Last-Modified`/`ETag`, and `updated` timestamps in UTC. Findings and packet captures parked in `docs/concelier-connector-research-20251011.md`; retention sweep follow-up tracked in 02-007.| -|FEEDCONN-CCCS-02-002 Implement fetch & source state handling|BE-Conn-CCCS|Source.Common, Storage.Mongo|**DONE (2025-10-14)** – `CccsConnector.FetchAsync` now hydrates feeds via `CccsFeedClient`, persists per-entry JSON payloads with SHA256 dedupe and cursor state, throttles requests, and records taxonomy + language metadata in document state.| -|FEEDCONN-CCCS-02-003 DTO/parser implementation|BE-Conn-CCCS|Source.Common|**DONE (2025-10-14)** – Added `CccsHtmlParser` to sanitize Atom body HTML, extract serial/date/product bullets, collapse whitespace, and emit normalized reference URLs; `ParseAsync` now persists DTO records under schema `cccs.dto.v1`.| -|FEEDCONN-CCCS-02-004 Canonical mapping & range primitives|BE-Conn-CCCS|Models|**DONE (2025-10-14)** – `CccsMapper` now materializes canonical advisories (aliases from serial/source/CVEs, references incl. canonical URL, vendor package records) with provenance masks; `MapAsync` stores results in `AdvisoryStore`.| -|FEEDCONN-CCCS-02-005 Deterministic fixtures & tests|QA|Testing|**DONE (2025-10-14)** – Added English/French fixtures plus parser + connector end-to-end tests (`StellaOps.Concelier.Connector.Cccs.Tests`). Canned HTTP handler + Mongo fixture enables fetch→parse→map regression; fixtures refresh via `UPDATE_CCCS_FIXTURES=1`.| -|FEEDCONN-CCCS-02-006 Observability & documentation|DevEx|Docs|**DONE (2025-10-15)** – Added `CccsDiagnostics` meter (fetch/parse/map counters), enriched connector logs with document counts, and published `docs/modules/concelier/operations/connectors/cccs.md` covering config, telemetry, and sanitiser guidance.| -|FEEDCONN-CCCS-02-007 Historical advisory harvesting plan|BE-Conn-CCCS|Research|**DONE (2025-10-15)** – Measured `/api/cccs/threats/v1/get` inventory (~5.1k rows/lang; earliest 2018-06-08), documented backfill workflow + language split strategy, and linked the runbook for Offline Kit execution.| -|FEEDCONN-CCCS-02-008 Raw DOM parsing refinement|BE-Conn-CCCS|Source.Common|**DONE (2025-10-15)** – Parser now walks unsanitised DOM (heading + nested list coverage), sanitizer keeps ``/`section` nodes, and regression fixtures/tests assert EN/FR list handling + preserved HTML structure.| |FEEDCONN-CCCS-02-009 Normalized versions rollout (Oct 2025)|BE-Conn-CCCS|Merge coordination (`FEEDMERGE-COORD-02-900`)|**TODO (due 2025-10-21)** – Implement trailing-version split helper per Merge guidance (see `../Merge/RANGE_PRIMITIVES_COORDINATION.md` “Helper snippets”) to emit `NormalizedVersions` via `SemVerRangeRuleBuilder`; refresh mapper tests/fixtures to assert provenance notes (`cccs:{serial}:{index}`) and confirm merge counters drop.
2025-10-29: See `docs/dev/normalized-rule-recipes.md` for ready-made helper + regex snippet; wire into `BuildPackages` and update fixtures with `UPDATE_CCCS_FIXTURES=1`.| diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertBund/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertBund/TASKS.completed.md new file mode 100644 index 00000000..1eac38e1 --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertBund/TASKS.completed.md @@ -0,0 +1,20 @@ +# Completed Tasks + +|FEEDCONN-CERTBUND-02-001 Research CERT-Bund advisory endpoints|BE-Conn-CERTBUND|Research|**DONE (2025-10-11)** – Confirmed public RSS at `https://wid.cert-bund.de/content/public/securityAdvisory/rss` (HTTP 200 w/out cookies), 250-item window, German titles/categories, and detail links pointing to Angular SPA. Captured header profile (no cache hints) and logged open item to discover the JSON API used by `portal` frontend.| + +|FEEDCONN-CERTBUND-02-002 Fetch job & state persistence|BE-Conn-CERTBUND|Source.Common, Storage.Mongo|**DONE (2025-10-14)** – `CertBundConnector.FetchAsync` consumes RSS via session-bootstrapped client, stores per-advisory JSON documents with metadata + SHA, throttles detail requests, and maintains cursor state (pending docs/mappings, known advisory IDs, last published).| + +|FEEDCONN-CERTBUND-02-003 Parser/DTO implementation|BE-Conn-CERTBUND|Source.Common|**DONE (2025-10-14)** – Detail JSON piped through `CertBundDetailParser` (raw DOM sanitised to HTML), capturing severity, CVEs, product list, and references into DTO records (`cert-bund.detail.v1`).| + +|FEEDCONN-CERTBUND-02-004 Canonical mapping & range primitives|BE-Conn-CERTBUND|Models|**DONE (2025-10-14)** – `CertBundMapper` emits canonical advisories (aliases, references, vendor package ranges, provenance) with severity normalisation and deterministic ordering.| + +|FEEDCONN-CERTBUND-02-005 Regression fixtures & tests|QA|Testing|**DONE (2025-10-14)** – Added `StellaOps.Concelier.Connector.CertBund.Tests` covering fetch→parse→map against canned RSS/JSON fixtures; integration harness uses Mongo2Go + canned HTTP handler; fixtures regenerate via `UPDATE_CERTBUND_FIXTURES=1`.| + +|FEEDCONN-CERTBUND-02-006 Telemetry & documentation|DevEx|Docs|**DONE (2025-10-15)** – Added `CertBundDiagnostics` (meter `StellaOps.Concelier.Connector.CertBund`) with fetch/parse/map counters + histograms, recorded coverage days, wired stage summary logs, and published the ops runbook (`docs/modules/concelier/operations/connectors/certbund.md`).| + +|FEEDCONN-CERTBUND-02-007 Feed history & locale assessment|BE-Conn-CERTBUND|Research|**DONE (2025-10-15)** – Measured RSS retention (~6 days/≈250 items), captured connector-driven backfill guidance in the runbook, and aligned locale guidance (preserve `language=de`, Docs glossary follow-up). **Next:** coordinate with Tools to land the state-seeding helper so scripted backfills replace manual Mongo tweaks.| + +|FEEDCONN-CERTBUND-02-008 Session bootstrap & cookie strategy|BE-Conn-CERTBUND|Source.Common|**DONE (2025-10-14)** – Feed client primes the portal session (cookie container via `SocketsHttpHandler`), shares cookies across detail requests, and documents bootstrap behaviour in options (`PortalBootstrapUri`).| + +|FEEDCONN-CERTBUND-02-009 Offline Kit export packaging|BE-Conn-CERTBUND, Docs|Offline Kit|**DONE (2025-10-17)** – Added `src/Tools/certbund_offline_snapshot.py` to capture search/export JSON, emit deterministic manifests + SHA files, and refreshed docs (`docs/modules/concelier/operations/connectors/certbund.md`, `docs/24_OFFLINE_KIT.md`) with offline-kit instructions and manifest layout guidance. Seed data README/ignore rules cover local snapshot hygiene.| + diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertBund/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertBund/TASKS.md index 4cb60363..0bb837c6 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertBund/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertBund/TASKS.md @@ -1,13 +1,4 @@ # TASKS | Task | Owner(s) | Depends on | Notes | |---|---|---|---| -|FEEDCONN-CERTBUND-02-001 Research CERT-Bund advisory endpoints|BE-Conn-CERTBUND|Research|**DONE (2025-10-11)** – Confirmed public RSS at `https://wid.cert-bund.de/content/public/securityAdvisory/rss` (HTTP 200 w/out cookies), 250-item window, German titles/categories, and detail links pointing to Angular SPA. Captured header profile (no cache hints) and logged open item to discover the JSON API used by `portal` frontend.| -|FEEDCONN-CERTBUND-02-002 Fetch job & state persistence|BE-Conn-CERTBUND|Source.Common, Storage.Mongo|**DONE (2025-10-14)** – `CertBundConnector.FetchAsync` consumes RSS via session-bootstrapped client, stores per-advisory JSON documents with metadata + SHA, throttles detail requests, and maintains cursor state (pending docs/mappings, known advisory IDs, last published).| -|FEEDCONN-CERTBUND-02-003 Parser/DTO implementation|BE-Conn-CERTBUND|Source.Common|**DONE (2025-10-14)** – Detail JSON piped through `CertBundDetailParser` (raw DOM sanitised to HTML), capturing severity, CVEs, product list, and references into DTO records (`cert-bund.detail.v1`).| -|FEEDCONN-CERTBUND-02-004 Canonical mapping & range primitives|BE-Conn-CERTBUND|Models|**DONE (2025-10-14)** – `CertBundMapper` emits canonical advisories (aliases, references, vendor package ranges, provenance) with severity normalisation and deterministic ordering.| -|FEEDCONN-CERTBUND-02-005 Regression fixtures & tests|QA|Testing|**DONE (2025-10-14)** – Added `StellaOps.Concelier.Connector.CertBund.Tests` covering fetch→parse→map against canned RSS/JSON fixtures; integration harness uses Mongo2Go + canned HTTP handler; fixtures regenerate via `UPDATE_CERTBUND_FIXTURES=1`.| -|FEEDCONN-CERTBUND-02-006 Telemetry & documentation|DevEx|Docs|**DONE (2025-10-15)** – Added `CertBundDiagnostics` (meter `StellaOps.Concelier.Connector.CertBund`) with fetch/parse/map counters + histograms, recorded coverage days, wired stage summary logs, and published the ops runbook (`docs/modules/concelier/operations/connectors/certbund.md`).| -|FEEDCONN-CERTBUND-02-007 Feed history & locale assessment|BE-Conn-CERTBUND|Research|**DONE (2025-10-15)** – Measured RSS retention (~6 days/≈250 items), captured connector-driven backfill guidance in the runbook, and aligned locale guidance (preserve `language=de`, Docs glossary follow-up). **Next:** coordinate with Tools to land the state-seeding helper so scripted backfills replace manual Mongo tweaks.| -|FEEDCONN-CERTBUND-02-008 Session bootstrap & cookie strategy|BE-Conn-CERTBUND|Source.Common|**DONE (2025-10-14)** – Feed client primes the portal session (cookie container via `SocketsHttpHandler`), shares cookies across detail requests, and documents bootstrap behaviour in options (`PortalBootstrapUri`).| -|FEEDCONN-CERTBUND-02-009 Offline Kit export packaging|BE-Conn-CERTBUND, Docs|Offline Kit|**DONE (2025-10-17)** – Added `src/Tools/certbund_offline_snapshot.py` to capture search/export JSON, emit deterministic manifests + SHA files, and refreshed docs (`docs/modules/concelier/operations/connectors/certbund.md`, `docs/24_OFFLINE_KIT.md`) with offline-kit instructions and manifest layout guidance. Seed data README/ignore rules cover local snapshot hygiene.| |FEEDCONN-CERTBUND-02-010 Normalized range translator|BE-Conn-CERTBUND|Merge coordination (`FEEDMERGE-COORD-02-900`)|**TODO (due 2025-10-22)** – Translate `product.Versions` phrases (e.g., `2023.1 bis 2024.2`, `alle`) into comparator strings for `SemVerRangeRuleBuilder`, emit `NormalizedVersions` with `certbund:{advisoryId}:{vendor}` provenance, and extend tests/README with localisation notes.| diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertCc/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertCc/TASKS.completed.md new file mode 100644 index 00000000..b1439b5d --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertCc/TASKS.completed.md @@ -0,0 +1,24 @@ +# Completed Tasks + +|Document CERT/CC advisory sources|BE-Conn-CERTCC|Research|**DONE (2025-10-10)** – Catalogued Atom feed + VINCE API endpoints and archive references in `README.md`; include polling/backoff guidance.| + +|Fetch pipeline & state tracking|BE-Conn-CERTCC|Source.Common, Storage.Mongo|**DONE (2025-10-12)** – Summary planner + fetch job persist monthly/yearly VINCE JSON to `DocumentStore`, hydrate the `TimeWindowCursorState`, and snapshot regression (`dotnet test` 2025-10-12) confirmed deterministic resume behaviour.| + +|VINCE note detail fetcher|BE-Conn-CERTCC|Source.Common, Storage.Mongo|**DONE (2025-10-12)** – Detail bundle fetch now enqueues VU identifiers and persists note/vendors/vuls/vendors-vuls documents with ETag/Last-Modified metadata, tolerating missing optional endpoints without wedging the cursor.| + +|DTO & parser implementation|BE-Conn-CERTCC|Source.Common|**DONE (2025-10-12)** – VINCE DTO aggregate materialises note/vendor/vulnerability payloads, normalises markdown to HTML-safe fragments, and surfaces vendor impact statements covered by parser unit tests.| + +|Canonical mapping & range primitives|BE-Conn-CERTCC|Models|**DONE (2025-10-12)** – Mapper emits aliases (VU#, CVE), vendor range primitives, and normalizedVersions (`scheme=certcc.vendor`) with provenance masks; `certcc-advisories.snapshot.json` validates canonical output after schema sync.| + +|Deterministic fixtures/tests|QA|Testing|**DONE (2025-10-11)** – Snapshot harness regenerated (`certcc-*.snapshot.json`), request ordering assertions added, and `UPDATE_CERTCC_FIXTURES` workflow verified for CI determinism.| + +|Connector test harness remediation|BE-Conn-CERTCC, QA|Testing|**DONE (2025-10-11)** – Connector test harness now rebuilds `FakeTimeProvider`, wires `AddSourceCommon`, and drives canned VINCE responses across fetch→parse→map with recorded-request assertions.| + +|Snapshot coverage handoff|QA|Models, Merge|**DONE (2025-10-11)** – Fixtures + request/advisory snapshots refreshed, README documents `UPDATE_CERTCC_FIXTURES` workflow, and recorded-request ordering is enforced for QA handoff.| + +|FEEDCONN-CERTCC-02-010 Partial-detail graceful degradation|BE-Conn-CERTCC|Connector plan|**DONE (2025-10-12)** – Detail fetch now catches 404/410/403 responses for optional endpoints, logs missing bundles, feeds empty payloads into parsing, and ships regression coverage for mixed responses.| + +|FEEDCONN-CERTCC-02-012 Schema sync & snapshot regen follow-up|QA, BE-Conn-CERTCC|Models `FEEDMODELS-SCHEMA-01-001`/`-002`/`-003`, Storage `FEEDSTORAGE-DATA-02-001`|**DONE (2025-10-12)** – Snapshot suite rerun, fixtures updated, and handoff notes (`FEEDCONN-CERTCC-02-012_HANDOFF.md`) document normalizedVersions/provenance expectations for Merge backfill.| + +|Telemetry & documentation|DevEx|Docs|**DONE (2025-10-12)** – `CertCcDiagnostics` now publishes summary/detail/parse/map metrics, README documents meter names, and structured logging guidance is captured for Ops handoff.| + diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertCc/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertCc/TASKS.md index c1e05929..2265b2fc 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertCc/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertCc/TASKS.md @@ -1,14 +1,3 @@ -# TASKS -| Task | Owner(s) | Depends on | Notes | -|---|---|---|---| -|Document CERT/CC advisory sources|BE-Conn-CERTCC|Research|**DONE (2025-10-10)** – Catalogued Atom feed + VINCE API endpoints and archive references in `README.md`; include polling/backoff guidance.| -|Fetch pipeline & state tracking|BE-Conn-CERTCC|Source.Common, Storage.Mongo|**DONE (2025-10-12)** – Summary planner + fetch job persist monthly/yearly VINCE JSON to `DocumentStore`, hydrate the `TimeWindowCursorState`, and snapshot regression (`dotnet test` 2025-10-12) confirmed deterministic resume behaviour.| -|VINCE note detail fetcher|BE-Conn-CERTCC|Source.Common, Storage.Mongo|**DONE (2025-10-12)** – Detail bundle fetch now enqueues VU identifiers and persists note/vendors/vuls/vendors-vuls documents with ETag/Last-Modified metadata, tolerating missing optional endpoints without wedging the cursor.| -|DTO & parser implementation|BE-Conn-CERTCC|Source.Common|**DONE (2025-10-12)** – VINCE DTO aggregate materialises note/vendor/vulnerability payloads, normalises markdown to HTML-safe fragments, and surfaces vendor impact statements covered by parser unit tests.| -|Canonical mapping & range primitives|BE-Conn-CERTCC|Models|**DONE (2025-10-12)** – Mapper emits aliases (VU#, CVE), vendor range primitives, and normalizedVersions (`scheme=certcc.vendor`) with provenance masks; `certcc-advisories.snapshot.json` validates canonical output after schema sync.| -|Deterministic fixtures/tests|QA|Testing|**DONE (2025-10-11)** – Snapshot harness regenerated (`certcc-*.snapshot.json`), request ordering assertions added, and `UPDATE_CERTCC_FIXTURES` workflow verified for CI determinism.| -|Connector test harness remediation|BE-Conn-CERTCC, QA|Testing|**DONE (2025-10-11)** – Connector test harness now rebuilds `FakeTimeProvider`, wires `AddSourceCommon`, and drives canned VINCE responses across fetch→parse→map with recorded-request assertions.| -|Snapshot coverage handoff|QA|Models, Merge|**DONE (2025-10-11)** – Fixtures + request/advisory snapshots refreshed, README documents `UPDATE_CERTCC_FIXTURES` workflow, and recorded-request ordering is enforced for QA handoff.| -|FEEDCONN-CERTCC-02-010 Partial-detail graceful degradation|BE-Conn-CERTCC|Connector plan|**DONE (2025-10-12)** – Detail fetch now catches 404/410/403 responses for optional endpoints, logs missing bundles, feeds empty payloads into parsing, and ships regression coverage for mixed responses.| -|FEEDCONN-CERTCC-02-012 Schema sync & snapshot regen follow-up|QA, BE-Conn-CERTCC|Models `FEEDMODELS-SCHEMA-01-001`/`-002`/`-003`, Storage `FEEDSTORAGE-DATA-02-001`|**DONE (2025-10-12)** – Snapshot suite rerun, fixtures updated, and handoff notes (`FEEDCONN-CERTCC-02-012_HANDOFF.md`) document normalizedVersions/provenance expectations for Merge backfill.| -|Telemetry & documentation|DevEx|Docs|**DONE (2025-10-12)** – `CertCcDiagnostics` now publishes summary/detail/parse/map metrics, README documents meter names, and structured logging guidance is captured for Ops handoff.| +# TASKS +| Task | Owner(s) | Depends on | Notes | +|---|---|---|---| diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertFr/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertFr/TASKS.completed.md new file mode 100644 index 00000000..7744a7ca --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertFr/TASKS.completed.md @@ -0,0 +1,18 @@ +# Completed Tasks + +|RSS/list fetcher with sliding window|BE-Conn-CertFr|Source.Common|**DONE** – RSS/list ingestion implemented with sliding date cursor.| + +|Detail page fetch and sanitizer|BE-Conn-CertFr|Source.Common|**DONE** – HTML sanitizer trims boilerplate prior to DTO mapping.| + +|Extractor and schema validation of DTO|BE-Conn-CertFr, QA|Source.Common|**DONE** – DTO parsing validates structure before persistence.| + +|Canonical mapping (aliases, refs, severity text)|BE-Conn-CertFr|Models|**DONE** – mapper emits enrichment references with severity text.| + +|Watermark plus dedupe by sha256|BE-Conn-CertFr|Storage.Mongo|**DONE** – SHA comparisons skip unchanged docs; covered by duplicate/not-modified connector tests.| + +|Golden fixtures and determinism tests|QA|Source.CertFr|**DONE** – snapshot fixtures added in `CertFrConnectorTests` to enforce deterministic output.| + +|Mark failure/backoff on fetch errors|BE-Conn-CertFr|Storage.Mongo|**DONE** – fetch path now marks failures/backoff and tests assert state repository updates.| + +|Conditional fetch caching|BE-Conn-CertFr|Source.Common|**DONE** – ETag/Last-Modified support wired via `SourceFetchService` and verified in not-modified test.| + diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertFr/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertFr/TASKS.md index c15ae640..2265b2fc 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertFr/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertFr/TASKS.md @@ -1,11 +1,3 @@ -# TASKS -| Task | Owner(s) | Depends on | Notes | -|---|---|---|---| -|RSS/list fetcher with sliding window|BE-Conn-CertFr|Source.Common|**DONE** – RSS/list ingestion implemented with sliding date cursor.| -|Detail page fetch and sanitizer|BE-Conn-CertFr|Source.Common|**DONE** – HTML sanitizer trims boilerplate prior to DTO mapping.| -|Extractor and schema validation of DTO|BE-Conn-CertFr, QA|Source.Common|**DONE** – DTO parsing validates structure before persistence.| -|Canonical mapping (aliases, refs, severity text)|BE-Conn-CertFr|Models|**DONE** – mapper emits enrichment references with severity text.| -|Watermark plus dedupe by sha256|BE-Conn-CertFr|Storage.Mongo|**DONE** – SHA comparisons skip unchanged docs; covered by duplicate/not-modified connector tests.| -|Golden fixtures and determinism tests|QA|Source.CertFr|**DONE** – snapshot fixtures added in `CertFrConnectorTests` to enforce deterministic output.| -|Mark failure/backoff on fetch errors|BE-Conn-CertFr|Storage.Mongo|**DONE** – fetch path now marks failures/backoff and tests assert state repository updates.| -|Conditional fetch caching|BE-Conn-CertFr|Source.Common|**DONE** – ETag/Last-Modified support wired via `SourceFetchService` and verified in not-modified test.| +# TASKS +| Task | Owner(s) | Depends on | Notes | +|---|---|---|---| diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertIn/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertIn/TASKS.completed.md new file mode 100644 index 00000000..9a8c5a1e --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertIn/TASKS.completed.md @@ -0,0 +1,16 @@ +# Completed Tasks + +|Index/detail crawler with windowing|BE-Conn-CertIn|Source.Common|**DONE** – index/detail fetch implemented with window overlap and pagination.| + +|Extractor (title/CVEs/mitigation)|BE-Conn-CertIn|Source.Common|**DONE** – parser normalizes encodings, CVE lists, and mitigation text.| + +|DTO validation and sanitizer|BE-Conn-CertIn, QA|Source.Common|**DONE** – HTML sanitizer produces DTO before persistence.| + +|Canonical mapping (aliases, refs)|BE-Conn-CertIn|Models|**DONE** – mapper creates CERT-IN aliases plus typed references.| + +|State/dedupe and fixtures|BE-Conn-CertIn, QA|Storage.Mongo|**DONE** – snapshot/resume tests cover dedupe and cursor handling.| + +|Mark failure/backoff on fetch errors|BE-Conn-CertIn|Storage.Mongo|**DONE** – fetch pipeline marks failures/backoff with unit coverage.| + +|Conditional fetch caching|BE-Conn-CertIn|Source.Common|**DONE** – connector reuses ETag/Last-Modified; tests verify not-modified flow.| + diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertIn/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertIn/TASKS.md index c821b649..2265b2fc 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertIn/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertIn/TASKS.md @@ -1,10 +1,3 @@ -# TASKS -| Task | Owner(s) | Depends on | Notes | -|---|---|---|---| -|Index/detail crawler with windowing|BE-Conn-CertIn|Source.Common|**DONE** – index/detail fetch implemented with window overlap and pagination.| -|Extractor (title/CVEs/mitigation)|BE-Conn-CertIn|Source.Common|**DONE** – parser normalizes encodings, CVE lists, and mitigation text.| -|DTO validation and sanitizer|BE-Conn-CertIn, QA|Source.Common|**DONE** – HTML sanitizer produces DTO before persistence.| -|Canonical mapping (aliases, refs)|BE-Conn-CertIn|Models|**DONE** – mapper creates CERT-IN aliases plus typed references.| -|State/dedupe and fixtures|BE-Conn-CertIn, QA|Storage.Mongo|**DONE** – snapshot/resume tests cover dedupe and cursor handling.| -|Mark failure/backoff on fetch errors|BE-Conn-CertIn|Storage.Mongo|**DONE** – fetch pipeline marks failures/backoff with unit coverage.| -|Conditional fetch caching|BE-Conn-CertIn|Source.Common|**DONE** – connector reuses ETag/Last-Modified; tests verify not-modified flow.| +# TASKS +| Task | Owner(s) | Depends on | Notes | +|---|---|---|---| diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Common/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Common/TASKS.completed.md new file mode 100644 index 00000000..497caa0e --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Common/TASKS.completed.md @@ -0,0 +1,32 @@ +# Completed Tasks + +|Register source HTTP clients with allowlists and timeouts|BE-Conn-Shared|Source.Common|**DONE** – `AddSourceHttpClient` wires named clients with host allowlists/timeouts.| + +|Implement retry/backoff with jitter and 429 handling|BE-Conn-Shared|Source.Common|**DONE** – `SourceRetryPolicy` retries with 429/5xx handling and exponential backoff.| + +|Conditional GET helpers (ETag/Last-Modified)|BE-Conn-Shared|Source.Common|**DONE** – `SourceFetchRequest` + fetch result propagate etag/last-modified for NotModified handling.| + +|Windowed cursor and pagination utilities|BE-Conn-Shared|Source.Common|**DONE** – `TimeWindowCursorPlanner` + `PaginationPlanner` centralize sliding windows and additional page indices.| + +|JSON/XML schema validators with rich errors|BE-Conn-Shared, QA|Source.Common|DONE – JsonSchemaValidator surfaces keyword/path/message details + tests.| + +|Raw document capture helper|BE-Conn-Shared|Storage.Mongo|**DONE** – `SourceFetchService` stores raw payload + headers with sha256 metadata.| + +|Canned HTTP test harness|QA|Source.Common|DONE – enriched `CannedHttpMessageHandler` with method-aware queues, request capture, fallbacks, and helpers + unit coverage.| + +|HTML sanitization and URL normalization utilities|BE-Conn-Shared|Source.Common|DONE – `HtmlContentSanitizer` + `UrlNormalizer` provide safe fragments and canonical links for connectors.| + +|PDF-to-text sandbox helper|BE-Conn-Shared|Source.Common|DONE – `PdfTextExtractor` uses PdfPig to yield deterministic text with options + tests.| + +|PURL and SemVer helper library|BE-Conn-Shared|Models|DONE – `PackageCoordinateHelper` exposes normalized purl + SemVer parsing utilities backed by normalization.| + +|Telemetry wiring (logs/metrics/traces)|BE-Conn-Shared|Observability|DONE – `SourceDiagnostics` emits Activity/Meter signals integrated into fetch pipeline and WebService OTEL setup.| + +|Shared jitter source in retry policy|BE-Conn-Shared|Source.Common|**DONE** – `SourceRetryPolicy` now consumes injected `CryptoJitterSource` for thread-safe jitter.| + +|Allow per-request Accept header overrides|BE-Conn-Shared|Source.Common|**DONE** – `SourceFetchRequest.AcceptHeaders` honored by `SourceFetchService` plus unit tests for overrides.| + +|FEEDCONN-SHARED-HTTP2-001 HTTP version fallback policy|BE-Conn-Shared, Source.Common|Source.Common|**DONE (2025-10-11)** – `AddSourceHttpClient` now honours per-connector HTTP version/ policy, exposes handler customisation, and defaults to downgrade-friendly settings; unit tests cover handler configuration hook.| + +|FEEDCONN-SHARED-TLS-001 Sovereign trust store support|BE-Conn-Shared, Ops|Source.Common|**DONE (2025-10-11)** – `SourceHttpClientOptions` now exposes `TrustedRootCertificates`, `ServerCertificateCustomValidation`, and `AllowInvalidServerCertificates`, and `AddSourceHttpClient` runs the shared configuration binder so connectors can pull `concelier:httpClients|sources::http` settings (incl. Offline Kit relative PEM paths via `concelier:offline:root`). Tests cover handler wiring. Ops follow-up: package RU trust roots for Offline Kit distribution.| + diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Common/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Common/TASKS.md index 65ad7875..87124829 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Common/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Common/TASKS.md @@ -1,19 +1,4 @@ -# TASKS -| Task | Owner(s) | Depends on | Notes | -|---|---|---|---| -|Register source HTTP clients with allowlists and timeouts|BE-Conn-Shared|Source.Common|**DONE** – `AddSourceHttpClient` wires named clients with host allowlists/timeouts.| -|Implement retry/backoff with jitter and 429 handling|BE-Conn-Shared|Source.Common|**DONE** – `SourceRetryPolicy` retries with 429/5xx handling and exponential backoff.| -|Conditional GET helpers (ETag/Last-Modified)|BE-Conn-Shared|Source.Common|**DONE** – `SourceFetchRequest` + fetch result propagate etag/last-modified for NotModified handling.| -|Windowed cursor and pagination utilities|BE-Conn-Shared|Source.Common|**DONE** – `TimeWindowCursorPlanner` + `PaginationPlanner` centralize sliding windows and additional page indices.| -|JSON/XML schema validators with rich errors|BE-Conn-Shared, QA|Source.Common|DONE – JsonSchemaValidator surfaces keyword/path/message details + tests.| -|Raw document capture helper|BE-Conn-Shared|Storage.Mongo|**DONE** – `SourceFetchService` stores raw payload + headers with sha256 metadata.| -|Canned HTTP test harness|QA|Source.Common|DONE – enriched `CannedHttpMessageHandler` with method-aware queues, request capture, fallbacks, and helpers + unit coverage.| -|HTML sanitization and URL normalization utilities|BE-Conn-Shared|Source.Common|DONE – `HtmlContentSanitizer` + `UrlNormalizer` provide safe fragments and canonical links for connectors.| -|PDF-to-text sandbox helper|BE-Conn-Shared|Source.Common|DONE – `PdfTextExtractor` uses PdfPig to yield deterministic text with options + tests.| -|PURL and SemVer helper library|BE-Conn-Shared|Models|DONE – `PackageCoordinateHelper` exposes normalized purl + SemVer parsing utilities backed by normalization.| -|Telemetry wiring (logs/metrics/traces)|BE-Conn-Shared|Observability|DONE – `SourceDiagnostics` emits Activity/Meter signals integrated into fetch pipeline and WebService OTEL setup.| -|Shared jitter source in retry policy|BE-Conn-Shared|Source.Common|**DONE** – `SourceRetryPolicy` now consumes injected `CryptoJitterSource` for thread-safe jitter.| -|Allow per-request Accept header overrides|BE-Conn-Shared|Source.Common|**DONE** – `SourceFetchRequest.AcceptHeaders` honored by `SourceFetchService` plus unit tests for overrides.| -|FEEDCONN-SHARED-HTTP2-001 HTTP version fallback policy|BE-Conn-Shared, Source.Common|Source.Common|**DONE (2025-10-11)** – `AddSourceHttpClient` now honours per-connector HTTP version/ policy, exposes handler customisation, and defaults to downgrade-friendly settings; unit tests cover handler configuration hook.| -|FEEDCONN-SHARED-TLS-001 Sovereign trust store support|BE-Conn-Shared, Ops|Source.Common|**DONE (2025-10-11)** – `SourceHttpClientOptions` now exposes `TrustedRootCertificates`, `ServerCertificateCustomValidation`, and `AllowInvalidServerCertificates`, and `AddSourceHttpClient` runs the shared configuration binder so connectors can pull `concelier:httpClients|sources::http` settings (incl. Offline Kit relative PEM paths via `concelier:offline:root`). Tests cover handler wiring. Ops follow-up: package RU trust roots for Offline Kit distribution.| +# TASKS +| Task | Owner(s) | Depends on | Notes | +|---|---|---|---| |FEEDCONN-SHARED-STATE-003 Source state seeding helper|Tools Guild, BE-Conn-MSRC|Tools|**DOING (2025-10-19)** – Provide a reusable CLI/utility to seed `pendingDocuments`/`pendingMappings` for connectors (MSRC backfills require scripted CVRF + detail injection). Coordinate with MSRC team for expected JSON schema and handoff once prototype lands. Prereqs confirmed none (2025-10-19).| diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Cve/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Cve/TASKS.completed.md new file mode 100644 index 00000000..3631a5cf --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Cve/TASKS.completed.md @@ -0,0 +1,20 @@ +# Completed Tasks + +|Define CVE data source + contract|BE-Conn-CVE|Research|**DONE (2025-10-10)** – Connector targets the CVE Services JSON 5 API with authenticated windowed queries documented in `CveOptions` (`CVE-API-*` headers, pagination semantics, failure backoff).| + +|Fetch/cursor implementation|BE-Conn-CVE|Source.Common, Storage.Mongo|**DONE (2025-10-10)** – Time-window + page-aware cursor with SourceFetchService fetching list/detail pairs, resumable state persisted via `CveCursor`.| + +|DTOs & parser|BE-Conn-CVE|Source.Common|**DONE (2025-10-10)** – `CveRecordParser` and DTOs capture aliases, references, metrics, vendor ranges; sanitises text and timestamps.| + +|Canonical mapping & range primitives|BE-Conn-CVE|Models|**DONE (2025-10-10)** – `CveMapper` emits canonical advisories, vendor range primitives, SemVer/range statuses, references, CVSS normalization.
2025-10-11 research trail: confirm subsequent MR adds `NormalizedVersions` shaped like `[{"scheme":"semver","type":"range","min":"","minInclusive":true,"max":"","maxInclusive":false,"notes":"nvd:CVE-2025-XXXX"}]` so storage provenance joins continue to work.| + +|Deterministic tests & fixtures|QA|Testing|**DONE (2025-10-10)** – Added `StellaOps.Concelier.Connector.Cve.Tests` harness with canned fixtures + snapshot regression covering fetch/parse/map.| + +|Observability & docs|DevEx|Docs|**DONE (2025-10-10)** – Diagnostics meter (`cve.fetch.*`, etc.) wired; options/usage documented via `CveServiceCollectionExtensions`.| + +|Operator rollout playbook|BE-Conn-CVE, Ops|Docs|**DONE (2025-10-12)** – Refreshed `docs/modules/concelier/operations/connectors/cve-kev.md` with credential checklist, smoke book, PromQL guardrails, and linked Grafana pack (`docs/modules/concelier/operations/connectors/cve-kev-grafana-dashboard.json`).| + +|Live smoke & monitoring|QA, BE-Conn-CVE|WebService, Observability|**DONE (2025-10-15)** – Executed connector harness smoke using CVE Services sample window (CVE-2024-0001), confirmed fetch/parse/map telemetry (`cve.fetch.*`, `cve.map.success`) all incremented once, and archived the summary log + Grafana import guidance in `docs/modules/concelier/operations/connectors/cve-kev.md` (“Staging smoke 2025-10-15”).| + +|FEEDCONN-CVE-02-003 Normalized versions rollout|BE-Conn-CVE|Models `FEEDMODELS-SCHEMA-01-003`, Normalization playbook|**DONE (2025-10-12)** – Confirmed SemVer primitives map to normalized rules with `cve:{cveId}:{identifier}` notes and refreshed snapshots; `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Cve.Tests` passes on net10 preview.| + diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Cve/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Cve/TASKS.md index 583bd862..2265b2fc 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Cve/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Cve/TASKS.md @@ -1,12 +1,3 @@ # TASKS | Task | Owner(s) | Depends on | Notes | |---|---|---|---| -|Define CVE data source + contract|BE-Conn-CVE|Research|**DONE (2025-10-10)** – Connector targets the CVE Services JSON 5 API with authenticated windowed queries documented in `CveOptions` (`CVE-API-*` headers, pagination semantics, failure backoff).| -|Fetch/cursor implementation|BE-Conn-CVE|Source.Common, Storage.Mongo|**DONE (2025-10-10)** – Time-window + page-aware cursor with SourceFetchService fetching list/detail pairs, resumable state persisted via `CveCursor`.| -|DTOs & parser|BE-Conn-CVE|Source.Common|**DONE (2025-10-10)** – `CveRecordParser` and DTOs capture aliases, references, metrics, vendor ranges; sanitises text and timestamps.| -|Canonical mapping & range primitives|BE-Conn-CVE|Models|**DONE (2025-10-10)** – `CveMapper` emits canonical advisories, vendor range primitives, SemVer/range statuses, references, CVSS normalization.
2025-10-11 research trail: confirm subsequent MR adds `NormalizedVersions` shaped like `[{"scheme":"semver","type":"range","min":"","minInclusive":true,"max":"","maxInclusive":false,"notes":"nvd:CVE-2025-XXXX"}]` so storage provenance joins continue to work.| -|Deterministic tests & fixtures|QA|Testing|**DONE (2025-10-10)** – Added `StellaOps.Concelier.Connector.Cve.Tests` harness with canned fixtures + snapshot regression covering fetch/parse/map.| -|Observability & docs|DevEx|Docs|**DONE (2025-10-10)** – Diagnostics meter (`cve.fetch.*`, etc.) wired; options/usage documented via `CveServiceCollectionExtensions`.| -|Operator rollout playbook|BE-Conn-CVE, Ops|Docs|**DONE (2025-10-12)** – Refreshed `docs/modules/concelier/operations/connectors/cve-kev.md` with credential checklist, smoke book, PromQL guardrails, and linked Grafana pack (`docs/modules/concelier/operations/connectors/cve-kev-grafana-dashboard.json`).| -|Live smoke & monitoring|QA, BE-Conn-CVE|WebService, Observability|**DONE (2025-10-15)** – Executed connector harness smoke using CVE Services sample window (CVE-2024-0001), confirmed fetch/parse/map telemetry (`cve.fetch.*`, `cve.map.success`) all incremented once, and archived the summary log + Grafana import guidance in `docs/modules/concelier/operations/connectors/cve-kev.md` (“Staging smoke 2025-10-15”).| -|FEEDCONN-CVE-02-003 Normalized versions rollout|BE-Conn-CVE|Models `FEEDMODELS-SCHEMA-01-003`, Normalization playbook|**DONE (2025-10-12)** – Confirmed SemVer primitives map to normalized rules with `cve:{cveId}:{identifier}` notes and refreshed snapshots; `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Cve.Tests` passes on net10 preview.| diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Distro.RedHat/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Distro.RedHat/TASKS.completed.md new file mode 100644 index 00000000..d055d1df --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Distro.RedHat/TASKS.completed.md @@ -0,0 +1,26 @@ +# Completed Tasks + +|Hydra fetch with after= cursor|BE-Conn-RH|Source.Common|**DONE** – windowed paging with overlap, ETag/Last-Modified persisted.| + +|DTOs for Security Data + OVAL|BE-Conn-RH|Tests|**DONE** – CSAF payloads serialized into `redhat.csaf.v2` DTOs.| + +|NEVRA parser/comparer (complete)|BE-Conn-RH|Models|**DONE** – parser/comparer shipped with coverage; add edge cases as needed.| + +|Mapper to canonical rpm/cpe affected|BE-Conn-RH|Models|**DONE** – maps fixed/known ranges, CPE provenance, status ranges.| + +|Job scheduler registration aligns with Options pipeline|BE-Conn-RH|Core|**DONE** – registered fetch/parse/map via JobSchedulerBuilder, preserving option overrides and tightening cron/timeouts.| + +|Watermark persistence + resume|BE-Conn-RH|Storage.Mongo|**DONE** – cursor updates via SourceStateRepository.| + +|Precedence tests vs NVD|QA|Merge|**DONE** – Added AffectedPackagePrecedenceResolver + tests ensuring Red Hat CPEs override NVD ranges.| + +|Golden mapping fixtures|QA|Fixtures|**DONE** – fixture validation test now snapshots RHSA-2025:0001/0002/0003 with env-driven regeneration.| + +|Job scheduling defaults for source:redhat tasks|BE-Core|JobScheduler|**DONE** – Cron windows + per-job timeouts defined for fetch/parse/map.| + +|Express unaffected/investigation statuses without overloading range fields|BE-Conn-RH|Models|**DONE** – Introduced AffectedPackageStatus collection and updated mapper/tests.| + +|Reference dedupe & ordering in mapper|BE-Conn-RH|Models|DONE – mapper consolidates by URL, merges metadata, deterministic ordering validated in tests.| + +|Hydra summary fetch through SourceFetchService|BE-Conn-RH|Source.Common|DONE – summary pages now fetched via SourceFetchService with cache + conditional headers.| + diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Distro.RedHat/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Distro.RedHat/TASKS.md index f96b914e..e0b66ab7 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Distro.RedHat/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Distro.RedHat/TASKS.md @@ -1,16 +1,4 @@ -# TASKS -| Task | Owner(s) | Depends on | Notes | -|---|---|---|---| -|Hydra fetch with after= cursor|BE-Conn-RH|Source.Common|**DONE** – windowed paging with overlap, ETag/Last-Modified persisted.| -|DTOs for Security Data + OVAL|BE-Conn-RH|Tests|**DONE** – CSAF payloads serialized into `redhat.csaf.v2` DTOs.| -|NEVRA parser/comparer (complete)|BE-Conn-RH|Models|**DONE** – parser/comparer shipped with coverage; add edge cases as needed.| -|Mapper to canonical rpm/cpe affected|BE-Conn-RH|Models|**DONE** – maps fixed/known ranges, CPE provenance, status ranges.| -|Job scheduler registration aligns with Options pipeline|BE-Conn-RH|Core|**DONE** – registered fetch/parse/map via JobSchedulerBuilder, preserving option overrides and tightening cron/timeouts.| -|Watermark persistence + resume|BE-Conn-RH|Storage.Mongo|**DONE** – cursor updates via SourceStateRepository.| -|Precedence tests vs NVD|QA|Merge|**DONE** – Added AffectedPackagePrecedenceResolver + tests ensuring Red Hat CPEs override NVD ranges.| -|Golden mapping fixtures|QA|Fixtures|**DONE** – fixture validation test now snapshots RHSA-2025:0001/0002/0003 with env-driven regeneration.| -|Job scheduling defaults for source:redhat tasks|BE-Core|JobScheduler|**DONE** – Cron windows + per-job timeouts defined for fetch/parse/map.| -|Express unaffected/investigation statuses without overloading range fields|BE-Conn-RH|Models|**DONE** – Introduced AffectedPackageStatus collection and updated mapper/tests.| -|Reference dedupe & ordering in mapper|BE-Conn-RH|Models|DONE – mapper consolidates by URL, merges metadata, deterministic ordering validated in tests.| -|Hydra summary fetch through SourceFetchService|BE-Conn-RH|Source.Common|DONE – summary pages now fetched via SourceFetchService with cache + conditional headers.| +# TASKS +| Task | Owner(s) | Depends on | Notes | +|---|---|---|---| |Fixture validation sweep|QA|None|**DOING (2025-10-19)** – Prereqs confirmed none; continuing RHSA fixture regeneration and diff review alongside mapper provenance updates.
2025-10-29: Added `scripts/update-redhat-fixtures.sh` to regenerate golden snapshots with `UPDATE_GOLDENS=1`; run it before reviews to capture CSAF contract deltas.| diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Distro.Ubuntu/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Distro.Ubuntu/TASKS.completed.md new file mode 100644 index 00000000..06cf7139 --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Distro.Ubuntu/TASKS.completed.md @@ -0,0 +1,10 @@ +# Completed Tasks + +| Task | Status | Notes | +|---|---|---| +|Discover data model & pagination for `notices.json`|DONE|Connector now walks `offset`/`limit` pages (configurable page size) until MaxNoticesPerFetch satisfied, reusing cached pages when unchanged.| +|Design cursor & state model|DONE|Cursor tracks last published timestamp plus processed USN identifiers with overlap logic.| +|Implement fetch/parse pipeline|DONE|Index fetch hydrates per-notice DTOs, stores metadata, and maps without dedicated detail fetches.| +|Emit RangePrimitives + telemetry|DONE|Each package emits EVR primitives with `ubuntu.release` and `ubuntu.pocket` extensions for dashboards.| +|Add integration tests|DONE|Fixture-driven fetch→map suite covers resolved and ESM pockets, including conditional GET behaviour.| +|NormalizedVersions rollout|DONE (2025-10-11)|EVR ranges now project `normalizedVersions` with `ubuntu:` notes; tests assert canonical rule emission.| diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Distro.Ubuntu/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Distro.Ubuntu/TASKS.md index a1bfff56..5985d083 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Distro.Ubuntu/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Distro.Ubuntu/TASKS.md @@ -1,10 +1,4 @@ -# Ubuntu Connector TODOs - -| Task | Status | Notes | -|---|---|---| -|Discover data model & pagination for `notices.json`|DONE|Connector now walks `offset`/`limit` pages (configurable page size) until MaxNoticesPerFetch satisfied, reusing cached pages when unchanged.| -|Design cursor & state model|DONE|Cursor tracks last published timestamp plus processed USN identifiers with overlap logic.| -|Implement fetch/parse pipeline|DONE|Index fetch hydrates per-notice DTOs, stores metadata, and maps without dedicated detail fetches.| -|Emit RangePrimitives + telemetry|DONE|Each package emits EVR primitives with `ubuntu.release` and `ubuntu.pocket` extensions for dashboards.| -|Add integration tests|DONE|Fixture-driven fetch→map suite covers resolved and ESM pockets, including conditional GET behaviour.| -|NormalizedVersions rollout|DONE (2025-10-11)|EVR ranges now project `normalizedVersions` with `ubuntu:` notes; tests assert canonical rule emission.| +# Ubuntu Connector TODOs + +| Task | Status | Notes | +|---|---|---| diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ghsa/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ghsa/TASKS.completed.md new file mode 100644 index 00000000..d7aabd24 --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ghsa/TASKS.completed.md @@ -0,0 +1,34 @@ +# Completed Tasks + +|Select GHSA data source & auth model|BE-Conn-GHSA|Research|**DONE (2025-10-10)** – Adopted GitHub Security Advisories REST (global) endpoint with bearer token + API version headers documented in `GhsaOptions`.| + +|Fetch pipeline & state management|BE-Conn-GHSA|Source.Common, Storage.Mongo|**DONE (2025-10-10)** – Implemented list/detail fetch using `GhsaCursor` (time window + page), resumable SourceState and backoff controls.| + +|DTO & parser implementation|BE-Conn-GHSA|Source.Common|**DONE (2025-10-10)** – Added `GhsaRecordParser`/DTOs extracting aliases, references, severity, vulnerable ranges, patched versions.| + +|Canonical mapping & range primitives|BE-Conn-GHSA|Models|**DONE (2025-10-10)** – `GhsaMapper` emits GHSA advisories with SemVer packages, vendor extensions (ecosystem/package) and deterministic references.
2025-10-11 research trail: upcoming normalized array should follow `[{"scheme":"semver","type":"range","min":"","minInclusive":true,"max":"","maxInclusive":false,"notes":"ghsa:GHSA-xxxx"}]`; include patched-only advisories as `lt`/`lte` when no explicit floor.| + +|Deterministic fixtures & tests|QA|Testing|**DONE (2025-10-10)** – New `StellaOps.Concelier.Connector.Ghsa.Tests` regression covers fetch/parse/map via canned GHSA fixtures and snapshot assertions.| + +|Telemetry & documentation|DevEx|Docs|**DONE (2025-10-10)** – Diagnostics meter (`ghsa.fetch.*`) wired; DI extension documents token/headers and job registrations.| + +|GitHub quota monitoring & retries|BE-Conn-GHSA, Observability|Source.Common|**DONE (2025-10-12)** – Rate-limit metrics/logs added, retry/backoff handles 403 secondary limits, and ops runbook documents dashboards + mitigation steps.| + +|Production credential & scheduler rollout|Ops, BE-Conn-GHSA|Docs, WebService|**DONE (2025-10-12)** – Scheduler defaults registered via `JobSchedulerBuilder`, credential provisioning documented (Compose/Helm samples), and staged backfill guidance captured in `docs/modules/concelier/operations/connectors/ghsa.md`.| + +|FEEDCONN-GHSA-04-002 Conflict regression fixtures|BE-Conn-GHSA, QA|Merge `FEEDMERGE-ENGINE-04-001`|**DONE (2025-10-12)** – Added `conflict-ghsa.canonical.json` + `GhsaConflictFixtureTests`; SemVer ranges and credits align with merge precedence triple and shareable with QA. Validation: `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa.Tests/StellaOps.Concelier.Connector.Ghsa.Tests.csproj --filter GhsaConflictFixtureTests`.| + +|FEEDCONN-GHSA-02-004 GHSA credits & ecosystem severity mapping|BE-Conn-GHSA|Models `FEEDMODELS-SCHEMA-01-002`|**DONE (2025-10-11)** – Mapper emits advisory credits with provenance masks, fixtures assert role/contact ordering, and severity normalization remains unchanged.| + +|FEEDCONN-GHSA-02-007 Credit parity regression fixtures|BE-Conn-GHSA, QA|Source.Nvd, Source.Osv|**DONE (2025-10-12)** – Parity fixtures regenerated via `src/Tools/FixtureUpdater`, normalized SemVer notes verified against GHSA/NVD/OSV snapshots, and the fixtures guide now documents the headroom checks.| + +|FEEDCONN-GHSA-02-001 Normalized versions rollout|BE-Conn-GHSA|Models `FEEDMODELS-SCHEMA-01-003`, Normalization playbook|**DONE (2025-10-11)** – GHSA mapper now emits SemVer primitives + normalized ranges, fixtures refreshed, connector tests passing; report logged via FEEDMERGE-COORD-02-900.| + +|FEEDCONN-GHSA-02-005 Quota monitoring hardening|BE-Conn-GHSA, Observability|Source.Common metrics|**DONE (2025-10-12)** – Diagnostics expose headroom histograms/gauges, warning logs dedupe below the configured threshold, and the ops runbook gained alerting and mitigation guidance.| + +|FEEDCONN-GHSA-02-006 Scheduler rollout integration|BE-Conn-GHSA, Ops|Job scheduler|**DONE (2025-10-12)** – Dependency routine tests assert cron/timeouts, and the runbook highlights cron overrides plus backoff toggles for staged rollouts.| + +|FEEDCONN-GHSA-04-003 Description/CWE/metric parity rollout|BE-Conn-GHSA|Models, Core|**DONE (2025-10-15)** – Mapper emits advisory description, CWE weaknesses, and canonical CVSS metric id with updated fixtures (`osv-ghsa.osv.json` parity suite) and connector regression covers the new fields. Reported completion to Merge coordination.| + +|FEEDCONN-GHSA-04-004 Canonical metric fallback coverage|BE-Conn-GHSA|Models, Merge|**DONE (2025-10-16)** – Ensure canonical metric ids remain populated when GitHub omits CVSS vectors/scores; add fixtures capturing severity-only advisories, document precedence with Merge, and emit analytics to track fallback usage.
2025-10-16: Mapper now emits `ghsa:severity/` canonical ids when vectors are missing, diagnostics expose `ghsa.map.canonical_metric_fallbacks`, conflict/mapper fixtures updated, and runbook documents Merge precedence. Tests: `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa.Tests/StellaOps.Concelier.Connector.Ghsa.Tests.csproj`.| + diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ghsa/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ghsa/TASKS.md index 223b1a53..2265b2fc 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ghsa/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ghsa/TASKS.md @@ -1,19 +1,3 @@ # TASKS | Task | Owner(s) | Depends on | Notes | |---|---|---|---| -|Select GHSA data source & auth model|BE-Conn-GHSA|Research|**DONE (2025-10-10)** – Adopted GitHub Security Advisories REST (global) endpoint with bearer token + API version headers documented in `GhsaOptions`.| -|Fetch pipeline & state management|BE-Conn-GHSA|Source.Common, Storage.Mongo|**DONE (2025-10-10)** – Implemented list/detail fetch using `GhsaCursor` (time window + page), resumable SourceState and backoff controls.| -|DTO & parser implementation|BE-Conn-GHSA|Source.Common|**DONE (2025-10-10)** – Added `GhsaRecordParser`/DTOs extracting aliases, references, severity, vulnerable ranges, patched versions.| -|Canonical mapping & range primitives|BE-Conn-GHSA|Models|**DONE (2025-10-10)** – `GhsaMapper` emits GHSA advisories with SemVer packages, vendor extensions (ecosystem/package) and deterministic references.
2025-10-11 research trail: upcoming normalized array should follow `[{"scheme":"semver","type":"range","min":"","minInclusive":true,"max":"","maxInclusive":false,"notes":"ghsa:GHSA-xxxx"}]`; include patched-only advisories as `lt`/`lte` when no explicit floor.| -|Deterministic fixtures & tests|QA|Testing|**DONE (2025-10-10)** – New `StellaOps.Concelier.Connector.Ghsa.Tests` regression covers fetch/parse/map via canned GHSA fixtures and snapshot assertions.| -|Telemetry & documentation|DevEx|Docs|**DONE (2025-10-10)** – Diagnostics meter (`ghsa.fetch.*`) wired; DI extension documents token/headers and job registrations.| -|GitHub quota monitoring & retries|BE-Conn-GHSA, Observability|Source.Common|**DONE (2025-10-12)** – Rate-limit metrics/logs added, retry/backoff handles 403 secondary limits, and ops runbook documents dashboards + mitigation steps.| -|Production credential & scheduler rollout|Ops, BE-Conn-GHSA|Docs, WebService|**DONE (2025-10-12)** – Scheduler defaults registered via `JobSchedulerBuilder`, credential provisioning documented (Compose/Helm samples), and staged backfill guidance captured in `docs/modules/concelier/operations/connectors/ghsa.md`.| -|FEEDCONN-GHSA-04-002 Conflict regression fixtures|BE-Conn-GHSA, QA|Merge `FEEDMERGE-ENGINE-04-001`|**DONE (2025-10-12)** – Added `conflict-ghsa.canonical.json` + `GhsaConflictFixtureTests`; SemVer ranges and credits align with merge precedence triple and shareable with QA. Validation: `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa.Tests/StellaOps.Concelier.Connector.Ghsa.Tests.csproj --filter GhsaConflictFixtureTests`.| -|FEEDCONN-GHSA-02-004 GHSA credits & ecosystem severity mapping|BE-Conn-GHSA|Models `FEEDMODELS-SCHEMA-01-002`|**DONE (2025-10-11)** – Mapper emits advisory credits with provenance masks, fixtures assert role/contact ordering, and severity normalization remains unchanged.| -|FEEDCONN-GHSA-02-007 Credit parity regression fixtures|BE-Conn-GHSA, QA|Source.Nvd, Source.Osv|**DONE (2025-10-12)** – Parity fixtures regenerated via `src/Tools/FixtureUpdater`, normalized SemVer notes verified against GHSA/NVD/OSV snapshots, and the fixtures guide now documents the headroom checks.| -|FEEDCONN-GHSA-02-001 Normalized versions rollout|BE-Conn-GHSA|Models `FEEDMODELS-SCHEMA-01-003`, Normalization playbook|**DONE (2025-10-11)** – GHSA mapper now emits SemVer primitives + normalized ranges, fixtures refreshed, connector tests passing; report logged via FEEDMERGE-COORD-02-900.| -|FEEDCONN-GHSA-02-005 Quota monitoring hardening|BE-Conn-GHSA, Observability|Source.Common metrics|**DONE (2025-10-12)** – Diagnostics expose headroom histograms/gauges, warning logs dedupe below the configured threshold, and the ops runbook gained alerting and mitigation guidance.| -|FEEDCONN-GHSA-02-006 Scheduler rollout integration|BE-Conn-GHSA, Ops|Job scheduler|**DONE (2025-10-12)** – Dependency routine tests assert cron/timeouts, and the runbook highlights cron overrides plus backoff toggles for staged rollouts.| -|FEEDCONN-GHSA-04-003 Description/CWE/metric parity rollout|BE-Conn-GHSA|Models, Core|**DONE (2025-10-15)** – Mapper emits advisory description, CWE weaknesses, and canonical CVSS metric id with updated fixtures (`osv-ghsa.osv.json` parity suite) and connector regression covers the new fields. Reported completion to Merge coordination.| -|FEEDCONN-GHSA-04-004 Canonical metric fallback coverage|BE-Conn-GHSA|Models, Merge|**DONE (2025-10-16)** – Ensure canonical metric ids remain populated when GitHub omits CVSS vectors/scores; add fixtures capturing severity-only advisories, document precedence with Merge, and emit analytics to track fallback usage.
2025-10-16: Mapper now emits `ghsa:severity/` canonical ids when vectors are missing, diagnostics expose `ghsa.map.canonical_metric_fallbacks`, conflict/mapper fixtures updated, and runbook documents Merge precedence. Tests: `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa.Tests/StellaOps.Concelier.Connector.Ghsa.Tests.csproj`.| diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ics.Cisa/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ics.Cisa/TASKS.completed.md new file mode 100644 index 00000000..16041cf7 --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ics.Cisa/TASKS.completed.md @@ -0,0 +1,24 @@ +# Completed Tasks + +|FEEDCONN-ICSCISA-02-001 Document CISA ICS feed contract|BE-Conn-ICS-CISA|Research|**DONE (2025-10-11)** – `https://www.cisa.gov/cybersecurity-advisories/ics-advisories.xml` and legacy `/sites/default/files/feeds/...` return Akamai 403 even with browser UA; HTML landing page blocked as well. Logged full headers (x-reference-error, AkamaiGHost) in `docs/concelier-connector-research-20251011.md` and initiated GovDelivery access request.| + +|FEEDCONN-ICSCISA-02-002 Fetch pipeline & cursor storage|BE-Conn-ICS-CISA|Source.Common, Storage.Mongo|**DONE (2025-10-16)** – Confirmed proxy knobs + cursor state behave with the refreshed fixtures; ops runbook now captures proxy usage/validation so the fetch stage is production-ready.| + +|FEEDCONN-ICSCISA-02-003 DTO/parser implementation|BE-Conn-ICS-CISA|Source.Common|**DONE (2025-10-16)** – Feed parser fixtures updated to retain vendor PDFs as attachments while maintaining reference coverage; console diagnostics removed.| + +|FEEDCONN-ICSCISA-02-004 Canonical mapping & range primitives|BE-Conn-ICS-CISA|Models|**DONE (2025-10-16)** – `TryCreateSemVerPrimitive` flow + Mongo deserialiser now persist `exactValue` (`4.2` → `4.2.0`), unblocking canonical snapshots.| + +|FEEDCONN-ICSCISA-02-005 Deterministic fixtures/tests|QA|Testing|**DONE (2025-10-16)** – `dotnet test src/Concelier/__Tests/StellaOps.Concelier.Connector.Ics.Cisa.Tests/...` passes; fixtures assert attachment handling + SemVer semantics.| + +|FEEDCONN-ICSCISA-02-006 Telemetry & documentation|DevEx|Docs|**DONE (2025-10-16)** – Ops guide documents attachment checks, SemVer exact values, and proxy guidance; diagnostics remain unchanged.| + +|FEEDCONN-ICSCISA-02-007 Detail document inventory|BE-Conn-ICS-CISA|Research|**DONE (2025-10-16)** – Validated canned detail pages vs feed output so attachment inventories stay aligned; archived expectations noted in `HANDOVER.md`.| + +|FEEDCONN-ICSCISA-02-008 Distribution fallback strategy|BE-Conn-ICS-CISA|Research|**DONE (2025-10-11)** – Outlined GovDelivery token request, HTML scrape + email digest fallback, and dependency on Ops for credential workflow; awaiting decision before fetch implementation.| + +|FEEDCONN-ICSCISA-02-009 GovDelivery credential onboarding|Ops, BE-Conn-ICS-CISA|Ops|**DONE (2025-10-14)** – GovDelivery onboarding runbook captured in `docs/modules/concelier/operations/connectors/ics-cisa.md`; secret vault path and Offline Kit handling documented.| + +|FEEDCONN-ICSCISA-02-010 Mitigation & SemVer polish|BE-Conn-ICS-CISA|02-003, 02-004|**DONE (2025-10-16)** – Attachment + mitigation references now land as expected and SemVer primitives carry exact values; end-to-end suite green (see `HANDOVER.md`).| + +|FEEDCONN-ICSCISA-02-011 Docs & telemetry refresh|DevEx|02-006|**DONE (2025-10-16)** – Ops documentation refreshed (attachments, SemVer validation, proxy knobs) and telemetry notes verified.| + diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ics.Cisa/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ics.Cisa/TASKS.md index 9acef929..e06c5339 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ics.Cisa/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ics.Cisa/TASKS.md @@ -1,15 +1,4 @@ # TASKS | Task | Owner(s) | Depends on | Notes | |---|---|---|---| -|FEEDCONN-ICSCISA-02-001 Document CISA ICS feed contract|BE-Conn-ICS-CISA|Research|**DONE (2025-10-11)** – `https://www.cisa.gov/cybersecurity-advisories/ics-advisories.xml` and legacy `/sites/default/files/feeds/...` return Akamai 403 even with browser UA; HTML landing page blocked as well. Logged full headers (x-reference-error, AkamaiGHost) in `docs/concelier-connector-research-20251011.md` and initiated GovDelivery access request.| -|FEEDCONN-ICSCISA-02-002 Fetch pipeline & cursor storage|BE-Conn-ICS-CISA|Source.Common, Storage.Mongo|**DONE (2025-10-16)** – Confirmed proxy knobs + cursor state behave with the refreshed fixtures; ops runbook now captures proxy usage/validation so the fetch stage is production-ready.| -|FEEDCONN-ICSCISA-02-003 DTO/parser implementation|BE-Conn-ICS-CISA|Source.Common|**DONE (2025-10-16)** – Feed parser fixtures updated to retain vendor PDFs as attachments while maintaining reference coverage; console diagnostics removed.| -|FEEDCONN-ICSCISA-02-004 Canonical mapping & range primitives|BE-Conn-ICS-CISA|Models|**DONE (2025-10-16)** – `TryCreateSemVerPrimitive` flow + Mongo deserialiser now persist `exactValue` (`4.2` → `4.2.0`), unblocking canonical snapshots.| -|FEEDCONN-ICSCISA-02-005 Deterministic fixtures/tests|QA|Testing|**DONE (2025-10-16)** – `dotnet test src/Concelier/__Tests/StellaOps.Concelier.Connector.Ics.Cisa.Tests/...` passes; fixtures assert attachment handling + SemVer semantics.| -|FEEDCONN-ICSCISA-02-006 Telemetry & documentation|DevEx|Docs|**DONE (2025-10-16)** – Ops guide documents attachment checks, SemVer exact values, and proxy guidance; diagnostics remain unchanged.| -|FEEDCONN-ICSCISA-02-007 Detail document inventory|BE-Conn-ICS-CISA|Research|**DONE (2025-10-16)** – Validated canned detail pages vs feed output so attachment inventories stay aligned; archived expectations noted in `HANDOVER.md`.| -|FEEDCONN-ICSCISA-02-008 Distribution fallback strategy|BE-Conn-ICS-CISA|Research|**DONE (2025-10-11)** – Outlined GovDelivery token request, HTML scrape + email digest fallback, and dependency on Ops for credential workflow; awaiting decision before fetch implementation.| -|FEEDCONN-ICSCISA-02-009 GovDelivery credential onboarding|Ops, BE-Conn-ICS-CISA|Ops|**DONE (2025-10-14)** – GovDelivery onboarding runbook captured in `docs/modules/concelier/operations/connectors/ics-cisa.md`; secret vault path and Offline Kit handling documented.| -|FEEDCONN-ICSCISA-02-010 Mitigation & SemVer polish|BE-Conn-ICS-CISA|02-003, 02-004|**DONE (2025-10-16)** – Attachment + mitigation references now land as expected and SemVer primitives carry exact values; end-to-end suite green (see `HANDOVER.md`).| -|FEEDCONN-ICSCISA-02-011 Docs & telemetry refresh|DevEx|02-006|**DONE (2025-10-16)** – Ops documentation refreshed (attachments, SemVer validation, proxy knobs) and telemetry notes verified.| |FEEDCONN-ICSCISA-02-012 Normalized version decision|BE-Conn-ICS-CISA|Merge coordination (`FEEDMERGE-COORD-02-900`)|**TODO (due 2025-10-23)** – Promote existing `SemVerPrimitive` exact values into `NormalizedVersions` via `.ToNormalizedVersionRule("ics-cisa:{advisoryId}:{product}")`, add regression coverage, and open Models ticket if non-SemVer firmware requires a new scheme.
2025-10-29: Follow `docs/dev/normalized-rule-recipes.md` §2 to call `ToNormalizedVersionRule` and ensure mixed firmware strings log a Models ticket when regex extraction fails.| diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ics.Kaspersky/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ics.Kaspersky/TASKS.completed.md new file mode 100644 index 00000000..3d215a77 --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ics.Kaspersky/TASKS.completed.md @@ -0,0 +1,16 @@ +# Completed Tasks + +|List/detail fetcher with windowing|BE-Conn-ICS-Kaspersky|Source.Common|**DONE** – feed client paginates and fetches detail pages with window overlap.| + +|Extractor (vendors/models/CVEs)|BE-Conn-ICS-Kaspersky|Source.Common|**DONE** – parser normalizes vendor/model taxonomy into DTO.| + +|DTO validation and sanitizer|BE-Conn-ICS-Kaspersky, QA|Source.Common|**DONE** – HTML parsed into DTO with sanitizer guardrails.| + +|Canonical mapping (affected, refs)|BE-Conn-ICS-Kaspersky|Models|**DONE** – mapper outputs `ics-vendor` affected entries with provenance.| + +|State/dedupe and fixtures|BE-Conn-ICS-Kaspersky, QA|Storage.Mongo|**DONE** – duplicate-content and resume tests exercise SHA gating + cursor hygiene.| + +|Backoff on fetch failures|BE-Conn-ICS-Kaspersky|Storage.Mongo|**DONE** – feed/page failures mark source_state with timed backoff.| + +|Conditional fetch caching|BE-Conn-ICS-Kaspersky|Source.Common|**DONE** – fetch cache persists ETag/Last-Modified; not-modified scenarios validated in tests.| + diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ics.Kaspersky/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ics.Kaspersky/TASKS.md index d8df0fcd..2265b2fc 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ics.Kaspersky/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ics.Kaspersky/TASKS.md @@ -1,10 +1,3 @@ -# TASKS -| Task | Owner(s) | Depends on | Notes | -|---|---|---|---| -|List/detail fetcher with windowing|BE-Conn-ICS-Kaspersky|Source.Common|**DONE** – feed client paginates and fetches detail pages with window overlap.| -|Extractor (vendors/models/CVEs)|BE-Conn-ICS-Kaspersky|Source.Common|**DONE** – parser normalizes vendor/model taxonomy into DTO.| -|DTO validation and sanitizer|BE-Conn-ICS-Kaspersky, QA|Source.Common|**DONE** – HTML parsed into DTO with sanitizer guardrails.| -|Canonical mapping (affected, refs)|BE-Conn-ICS-Kaspersky|Models|**DONE** – mapper outputs `ics-vendor` affected entries with provenance.| -|State/dedupe and fixtures|BE-Conn-ICS-Kaspersky, QA|Storage.Mongo|**DONE** – duplicate-content and resume tests exercise SHA gating + cursor hygiene.| -|Backoff on fetch failures|BE-Conn-ICS-Kaspersky|Storage.Mongo|**DONE** – feed/page failures mark source_state with timed backoff.| -|Conditional fetch caching|BE-Conn-ICS-Kaspersky|Source.Common|**DONE** – fetch cache persists ETag/Last-Modified; not-modified scenarios validated in tests.| +# TASKS +| Task | Owner(s) | Depends on | Notes | +|---|---|---|---| diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Jvn/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Jvn/TASKS.completed.md new file mode 100644 index 00000000..5df55d09 --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Jvn/TASKS.completed.md @@ -0,0 +1,22 @@ +# Completed Tasks + +|MyJVN client (JVNRSS+VULDEF) with windowing|BE-Conn-JVN|Source.Common|**DONE** – windowed overview/detail fetch with rate limit handling implemented.| + +|Schema/XSD validation and DTO sanitizer|BE-Conn-JVN, QA|Source.Common|**DONE** – parser validates XML against schema before DTO persistence.| + +|Canonical mapping (aliases, jp_flags, refs)|BE-Conn-JVN|Models|**DONE** – mapper populates aliases, jp_flags, references while skipping non-actionable affected entries.| + +|SourceState and idempotent dedupe|BE-Conn-JVN|Storage.Mongo|**DONE** – cursor tracks pending docs/mappings with resume support.| + +|Golden fixtures and determinism tests|QA|Source.Jvn|**DONE** – deterministic snapshot test in `JvnConnectorTests` now passes with offline fixtures.| + +|Async-safe overview query building|BE-Conn-JVN|Source.Common|DONE – `MyJvnClient` now builds query strings synchronously without blocking calls.| + +|Reference dedupe + deterministic ordering|BE-Conn-JVN|Models|DONE – mapper merges by URL, retains richer metadata, sorts deterministically.| + +|Console logging remediation|BE-Conn-JVN|Observability|**DONE** – connector now uses structured `ILogger` debug entries instead of console writes.| + +|Offline fixtures for connector tests|QA|Source.Jvn|**DONE** – tests rely solely on canned HTTP responses and local fixtures.| + +|Update VULDEF schema for vendor attribute|BE-Conn-JVN, QA|Source.Jvn|**DONE** – embedded XSD updated (vendor/product attrs, impact item), parser tightened, fixtures & snapshots refreshed.| + diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Jvn/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Jvn/TASKS.md index 31e3b3c8..2265b2fc 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Jvn/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Jvn/TASKS.md @@ -1,13 +1,3 @@ -# TASKS -| Task | Owner(s) | Depends on | Notes | -|---|---|---|---| -|MyJVN client (JVNRSS+VULDEF) with windowing|BE-Conn-JVN|Source.Common|**DONE** – windowed overview/detail fetch with rate limit handling implemented.| -|Schema/XSD validation and DTO sanitizer|BE-Conn-JVN, QA|Source.Common|**DONE** – parser validates XML against schema before DTO persistence.| -|Canonical mapping (aliases, jp_flags, refs)|BE-Conn-JVN|Models|**DONE** – mapper populates aliases, jp_flags, references while skipping non-actionable affected entries.| -|SourceState and idempotent dedupe|BE-Conn-JVN|Storage.Mongo|**DONE** – cursor tracks pending docs/mappings with resume support.| -|Golden fixtures and determinism tests|QA|Source.Jvn|**DONE** – deterministic snapshot test in `JvnConnectorTests` now passes with offline fixtures.| -|Async-safe overview query building|BE-Conn-JVN|Source.Common|DONE – `MyJvnClient` now builds query strings synchronously without blocking calls.| -|Reference dedupe + deterministic ordering|BE-Conn-JVN|Models|DONE – mapper merges by URL, retains richer metadata, sorts deterministically.| -|Console logging remediation|BE-Conn-JVN|Observability|**DONE** – connector now uses structured `ILogger` debug entries instead of console writes.| -|Offline fixtures for connector tests|QA|Source.Jvn|**DONE** – tests rely solely on canned HTTP responses and local fixtures.| -|Update VULDEF schema for vendor attribute|BE-Conn-JVN, QA|Source.Jvn|**DONE** – embedded XSD updated (vendor/product attrs, impact item), parser tightened, fixtures & snapshots refreshed.| +# TASKS +| Task | Owner(s) | Depends on | Notes | +|---|---|---|---| diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Kev/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Kev/TASKS.completed.md new file mode 100644 index 00000000..11fc3311 --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Kev/TASKS.completed.md @@ -0,0 +1,20 @@ +# Completed Tasks + +|Review KEV JSON schema & cadence|BE-Conn-KEV|Research|**DONE** – Feed defaults lock to the public JSON catalog; AGENTS notes call out daily cadence and allowlist requirements.| + +|Fetch & cursor implementation|BE-Conn-KEV|Source.Common, Storage.Mongo|**DONE** – SourceFetchService drives ETag/Last-Modified aware fetches with SourceState cursor tracking documents + catalog metadata.| + +|DTO/parser implementation|BE-Conn-KEV|Source.Common|**DONE** – `KevCatalogDto`/`KevVulnerabilityDto` deserialize payloads with logging for catalog version/releases before DTO persistence.| + +|Canonical mapping & range primitives|BE-Conn-KEV|Models|**DONE** – Mapper produces vendor RangePrimitives (due dates, CWE list, ransomware flag, catalog metadata) and deduplicated references.| + +|Deterministic fixtures/tests|QA|Testing|**DONE** – End-to-end fetch→parse→map test with canned catalog + snapshot (`UPDATE_KEV_FIXTURES=1`) guards determinism.| + +|Telemetry & docs|DevEx|Docs|**DONE** – Connector emits structured logs + meters for catalog entries/advisories and AGENTS docs cover cadence/allowlist guidance.| + +|Schema validation & anomaly surfacing|BE-Conn-KEV, QA|Source.Common|**DONE (2025-10-12)** – Wired `IJsonSchemaValidator` + embedded schema, added failure reasons (`schema`, `download`, `invalidJson`, etc.), anomaly counters (`missingCveId`, `countMismatch`, `nullEntry`), and kept `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Kev.Tests` passing.| + +|Metrics export wiring|DevOps, DevEx|Observability|**DONE (2025-10-12)** – Added `kev.fetch.*` counters, parse failure/anomaly tags, refreshed ops runbook + Grafana dashboard (`docs/modules/concelier/operations/connectors/cve-kev-grafana-dashboard.json`) with PromQL guidance.| + +|FEEDCONN-KEV-02-003 Normalized versions propagation|BE-Conn-KEV|Models `FEEDMODELS-SCHEMA-01-003`, Normalization playbook|**DONE (2025-10-12)** – Validated catalog/date/due normalized rules emission + ordering; fixtures assert rule set and `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Kev.Tests` remains green.| + diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Kev/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Kev/TASKS.md index 22a55558..2265b2fc 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Kev/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Kev/TASKS.md @@ -1,12 +1,3 @@ # TASKS | Task | Owner(s) | Depends on | Notes | |---|---|---|---| -|Review KEV JSON schema & cadence|BE-Conn-KEV|Research|**DONE** – Feed defaults lock to the public JSON catalog; AGENTS notes call out daily cadence and allowlist requirements.| -|Fetch & cursor implementation|BE-Conn-KEV|Source.Common, Storage.Mongo|**DONE** – SourceFetchService drives ETag/Last-Modified aware fetches with SourceState cursor tracking documents + catalog metadata.| -|DTO/parser implementation|BE-Conn-KEV|Source.Common|**DONE** – `KevCatalogDto`/`KevVulnerabilityDto` deserialize payloads with logging for catalog version/releases before DTO persistence.| -|Canonical mapping & range primitives|BE-Conn-KEV|Models|**DONE** – Mapper produces vendor RangePrimitives (due dates, CWE list, ransomware flag, catalog metadata) and deduplicated references.| -|Deterministic fixtures/tests|QA|Testing|**DONE** – End-to-end fetch→parse→map test with canned catalog + snapshot (`UPDATE_KEV_FIXTURES=1`) guards determinism.| -|Telemetry & docs|DevEx|Docs|**DONE** – Connector emits structured logs + meters for catalog entries/advisories and AGENTS docs cover cadence/allowlist guidance.| -|Schema validation & anomaly surfacing|BE-Conn-KEV, QA|Source.Common|**DONE (2025-10-12)** – Wired `IJsonSchemaValidator` + embedded schema, added failure reasons (`schema`, `download`, `invalidJson`, etc.), anomaly counters (`missingCveId`, `countMismatch`, `nullEntry`), and kept `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Kev.Tests` passing.| -|Metrics export wiring|DevOps, DevEx|Observability|**DONE (2025-10-12)** – Added `kev.fetch.*` counters, parse failure/anomaly tags, refreshed ops runbook + Grafana dashboard (`docs/modules/concelier/operations/connectors/cve-kev-grafana-dashboard.json`) with PromQL guidance.| -|FEEDCONN-KEV-02-003 Normalized versions propagation|BE-Conn-KEV|Models `FEEDMODELS-SCHEMA-01-003`, Normalization playbook|**DONE (2025-10-12)** – Validated catalog/date/due normalized rules emission + ordering; fixtures assert rule set and `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Kev.Tests` remains green.| diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Kisa/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Kisa/TASKS.completed.md new file mode 100644 index 00000000..9a274143 --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Kisa/TASKS.completed.md @@ -0,0 +1,16 @@ +# Completed Tasks + +|FEEDCONN-KISA-02-001 Research KISA advisory feeds|BE-Conn-KISA|Research|**DONE (2025-10-11)** – Located public RSS endpoints (`https://knvd.krcert.or.kr/rss/securityInfo.do`, `.../securityNotice.do`) returning UTF-8 XML with 10-item windows and canonical `detailDos.do?IDX=` links. Logged output structure + header profile in `docs/concelier-connector-research-20251011.md`; outstanding work is parsing the SPA detail payload.| + +|FEEDCONN-KISA-02-002 Fetch pipeline & source state|BE-Conn-KISA|Source.Common, Storage.Mongo|**DONE (2025-10-14)** – `KisaConnector.FetchAsync` pulls RSS, sets `Accept-Language: ko-KR`, persists detail JSON with IDX metadata, throttles requests, and tracks cursor state (pending docs/mappings, known IDs, published timestamp).| + +|FEEDCONN-KISA-02-003 Parser & DTO implementation|BE-Conn-KISA|Source.Common|**DONE (2025-10-14)** – Detail API parsed via `KisaDetailParser` (Hangul NFC normalisation, sanitised HTML, CVE extraction, references/products captured into DTO `kisa.detail.v1`).| + +|FEEDCONN-KISA-02-004 Canonical mapping & range primitives|BE-Conn-KISA|Models|**DONE (2025-10-14)** – `KisaMapper` emits vendor packages with range strings, aliases (IDX/CVEs), references, and provenance; advisories default to `ko` language and normalised severity.| + +|FEEDCONN-KISA-02-005 Deterministic fixtures & tests|QA|Testing|**DONE (2025-10-14)** – Added `StellaOps.Concelier.Connector.Kisa.Tests` with Korean fixtures and fetch→parse→map regression; fixtures regenerate via `UPDATE_KISA_FIXTURES=1`.| + +|FEEDCONN-KISA-02-006 Telemetry & documentation|DevEx|Docs|**DONE (2025-10-14)** – Added diagnostics-backed telemetry, structured logs, regression coverage, and published localisation notes in `docs/dev/kisa_connector_notes.md` + fixture guidance for Docs/QA.| + +|FEEDCONN-KISA-02-007 RSS contract & localisation brief|BE-Conn-KISA|Research|**DONE (2025-10-11)** – Documented RSS URLs, confirmed UTF-8 payload (no additional cookies required), and drafted localisation plan (Hangul glossary + optional MT plugin). Remaining open item: capture SPA detail API contract for full-text translations.| + diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Kisa/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Kisa/TASKS.md index 8abdbdf1..84ed2732 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Kisa/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Kisa/TASKS.md @@ -1,11 +1,4 @@ -# TASKS -| Task | Owner(s) | Depends on | Notes | -|---|---|---|---| -|FEEDCONN-KISA-02-001 Research KISA advisory feeds|BE-Conn-KISA|Research|**DONE (2025-10-11)** – Located public RSS endpoints (`https://knvd.krcert.or.kr/rss/securityInfo.do`, `.../securityNotice.do`) returning UTF-8 XML with 10-item windows and canonical `detailDos.do?IDX=` links. Logged output structure + header profile in `docs/concelier-connector-research-20251011.md`; outstanding work is parsing the SPA detail payload.| -|FEEDCONN-KISA-02-002 Fetch pipeline & source state|BE-Conn-KISA|Source.Common, Storage.Mongo|**DONE (2025-10-14)** – `KisaConnector.FetchAsync` pulls RSS, sets `Accept-Language: ko-KR`, persists detail JSON with IDX metadata, throttles requests, and tracks cursor state (pending docs/mappings, known IDs, published timestamp).| -|FEEDCONN-KISA-02-003 Parser & DTO implementation|BE-Conn-KISA|Source.Common|**DONE (2025-10-14)** – Detail API parsed via `KisaDetailParser` (Hangul NFC normalisation, sanitised HTML, CVE extraction, references/products captured into DTO `kisa.detail.v1`).| -|FEEDCONN-KISA-02-004 Canonical mapping & range primitives|BE-Conn-KISA|Models|**DONE (2025-10-14)** – `KisaMapper` emits vendor packages with range strings, aliases (IDX/CVEs), references, and provenance; advisories default to `ko` language and normalised severity.| -|FEEDCONN-KISA-02-005 Deterministic fixtures & tests|QA|Testing|**DONE (2025-10-14)** – Added `StellaOps.Concelier.Connector.Kisa.Tests` with Korean fixtures and fetch→parse→map regression; fixtures regenerate via `UPDATE_KISA_FIXTURES=1`.| -|FEEDCONN-KISA-02-006 Telemetry & documentation|DevEx|Docs|**DONE (2025-10-14)** – Added diagnostics-backed telemetry, structured logs, regression coverage, and published localisation notes in `docs/dev/kisa_connector_notes.md` + fixture guidance for Docs/QA.| -|FEEDCONN-KISA-02-007 RSS contract & localisation brief|BE-Conn-KISA|Research|**DONE (2025-10-11)** – Documented RSS URLs, confirmed UTF-8 payload (no additional cookies required), and drafted localisation plan (Hangul glossary + optional MT plugin). Remaining open item: capture SPA detail API contract for full-text translations.| -|FEEDCONN-KISA-02-008 Firmware scheme proposal|BE-Conn-KISA, Models|Merge coordination (`FEEDMERGE-COORD-02-900`)|**TODO (due 2025-10-24)** – Define transformation for Hangul-labelled firmware ranges (`XFU 1.0.1.0084 ~ 2.0.1.0034`), propose `kisa.build` (or equivalent) scheme to Models, implement normalized rule emission/tests once scheme approved, and update localisation notes.| +# TASKS +| Task | Owner(s) | Depends on | Notes | +|---|---|---|---| +|FEEDCONN-KISA-02-008 Firmware scheme proposal|BE-Conn-KISA, Models|Merge coordination (`FEEDMERGE-COORD-02-900`)|**TODO (due 2025-10-24)** – Define transformation for Hangul-labelled firmware ranges (`XFU 1.0.1.0084 ~ 2.0.1.0034`), propose `kisa.build` (or equivalent) scheme to Models, implement normalized rule emission/tests once scheme approved, and update localisation notes.| diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Nvd/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Nvd/TASKS.completed.md new file mode 100644 index 00000000..b4d8a05d --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Nvd/TASKS.completed.md @@ -0,0 +1,32 @@ +# Completed Tasks + +|Fetch job with sliding modified windows|BE-Conn-Nvd|Source.Common|**DONE** – windowed fetch implemented with overlap and raw doc persistence.| + +|DTO schema + validation|BE-Conn-Nvd|Source.Common|**DONE** – schema validator enforced before DTO persistence.| + +|Mapper to canonical model|BE-Conn-Nvd|Models|**DONE** – `NvdMapper` populates CVSS/CWE/CPE data.
2025-10-11 research trail: upcoming normalized rules must serialize as `[{"scheme":"semver","type":"range","min":"","minInclusive":true,"max":"","maxInclusive":false,"notes":"nvd:CVE-2025-XXXX"}]`; keep notes consistent with CVE IDs for provenance joins.| + +|Watermark repo usage|BE-Conn-Nvd|Storage.Mongo|**DONE** – cursor tracks windowStart/windowEnd and updates SourceState.| + +|Integration test fixture isolation|QA|Storage.Mongo|**DONE** – connector tests reset Mongo/time fixtures between runs to avoid cross-test bleed.| + +|Tests: golden pages + resume|QA|Tests|**DONE** – snapshot and resume coverage added across `NvdConnectorTests`.| + +|Observability|BE-Conn-Nvd|Core|**DONE** – `NvdDiagnostics` meter tracks attempts/documents/failures with collector tests.| + +|Change history snapshotting|BE-Conn-Nvd|Storage.Mongo|DONE – connector now records per-CVE snapshots with top-level diff metadata whenever canonical advisories change.| + +|Pagination for windows over page limit|BE-Conn-Nvd|Source.Common|**DONE** – additional page fetcher honors `startIndex`; covered by multipage tests.| + +|Schema validation quarantine path|BE-Conn-Nvd|Storage.Mongo|**DONE** – schema failures mark documents failed and metrics assert quarantine.| + +|FEEDCONN-NVD-04-002 Conflict regression fixtures|BE-Conn-Nvd, QA|Merge `FEEDMERGE-ENGINE-04-001`|**DONE (2025-10-12)** – Published `conflict-nvd.canonical.json` + mapper test; includes CVSS 3.1 + CWE reference and normalized CPE range feeding the conflict triple. Validation: `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Nvd.Tests/StellaOps.Concelier.Connector.Nvd.Tests.csproj --filter NvdConflictFixtureTests`.| + +|FEEDCONN-NVD-02-004 NVD CVSS & CWE precedence payloads|BE-Conn-Nvd|Models `FEEDMODELS-SCHEMA-01-002`|**DONE (2025-10-11)** – CVSS metrics now carry provenance masks, CWE weaknesses emit normalized references, and fixtures cover the additional precedence data.| + +|FEEDCONN-NVD-02-005 NVD merge/export parity regression|BE-Conn-Nvd, BE-Merge|Merge `FEEDMERGE-ENGINE-04-003`|**DONE (2025-10-12)** – Canonical merge parity fixtures captured, regression test validates credit/reference union, and exporter snapshot check guarantees parity through JSON exports.| + +|FEEDCONN-NVD-02-002 Normalized versions rollout|BE-Conn-Nvd|Models `FEEDMODELS-SCHEMA-01-003`, Normalization playbook|**DONE (2025-10-11)** – SemVer primitives + normalized rules emitting for parseable ranges, fixtures/tests refreshed, coordination pinged via FEEDMERGE-COORD-02-900.| + +|FEEDCONN-NVD-04-003 Description/CWE/metric parity rollout|BE-Conn-Nvd|Models, Core|**DONE (2025-10-15)** – Mapper now surfaces normalized description text, CWE weaknesses, and canonical CVSS metric id. Snapshots (`conflict-nvd.canonical.json`) refreshed and completion relayed to Merge coordination.| + diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Nvd/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Nvd/TASKS.md index 13fefe22..2265b2fc 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Nvd/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Nvd/TASKS.md @@ -1,18 +1,3 @@ # TASKS | Task | Owner(s) | Depends on | Notes | |---|---|---|---| -|Fetch job with sliding modified windows|BE-Conn-Nvd|Source.Common|**DONE** – windowed fetch implemented with overlap and raw doc persistence.| -|DTO schema + validation|BE-Conn-Nvd|Source.Common|**DONE** – schema validator enforced before DTO persistence.| -|Mapper to canonical model|BE-Conn-Nvd|Models|**DONE** – `NvdMapper` populates CVSS/CWE/CPE data.
2025-10-11 research trail: upcoming normalized rules must serialize as `[{"scheme":"semver","type":"range","min":"","minInclusive":true,"max":"","maxInclusive":false,"notes":"nvd:CVE-2025-XXXX"}]`; keep notes consistent with CVE IDs for provenance joins.| -|Watermark repo usage|BE-Conn-Nvd|Storage.Mongo|**DONE** – cursor tracks windowStart/windowEnd and updates SourceState.| -|Integration test fixture isolation|QA|Storage.Mongo|**DONE** – connector tests reset Mongo/time fixtures between runs to avoid cross-test bleed.| -|Tests: golden pages + resume|QA|Tests|**DONE** – snapshot and resume coverage added across `NvdConnectorTests`.| -|Observability|BE-Conn-Nvd|Core|**DONE** – `NvdDiagnostics` meter tracks attempts/documents/failures with collector tests.| -|Change history snapshotting|BE-Conn-Nvd|Storage.Mongo|DONE – connector now records per-CVE snapshots with top-level diff metadata whenever canonical advisories change.| -|Pagination for windows over page limit|BE-Conn-Nvd|Source.Common|**DONE** – additional page fetcher honors `startIndex`; covered by multipage tests.| -|Schema validation quarantine path|BE-Conn-Nvd|Storage.Mongo|**DONE** – schema failures mark documents failed and metrics assert quarantine.| -|FEEDCONN-NVD-04-002 Conflict regression fixtures|BE-Conn-Nvd, QA|Merge `FEEDMERGE-ENGINE-04-001`|**DONE (2025-10-12)** – Published `conflict-nvd.canonical.json` + mapper test; includes CVSS 3.1 + CWE reference and normalized CPE range feeding the conflict triple. Validation: `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Nvd.Tests/StellaOps.Concelier.Connector.Nvd.Tests.csproj --filter NvdConflictFixtureTests`.| -|FEEDCONN-NVD-02-004 NVD CVSS & CWE precedence payloads|BE-Conn-Nvd|Models `FEEDMODELS-SCHEMA-01-002`|**DONE (2025-10-11)** – CVSS metrics now carry provenance masks, CWE weaknesses emit normalized references, and fixtures cover the additional precedence data.| -|FEEDCONN-NVD-02-005 NVD merge/export parity regression|BE-Conn-Nvd, BE-Merge|Merge `FEEDMERGE-ENGINE-04-003`|**DONE (2025-10-12)** – Canonical merge parity fixtures captured, regression test validates credit/reference union, and exporter snapshot check guarantees parity through JSON exports.| -|FEEDCONN-NVD-02-002 Normalized versions rollout|BE-Conn-Nvd|Models `FEEDMODELS-SCHEMA-01-003`, Normalization playbook|**DONE (2025-10-11)** – SemVer primitives + normalized rules emitting for parseable ranges, fixtures/tests refreshed, coordination pinged via FEEDMERGE-COORD-02-900.| -|FEEDCONN-NVD-04-003 Description/CWE/metric parity rollout|BE-Conn-Nvd|Models, Core|**DONE (2025-10-15)** – Mapper now surfaces normalized description text, CWE weaknesses, and canonical CVSS metric id. Snapshots (`conflict-nvd.canonical.json`) refreshed and completion relayed to Merge coordination.| diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Osv/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Osv/TASKS.completed.md new file mode 100644 index 00000000..a42148d0 --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Osv/TASKS.completed.md @@ -0,0 +1,36 @@ +# Completed Tasks + +|Ecosystem fetchers (npm, pypi, maven, go, crates)|BE-Conn-OSV|Source.Common|**DONE** – archive fetch loop iterates ecosystems with pagination + change gating.| + +|OSV options & HttpClient configuration|BE-Conn-OSV|Source.Common|**DONE** – `OsvOptions` + `AddOsvConnector` configure allowlisted HttpClient.| + +|DTO validation + sanitizer|BE-Conn-OSV|Source.Common|**DONE** – JSON deserialization sanitizes payloads before persistence; schema enforcement deferred.| + +|Mapper to canonical SemVer ranges|BE-Conn-OSV|Models|**DONE** – `OsvMapper` emits SemVer ranges with provenance metadata.
2025-10-11 research trail: ensure `NormalizedVersions` array uses payloads such as `[{"scheme":"semver","type":"range","min":"","minInclusive":true,"max":"","maxInclusive":false,"notes":"osv:GHI-2025-0001"}]` so storage merges align with GHSA parity tests.| + +|Alias consolidation (GHSA/CVE)|BE-Merge|Merge|DONE – OSV advisory records now emit GHSA/CVE aliases captured by alias graph tests.| + +|Tests: snapshot per ecosystem|QA|Tests|DONE – deterministic snapshots added for npm and PyPI advisories.| + +|Cursor persistence and hash gating|BE-Conn-OSV|Storage.Mongo|**DONE** – `OsvCursor` tracks per-ecosystem metadata and SHA gating.| + +|Parity checks vs GHSA data|QA|Merge|DONE – `OsvGhsaParityRegressionTests` keep OSV ↔ GHSA fixtures green; regeneration workflow documented in docs/19_TEST_SUITE_OVERVIEW.md.| + +|Connector DI routine & job registration|BE-Conn-OSV|Core|**DONE** – DI routine registers fetch/parse/map jobs with scheduler.| + +|Implement OSV fetch/parse/map skeleton|BE-Conn-OSV|Source.Common|**DONE** – connector now persists documents, DTOs, and canonical advisories.| + +|FEEDCONN-OSV-02-004 OSV references & credits alignment|BE-Conn-OSV|Models `FEEDMODELS-SCHEMA-01-002`|**DONE (2025-10-11)** – Mapper normalizes references with provenance masks, emits advisory credits, and regression fixtures/assertions cover the new fields.| + +|FEEDCONN-OSV-02-005 Fixture updater workflow|BE-Conn-OSV, QA|Docs|**DONE (2025-10-12)** – Canonical PURL derivation now covers Go + scoped npm advisories without upstream `purl`; legacy invalid npm names still fall back to `ecosystem:name`. OSV/GHSA/NVD suites and normalization/storage tests rerun clean.| + +|FEEDCONN-OSV-02-003 Normalized versions rollout|BE-Conn-OSV|Models `FEEDMODELS-SCHEMA-01-003`, Normalization playbook|**DONE (2025-10-11)** – `OsvMapper` now emits SemVer primitives + normalized rules with `osv:{ecosystem}:{advisoryId}:{identifier}` notes; npm/PyPI/Parity fixtures refreshed; merge coordination pinged (OSV handoff).| + +|FEEDCONN-OSV-04-003 Parity fixture refresh|QA, BE-Conn-OSV|Normalized versions rollout, GHSA parity tests|**DONE (2025-10-12)** – Parity fixtures include normalizedVersions notes (`osv:::`); regression math rerun via `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv.Tests` and docs flagged for workflow sync.| + +|FEEDCONN-OSV-04-002 Conflict regression fixtures|BE-Conn-OSV, QA|Merge `FEEDMERGE-ENGINE-04-001`|**DONE (2025-10-12)** – Added `conflict-osv.canonical.json` + regression asserting SemVer range + CVSS medium severity; dataset matches GHSA/NVD fixtures for merge tests. Validation: `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv.Tests/StellaOps.Concelier.Connector.Osv.Tests.csproj --filter OsvConflictFixtureTests`.| + +|FEEDCONN-OSV-04-004 Description/CWE/metric parity rollout|BE-Conn-OSV|Models, Core|**DONE (2025-10-15)** – OSV mapper writes advisory descriptions, `database_specific.cwe_ids` weaknesses, and canonical CVSS metric id. Parity fixtures (`osv-ghsa.*`, `osv-npm.snapshot.json`, `osv-pypi.snapshot.json`) refreshed and status communicated to Merge coordination.| + +|FEEDCONN-OSV-04-005 Canonical metric fallbacks & CWE notes|BE-Conn-OSV|Models, Merge|**DONE (2025-10-16)** – Add fallback logic and metrics for advisories lacking CVSS vectors, enrich CWE provenance notes, and document merge/export expectations; refresh parity fixtures accordingly.
2025-10-16: Mapper now emits `osv:severity/` canonical ids for severity-only advisories, weakness provenance carries `database_specific.cwe_ids`, diagnostics expose `osv.map.canonical_metric_fallbacks`, parity fixtures regenerated, and ops notes added in `docs/modules/concelier/operations/connectors/osv.md`. Tests: `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv.Tests/StellaOps.Concelier.Connector.Osv.Tests.csproj`.| + diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Osv/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Osv/TASKS.md index 513e3688..2265b2fc 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Osv/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Osv/TASKS.md @@ -1,20 +1,3 @@ # TASKS | Task | Owner(s) | Depends on | Notes | |---|---|---|---| -|Ecosystem fetchers (npm, pypi, maven, go, crates)|BE-Conn-OSV|Source.Common|**DONE** – archive fetch loop iterates ecosystems with pagination + change gating.| -|OSV options & HttpClient configuration|BE-Conn-OSV|Source.Common|**DONE** – `OsvOptions` + `AddOsvConnector` configure allowlisted HttpClient.| -|DTO validation + sanitizer|BE-Conn-OSV|Source.Common|**DONE** – JSON deserialization sanitizes payloads before persistence; schema enforcement deferred.| -|Mapper to canonical SemVer ranges|BE-Conn-OSV|Models|**DONE** – `OsvMapper` emits SemVer ranges with provenance metadata.
2025-10-11 research trail: ensure `NormalizedVersions` array uses payloads such as `[{"scheme":"semver","type":"range","min":"","minInclusive":true,"max":"","maxInclusive":false,"notes":"osv:GHI-2025-0001"}]` so storage merges align with GHSA parity tests.| -|Alias consolidation (GHSA/CVE)|BE-Merge|Merge|DONE – OSV advisory records now emit GHSA/CVE aliases captured by alias graph tests.| -|Tests: snapshot per ecosystem|QA|Tests|DONE – deterministic snapshots added for npm and PyPI advisories.| -|Cursor persistence and hash gating|BE-Conn-OSV|Storage.Mongo|**DONE** – `OsvCursor` tracks per-ecosystem metadata and SHA gating.| -|Parity checks vs GHSA data|QA|Merge|DONE – `OsvGhsaParityRegressionTests` keep OSV ↔ GHSA fixtures green; regeneration workflow documented in docs/19_TEST_SUITE_OVERVIEW.md.| -|Connector DI routine & job registration|BE-Conn-OSV|Core|**DONE** – DI routine registers fetch/parse/map jobs with scheduler.| -|Implement OSV fetch/parse/map skeleton|BE-Conn-OSV|Source.Common|**DONE** – connector now persists documents, DTOs, and canonical advisories.| -|FEEDCONN-OSV-02-004 OSV references & credits alignment|BE-Conn-OSV|Models `FEEDMODELS-SCHEMA-01-002`|**DONE (2025-10-11)** – Mapper normalizes references with provenance masks, emits advisory credits, and regression fixtures/assertions cover the new fields.| -|FEEDCONN-OSV-02-005 Fixture updater workflow|BE-Conn-OSV, QA|Docs|**DONE (2025-10-12)** – Canonical PURL derivation now covers Go + scoped npm advisories without upstream `purl`; legacy invalid npm names still fall back to `ecosystem:name`. OSV/GHSA/NVD suites and normalization/storage tests rerun clean.| -|FEEDCONN-OSV-02-003 Normalized versions rollout|BE-Conn-OSV|Models `FEEDMODELS-SCHEMA-01-003`, Normalization playbook|**DONE (2025-10-11)** – `OsvMapper` now emits SemVer primitives + normalized rules with `osv:{ecosystem}:{advisoryId}:{identifier}` notes; npm/PyPI/Parity fixtures refreshed; merge coordination pinged (OSV handoff).| -|FEEDCONN-OSV-04-003 Parity fixture refresh|QA, BE-Conn-OSV|Normalized versions rollout, GHSA parity tests|**DONE (2025-10-12)** – Parity fixtures include normalizedVersions notes (`osv:::`); regression math rerun via `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv.Tests` and docs flagged for workflow sync.| -|FEEDCONN-OSV-04-002 Conflict regression fixtures|BE-Conn-OSV, QA|Merge `FEEDMERGE-ENGINE-04-001`|**DONE (2025-10-12)** – Added `conflict-osv.canonical.json` + regression asserting SemVer range + CVSS medium severity; dataset matches GHSA/NVD fixtures for merge tests. Validation: `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv.Tests/StellaOps.Concelier.Connector.Osv.Tests.csproj --filter OsvConflictFixtureTests`.| -|FEEDCONN-OSV-04-004 Description/CWE/metric parity rollout|BE-Conn-OSV|Models, Core|**DONE (2025-10-15)** – OSV mapper writes advisory descriptions, `database_specific.cwe_ids` weaknesses, and canonical CVSS metric id. Parity fixtures (`osv-ghsa.*`, `osv-npm.snapshot.json`, `osv-pypi.snapshot.json`) refreshed and status communicated to Merge coordination.| -|FEEDCONN-OSV-04-005 Canonical metric fallbacks & CWE notes|BE-Conn-OSV|Models, Merge|**DONE (2025-10-16)** – Add fallback logic and metrics for advisories lacking CVSS vectors, enrich CWE provenance notes, and document merge/export expectations; refresh parity fixtures accordingly.
2025-10-16: Mapper now emits `osv:severity/` canonical ids for severity-only advisories, weakness provenance carries `database_specific.cwe_ids`, diagnostics expose `osv.map.canonical_metric_fallbacks`, parity fixtures regenerated, and ops notes added in `docs/modules/concelier/operations/connectors/osv.md`. Tests: `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv.Tests/StellaOps.Concelier.Connector.Osv.Tests.csproj`.| diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ru.Bdu/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ru.Bdu/TASKS.completed.md new file mode 100644 index 00000000..abe8cd2d --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ru.Bdu/TASKS.completed.md @@ -0,0 +1,18 @@ +# Completed Tasks + +|FEEDCONN-RUBDU-02-001 Identify BDU data source & schema|BE-Conn-BDU|Research|**DONE (2025-10-11)** – Candidate endpoints (`https://bdu.fstec.ru/component/rsform/form/7-bdu?format=xml`, `...?format=json`) return 403/404 even with `--insecure` because TLS chain requires Russian Trusted Sub CA and WAF expects referer/session headers. Documented request/response samples in `docs/concelier-connector-research-20251011.md`; blocked until trusted root + access strategy from Ops.| + +|FEEDCONN-RUBDU-02-002 Fetch pipeline & cursor handling|BE-Conn-BDU|Source.Common, Storage.Mongo|**DONE (2025-10-14)** – Connector streams `vulxml.zip` through cached fetches, persists JSON payloads via `RawDocumentStorage`, and tracks cursor pending sets. Added cache fallback + deterministic SHA logging and state updates tied to `TimeProvider`.| + +|FEEDCONN-RUBDU-02-003 DTO/parser implementation|BE-Conn-BDU|Source.Common|**DONE (2025-10-14)** – `RuBduXmlParser` now captures identifiers, source links, CVSS 2/3 metrics, CWE arrays, and environment/software metadata with coverage for multi-entry fixtures.| + +|FEEDCONN-RUBDU-02-004 Canonical mapping & range primitives|BE-Conn-BDU|Models|**DONE (2025-10-14)** – `RuBduMapper` emits vendor/ICS packages with normalized `ru-bdu.raw` rules, dual status provenance, alias/reference hydration (CVE, external, source), and CVSS severity normalisation.| + +|FEEDCONN-RUBDU-02-005 Deterministic fixtures & regression tests|QA|Testing|**DONE (2025-10-14)** – Added connector harness snapshot suite with canned archive, state/documents/dtos/advisories snapshots under `Fixtures/`, gated by `UPDATE_BDU_FIXTURES`.| + +|FEEDCONN-RUBDU-02-006 Telemetry & documentation|DevEx|Docs|**DONE (2025-10-14)** – Introduced `RuBduDiagnostics` meter (fetch/parse/map counters & histograms) and authored connector README covering configuration, trusted roots, telemetry, and offline behaviour.| + +|FEEDCONN-RUBDU-02-007 Access & export options assessment|BE-Conn-BDU|Research|**DONE (2025-10-14)** – Documented archive access constraints, offline mirroring expectations, and export packaging in `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ru.Bdu/README.md` + flagged Offline Kit bundling requirements.| + +|FEEDCONN-RUBDU-02-008 Trusted root onboarding plan|BE-Conn-BDU|Source.Common|**DONE (2025-10-14)** – Validated Russian Trusted Root/Sub CA bundle wiring (`certificates/russian_trusted_bundle.pem`), updated Offline Kit guidance, and surfaced `concelier:httpClients:source.bdu:trustedRootPaths` sample configuration.| + diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ru.Bdu/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ru.Bdu/TASKS.md index f14b7cae..2265b2fc 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ru.Bdu/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ru.Bdu/TASKS.md @@ -1,11 +1,3 @@ # TASKS | Task | Owner(s) | Depends on | Notes | |---|---|---|---| -|FEEDCONN-RUBDU-02-001 Identify BDU data source & schema|BE-Conn-BDU|Research|**DONE (2025-10-11)** – Candidate endpoints (`https://bdu.fstec.ru/component/rsform/form/7-bdu?format=xml`, `...?format=json`) return 403/404 even with `--insecure` because TLS chain requires Russian Trusted Sub CA and WAF expects referer/session headers. Documented request/response samples in `docs/concelier-connector-research-20251011.md`; blocked until trusted root + access strategy from Ops.| -|FEEDCONN-RUBDU-02-002 Fetch pipeline & cursor handling|BE-Conn-BDU|Source.Common, Storage.Mongo|**DONE (2025-10-14)** – Connector streams `vulxml.zip` through cached fetches, persists JSON payloads via `RawDocumentStorage`, and tracks cursor pending sets. Added cache fallback + deterministic SHA logging and state updates tied to `TimeProvider`.| -|FEEDCONN-RUBDU-02-003 DTO/parser implementation|BE-Conn-BDU|Source.Common|**DONE (2025-10-14)** – `RuBduXmlParser` now captures identifiers, source links, CVSS 2/3 metrics, CWE arrays, and environment/software metadata with coverage for multi-entry fixtures.| -|FEEDCONN-RUBDU-02-004 Canonical mapping & range primitives|BE-Conn-BDU|Models|**DONE (2025-10-14)** – `RuBduMapper` emits vendor/ICS packages with normalized `ru-bdu.raw` rules, dual status provenance, alias/reference hydration (CVE, external, source), and CVSS severity normalisation.| -|FEEDCONN-RUBDU-02-005 Deterministic fixtures & regression tests|QA|Testing|**DONE (2025-10-14)** – Added connector harness snapshot suite with canned archive, state/documents/dtos/advisories snapshots under `Fixtures/`, gated by `UPDATE_BDU_FIXTURES`.| -|FEEDCONN-RUBDU-02-006 Telemetry & documentation|DevEx|Docs|**DONE (2025-10-14)** – Introduced `RuBduDiagnostics` meter (fetch/parse/map counters & histograms) and authored connector README covering configuration, trusted roots, telemetry, and offline behaviour.| -|FEEDCONN-RUBDU-02-007 Access & export options assessment|BE-Conn-BDU|Research|**DONE (2025-10-14)** – Documented archive access constraints, offline mirroring expectations, and export packaging in `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ru.Bdu/README.md` + flagged Offline Kit bundling requirements.| -|FEEDCONN-RUBDU-02-008 Trusted root onboarding plan|BE-Conn-BDU|Source.Common|**DONE (2025-10-14)** – Validated Russian Trusted Root/Sub CA bundle wiring (`certificates/russian_trusted_bundle.pem`), updated Offline Kit guidance, and surfaced `concelier:httpClients:source.bdu:trustedRootPaths` sample configuration.| diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ru.Nkcki/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ru.Nkcki/TASKS.completed.md new file mode 100644 index 00000000..3eb7f969 --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ru.Nkcki/TASKS.completed.md @@ -0,0 +1,18 @@ +# Completed Tasks + +|FEEDCONN-NKCKI-02-001 Research NKTsKI advisory feeds|BE-Conn-Nkcki|Research|**DONE (2025-10-11)** – Candidate RSS locations (`https://cert.gov.ru/rss/advisories.xml`, `https://www.cert.gov.ru/...`) return 403/404 even with `Accept-Language: ru-RU` and `--insecure`; site is Bitrix-backed and expects Russian Trusted Sub CA plus session cookies. Logged packet captures + needed cert list in `docs/concelier-connector-research-20251011.md`; waiting on Ops for sanctioned trust bundle.| + +|FEEDCONN-NKCKI-02-002 Fetch pipeline & state persistence|BE-Conn-Nkcki|Source.Common, Storage.Mongo|**DONE (2025-10-13)** – Listing fetch now honours `maxListingPagesPerFetch`, persists cache hits when listing access fails, and records telemetry via `RuNkckiDiagnostics`. Cursor tracking covers pending documents/mappings and the known bulletin ring buffer.| + +|FEEDCONN-NKCKI-02-003 DTO & parser implementation|BE-Conn-Nkcki|Source.Common|**DONE (2025-10-13)** – Parser normalises nested arrays (ICS categories, vulnerable software lists, optional tags), flattens multiline `software_text`, and guarantees deterministic ordering for URLs and tags.| + +|FEEDCONN-NKCKI-02-004 Canonical mapping & range primitives|BE-Conn-Nkcki|Models|**DONE (2025-10-13)** – Mapper splits structured software entries, emits SemVer range primitives + normalized rules, deduplicates references, and surfaces CVSS v4 metadata alongside existing metrics.| + +|FEEDCONN-NKCKI-02-005 Deterministic fixtures & tests|QA|Testing|**DONE (2025-10-13)** – Fixtures refreshed with multi-page pagination + multi-entry bulletins. Tests exercise cache replay and rely on bundled OpenSSL 1.1 libs in `src/Tools/openssl/linux-x64` to keep Mongo2Go green on modern distros.| + +|FEEDCONN-NKCKI-02-006 Telemetry & documentation|DevEx|Docs|**DONE (2025-10-13)** – Added connector-specific metrics (`nkcki.*`) and documented configuration/operational guidance in `docs/modules/concelier/operations/connectors/nkcki.md`.| + +|FEEDCONN-NKCKI-02-007 Archive ingestion strategy|BE-Conn-Nkcki|Research|**DONE (2025-10-13)** – Documented Bitrix pagination/backfill plan (cache-first, offline replay, HTML/PDF capture) in `docs/modules/concelier/operations/connectors/nkcki.md`.| + +|FEEDCONN-NKCKI-02-008 Access enablement plan|BE-Conn-Nkcki|Source.Common|**DONE (2025-10-11)** – Documented trust-store requirement, optional SOCKS proxy fallback, and monitoring plan; shared TLS support now available via `SourceHttpClientOptions.TrustedRootCertificates` (`concelier:httpClients:source.nkcki:*`), awaiting Ops-sourced cert bundle before fetch implementation.| + diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ru.Nkcki/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ru.Nkcki/TASKS.md index 16b1bb90..2265b2fc 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ru.Nkcki/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ru.Nkcki/TASKS.md @@ -1,11 +1,3 @@ # TASKS | Task | Owner(s) | Depends on | Notes | |---|---|---|---| -|FEEDCONN-NKCKI-02-001 Research NKTsKI advisory feeds|BE-Conn-Nkcki|Research|**DONE (2025-10-11)** – Candidate RSS locations (`https://cert.gov.ru/rss/advisories.xml`, `https://www.cert.gov.ru/...`) return 403/404 even with `Accept-Language: ru-RU` and `--insecure`; site is Bitrix-backed and expects Russian Trusted Sub CA plus session cookies. Logged packet captures + needed cert list in `docs/concelier-connector-research-20251011.md`; waiting on Ops for sanctioned trust bundle.| -|FEEDCONN-NKCKI-02-002 Fetch pipeline & state persistence|BE-Conn-Nkcki|Source.Common, Storage.Mongo|**DONE (2025-10-13)** – Listing fetch now honours `maxListingPagesPerFetch`, persists cache hits when listing access fails, and records telemetry via `RuNkckiDiagnostics`. Cursor tracking covers pending documents/mappings and the known bulletin ring buffer.| -|FEEDCONN-NKCKI-02-003 DTO & parser implementation|BE-Conn-Nkcki|Source.Common|**DONE (2025-10-13)** – Parser normalises nested arrays (ICS categories, vulnerable software lists, optional tags), flattens multiline `software_text`, and guarantees deterministic ordering for URLs and tags.| -|FEEDCONN-NKCKI-02-004 Canonical mapping & range primitives|BE-Conn-Nkcki|Models|**DONE (2025-10-13)** – Mapper splits structured software entries, emits SemVer range primitives + normalized rules, deduplicates references, and surfaces CVSS v4 metadata alongside existing metrics.| -|FEEDCONN-NKCKI-02-005 Deterministic fixtures & tests|QA|Testing|**DONE (2025-10-13)** – Fixtures refreshed with multi-page pagination + multi-entry bulletins. Tests exercise cache replay and rely on bundled OpenSSL 1.1 libs in `src/Tools/openssl/linux-x64` to keep Mongo2Go green on modern distros.| -|FEEDCONN-NKCKI-02-006 Telemetry & documentation|DevEx|Docs|**DONE (2025-10-13)** – Added connector-specific metrics (`nkcki.*`) and documented configuration/operational guidance in `docs/modules/concelier/operations/connectors/nkcki.md`.| -|FEEDCONN-NKCKI-02-007 Archive ingestion strategy|BE-Conn-Nkcki|Research|**DONE (2025-10-13)** – Documented Bitrix pagination/backfill plan (cache-first, offline replay, HTML/PDF capture) in `docs/modules/concelier/operations/connectors/nkcki.md`.| -|FEEDCONN-NKCKI-02-008 Access enablement plan|BE-Conn-Nkcki|Source.Common|**DONE (2025-10-11)** – Documented trust-store requirement, optional SOCKS proxy fallback, and monitoring plan; shared TLS support now available via `SourceHttpClientOptions.TrustedRootCertificates` (`concelier:httpClients:source.nkcki:*`), awaiting Ops-sourced cert bundle before fetch implementation.| diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.StellaOpsMirror/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.StellaOpsMirror/TASKS.completed.md new file mode 100644 index 00000000..8f9e0b9a --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.StellaOpsMirror/TASKS.completed.md @@ -0,0 +1,7 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| FEEDCONN-STELLA-08-001 | DONE (2025-10-20) | BE-Conn-Stella | CONCELIER-EXPORT-08-201 | Implement Concelier mirror fetcher hitting `https://.stella-ops.org/concelier/exports/index.json`, verify signatures/digests, and persist raw documents with provenance. | Fetch job downloads mirror manifest, verifies digest/signature, stores raw docs with tests covering happy-path + tampered manifest. *(Completed 2025-10-20: detached JWS + digest enforcement, metadata persisted, and regression coverage via `dotnet test src/Concelier/__Tests/StellaOps.Concelier.Connector.StellaOpsMirror.Tests/StellaOps.Concelier.Connector.StellaOpsMirror.Tests.csproj`.)* | +| FEEDCONN-STELLA-08-002 | DONE (2025-10-20) | BE-Conn-Stella | FEEDCONN-STELLA-08-001 | Map mirror payloads into canonical advisory DTOs with provenance referencing mirror domain + original source metadata. | Mapper produces advisories/aliases/affected with mirror provenance; fixtures assert canonical parity with upstream JSON exporters. | +| FEEDCONN-STELLA-08-003 | DONE (2025-10-20) | BE-Conn-Stella | FEEDCONN-STELLA-08-002 | Add incremental cursor + resume support (per-export fingerprint) and document configuration for downstream Concelier instances. | Connector resumes from last export, handles deletion/delta cases, docs updated with config sample; integration test covers resume + new export scenario. | diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.StellaOpsMirror/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.StellaOpsMirror/TASKS.md index 34eb94a6..07f04678 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.StellaOpsMirror/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.StellaOpsMirror/TASKS.md @@ -2,6 +2,3 @@ | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| FEEDCONN-STELLA-08-001 | DONE (2025-10-20) | BE-Conn-Stella | CONCELIER-EXPORT-08-201 | Implement Concelier mirror fetcher hitting `https://.stella-ops.org/concelier/exports/index.json`, verify signatures/digests, and persist raw documents with provenance. | Fetch job downloads mirror manifest, verifies digest/signature, stores raw docs with tests covering happy-path + tampered manifest. *(Completed 2025-10-20: detached JWS + digest enforcement, metadata persisted, and regression coverage via `dotnet test src/Concelier/__Tests/StellaOps.Concelier.Connector.StellaOpsMirror.Tests/StellaOps.Concelier.Connector.StellaOpsMirror.Tests.csproj`.)* | -| FEEDCONN-STELLA-08-002 | DONE (2025-10-20) | BE-Conn-Stella | FEEDCONN-STELLA-08-001 | Map mirror payloads into canonical advisory DTOs with provenance referencing mirror domain + original source metadata. | Mapper produces advisories/aliases/affected with mirror provenance; fixtures assert canonical parity with upstream JSON exporters. | -| FEEDCONN-STELLA-08-003 | DONE (2025-10-20) | BE-Conn-Stella | FEEDCONN-STELLA-08-002 | Add incremental cursor + resume support (per-export fingerprint) and document configuration for downstream Concelier instances. | Connector resumes from last export, handles deletion/delta cases, docs updated with config sample; integration test covers resume + new export scenario. | diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Adobe/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Adobe/TASKS.completed.md new file mode 100644 index 00000000..2446f150 --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Adobe/TASKS.completed.md @@ -0,0 +1,20 @@ +# Completed Tasks + +|Index discovery and sliding window fetch|BE-Conn-Adobe|Source.Common|DONE — Support backfill; honor robots/ToS.| + +|Detail extractor (products/components/fixes)|BE-Conn-Adobe|Source.Common|DONE — Normalizes metadata and CVE/product capture.| + +|DTO schema and validation pipeline|BE-Conn-Adobe, QA|Source.Common|DONE — JSON schema enforced during parse.| + +|Canonical mapping plus psirt_flags|BE-Conn-Adobe|Models|DONE — Emits canonical advisory and Adobe psirt flag.| + +|SourceState plus sha256 short-circuit|BE-Conn-Adobe|Storage.Mongo|DONE — Idempotence guarantee.| + +|Golden fixtures and determinism tests|QA|Source.Vndr.Adobe|**DONE** — connector tests assert snapshot determinism for dual advisories.| + +|Mark failed parse DTOs|BE-Conn-Adobe|Storage.Mongo|**DONE** — parse failures now mark documents `Failed` and tests cover the path.| + +|Reference dedupe & ordering|BE-Conn-Adobe|Models|**DONE** — mapper groups references by URL with deterministic ordering.| + +|NormalizedVersions emission|BE-Conn-Adobe|Models|**DONE** (2025-10-11) — EVR-like version metadata now projects `normalizedVersions` with `adobe::` notes; regression fixtures refreshed to assert rule output.| + diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Adobe/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Adobe/TASKS.md index a72f8690..2265b2fc 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Adobe/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Adobe/TASKS.md @@ -1,12 +1,3 @@ -# TASKS -| Task | Owner(s) | Depends on | Notes | -|---|---|---|---| -|Index discovery and sliding window fetch|BE-Conn-Adobe|Source.Common|DONE — Support backfill; honor robots/ToS.| -|Detail extractor (products/components/fixes)|BE-Conn-Adobe|Source.Common|DONE — Normalizes metadata and CVE/product capture.| -|DTO schema and validation pipeline|BE-Conn-Adobe, QA|Source.Common|DONE — JSON schema enforced during parse.| -|Canonical mapping plus psirt_flags|BE-Conn-Adobe|Models|DONE — Emits canonical advisory and Adobe psirt flag.| -|SourceState plus sha256 short-circuit|BE-Conn-Adobe|Storage.Mongo|DONE — Idempotence guarantee.| -|Golden fixtures and determinism tests|QA|Source.Vndr.Adobe|**DONE** — connector tests assert snapshot determinism for dual advisories.| -|Mark failed parse DTOs|BE-Conn-Adobe|Storage.Mongo|**DONE** — parse failures now mark documents `Failed` and tests cover the path.| -|Reference dedupe & ordering|BE-Conn-Adobe|Models|**DONE** — mapper groups references by URL with deterministic ordering.| -|NormalizedVersions emission|BE-Conn-Adobe|Models|**DONE** (2025-10-11) — EVR-like version metadata now projects `normalizedVersions` with `adobe::` notes; regression fixtures refreshed to assert rule output.| +# TASKS +| Task | Owner(s) | Depends on | Notes | +|---|---|---|---| diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Apple/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Apple/TASKS.completed.md new file mode 100644 index 00000000..9cb39c57 --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Apple/TASKS.completed.md @@ -0,0 +1,18 @@ +# Completed Tasks + +|Catalogue Apple security bulletin sources|BE-Conn-Apple|Research|**DONE** – Feed contract documented in README (Software Lookup Service JSON + HT article hub) with rate-limit notes.| + +|Fetch pipeline & state persistence|BE-Conn-Apple|Source.Common, Storage.Mongo|**DONE** – Index fetch + detail ingestion with SourceState cursoring/allowlists committed; awaiting live smoke run before enabling in scheduler defaults.| + +|Parser & DTO implementation|BE-Conn-Apple|Source.Common|**DONE** – AngleSharp detail parser produces canonical DTO payloads (CVE list, timestamps, affected tables) persisted via DTO store.| + +|Canonical mapping & range primitives|BE-Conn-Apple|Models|**DONE** – Mapper now emits SemVer-derived normalizedVersions with `apple::` notes; fixtures updated to assert canonical rules while we continue tracking multi-device coverage in follow-up tasks.
2025-10-11 research trail: confirmed payload aligns with `[{"scheme":"semver","type":"range","min":"","minInclusive":true,"max":"","maxInclusive":false,"notes":"apple:ios:17.1"}]`; continue using `notes` to surface build identifiers for storage provenance.| + +|Deterministic fixtures/tests|QA|Testing|**DONE (2025-10-12)** – Parser now scopes references to article content, sorts affected rows deterministically, and regenerated fixtures (125326/125328/106355/HT214108/HT215500) produce stable JSON + sanitizer HTML in English.| + +|Telemetry & documentation|DevEx|Docs|**DONE (2025-10-12)** – OpenTelemetry pipeline exports `StellaOps.Concelier.Connector.Vndr.Apple`; runbook `docs/modules/concelier/operations/connectors/apple.md` added with metrics + monitoring guidance.| + +|Live HTML regression sweep|QA|Source.Common|**DONE (2025-10-12)** – Captured latest support.apple.com articles for 125326/125328/106355/HT214108/HT215500, trimmed nav noise, and committed sanitized HTML + expected DTOs with invariant timestamps.| + +|Fixture regeneration tooling|DevEx|Testing|**DONE (2025-10-12)** – `scripts/update-apple-fixtures.(sh|ps1)` set the env flag + sentinel, forward through WSLENV, and clean up after regeneration; README references updated usage.| + diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Apple/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Apple/TASKS.md index ff9fe4aa..2265b2fc 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Apple/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Apple/TASKS.md @@ -1,11 +1,3 @@ # TASKS | Task | Owner(s) | Depends on | Notes | |---|---|---|---| -|Catalogue Apple security bulletin sources|BE-Conn-Apple|Research|**DONE** – Feed contract documented in README (Software Lookup Service JSON + HT article hub) with rate-limit notes.| -|Fetch pipeline & state persistence|BE-Conn-Apple|Source.Common, Storage.Mongo|**DONE** – Index fetch + detail ingestion with SourceState cursoring/allowlists committed; awaiting live smoke run before enabling in scheduler defaults.| -|Parser & DTO implementation|BE-Conn-Apple|Source.Common|**DONE** – AngleSharp detail parser produces canonical DTO payloads (CVE list, timestamps, affected tables) persisted via DTO store.| -|Canonical mapping & range primitives|BE-Conn-Apple|Models|**DONE** – Mapper now emits SemVer-derived normalizedVersions with `apple::` notes; fixtures updated to assert canonical rules while we continue tracking multi-device coverage in follow-up tasks.
2025-10-11 research trail: confirmed payload aligns with `[{"scheme":"semver","type":"range","min":"","minInclusive":true,"max":"","maxInclusive":false,"notes":"apple:ios:17.1"}]`; continue using `notes` to surface build identifiers for storage provenance.| -|Deterministic fixtures/tests|QA|Testing|**DONE (2025-10-12)** – Parser now scopes references to article content, sorts affected rows deterministically, and regenerated fixtures (125326/125328/106355/HT214108/HT215500) produce stable JSON + sanitizer HTML in English.| -|Telemetry & documentation|DevEx|Docs|**DONE (2025-10-12)** – OpenTelemetry pipeline exports `StellaOps.Concelier.Connector.Vndr.Apple`; runbook `docs/modules/concelier/operations/connectors/apple.md` added with metrics + monitoring guidance.| -|Live HTML regression sweep|QA|Source.Common|**DONE (2025-10-12)** – Captured latest support.apple.com articles for 125326/125328/106355/HT214108/HT215500, trimmed nav noise, and committed sanitized HTML + expected DTOs with invariant timestamps.| -|Fixture regeneration tooling|DevEx|Testing|**DONE (2025-10-12)** – `scripts/update-apple-fixtures.(sh|ps1)` set the env flag + sentinel, forward through WSLENV, and clean up after regeneration; README references updated usage.| diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Chromium/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Chromium/TASKS.completed.md new file mode 100644 index 00000000..6c21bfe0 --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Chromium/TASKS.completed.md @@ -0,0 +1,22 @@ +# Completed Tasks + +| CH1 | Blog crawl + cursor | Conn | DONE | Common | Sliding window feed reader with cursor persisted. | + +| CH2 | Post parser → DTO (CVEs, versions, refs) | QA | DONE | | AngleSharp parser normalizes CVEs, versions, references. | + +| CH3 | Canonical mapping (aliases/refs/affected-hint)| Conn | DONE | Models | Deterministic advisory mapping with psirt flags. | + +| CH4 | Snapshot tests + resume | QA | DONE | Storage | Deterministic snapshot plus resume scenario via Mongo state. | + +| CH5 | Observability | QA | DONE | | Metered fetch/parse/map counters. | + +| CH6 | SourceState + SHA dedupe | Conn | DONE | Storage | Cursor tracks SHA cache to skip unchanged posts. | + +| CH7 | Stabilize resume integration (preserve pending docs across provider instances) | QA | DONE | Storage.Mongo | Resume integration test exercises pending docs across providers via shared Mongo. | + +| CH8 | Mark failed parse documents | Conn | DONE | Storage.Mongo | Parse pipeline marks failures; unit tests assert status transitions. | + +| CH9 | Reference dedupe & ordering | Conn | DONE | Models | Mapper groups references by URL and sorts deterministically. | + +| CH10 | Range primitives + provenance instrumentation | Conn | DONE | Models, Storage.Mongo | Vendor primitives + logging in place, resume metrics updated, snapshots refreshed. | + diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Chromium/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Chromium/TASKS.md index 70a5cf6c..e2836c51 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Chromium/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Chromium/TASKS.md @@ -1,17 +1,7 @@ -# Source.Vndr.Chromium — Task Board - -| ID | Task | Owner | Status | Depends On | Notes | -|------|-----------------------------------------------|-------|--------|------------|-------| -| CH1 | Blog crawl + cursor | Conn | DONE | Common | Sliding window feed reader with cursor persisted. | -| CH2 | Post parser → DTO (CVEs, versions, refs) | QA | DONE | | AngleSharp parser normalizes CVEs, versions, references. | -| CH3 | Canonical mapping (aliases/refs/affected-hint)| Conn | DONE | Models | Deterministic advisory mapping with psirt flags. | -| CH4 | Snapshot tests + resume | QA | DONE | Storage | Deterministic snapshot plus resume scenario via Mongo state. | -| CH5 | Observability | QA | DONE | | Metered fetch/parse/map counters. | -| CH6 | SourceState + SHA dedupe | Conn | DONE | Storage | Cursor tracks SHA cache to skip unchanged posts. | -| CH7 | Stabilize resume integration (preserve pending docs across provider instances) | QA | DONE | Storage.Mongo | Resume integration test exercises pending docs across providers via shared Mongo. | -| CH8 | Mark failed parse documents | Conn | DONE | Storage.Mongo | Parse pipeline marks failures; unit tests assert status transitions. | -| CH9 | Reference dedupe & ordering | Conn | DONE | Models | Mapper groups references by URL and sorts deterministically. | -| CH10 | Range primitives + provenance instrumentation | Conn | DONE | Models, Storage.Mongo | Vendor primitives + logging in place, resume metrics updated, snapshots refreshed. | - -## Changelog -- YYYY-MM-DD: Created. +# Source.Vndr.Chromium — Task Board + +| ID | Task | Owner | Status | Depends On | Notes | +|------|-----------------------------------------------|-------|--------|------------|-------| + +## Changelog +- YYYY-MM-DD: Created. diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Cisco/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Cisco/TASKS.completed.md new file mode 100644 index 00000000..4b163e1c --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Cisco/TASKS.completed.md @@ -0,0 +1,18 @@ +# Completed Tasks + +|FEEDCONN-CISCO-02-001 Confirm Cisco PSIRT data source|BE-Conn-Cisco|Research|**DONE (2025-10-11)** – Selected openVuln REST API (`https://apix.cisco.com/security/advisories/v2/…`) as primary (structured JSON, CSAF/CVRF links) with RSS as fallback. Documented OAuth2 client-credentials flow (`cloudsso.cisco.com/as/token.oauth2`), baseline quotas (5 req/s, 30 req/min, 5 000 req/day), and pagination contract (`pageIndex`, `pageSize≤100`) in `docs/concelier-connector-research-20251011.md`.| + +|FEEDCONN-CISCO-02-002 Fetch pipeline & state persistence|BE-Conn-Cisco|Source.Common, Storage.Mongo|**DONE (2025-10-14)** – Fetch job now streams openVuln pages with OAuth bearer handler, honours 429 `Retry-After`, persists per-advisory JSON + metadata into GridFS, and updates cursor (`lastModified`, advisory ID, pending docs).| + +|FEEDCONN-CISCO-02-003 Parser & DTO implementation|BE-Conn-Cisco|Source.Common|**DONE (2025-10-14)** – DTO factory normalizes SIR, folds CSAF product statuses, and persists `cisco.dto.v1` payloads (see `CiscoDtoFactory`).| + +|FEEDCONN-CISCO-02-004 Canonical mapping & range primitives|BE-Conn-Cisco|Models|**DONE (2025-10-14)** – `CiscoMapper` emits canonical advisories with vendor + SemVer primitives, provenance, and status tags.| + +|FEEDCONN-CISCO-02-005 Deterministic fixtures & tests|QA|Testing|**DONE (2025-10-14)** – Added unit tests (`StellaOps.Concelier.Connector.Vndr.Cisco.Tests`) exercising DTO/mapper pipelines; `dotnet test` validated.| + +|FEEDCONN-CISCO-02-006 Telemetry & documentation|DevEx|Docs|**DONE (2025-10-14)** – Cisco diagnostics counters exposed and ops runbook updated with telemetry guidance (`docs/modules/concelier/operations/connectors/cisco.md`).| + +|FEEDCONN-CISCO-02-007 API selection decision memo|BE-Conn-Cisco|Research|**DONE (2025-10-11)** – Drafted decision matrix: openVuln (structured/delta filters, OAuth throttle) vs RSS (delayed/minimal metadata). Pending OAuth onboarding (`FEEDCONN-CISCO-02-008`) before final recommendation circulated.| + +|FEEDCONN-CISCO-02-008 OAuth client provisioning|Ops, BE-Conn-Cisco|Ops|**DONE (2025-10-14)** – `docs/modules/concelier/operations/connectors/cisco.md` documents OAuth provisioning/rotation, quotas, and Offline Kit distribution guidance.| + diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Cisco/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Cisco/TASKS.md index 710e67a3..45e8f095 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Cisco/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Cisco/TASKS.md @@ -1,12 +1,4 @@ # TASKS | Task | Owner(s) | Depends on | Notes | |---|---|---|---| -|FEEDCONN-CISCO-02-001 Confirm Cisco PSIRT data source|BE-Conn-Cisco|Research|**DONE (2025-10-11)** – Selected openVuln REST API (`https://apix.cisco.com/security/advisories/v2/…`) as primary (structured JSON, CSAF/CVRF links) with RSS as fallback. Documented OAuth2 client-credentials flow (`cloudsso.cisco.com/as/token.oauth2`), baseline quotas (5 req/s, 30 req/min, 5 000 req/day), and pagination contract (`pageIndex`, `pageSize≤100`) in `docs/concelier-connector-research-20251011.md`.| -|FEEDCONN-CISCO-02-002 Fetch pipeline & state persistence|BE-Conn-Cisco|Source.Common, Storage.Mongo|**DONE (2025-10-14)** – Fetch job now streams openVuln pages with OAuth bearer handler, honours 429 `Retry-After`, persists per-advisory JSON + metadata into GridFS, and updates cursor (`lastModified`, advisory ID, pending docs).| -|FEEDCONN-CISCO-02-003 Parser & DTO implementation|BE-Conn-Cisco|Source.Common|**DONE (2025-10-14)** – DTO factory normalizes SIR, folds CSAF product statuses, and persists `cisco.dto.v1` payloads (see `CiscoDtoFactory`).| -|FEEDCONN-CISCO-02-004 Canonical mapping & range primitives|BE-Conn-Cisco|Models|**DONE (2025-10-14)** – `CiscoMapper` emits canonical advisories with vendor + SemVer primitives, provenance, and status tags.| -|FEEDCONN-CISCO-02-005 Deterministic fixtures & tests|QA|Testing|**DONE (2025-10-14)** – Added unit tests (`StellaOps.Concelier.Connector.Vndr.Cisco.Tests`) exercising DTO/mapper pipelines; `dotnet test` validated.| -|FEEDCONN-CISCO-02-006 Telemetry & documentation|DevEx|Docs|**DONE (2025-10-14)** – Cisco diagnostics counters exposed and ops runbook updated with telemetry guidance (`docs/modules/concelier/operations/connectors/cisco.md`).| -|FEEDCONN-CISCO-02-007 API selection decision memo|BE-Conn-Cisco|Research|**DONE (2025-10-11)** – Drafted decision matrix: openVuln (structured/delta filters, OAuth throttle) vs RSS (delayed/minimal metadata). Pending OAuth onboarding (`FEEDCONN-CISCO-02-008`) before final recommendation circulated.| -|FEEDCONN-CISCO-02-008 OAuth client provisioning|Ops, BE-Conn-Cisco|Ops|**DONE (2025-10-14)** – `docs/modules/concelier/operations/connectors/cisco.md` documents OAuth provisioning/rotation, quotas, and Offline Kit distribution guidance.| |FEEDCONN-CISCO-02-009 Normalized SemVer promotion|BE-Conn-Cisco|Merge coordination (`FEEDMERGE-COORD-02-900`)|**TODO (due 2025-10-21)** – Use helper from `../Merge/RANGE_PRIMITIVES_COORDINATION.md` to convert `SemVerPrimitive` outputs into `NormalizedVersionRule` with provenance (`cisco:{productId}`), update mapper/tests, and confirm merge normalized-rule counters drop.| diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Msrc/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Msrc/TASKS.completed.md new file mode 100644 index 00000000..77cc9577 --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Msrc/TASKS.completed.md @@ -0,0 +1,18 @@ +# Completed Tasks + +|FEEDCONN-MSRC-02-001 Document MSRC Security Update Guide API|BE-Conn-MSRC|Research|**DONE (2025-10-11)** – Confirmed REST endpoint (`https://api.msrc.microsoft.com/sug/v2.0/en-US/vulnerabilities`) + CVRF ZIP download flow, required Azure AD client-credentials scope (`api://api.msrc.microsoft.com/.default`), mandatory `api-version=2024-08-01` header, and delta params (`lastModifiedStartDateTime`, `lastModifiedEndDateTime`). Findings recorded in `docs/concelier-connector-research-20251011.md`.| + +|FEEDCONN-MSRC-02-002 Fetch pipeline & source state|BE-Conn-MSRC|Source.Common, Storage.Mongo|**DONE (2025-10-15)** – Added `MsrcApiClient` + token provider, cursor overlap handling, and detail persistence via GridFS (metadata carries CVRF URL + timestamps). State tracks `lastModifiedCursor` with configurable overlap/backoff. **Next:** coordinate with Tools on shared state-seeding helper once CVRF download flag stabilises.| + +|FEEDCONN-MSRC-02-003 Parser & DTO implementation|BE-Conn-MSRC|Source.Common|**DONE (2025-10-15)** – Implemented `MsrcDetailParser`/DTOs capturing threats, remediations, KB IDs, CVEs, CVSS, and affected products (build/platform metadata preserved).| + +|FEEDCONN-MSRC-02-004 Canonical mapping & range primitives|BE-Conn-MSRC|Models|**DONE (2025-10-15)** – `MsrcMapper` emits aliases (MSRC ID/CVE/KB), references (release notes + CVRF), vendor packages with `msrc.build` normalized rules, and CVSS provenance.| + +|FEEDCONN-MSRC-02-005 Deterministic fixtures/tests|QA|Testing|**DONE (2025-10-15)** – Added `StellaOps.Concelier.Connector.Vndr.Msrc.Tests` with canned token/summary/detail responses and snapshot assertions via Mongo2Go. Fixtures regenerate via `UPDATE_MSRC_FIXTURES`.| + +|FEEDCONN-MSRC-02-006 Telemetry & documentation|DevEx|Docs|**DONE (2025-10-15)** – Introduced `MsrcDiagnostics` meter (summary/detail/parse/map metrics), structured fetch logs, README updates, and Ops brief `docs/modules/concelier/operations/connectors/msrc.md` covering AAD onboarding + CVRF handling.| + +|FEEDCONN-MSRC-02-007 API contract comparison memo|BE-Conn-MSRC|Research|**DONE (2025-10-11)** – Completed memo outline recommending dual-path (REST for incremental, CVRF for offline); implementation hinges on `FEEDCONN-MSRC-02-008` AAD onboarding for token acquisition.| + +|FEEDCONN-MSRC-02-008 Azure AD application onboarding|Ops, BE-Conn-MSRC|Ops|**DONE (2025-10-15)** – Coordinated Ops handoff; drafted AAD onboarding brief (`docs/modules/concelier/operations/connectors/msrc.md`) with app registration requirements, secret rotation policy, sample configuration, and CVRF mirroring guidance for Offline Kit.| + diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Msrc/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Msrc/TASKS.md index 6b8e9c8c..2265b2fc 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Msrc/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Msrc/TASKS.md @@ -1,11 +1,3 @@ # TASKS | Task | Owner(s) | Depends on | Notes | |---|---|---|---| -|FEEDCONN-MSRC-02-001 Document MSRC Security Update Guide API|BE-Conn-MSRC|Research|**DONE (2025-10-11)** – Confirmed REST endpoint (`https://api.msrc.microsoft.com/sug/v2.0/en-US/vulnerabilities`) + CVRF ZIP download flow, required Azure AD client-credentials scope (`api://api.msrc.microsoft.com/.default`), mandatory `api-version=2024-08-01` header, and delta params (`lastModifiedStartDateTime`, `lastModifiedEndDateTime`). Findings recorded in `docs/concelier-connector-research-20251011.md`.| -|FEEDCONN-MSRC-02-002 Fetch pipeline & source state|BE-Conn-MSRC|Source.Common, Storage.Mongo|**DONE (2025-10-15)** – Added `MsrcApiClient` + token provider, cursor overlap handling, and detail persistence via GridFS (metadata carries CVRF URL + timestamps). State tracks `lastModifiedCursor` with configurable overlap/backoff. **Next:** coordinate with Tools on shared state-seeding helper once CVRF download flag stabilises.| -|FEEDCONN-MSRC-02-003 Parser & DTO implementation|BE-Conn-MSRC|Source.Common|**DONE (2025-10-15)** – Implemented `MsrcDetailParser`/DTOs capturing threats, remediations, KB IDs, CVEs, CVSS, and affected products (build/platform metadata preserved).| -|FEEDCONN-MSRC-02-004 Canonical mapping & range primitives|BE-Conn-MSRC|Models|**DONE (2025-10-15)** – `MsrcMapper` emits aliases (MSRC ID/CVE/KB), references (release notes + CVRF), vendor packages with `msrc.build` normalized rules, and CVSS provenance.| -|FEEDCONN-MSRC-02-005 Deterministic fixtures/tests|QA|Testing|**DONE (2025-10-15)** – Added `StellaOps.Concelier.Connector.Vndr.Msrc.Tests` with canned token/summary/detail responses and snapshot assertions via Mongo2Go. Fixtures regenerate via `UPDATE_MSRC_FIXTURES`.| -|FEEDCONN-MSRC-02-006 Telemetry & documentation|DevEx|Docs|**DONE (2025-10-15)** – Introduced `MsrcDiagnostics` meter (summary/detail/parse/map metrics), structured fetch logs, README updates, and Ops brief `docs/modules/concelier/operations/connectors/msrc.md` covering AAD onboarding + CVRF handling.| -|FEEDCONN-MSRC-02-007 API contract comparison memo|BE-Conn-MSRC|Research|**DONE (2025-10-11)** – Completed memo outline recommending dual-path (REST for incremental, CVRF for offline); implementation hinges on `FEEDCONN-MSRC-02-008` AAD onboarding for token acquisition.| -|FEEDCONN-MSRC-02-008 Azure AD application onboarding|Ops, BE-Conn-MSRC|Ops|**DONE (2025-10-15)** – Coordinated Ops handoff; drafted AAD onboarding brief (`docs/modules/concelier/operations/connectors/msrc.md`) with app registration requirements, secret rotation policy, sample configuration, and CVRF mirroring guidance for Offline Kit.| diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Oracle/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Oracle/TASKS.completed.md new file mode 100644 index 00000000..bccb2ea6 --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Oracle/TASKS.completed.md @@ -0,0 +1,22 @@ +# Completed Tasks + +|Oracle options & HttpClient configuration|BE-Conn-Oracle|Source.Common|**DONE** – `AddOracleConnector` wires options and allowlisted HttpClient.| + +|CPU calendar plus advisory fetchers|BE-Conn-Oracle|Source.Common|**DONE** – resume/backfill scenario covered with new integration test and fetch cache pruning verified.| + +|Extractor for products/components/fix levels|BE-Conn-Oracle|Source.Common|**DONE** – HTML risk matrices parsed into vendor packages with fix heuristics and normalized versions.| + +|DTO schema and validation|BE-Conn-Oracle, QA|Source.Common|**DONE** – `OracleDtoValidator` enforces required fields and quarantines malformed payloads.| + +|Canonical mapping with psirt_flags|BE-Conn-Oracle|Models|**DONE** – mapper now emits CVE aliases, patch references, and vendor affected packages under psirt flag provenance.| + +|SourceState and dedupe|BE-Conn-Oracle|Storage.Mongo|**DONE** – cursor fetch cache tracks SHA/ETag to skip unchanged advisories and clear pending work.| + +|Golden fixtures and precedence tests (later with merge)|QA|Source.Vndr.Oracle|**DONE** – snapshot fixtures and psirt flag assertions added in `OracleConnectorTests`.| + +|Dependency injection routine & job registration|BE-Conn-Oracle|Core|**DONE** – `OracleDependencyInjectionRoutine` registers connector and fetch/parse/map jobs with scheduler defaults.| + +|Implement Oracle connector skeleton|BE-Conn-Oracle|Source.Common|**DONE** – fetch/parse/map pipeline persists documents, DTOs, advisories, psirt flags.| + +|Range primitives & provenance backfill|BE-Conn-Oracle|Models, Storage.Mongo|**DONE** – vendor primitives emitted (extensions + fix parsing), provenance tagging/logging extended, snapshots refreshed.| + diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Oracle/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Oracle/TASKS.md index 7b670e97..2265b2fc 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Oracle/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Oracle/TASKS.md @@ -1,13 +1,3 @@ -# TASKS -| Task | Owner(s) | Depends on | Notes | -|---|---|---|---| -|Oracle options & HttpClient configuration|BE-Conn-Oracle|Source.Common|**DONE** – `AddOracleConnector` wires options and allowlisted HttpClient.| -|CPU calendar plus advisory fetchers|BE-Conn-Oracle|Source.Common|**DONE** – resume/backfill scenario covered with new integration test and fetch cache pruning verified.| -|Extractor for products/components/fix levels|BE-Conn-Oracle|Source.Common|**DONE** – HTML risk matrices parsed into vendor packages with fix heuristics and normalized versions.| -|DTO schema and validation|BE-Conn-Oracle, QA|Source.Common|**DONE** – `OracleDtoValidator` enforces required fields and quarantines malformed payloads.| -|Canonical mapping with psirt_flags|BE-Conn-Oracle|Models|**DONE** – mapper now emits CVE aliases, patch references, and vendor affected packages under psirt flag provenance.| -|SourceState and dedupe|BE-Conn-Oracle|Storage.Mongo|**DONE** – cursor fetch cache tracks SHA/ETag to skip unchanged advisories and clear pending work.| -|Golden fixtures and precedence tests (later with merge)|QA|Source.Vndr.Oracle|**DONE** – snapshot fixtures and psirt flag assertions added in `OracleConnectorTests`.| -|Dependency injection routine & job registration|BE-Conn-Oracle|Core|**DONE** – `OracleDependencyInjectionRoutine` registers connector and fetch/parse/map jobs with scheduler defaults.| -|Implement Oracle connector skeleton|BE-Conn-Oracle|Source.Common|**DONE** – fetch/parse/map pipeline persists documents, DTOs, advisories, psirt flags.| -|Range primitives & provenance backfill|BE-Conn-Oracle|Models, Storage.Mongo|**DONE** – vendor primitives emitted (extensions + fix parsing), provenance tagging/logging extended, snapshots refreshed.| +# TASKS +| Task | Owner(s) | Depends on | Notes | +|---|---|---|---| diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Vmware/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Vmware/TASKS.completed.md new file mode 100644 index 00000000..14fbf7c7 --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Vmware/TASKS.completed.md @@ -0,0 +1,22 @@ +# Completed Tasks + +| VM1 | Advisory listing discovery + cursor | Conn | DONE | Common | **DONE** – fetch pipeline uses index JSON with sliding cursor + processed id tracking. | + +| VM2 | VMSA parser → DTO | QA | DONE | | **DONE** – JSON DTO deserialization wired with sanitization. | + +| VM3 | Canonical mapping (aliases/affected/refs) | Conn | DONE | Models | **DONE** – `VmwareMapper` emits aliases/affected/reference ordering and persists PSIRT flags via `PsirtFlagStore`. | + +| VM4 | Snapshot tests + resume | QA | DONE | Storage | **DONE** – integration test validates snapshot output and resume flow with cached state. | + +| VM5 | Observability | QA | DONE | | **DONE** – diagnostics meter exposes fetch/parse/map metrics and structured logs. | + +| VM6 | SourceState + hash dedupe | Conn | DONE | Storage | **DONE** – fetch cache stores sha/etag to skip unchanged advisories during resume. | + +| VM6a | Options & HttpClient configuration | Conn | DONE | Source.Common | **DONE** – `AddVmwareConnector` configures allowlisted HttpClient + options. | + +| VM7 | Dependency injection routine & scheduler registration | Conn | DONE | Core | **DONE** – `VmwareDependencyInjectionRoutine` registers fetch/parse/map jobs. | + +| VM8 | Replace stub plugin with connector pipeline skeleton | Conn | DONE | Source.Common | **DONE** – connector implements fetch/parse/map persisting docs, DTOs, advisories. | + +| VM9 | Range primitives + provenance diagnostics refresh | Conn | DONE | Models, Storage.Mongo | Vendor primitives emitted (SemVer + vendor extensions), provenance tags/logging updated, snapshots refreshed. | + diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Vmware/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Vmware/TASKS.md index 3215f711..22d0f02e 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Vmware/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Vmware/TASKS.md @@ -1,17 +1,7 @@ -# Source.Vndr.Vmware — Task Board - -| ID | Task | Owner | Status | Depends On | Notes | -|------|-----------------------------------------------|-------|--------|------------|-------| -| VM1 | Advisory listing discovery + cursor | Conn | DONE | Common | **DONE** – fetch pipeline uses index JSON with sliding cursor + processed id tracking. | -| VM2 | VMSA parser → DTO | QA | DONE | | **DONE** – JSON DTO deserialization wired with sanitization. | -| VM3 | Canonical mapping (aliases/affected/refs) | Conn | DONE | Models | **DONE** – `VmwareMapper` emits aliases/affected/reference ordering and persists PSIRT flags via `PsirtFlagStore`. | -| VM4 | Snapshot tests + resume | QA | DONE | Storage | **DONE** – integration test validates snapshot output and resume flow with cached state. | -| VM5 | Observability | QA | DONE | | **DONE** – diagnostics meter exposes fetch/parse/map metrics and structured logs. | -| VM6 | SourceState + hash dedupe | Conn | DONE | Storage | **DONE** – fetch cache stores sha/etag to skip unchanged advisories during resume. | -| VM6a | Options & HttpClient configuration | Conn | DONE | Source.Common | **DONE** – `AddVmwareConnector` configures allowlisted HttpClient + options. | -| VM7 | Dependency injection routine & scheduler registration | Conn | DONE | Core | **DONE** – `VmwareDependencyInjectionRoutine` registers fetch/parse/map jobs. | -| VM8 | Replace stub plugin with connector pipeline skeleton | Conn | DONE | Source.Common | **DONE** – connector implements fetch/parse/map persisting docs, DTOs, advisories. | -| VM9 | Range primitives + provenance diagnostics refresh | Conn | DONE | Models, Storage.Mongo | Vendor primitives emitted (SemVer + vendor extensions), provenance tags/logging updated, snapshots refreshed. | - -## Changelog -- YYYY-MM-DD: Created. +# Source.Vndr.Vmware — Task Board + +| ID | Task | Owner | Status | Depends On | Notes | +|------|-----------------------------------------------|-------|--------|------------|-------| + +## Changelog +- YYYY-MM-DD: Created. diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Core/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Core/TASKS.completed.md new file mode 100644 index 00000000..e0433cc9 --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Core/TASKS.completed.md @@ -0,0 +1,11 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Notes | +|---|---|---|---|---| +| CONCELIER-CORE-AOC-19-001 `AOC write guard` | DONE (2025-10-29) | Concelier Core Guild | WEB-AOC-19-001 | Implement repository interceptor that inspects write payloads for forbidden AOC keys, validates provenance/signature presence, and maps violations to `ERR_AOC_00x`. | +| CONCELIER-CORE-AOC-19-002 `Deterministic linkset extraction` | DONE (2025-10-31) | Concelier Core Guild | CONCELIER-CORE-AOC-19-001 | Build canonical linkset mappers for CVE/GHSA/PURL/CPE/reference extraction from upstream raw payloads, ensuring reconciled-from metadata is tracked and deterministic. | +| CONCELIER-CORE-AOC-19-003 `Idempotent append-only upsert` | DONE (2025-10-28) | Concelier Core Guild | CONCELIER-STORE-AOC-19-002 | Implement idempotent upsert path using `(vendor, upstreamId, contentHash, tenant)` key, emitting supersedes pointers for new revisions and preventing duplicate inserts. | + +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| CONCELIER-GRAPH-24-001 `Advisory overlay inputs` | DONE (2025-10-29) | Concelier Core Guild | CONCELIER-POLICY-23-001 | Expose raw advisory observations/linksets with tenant filters for overlay services; no derived counts/severity in ingestion. | diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Core/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Core/TASKS.md index e606d555..e2a094fb 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Core/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Core/TASKS.md @@ -1,122 +1,118 @@ -# TASKS — Epic 1: Aggregation-Only Contract -> **AOC Reminder:** ingestion aggregates and links only—no precedence, normalization, or severity computation. Derived data lives in Policy/overlay services. -| ID | Status | Owner(s) | Depends on | Notes | -|---|---|---|---|---| -| CONCELIER-CORE-AOC-19-001 `AOC write guard` | DONE (2025-10-29) | Concelier Core Guild | WEB-AOC-19-001 | Implement repository interceptor that inspects write payloads for forbidden AOC keys, validates provenance/signature presence, and maps violations to `ERR_AOC_00x`. | -> Docs alignment (2025-10-26): Behaviour/spec captured in `docs/ingestion/aggregation-only-contract.md` and architecture overview §2. -> Implementation (2025-10-29): Added `AdvisoryRawWriteGuard` + DI extensions wrapping `AocWriteGuard`, throwing domain-specific `ConcelierAocGuardException` with `ERR_AOC_00x` mappings. Unit tests cover valid/missing-tenant/signature cases. -> Coordination (2025-10-27): Authority `dotnet test` run is currently blocked because `AdvisoryObservationQueryService.BuildAliasLookup` returns `ImmutableHashSet`; please normalise these lookups to `ImmutableHashSet` (trim nulls) so downstream builds succeed. -| CONCELIER-CORE-AOC-19-002 `Deterministic linkset extraction` | DONE (2025-10-31) | Concelier Core Guild | CONCELIER-CORE-AOC-19-001 | Build canonical linkset mappers for CVE/GHSA/PURL/CPE/reference extraction from upstream raw payloads, ensuring reconciled-from metadata is tracked and deterministic. | -> 2025-10-31: Added advisory linkset mapper + DI registration, normalized PURL/CPE canonicalization, persisted `reconciled_from` pointers, and refreshed observation factory/tests for new raw linkset shape. -> Docs alignment (2025-10-26): Linkset expectations detailed in AOC reference §4 and policy-engine architecture §2.1. -| CONCELIER-CORE-AOC-19-003 `Idempotent append-only upsert` | DONE (2025-10-28) | Concelier Core Guild | CONCELIER-STORE-AOC-19-002 | Implement idempotent upsert path using `(vendor, upstreamId, contentHash, tenant)` key, emitting supersedes pointers for new revisions and preventing duplicate inserts. | -> 2025-10-28: Advisory raw ingestion now strips client-supplied supersedes hints, logs ignored pointers, and surfaces repository-supplied supersedes identifiers; service tests cover duplicate handling and append-only semantics. -> Docs alignment (2025-10-26): Deployment guide + observability guide describe supersedes metrics; ensure implementation emits `aoc_violation_total` on failure. +# TASKS — Epic 1: Aggregation-Only Contract +> **AOC Reminder:** ingestion aggregates and links only—no precedence, normalization, or severity computation. Derived data lives in Policy/overlay services. +| ID | Status | Owner(s) | Depends on | Notes | +|---|---|---|---|---| +> Docs alignment (2025-10-26): Behaviour/spec captured in `docs/ingestion/aggregation-only-contract.md` and architecture overview §2. +> Implementation (2025-10-29): Added `AdvisoryRawWriteGuard` + DI extensions wrapping `AocWriteGuard`, throwing domain-specific `ConcelierAocGuardException` with `ERR_AOC_00x` mappings. Unit tests cover valid/missing-tenant/signature cases. +> Coordination (2025-10-27): Authority `dotnet test` run is currently blocked because `AdvisoryObservationQueryService.BuildAliasLookup` returns `ImmutableHashSet`; please normalise these lookups to `ImmutableHashSet` (trim nulls) so downstream builds succeed. +> 2025-10-31: Added advisory linkset mapper + DI registration, normalized PURL/CPE canonicalization, persisted `reconciled_from` pointers, and refreshed observation factory/tests for new raw linkset shape. +> Docs alignment (2025-10-26): Linkset expectations detailed in AOC reference §4 and policy-engine architecture §2.1. +> 2025-10-28: Advisory raw ingestion now strips client-supplied supersedes hints, logs ignored pointers, and surfaces repository-supplied supersedes identifiers; service tests cover duplicate handling and append-only semantics. +> Docs alignment (2025-10-26): Deployment guide + observability guide describe supersedes metrics; ensure implementation emits `aoc_violation_total` on failure. | CONCELIER-CORE-AOC-19-004 `Remove ingestion normalization` | DOING (2025-10-28) | Concelier Core Guild | CONCELIER-CORE-AOC-19-002, POLICY-AOC-19-003 | Strip normalization/dedup/severity logic from ingestion pipelines, delegate derived computations to Policy Engine, and update exporters/tests to consume raw documents only.
2025-10-29 19:05Z: Audit completed for `AdvisoryRawService`/Mongo repo to confirm alias order/dedup removal persists; identified remaining normalization in observation/linkset factory that will be revised to surface raw duplicates for Policy ingestion. Change sketch + regression matrix drafted under `docs/dev/aoc-normalization-removal-notes.md` (pending commit). | > Docs alignment (2025-10-26): Architecture overview emphasises policy-only derivation; coordinate with Policy Engine guild for rollout. > 2025-10-29: `AdvisoryRawService` now preserves upstream alias/linkset ordering (trim-only) and updated AOC documentation reflects the behaviour; follow-up to ensure policy consumers handle duplicates remains open. -| CONCELIER-CORE-AOC-19-013 `Authority tenant scope smoke coverage` | TODO | Concelier Core Guild | AUTH-AOC-19-002 | Extend Concelier smoke/e2e fixtures to configure `requiredTenants` and assert cross-tenant rejection with updated Authority tokens. | Coordinate deliverable so Authority docs (`AUTH-AOC-19-003`) can close once tests are in place. | - -## Policy Engine v2 - -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| CONCELIER-POLICY-20-002 `Linkset enrichment for policy` | TODO | Concelier Core Guild, Policy Guild | CONCELIER-CORE-AOC-19-002, POLICY-ENGINE-20-001 | Strengthen linkset builders with vendor-specific equivalence tables, NEVRA/PURL normalization, and version range parsing to maximize policy join recall; update fixtures + docs. | -> 2025-10-31: Base advisory linkset mapper landed under `CONCELIER-CORE-AOC-19-002`; policy enrichment work can now proceed with mapper outputs and observation schema fixtures. - -## Graph Explorer v1 - -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| +| CONCELIER-CORE-AOC-19-013 `Authority tenant scope smoke coverage` | TODO | Concelier Core Guild | AUTH-AOC-19-002 | Extend Concelier smoke/e2e fixtures to configure `requiredTenants` and assert cross-tenant rejection with updated Authority tokens. | Coordinate deliverable so Authority docs (`AUTH-AOC-19-003`) can close once tests are in place. | + +## Policy Engine v2 + +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| CONCELIER-POLICY-20-002 `Linkset enrichment for policy` | TODO | Concelier Core Guild, Policy Guild | CONCELIER-CORE-AOC-19-002, POLICY-ENGINE-20-001 | Strengthen linkset builders with vendor-specific equivalence tables, NEVRA/PURL normalization, and version range parsing to maximize policy join recall; update fixtures + docs. | +> 2025-10-31: Base advisory linkset mapper landed under `CONCELIER-CORE-AOC-19-002`; policy enrichment work can now proceed with mapper outputs and observation schema fixtures. + +## Graph Explorer v1 + +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| | CONCELIER-GRAPH-21-001 `SBOM projection enrichment` | BLOCKED (2025-10-27) | Concelier Core Guild, Cartographer Guild | CONCELIER-POLICY-20-002, CARTO-GRAPH-21-002 | Extend SBOM normalization to emit full relationship graph (depends_on/contains/provides), scope tags, entrypoint annotations, and component metadata required by Cartographer. | > 2025-10-27: Waiting on policy-driven linkset enrichment (`CONCELIER-POLICY-20-002`) and Cartographer API contract (`CARTO-GRAPH-21-002`) to define required relationship payloads. Without those schemas the projection changes cannot be implemented deterministically. > 2025-10-29: Cross-guild handshake captured in `docs/dev/cartographer-graph-handshake.md`; begin drafting enrichment plan once Cartographer ships the inspector schema/query patterns. | CONCELIER-GRAPH-21-002 `Change events` | BLOCKED (2025-10-27) | Concelier Core Guild, Scheduler Guild | CONCELIER-GRAPH-21-001 | Publish change events (new SBOM version, relationship delta) for Cartographer build queue; ensure events include tenant/context metadata. | > 2025-10-27: Depends on `CONCELIER-GRAPH-21-001`; event schema hinges on finalized projection output and Cartographer webhook contract, both pending. > 2025-10-29: Action item from handshake doc — prepare sample `sbom.relationship.changed` payload + replay notes once schema lands; coordinate with Scheduler for queue semantics. - -## Link-Not-Merge v1 - -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| CONCELIER-LNM-21-001 `Advisory observation schema` | TODO | Concelier Core Guild | CONCELIER-CORE-AOC-19-001 | Introduce immutable `advisory_observations` model with AOC metadata, raw payload pointers, normalized fields, and tenancy guardrails; publish schema definition. `DOCS-LNM-22-001` blocked pending this deliverable. | -| CONCELIER-LNM-21-002 `Linkset builder` | TODO | Concelier Core Guild, Data Science Guild | CONCELIER-LNM-21-001 | Implement correlation pipeline (alias graph, PURL overlap, CVSS vector equality, fuzzy title match) that produces `advisory_linksets` with confidence + conflict annotations. Docs note: unblock `DOCS-LNM-22-001` once builder lands. | -| CONCELIER-LNM-21-003 `Conflict annotator` | TODO | Concelier Core Guild | CONCELIER-LNM-21-002 | Detect field disagreements (severity, CVSS, ranges, references) and record structured conflicts on linksets; surface to API/UI. Docs awaiting structured conflict payloads. | -| CONCELIER-LNM-21-004 `Merge code removal` | TODO | Concelier Core Guild | CONCELIER-LNM-21-002 | Excise existing merge/dedup logic, enforce immutability on observations, and add guards/tests to prevent future merges. | -| CONCELIER-LNM-21-005 `Event emission` | TODO | Concelier Core Guild, Platform Events Guild | CONCELIER-LNM-21-002 | Emit `advisory.linkset.updated` events with delta payloads for downstream Policy Engine/Cartographer consumers; ensure idempotent delivery. | - -## Policy Engine + Editor v1 - -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| CONCELIER-POLICY-23-001 `Evidence indexes` | TODO | Concelier Core Guild | CONCELIER-LNM-21-002 | Add secondary indexes/materialized views to accelerate policy lookups (alias, severity per observation, correlation confidence). Document query contracts for runtime. | -| CONCELIER-POLICY-23-002 `Event guarantees` | TODO | Concelier Core Guild, Platform Events Guild | CONCELIER-LNM-21-005 | Ensure `advisory.linkset.updated` emits at-least-once with idempotent keys and include policy-relevant metadata (confidence, conflict summary). | - -## Graph & Vuln Explorer v1 - -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| CONCELIER-GRAPH-24-001 `Advisory overlay inputs` | DONE (2025-10-29) | Concelier Core Guild | CONCELIER-POLICY-23-001 | Expose raw advisory observations/linksets with tenant filters for overlay services; no derived counts/severity in ingestion. | -> 2025-10-29: Filter-aware lookup path and /concelier/observations coverage landed; overlay services can consume raw advisory feeds deterministically. - -## Reachability v1 - -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| CONCELIER-SIG-26-001 `Vulnerable symbol exposure` | TODO | Concelier Core Guild, Signals Guild | SIGNALS-24-002 | Expose advisory metadata (affected symbols/functions) via API to enrich reachability scoring; update fixtures. | - -## Orchestrator Dashboard - -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| CONCELIER-ORCH-32-001 `Source registry integration` | TODO | Concelier Core Guild | ORCH-SVC-32-001, AUTH-ORCH-32-001 | Register Concelier data sources with orchestrator (metadata, schedules, rate policies) and wire provenance IDs/security scopes. | -| CONCELIER-ORCH-32-002 `Worker SDK adoption` | TODO | Concelier Core Guild | CONCELIER-ORCH-32-001, WORKER-GO-32-001, WORKER-PY-32-001 | Embed orchestrator worker SDK in ingestion loops, emit heartbeats/progress/artifact hashes, and enforce idempotency keys. | -| CONCELIER-ORCH-33-001 `Control hook compliance` | TODO | Concelier Core Guild | CONCELIER-ORCH-32-002, ORCH-SVC-33-001, ORCH-SVC-33-002 | Honor orchestrator throttle/pause/retry actions, surface structured error classes, and persist safe checkpoints for resume. | -| CONCELIER-ORCH-34-001 `Backfill + ledger linkage` | TODO | Concelier Core Guild | CONCELIER-ORCH-33-001, ORCH-SVC-33-003, ORCH-SVC-34-001 | Execute orchestrator-driven backfills, reuse artifact hashes to avoid duplicates, and link provenance to run ledger exports. | - -## Authority-Backed Scopes & Tenancy (Epic 14) -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| CONCELIER-TEN-48-001 `Tenant-aware linking` | TODO | Concelier Core Guild | AUTH-TEN-47-001 | Ensure advisory normalization/linking runs per tenant with RLS enforcing isolation; emit capability endpoint reporting `merge=false`; update events with tenant context. | - -## Observability & Forensics (Epic 15) -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| CONCELIER-OBS-50-001 `Telemetry adoption` | TODO | Concelier Core Guild, Observability Guild | TELEMETRY-OBS-50-001, TELEMETRY-OBS-50-002 | Replace ad-hoc logging with telemetry core across ingestion/linking pipelines; ensure spans/logs include tenant, source vendor, upstream id, content hash, and trace IDs. | -| CONCELIER-OBS-51-001 `Metrics & SLOs` | TODO | Concelier Core Guild, DevOps Guild | CONCELIER-OBS-50-001, TELEMETRY-OBS-51-001 | Emit metrics for ingest latency (cold/warm), queue depth, aoc violation rate, and publish SLO burn-rate alerts (ingest P95 <30s cold / <5s warm). Ship dashboards + alert configs. | -| CONCELIER-OBS-52-001 `Timeline events` | TODO | Concelier Core Guild | CONCELIER-OBS-50-001, TIMELINE-OBS-52-002 | Emit `timeline_event` records for advisory ingest/normalization/linkset creation with provenance, trace IDs, conflict summaries, and evidence placeholders. | -| CONCELIER-OBS-53-001 `Evidence snapshots` | TODO | Concelier Core Guild, Evidence Locker Guild | CONCELIER-OBS-52-001, EVID-OBS-53-002 | Produce advisory evaluation bundle payloads (raw doc, linkset, normalization diff) for evidence locker; ensure Merkle manifests seeded with content hashes. | -| CONCELIER-OBS-54-001 `Attestation & verification` | TODO | Concelier Core Guild, Provenance Guild | CONCELIER-OBS-53-001, PROV-OBS-54-001 | Attach DSSE attestations for advisory processing batches, expose verification API to confirm bundle integrity, and link attestation IDs back to timeline + ledger. | -| CONCELIER-OBS-55-001 `Incident mode hooks` | TODO | Concelier Core Guild, DevOps Guild | CONCELIER-OBS-51-001, DEVOPS-OBS-55-001 | Increase sampling, capture raw payload snapshots, and extend retention under incident mode; emit activation events + guardrails against PII leak. | - -## Air-Gapped Mode (Epic 16) -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| CONCELIER-AIRGAP-56-001 `Mirror ingestion adapters` | TODO | Concelier Core Guild | AIRGAP-IMP-57-002, MIRROR-CRT-56-001 | Add mirror source adapters reading advisories from imported bundles, preserving source metadata and bundle IDs. Ensure ingestion remains append-only. | -| CONCELIER-AIRGAP-56-002 `Bundle catalog linking` | TODO | Concelier Core Guild, AirGap Importer Guild | CONCELIER-AIRGAP-56-001, AIRGAP-IMP-57-001 | Persist `bundle_id`, `merkle_root`, and time anchor references on observations/linksets for provenance. | -| CONCELIER-AIRGAP-57-001 `Sealed-mode source restrictions` | TODO | Concelier Core Guild, AirGap Policy Guild | CONCELIER-AIRGAP-56-001, AIRGAP-POL-56-001 | Enforce sealed-mode egress rules by disallowing non-mirror connectors and surfacing remediation errors. | -| CONCELIER-AIRGAP-57-002 `Staleness annotations` | TODO | Concelier Core Guild, AirGap Time Guild | CONCELIER-AIRGAP-56-002, AIRGAP-TIME-58-001 | Compute staleness metadata for advisories per bundle and expose via API for Console/CLI badges. | -| CONCELIER-AIRGAP-58-001 `Portable advisory evidence` | TODO | Concelier Core Guild, Evidence Locker Guild | CONCELIER-OBS-53-001, EVID-OBS-54-001 | Package advisory evidence fragments into portable evidence bundles for cross-domain transfer. | - -## SDKs & OpenAPI (Epic 17) -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| CONCELIER-OAS-61-001 `Spec coverage` | TODO | Concelier Core Guild, API Contracts Guild | OAS-61-001 | Update Concelier OAS with advisory observation/linkset endpoints, standard pagination, and source provenance fields. | -| CONCELIER-OAS-61-002 `Examples library` | TODO | Concelier Core Guild | CONCELIER-OAS-61-001 | Provide rich examples for advisories, linksets, conflict annotations used by SDK + docs. | -| CONCELIER-OAS-62-001 `SDK smoke tests` | TODO | Concelier Core Guild, SDK Generator Guild | CONCELIER-OAS-61-001, SDKGEN-63-001 | Add SDK tests covering advisory search, pagination, and conflict handling; ensure source metadata surfaced. | -| CONCELIER-OAS-63-001 `Deprecation headers` | TODO | Concelier Core Guild, API Governance Guild | APIGOV-63-001 | Implement deprecation header support and timeline events for retiring endpoints. | - -## Risk Profiles (Epic 18) -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| CONCELIER-RISK-66-001 `CVSS/KEV providers` | TODO | Concelier Core Guild, Risk Engine Guild | RISK-ENGINE-67-001 | Expose CVSS, KEV, fix availability data via provider APIs with source metadata preserved. | -| CONCELIER-RISK-66-002 `Fix availability signals` | TODO | Concelier Core Guild | CONCELIER-RISK-66-001 | Provide structured fix availability and release metadata consumable by risk engine; document provenance. | -| CONCELIER-RISK-67-001 `Source consensus metrics` | TODO | Concelier Core Guild | CONCELIER-RISK-66-001 | Add consensus counts and confidence scores for linked advisories; ensure explainability includes source digests. | -| CONCELIER-RISK-68-001 `Policy Studio integration` | TODO | Concelier Core Guild, Policy Studio Guild | POLICY-RISK-68-001 | Surface advisory fields in Policy Studio profile editor (signal pickers, reducers). | -| CONCELIER-RISK-69-001 `Notification hooks` | TODO | Concelier Core Guild, Notifications Guild | CONCELIER-RISK-66-002 | Emit events when advisory signals change impacting risk scores (e.g., fix available). | - -## Attestor Console (Epic 19) -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| CONCELIER-ATTEST-73-001 `ScanResults attestation inputs` | TODO | Concelier Core Guild, Attestor Service Guild | ATTEST-TYPES-72-001 | Provide normalized advisory data and linkset digests needed for ScanResults attestations. | -| CONCELIER-ATTEST-73-002 `Transparency metadata` | TODO | Concelier Core Guild | CONCELIER-ATTEST-73-001 | Ensure Conseiller exposes source digests for transparency proofs and explainability. | + +## Link-Not-Merge v1 + +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| CONCELIER-LNM-21-001 `Advisory observation schema` | TODO | Concelier Core Guild | CONCELIER-CORE-AOC-19-001 | Introduce immutable `advisory_observations` model with AOC metadata, raw payload pointers, normalized fields, and tenancy guardrails; publish schema definition. `DOCS-LNM-22-001` blocked pending this deliverable. | +| CONCELIER-LNM-21-002 `Linkset builder` | TODO | Concelier Core Guild, Data Science Guild | CONCELIER-LNM-21-001 | Implement correlation pipeline (alias graph, PURL overlap, CVSS vector equality, fuzzy title match) that produces `advisory_linksets` with confidence + conflict annotations. Docs note: unblock `DOCS-LNM-22-001` once builder lands. | +| CONCELIER-LNM-21-003 `Conflict annotator` | TODO | Concelier Core Guild | CONCELIER-LNM-21-002 | Detect field disagreements (severity, CVSS, ranges, references) and record structured conflicts on linksets; surface to API/UI. Docs awaiting structured conflict payloads. | +| CONCELIER-LNM-21-004 `Merge code removal` | TODO | Concelier Core Guild | CONCELIER-LNM-21-002 | Excise existing merge/dedup logic, enforce immutability on observations, and add guards/tests to prevent future merges. | +| CONCELIER-LNM-21-005 `Event emission` | TODO | Concelier Core Guild, Platform Events Guild | CONCELIER-LNM-21-002 | Emit `advisory.linkset.updated` events with delta payloads for downstream Policy Engine/Cartographer consumers; ensure idempotent delivery. | + +## Policy Engine + Editor v1 + +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| CONCELIER-POLICY-23-001 `Evidence indexes` | TODO | Concelier Core Guild | CONCELIER-LNM-21-002 | Add secondary indexes/materialized views to accelerate policy lookups (alias, severity per observation, correlation confidence). Document query contracts for runtime. | +| CONCELIER-POLICY-23-002 `Event guarantees` | TODO | Concelier Core Guild, Platform Events Guild | CONCELIER-LNM-21-005 | Ensure `advisory.linkset.updated` emits at-least-once with idempotent keys and include policy-relevant metadata (confidence, conflict summary). | + +## Graph & Vuln Explorer v1 + +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +> 2025-10-29: Filter-aware lookup path and /concelier/observations coverage landed; overlay services can consume raw advisory feeds deterministically. + +## Reachability v1 + +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| CONCELIER-SIG-26-001 `Vulnerable symbol exposure` | TODO | Concelier Core Guild, Signals Guild | SIGNALS-24-002 | Expose advisory metadata (affected symbols/functions) via API to enrich reachability scoring; update fixtures. | + +## Orchestrator Dashboard + +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| CONCELIER-ORCH-32-001 `Source registry integration` | TODO | Concelier Core Guild | ORCH-SVC-32-001, AUTH-ORCH-32-001 | Register Concelier data sources with orchestrator (metadata, schedules, rate policies) and wire provenance IDs/security scopes. | +| CONCELIER-ORCH-32-002 `Worker SDK adoption` | TODO | Concelier Core Guild | CONCELIER-ORCH-32-001, WORKER-GO-32-001, WORKER-PY-32-001 | Embed orchestrator worker SDK in ingestion loops, emit heartbeats/progress/artifact hashes, and enforce idempotency keys. | +| CONCELIER-ORCH-33-001 `Control hook compliance` | TODO | Concelier Core Guild | CONCELIER-ORCH-32-002, ORCH-SVC-33-001, ORCH-SVC-33-002 | Honor orchestrator throttle/pause/retry actions, surface structured error classes, and persist safe checkpoints for resume. | +| CONCELIER-ORCH-34-001 `Backfill + ledger linkage` | TODO | Concelier Core Guild | CONCELIER-ORCH-33-001, ORCH-SVC-33-003, ORCH-SVC-34-001 | Execute orchestrator-driven backfills, reuse artifact hashes to avoid duplicates, and link provenance to run ledger exports. | + +## Authority-Backed Scopes & Tenancy (Epic 14) +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| CONCELIER-TEN-48-001 `Tenant-aware linking` | TODO | Concelier Core Guild | AUTH-TEN-47-001 | Ensure advisory normalization/linking runs per tenant with RLS enforcing isolation; emit capability endpoint reporting `merge=false`; update events with tenant context. | + +## Observability & Forensics (Epic 15) +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| CONCELIER-OBS-50-001 `Telemetry adoption` | TODO | Concelier Core Guild, Observability Guild | TELEMETRY-OBS-50-001, TELEMETRY-OBS-50-002 | Replace ad-hoc logging with telemetry core across ingestion/linking pipelines; ensure spans/logs include tenant, source vendor, upstream id, content hash, and trace IDs. | +| CONCELIER-OBS-51-001 `Metrics & SLOs` | TODO | Concelier Core Guild, DevOps Guild | CONCELIER-OBS-50-001, TELEMETRY-OBS-51-001 | Emit metrics for ingest latency (cold/warm), queue depth, aoc violation rate, and publish SLO burn-rate alerts (ingest P95 <30s cold / <5s warm). Ship dashboards + alert configs. | +| CONCELIER-OBS-52-001 `Timeline events` | TODO | Concelier Core Guild | CONCELIER-OBS-50-001, TIMELINE-OBS-52-002 | Emit `timeline_event` records for advisory ingest/normalization/linkset creation with provenance, trace IDs, conflict summaries, and evidence placeholders. | +| CONCELIER-OBS-53-001 `Evidence snapshots` | TODO | Concelier Core Guild, Evidence Locker Guild | CONCELIER-OBS-52-001, EVID-OBS-53-002 | Produce advisory evaluation bundle payloads (raw doc, linkset, normalization diff) for evidence locker; ensure Merkle manifests seeded with content hashes. | +| CONCELIER-OBS-54-001 `Attestation & verification` | TODO | Concelier Core Guild, Provenance Guild | CONCELIER-OBS-53-001, PROV-OBS-54-001 | Attach DSSE attestations for advisory processing batches, expose verification API to confirm bundle integrity, and link attestation IDs back to timeline + ledger. | +| CONCELIER-OBS-55-001 `Incident mode hooks` | TODO | Concelier Core Guild, DevOps Guild | CONCELIER-OBS-51-001, DEVOPS-OBS-55-001 | Increase sampling, capture raw payload snapshots, and extend retention under incident mode; emit activation events + guardrails against PII leak. | + +## Air-Gapped Mode (Epic 16) +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| CONCELIER-AIRGAP-56-001 `Mirror ingestion adapters` | TODO | Concelier Core Guild | AIRGAP-IMP-57-002, MIRROR-CRT-56-001 | Add mirror source adapters reading advisories from imported bundles, preserving source metadata and bundle IDs. Ensure ingestion remains append-only. | +| CONCELIER-AIRGAP-56-002 `Bundle catalog linking` | TODO | Concelier Core Guild, AirGap Importer Guild | CONCELIER-AIRGAP-56-001, AIRGAP-IMP-57-001 | Persist `bundle_id`, `merkle_root`, and time anchor references on observations/linksets for provenance. | +| CONCELIER-AIRGAP-57-001 `Sealed-mode source restrictions` | TODO | Concelier Core Guild, AirGap Policy Guild | CONCELIER-AIRGAP-56-001, AIRGAP-POL-56-001 | Enforce sealed-mode egress rules by disallowing non-mirror connectors and surfacing remediation errors. | +| CONCELIER-AIRGAP-57-002 `Staleness annotations` | TODO | Concelier Core Guild, AirGap Time Guild | CONCELIER-AIRGAP-56-002, AIRGAP-TIME-58-001 | Compute staleness metadata for advisories per bundle and expose via API for Console/CLI badges. | +| CONCELIER-AIRGAP-58-001 `Portable advisory evidence` | TODO | Concelier Core Guild, Evidence Locker Guild | CONCELIER-OBS-53-001, EVID-OBS-54-001 | Package advisory evidence fragments into portable evidence bundles for cross-domain transfer. | + +## SDKs & OpenAPI (Epic 17) +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| CONCELIER-OAS-61-001 `Spec coverage` | TODO | Concelier Core Guild, API Contracts Guild | OAS-61-001 | Update Concelier OAS with advisory observation/linkset endpoints, standard pagination, and source provenance fields. | +| CONCELIER-OAS-61-002 `Examples library` | TODO | Concelier Core Guild | CONCELIER-OAS-61-001 | Provide rich examples for advisories, linksets, conflict annotations used by SDK + docs. | +| CONCELIER-OAS-62-001 `SDK smoke tests` | TODO | Concelier Core Guild, SDK Generator Guild | CONCELIER-OAS-61-001, SDKGEN-63-001 | Add SDK tests covering advisory search, pagination, and conflict handling; ensure source metadata surfaced. | +| CONCELIER-OAS-63-001 `Deprecation headers` | TODO | Concelier Core Guild, API Governance Guild | APIGOV-63-001 | Implement deprecation header support and timeline events for retiring endpoints. | + +## Risk Profiles (Epic 18) +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| CONCELIER-RISK-66-001 `CVSS/KEV providers` | TODO | Concelier Core Guild, Risk Engine Guild | RISK-ENGINE-67-001 | Expose CVSS, KEV, fix availability data via provider APIs with source metadata preserved. | +| CONCELIER-RISK-66-002 `Fix availability signals` | TODO | Concelier Core Guild | CONCELIER-RISK-66-001 | Provide structured fix availability and release metadata consumable by risk engine; document provenance. | +| CONCELIER-RISK-67-001 `Source consensus metrics` | TODO | Concelier Core Guild | CONCELIER-RISK-66-001 | Add consensus counts and confidence scores for linked advisories; ensure explainability includes source digests. | +| CONCELIER-RISK-68-001 `Policy Studio integration` | TODO | Concelier Core Guild, Policy Studio Guild | POLICY-RISK-68-001 | Surface advisory fields in Policy Studio profile editor (signal pickers, reducers). | +| CONCELIER-RISK-69-001 `Notification hooks` | TODO | Concelier Core Guild, Notifications Guild | CONCELIER-RISK-66-002 | Emit events when advisory signals change impacting risk scores (e.g., fix available). | + +## Attestor Console (Epic 19) +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| CONCELIER-ATTEST-73-001 `ScanResults attestation inputs` | TODO | Concelier Core Guild, Attestor Service Guild | ATTEST-TYPES-72-001 | Provide normalized advisory data and linkset digests needed for ScanResults attestations. | +| CONCELIER-ATTEST-73-002 `Transparency metadata` | TODO | Concelier Core Guild | CONCELIER-ATTEST-73-001 | Ensure Conseiller exposes source digests for transparency proofs and explainability. | diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Exporter.Json/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Exporter.Json/TASKS.completed.md new file mode 100644 index 00000000..89b6b57d --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Exporter.Json/TASKS.completed.md @@ -0,0 +1,22 @@ +# Completed Tasks + +|Directory layout strategy (vuln-list mirror)|BE-Export|Models|DONE – `VulnListJsonExportPathResolver` maps CVE, GHSA, distro, and vendor identifiers into vuln-list style paths.| + +|Deterministic serializer|BE-Export|Models|DONE – Canonical serializer + snapshot builder emit stable JSON across runs.| + +|ExportState read/write|BE-Export|Storage.Mongo|DONE – `JsonFeedExporter` reads prior state, stores digests/cursors, and skips unchanged exports.| + +|JsonExportJob wiring|BE-Export|Core|DONE – Job scheduler options now configurable via DI; JSON job registered with scheduler.| + +|Snapshot tests for file tree|QA|Exporters|DONE – Added resolver/exporter tests asserting tree layout and deterministic behavior.| + +|Parity smoke vs upstream vuln-list|QA|Exporters|DONE – `JsonExporterParitySmokeTests` covers common ecosystems against vuln-list layout.| + +|Stream advisories during export|BE-Export|Storage.Mongo|DONE – exporter + streaming-only test ensures single enumeration and per-file digest capture.| + +|Emit export manifest with digest metadata|BE-Export|Exporters|DONE – manifest now includes per-file digests/sizes alongside tree digest.| + +|Surface new advisory fields (description/CWEs/canonical metric)|BE-Export|Models, Core|DONE (2025-10-15) – JSON exporter validated with new fixtures ensuring description/CWEs/canonical metric are preserved in outputs; `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.Json.Tests` run 2025-10-15 for regression coverage.| + +|CONCELIER-EXPORT-08-201 – Mirror bundle + domain manifest|Team Concelier Export|FEEDCORE-ENGINE-07-001|DONE (2025-10-19) – Mirror bundle writer emits domain aggregates + manifests with cosign-compatible JWS signatures; index/tests updated via `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.Json.Tests/StellaOps.Concelier.Exporter.Json.Tests.csproj` (2025-10-19).| + diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Exporter.Json/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Exporter.Json/TASKS.md index f0417dca..2265b2fc 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Exporter.Json/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Exporter.Json/TASKS.md @@ -1,13 +1,3 @@ # TASKS | Task | Owner(s) | Depends on | Notes | |---|---|---|---| -|Directory layout strategy (vuln-list mirror)|BE-Export|Models|DONE – `VulnListJsonExportPathResolver` maps CVE, GHSA, distro, and vendor identifiers into vuln-list style paths.| -|Deterministic serializer|BE-Export|Models|DONE – Canonical serializer + snapshot builder emit stable JSON across runs.| -|ExportState read/write|BE-Export|Storage.Mongo|DONE – `JsonFeedExporter` reads prior state, stores digests/cursors, and skips unchanged exports.| -|JsonExportJob wiring|BE-Export|Core|DONE – Job scheduler options now configurable via DI; JSON job registered with scheduler.| -|Snapshot tests for file tree|QA|Exporters|DONE – Added resolver/exporter tests asserting tree layout and deterministic behavior.| -|Parity smoke vs upstream vuln-list|QA|Exporters|DONE – `JsonExporterParitySmokeTests` covers common ecosystems against vuln-list layout.| -|Stream advisories during export|BE-Export|Storage.Mongo|DONE – exporter + streaming-only test ensures single enumeration and per-file digest capture.| -|Emit export manifest with digest metadata|BE-Export|Exporters|DONE – manifest now includes per-file digests/sizes alongside tree digest.| -|Surface new advisory fields (description/CWEs/canonical metric)|BE-Export|Models, Core|DONE (2025-10-15) – JSON exporter validated with new fixtures ensuring description/CWEs/canonical metric are preserved in outputs; `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.Json.Tests` run 2025-10-15 for regression coverage.| -|CONCELIER-EXPORT-08-201 – Mirror bundle + domain manifest|Team Concelier Export|FEEDCORE-ENGINE-07-001|DONE (2025-10-19) – Mirror bundle writer emits domain aggregates + manifests with cosign-compatible JWS signatures; index/tests updated via `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.Json.Tests/StellaOps.Concelier.Exporter.Json.Tests.csproj` (2025-10-19).| diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Exporter.TrivyDb/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Exporter.TrivyDb/TASKS.completed.md new file mode 100644 index 00000000..3580b6c1 --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Exporter.TrivyDb/TASKS.completed.md @@ -0,0 +1,26 @@ +# Completed Tasks + +|Fix method name typo GetExportRoot' -> GetExportRoot|BE-Export|Exporters|DONE – `TrivyDbExportOptions.GetExportRoot` helper added with unit coverage.| + +|Implement BoltDB builder integration (v0 via trivy-db CLI)|BE-Export|Env|DONE – `TrivyDbBoltBuilder` shells `trivy-db build` against our JSON tree with deterministic packaging.| + +|Pack db.tar.gz + metadata.json|BE-Export|Exporters|DONE – Builder output re-packed with fixed timestamps and zeroed gzip mtime.| + +|ORAS push support|BE-Export|Exporters|DONE – Optional `TrivyDbOrasPusher` shells `oras cp --from-oci-layout` with configurable args/env.| + +|Offline bundle toggle|BE-Export|Exporters|DONE – Deterministic OCI layout bundle emitted when enabled.| + +|Deterministic ordering of advisories|BE-Export|Models|DONE – exporter now loads advisories, sorts by advisoryKey, and emits sorted JSON trees with deterministic OCI payloads.| + +|End-to-end tests with small dataset|QA|Exporters|DONE – added deterministic round-trip test covering OCI layout, media types, and digest stability w/ repeated inputs.| + +|ExportState persistence & idempotence|BE-Export|Storage.Mongo|DONE – baseline resets wired into `ExportStateManager`, planner signals resets after delta runs, and exporters update state w/ repository-aware baseline rotation + tests.| + +|Streamed package building to avoid large copies|BE-Export|Exporters|DONE – metadata/config now reuse backing arrays and OCI writer streams directly without double buffering.| + +|Plan incremental/delta exports|BE-Export|Exporters|DONE – state captures per-file manifests, planner schedules delta vs full resets, layer reuse smoke test verifies OCI reuse, and operator guide documents the validation flow.| + +|Advisory schema parity export (description/CWEs/canonical metric)|BE-Export|Models, Core|DONE (2025-10-15) – exporter/test fixtures updated to handle description/CWEs/canonical metric fields during Trivy DB packaging; `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.TrivyDb.Tests` re-run 2025-10-15 to confirm coverage.| + +|CONCELIER-EXPORT-08-202 – Mirror-ready Trivy DB bundles|Team Concelier Export|CONCELIER-EXPORT-08-201|**DONE (2025-10-19)** – Added mirror export options and writer emitting `mirror/index.json` plus per-domain `manifest.json`/`metadata.json`/`db.tar.gz` with deterministic SHA-256 digests; regression covered via `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.TrivyDb.Tests/StellaOps.Concelier.Exporter.TrivyDb.Tests.csproj`.| + diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Exporter.TrivyDb/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Exporter.TrivyDb/TASKS.md index 6b343ed0..2265b2fc 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Exporter.TrivyDb/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Exporter.TrivyDb/TASKS.md @@ -1,15 +1,3 @@ # TASKS | Task | Owner(s) | Depends on | Notes | |---|---|---|---| -|Fix method name typo GetExportRoot' -> GetExportRoot|BE-Export|Exporters|DONE – `TrivyDbExportOptions.GetExportRoot` helper added with unit coverage.| -|Implement BoltDB builder integration (v0 via trivy-db CLI)|BE-Export|Env|DONE – `TrivyDbBoltBuilder` shells `trivy-db build` against our JSON tree with deterministic packaging.| -|Pack db.tar.gz + metadata.json|BE-Export|Exporters|DONE – Builder output re-packed with fixed timestamps and zeroed gzip mtime.| -|ORAS push support|BE-Export|Exporters|DONE – Optional `TrivyDbOrasPusher` shells `oras cp --from-oci-layout` with configurable args/env.| -|Offline bundle toggle|BE-Export|Exporters|DONE – Deterministic OCI layout bundle emitted when enabled.| -|Deterministic ordering of advisories|BE-Export|Models|DONE – exporter now loads advisories, sorts by advisoryKey, and emits sorted JSON trees with deterministic OCI payloads.| -|End-to-end tests with small dataset|QA|Exporters|DONE – added deterministic round-trip test covering OCI layout, media types, and digest stability w/ repeated inputs.| -|ExportState persistence & idempotence|BE-Export|Storage.Mongo|DONE – baseline resets wired into `ExportStateManager`, planner signals resets after delta runs, and exporters update state w/ repository-aware baseline rotation + tests.| -|Streamed package building to avoid large copies|BE-Export|Exporters|DONE – metadata/config now reuse backing arrays and OCI writer streams directly without double buffering.| -|Plan incremental/delta exports|BE-Export|Exporters|DONE – state captures per-file manifests, planner schedules delta vs full resets, layer reuse smoke test verifies OCI reuse, and operator guide documents the validation flow.| -|Advisory schema parity export (description/CWEs/canonical metric)|BE-Export|Models, Core|DONE (2025-10-15) – exporter/test fixtures updated to handle description/CWEs/canonical metric fields during Trivy DB packaging; `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.TrivyDb.Tests` re-run 2025-10-15 to confirm coverage.| -|CONCELIER-EXPORT-08-202 – Mirror-ready Trivy DB bundles|Team Concelier Export|CONCELIER-EXPORT-08-201|**DONE (2025-10-19)** – Added mirror export options and writer emitting `mirror/index.json` plus per-domain `manifest.json`/`metadata.json`/`db.tar.gz` with deterministic SHA-256 digests; regression covered via `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.TrivyDb.Tests/StellaOps.Concelier.Exporter.TrivyDb.Tests.csproj`.| diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.completed.md new file mode 100644 index 00000000..49cb2e78 --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.completed.md @@ -0,0 +1,37 @@ +# Completed Tasks + +|Identity graph and alias resolver|BE-Merge|Models, Storage.Mongo|DONE – `AdvisoryIdentityResolver` builds alias-driven clusters with canonical key selection + unit coverage.| + +|Precedence policy engine|BE-Merge|Architecture|**DONE** – precedence defaults enforced by `AdvisoryPrecedenceMerger`/`AdvisoryPrecedenceDefaults` with distro/PSIRT overriding registry feeds and CERT/KEV enrichers.| + +|NEVRA comparer plus tests|BE-Merge (Distro WG)|Source.Distro fixtures|DONE – Added Nevra parser/comparer with tilde-aware rpm ordering and unit coverage.| + +|Debian EVR comparer plus tests|BE-Merge (Distro WG)|Debian fixtures|DONE – DebianEvr comparer mirrors dpkg ordering with tilde/epoch handling and unit coverage.| + +|SemVer range resolver plus tests|BE-Merge (OSS WG)|OSV/GHSA fixtures|DONE – SemanticVersionRangeResolver covers introduced/fixed/lastAffected semantics with SemVer ordering tests.| + +|Canonical hash and merge_event writer|BE-Merge|Models, Storage.Mongo|DONE – Hash calculator + MergeEventWriter compute canonical SHA-256 digests and persist merge events.| + +|Conflict detection and metrics|BE-Merge|Core|**DONE** – merge meters emit override/conflict counters and structured audits (`AdvisoryPrecedenceMerger`).| + +|FEEDMERGE-ENGINE-04-001 GHSA/NVD/OSV conflict rules|BE-Merge|Core, Storage.Mongo|DONE – `AdvisoryMergeService` applies `CanonicalMerger` output before precedence merge, replacing source advisories with the canonical transcript. **Coordination:** connector fixture owners should surface canonical deltas to Merge QA before regression sign-off.| + +|FEEDMERGE-ENGINE-04-002 Override metrics instrumentation|BE-Merge|Observability|DONE – merge events persist `MergeFieldDecision` records enabling analytics on precedence/freshness decisions. **Next:** hand off metrics schema to Ops for dashboard wiring.| + +|FEEDMERGE-ENGINE-04-003 Reference & credit union pipeline|BE-Merge|Models|DONE – canonical merge preserves union semantics while respecting precedence, validated via updated credit union tests.| + +|End-to-end determinism test|QA|Merge, key connectors|**DONE** – `MergePrecedenceIntegrationTests.MergePipeline_IsDeterministicAcrossRuns` guards determinism.| + +|FEEDMERGE-QA-04-001 End-to-end conflict regression suite|QA|Merge|DONE – `AdvisoryMergeServiceTests.MergeAsync_AppliesCanonicalRulesAndPersistsDecisions` exercises GHSA/NVD/OSV conflict path and merge-event analytics. **Reminder:** QA to sync with connector teams once new fixture triples land.| + +|Override audit logging|BE-Merge|Observability|DONE – override audits now emit structured logs plus bounded-tag metrics suitable for prod telemetry.| + +|Configurable precedence table|BE-Merge|Architecture|DONE – precedence options bind via concelier:merge:precedence:ranks with docs/tests covering operator workflow.| + +|Merge pipeline parity for new advisory fields|BE-Merge|Models, Core|DONE (2025-10-15) – merge service now surfaces description/CWE/canonical metric decisions with updated metrics/tests.| + +|Connector coordination for new advisory fields|Connector Leads, BE-Merge|Models, Core|**DONE (2025-10-15)** – GHSA, NVD, and OSV connectors now emit advisory descriptions, CWE weaknesses, and canonical metric ids. Fixtures refreshed (GHSA connector regression suite, `conflict-nvd.canonical.json`, OSV parity snapshots) and completion recorded in coordination log.| + +|FEEDMERGE-ENGINE-07-001 Conflict sets & explainers|BE-Merge|FEEDSTORAGE-DATA-07-001|**DONE (2025-10-20)** – Merge surfaces conflict explainers with replay hashes via `MergeConflictSummary`; API exposes structured payloads and integration tests cover deterministic `asOf` hashes.| +> Remark (2025-10-20): `AdvisoryMergeService` now returns conflict summaries with deterministic hashes; WebService replay endpoint emits typed explainers verified by new tests. + diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md index 99fb9575..7228af77 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md @@ -1,33 +1,15 @@ -# TASKS -| Task | Owner(s) | Depends on | Notes | -|---|---|---|---| -|Identity graph and alias resolver|BE-Merge|Models, Storage.Mongo|DONE – `AdvisoryIdentityResolver` builds alias-driven clusters with canonical key selection + unit coverage.| -|Precedence policy engine|BE-Merge|Architecture|**DONE** – precedence defaults enforced by `AdvisoryPrecedenceMerger`/`AdvisoryPrecedenceDefaults` with distro/PSIRT overriding registry feeds and CERT/KEV enrichers.| -|NEVRA comparer plus tests|BE-Merge (Distro WG)|Source.Distro fixtures|DONE – Added Nevra parser/comparer with tilde-aware rpm ordering and unit coverage.| -|Debian EVR comparer plus tests|BE-Merge (Distro WG)|Debian fixtures|DONE – DebianEvr comparer mirrors dpkg ordering with tilde/epoch handling and unit coverage.| -|SemVer range resolver plus tests|BE-Merge (OSS WG)|OSV/GHSA fixtures|DONE – SemanticVersionRangeResolver covers introduced/fixed/lastAffected semantics with SemVer ordering tests.| -|Canonical hash and merge_event writer|BE-Merge|Models, Storage.Mongo|DONE – Hash calculator + MergeEventWriter compute canonical SHA-256 digests and persist merge events.| -|Conflict detection and metrics|BE-Merge|Core|**DONE** – merge meters emit override/conflict counters and structured audits (`AdvisoryPrecedenceMerger`).| -|FEEDMERGE-ENGINE-04-001 GHSA/NVD/OSV conflict rules|BE-Merge|Core, Storage.Mongo|DONE – `AdvisoryMergeService` applies `CanonicalMerger` output before precedence merge, replacing source advisories with the canonical transcript. **Coordination:** connector fixture owners should surface canonical deltas to Merge QA before regression sign-off.| -|FEEDMERGE-ENGINE-04-002 Override metrics instrumentation|BE-Merge|Observability|DONE – merge events persist `MergeFieldDecision` records enabling analytics on precedence/freshness decisions. **Next:** hand off metrics schema to Ops for dashboard wiring.| -|FEEDMERGE-ENGINE-04-003 Reference & credit union pipeline|BE-Merge|Models|DONE – canonical merge preserves union semantics while respecting precedence, validated via updated credit union tests.| -|End-to-end determinism test|QA|Merge, key connectors|**DONE** – `MergePrecedenceIntegrationTests.MergePipeline_IsDeterministicAcrossRuns` guards determinism.| -|FEEDMERGE-QA-04-001 End-to-end conflict regression suite|QA|Merge|DONE – `AdvisoryMergeServiceTests.MergeAsync_AppliesCanonicalRulesAndPersistsDecisions` exercises GHSA/NVD/OSV conflict path and merge-event analytics. **Reminder:** QA to sync with connector teams once new fixture triples land.| -|Override audit logging|BE-Merge|Observability|DONE – override audits now emit structured logs plus bounded-tag metrics suitable for prod telemetry.| -|Configurable precedence table|BE-Merge|Architecture|DONE – precedence options bind via concelier:merge:precedence:ranks with docs/tests covering operator workflow.| +# TASKS +| Task | Owner(s) | Depends on | Notes | +|---|---|---|---| |Range primitives backlog|BE-Merge|Connector WGs|**DOING** – Coordinate remaining connectors (`Acsc`, `Cccs`, `CertBund`, `CertCc`, `Cve`, `Ghsa`, `Ics.Cisa`, `Kisa`, `Ru.Bdu`, `Ru.Nkcki`, `Vndr.Apple`, `Vndr.Cisco`, `Vndr.Msrc`) to emit canonical RangePrimitives with provenance tags; track progress/fixtures here.
2025-10-11: Storage alignment notes + sample normalized rule JSON now captured in `RANGE_PRIMITIVES_COORDINATION.md` (see “Storage alignment quick reference”).
2025-10-11 18:45Z: GHSA normalized rules landed; OSV connector picked up next for rollout.
2025-10-11 21:10Z: `docs/dev/merge_semver_playbook.md` Section 8 now documents the persisted Mongo projection (SemVer + NEVRA) for connector reviewers.
2025-10-11 21:30Z: Added `docs/dev/normalized_versions_rollout.md` dashboard to centralize connector status and upcoming milestones.
2025-10-11 21:55Z: Merge now emits `concelier.merge.normalized_rules*` counters and unions connector-provided normalized arrays; see new test coverage in `AdvisoryPrecedenceMergerTests.Merge_RecordsNormalizedRuleMetrics`.
2025-10-12 17:05Z: CVE + KEV normalized rule verification complete; OSV parity fixtures revalidated—downstream parity/monitoring tasks may proceed.
2025-10-19 14:35Z: Prerequisites reviewed (none outstanding); FEEDMERGE-COORD-02-900 remains in DOING with connector follow-ups unchanged.
2025-10-19 15:25Z: Refreshed `RANGE_PRIMITIVES_COORDINATION.md` matrix + added targeted follow-ups (Cccs, CertBund, ICS-CISA, Kisa, Vndr.Cisco) with delivery dates 2025-10-21 → 2025-10-25; monitoring merge counters for regression.
2025-10-29: Added merge-time warnings highlighting sources/package types when ranges emit without normalized rules to accelerate backlog triage.| -|Range primitives backlog|BE-Merge|Connector WGs|**DOING** – Coordinate remaining connectors (`Acsc`, `Cccs`, `CertBund`, `CertCc`, `Cve`, `Ghsa`, `Ics.Cisa`, `Kisa`, `Ru.Bdu`, `Ru.Nkcki`, `Vndr.Apple`, `Vndr.Cisco`, `Vndr.Msrc`) to emit canonical RangePrimitives with provenance tags; track progress/fixtures here.
2025-10-11: Storage alignment notes + sample normalized rule JSON now captured in `RANGE_PRIMITIVES_COORDINATION.md` (see “Storage alignment quick reference”).
2025-10-11 18:45Z: GHSA normalized rules landed; OSV connector picked up next for rollout.
2025-10-11 21:10Z: `docs/dev/merge_semver_playbook.md` Section 8 now documents the persisted Mongo projection (SemVer + NEVRA) for connector reviewers.
2025-10-11 21:30Z: Added `docs/dev/normalized_versions_rollout.md` dashboard to centralize connector status and upcoming milestones.
2025-10-11 21:55Z: Merge now emits `concelier.merge.normalized_rules*` counters and unions connector-provided normalized arrays; see new test coverage in `AdvisoryPrecedenceMergerTests.Merge_RecordsNormalizedRuleMetrics`.
2025-10-12 17:05Z: CVE + KEV normalized rule verification complete; OSV parity fixtures revalidated—downstream parity/monitoring tasks may proceed.
2025-10-19 14:35Z: Prerequisites reviewed (none outstanding); FEEDMERGE-COORD-02-900 remains in DOING with connector follow-ups unchanged.
2025-10-19 15:25Z: Refreshed `RANGE_PRIMITIVES_COORDINATION.md` matrix + added targeted follow-ups (Cccs, CertBund, ICS-CISA, Kisa, Vndr.Cisco) with delivery dates 2025-10-21 → 2025-10-25; monitoring merge counters for regression.
2025-10-20 19:30Z: Coordination matrix + rollout dashboard updated with current connector statuses and due dates; flagged Slack escalation plan if Cccs/Cisco miss 2025-10-21 and documented Acsc kickoff window for 2025-10-24.| -|Merge pipeline parity for new advisory fields|BE-Merge|Models, Core|DONE (2025-10-15) – merge service now surfaces description/CWE/canonical metric decisions with updated metrics/tests.| -|Connector coordination for new advisory fields|Connector Leads, BE-Merge|Models, Core|**DONE (2025-10-15)** – GHSA, NVD, and OSV connectors now emit advisory descriptions, CWE weaknesses, and canonical metric ids. Fixtures refreshed (GHSA connector regression suite, `conflict-nvd.canonical.json`, OSV parity snapshots) and completion recorded in coordination log.| -|FEEDMERGE-ENGINE-07-001 Conflict sets & explainers|BE-Merge|FEEDSTORAGE-DATA-07-001|**DONE (2025-10-20)** – Merge surfaces conflict explainers with replay hashes via `MergeConflictSummary`; API exposes structured payloads and integration tests cover deterministic `asOf` hashes.| -> Remark (2025-10-20): `AdvisoryMergeService` now returns conflict summaries with deterministic hashes; WebService replay endpoint emits typed explainers verified by new tests. +|Range primitives backlog|BE-Merge|Connector WGs|**DOING** – Coordinate remaining connectors (`Acsc`, `Cccs`, `CertBund`, `CertCc`, `Cve`, `Ghsa`, `Ics.Cisa`, `Kisa`, `Ru.Bdu`, `Ru.Nkcki`, `Vndr.Apple`, `Vndr.Cisco`, `Vndr.Msrc`) to emit canonical RangePrimitives with provenance tags; track progress/fixtures here.
2025-10-11: Storage alignment notes + sample normalized rule JSON now captured in `RANGE_PRIMITIVES_COORDINATION.md` (see “Storage alignment quick reference”).
2025-10-11 18:45Z: GHSA normalized rules landed; OSV connector picked up next for rollout.
2025-10-11 21:10Z: `docs/dev/merge_semver_playbook.md` Section 8 now documents the persisted Mongo projection (SemVer + NEVRA) for connector reviewers.
2025-10-11 21:30Z: Added `docs/dev/normalized_versions_rollout.md` dashboard to centralize connector status and upcoming milestones.
2025-10-11 21:55Z: Merge now emits `concelier.merge.normalized_rules*` counters and unions connector-provided normalized arrays; see new test coverage in `AdvisoryPrecedenceMergerTests.Merge_RecordsNormalizedRuleMetrics`.
2025-10-12 17:05Z: CVE + KEV normalized rule verification complete; OSV parity fixtures revalidated—downstream parity/monitoring tasks may proceed.
2025-10-19 14:35Z: Prerequisites reviewed (none outstanding); FEEDMERGE-COORD-02-900 remains in DOING with connector follow-ups unchanged.
2025-10-19 15:25Z: Refreshed `RANGE_PRIMITIVES_COORDINATION.md` matrix + added targeted follow-ups (Cccs, CertBund, ICS-CISA, Kisa, Vndr.Cisco) with delivery dates 2025-10-21 → 2025-10-25; monitoring merge counters for regression.
2025-10-20 19:30Z: Coordination matrix + rollout dashboard updated with current connector statuses and due dates; flagged Slack escalation plan if Cccs/Cisco miss 2025-10-21 and documented Acsc kickoff window for 2025-10-24.| |FEEDMERGE-COORD-02-901 Connector deadline check-ins|BE-Merge|FEEDMERGE-COORD-02-900|**TODO (due 2025-10-21)** – Confirm Cccs/Cisco normalized-rule branches land, capture `concelier.merge.normalized_rules*` counter screenshots, and update coordination docs with the results.
2025-10-29: Merge now emits `Normalized version rules missing...` warnings (see `docs/dev/normalized-rule-recipes.md` §4); include zero-warning excerpt plus Grafana counter snapshot when closing this task.| |FEEDMERGE-COORD-02-902 ICS-CISA normalized-rule decision support|BE-Merge, Models|FEEDMERGE-COORD-02-900|**TODO (due 2025-10-23)** – Review ICS-CISA sample advisories, confirm SemVer reuse vs new firmware scheme, pre-stage Models ticket template, and document outcome in coordination docs + tracker files.
2025-10-29: Recipes doc (§2–§3) outlines SemVer promotion + fallback logging—attach decision summary + log sample when handing off to Models.| -|FEEDMERGE-COORD-02-903 KISA firmware scheme review|BE-Merge, Models|FEEDMERGE-COORD-02-900|**TODO (due 2025-10-24)** – Pair with KISA team on proposed firmware scheme (`kisa.build` or variant), ensure builder alignment, open Models ticket if required, and log decision in coordination docs + tracker files.| - -## Link-Not-Merge v1 Transition -| Task | Owner(s) | Depends on | Notes | -|---|---|---|---| -|MERGE-LNM-21-001 Migration plan authoring|BE-Merge, Architecture Guild|CONCELIER-LNM-21-101|Draft `no-merge` migration playbook, documenting backfill strategy, feature flag rollout, and rollback steps for legacy merge pipeline deprecation.| -|MERGE-LNM-21-002 Merge service deprecation|BE-Merge|MERGE-LNM-21-001|Refactor or retire `AdvisoryMergeService` and related pipelines, ensuring callers transition to observation/linkset APIs; add compile-time analyzer preventing merge service usage.| -|MERGE-LNM-21-003 Determinism/test updates|QA Guild, BE-Merge|MERGE-LNM-21-002|Replace merge determinism suites with observation/linkset regression tests verifying no data mutation and conflicts remain visible.| +|FEEDMERGE-COORD-02-903 KISA firmware scheme review|BE-Merge, Models|FEEDMERGE-COORD-02-900|**TODO (due 2025-10-24)** – Pair with KISA team on proposed firmware scheme (`kisa.build` or variant), ensure builder alignment, open Models ticket if required, and log decision in coordination docs + tracker files.| + +## Link-Not-Merge v1 Transition +| Task | Owner(s) | Depends on | Notes | +|---|---|---|---| +|MERGE-LNM-21-001 Migration plan authoring|BE-Merge, Architecture Guild|CONCELIER-LNM-21-101|Draft `no-merge` migration playbook, documenting backfill strategy, feature flag rollout, and rollback steps for legacy merge pipeline deprecation.| +|MERGE-LNM-21-002 Merge service deprecation|BE-Merge|MERGE-LNM-21-001|Refactor or retire `AdvisoryMergeService` and related pipelines, ensuring callers transition to observation/linkset APIs; add compile-time analyzer preventing merge service usage.| +|MERGE-LNM-21-003 Determinism/test updates|QA Guild, BE-Merge|MERGE-LNM-21-002|Replace merge determinism suites with observation/linkset regression tests verifying no data mutation and conflicts remain visible.| diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Models/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Models/TASKS.completed.md new file mode 100644 index 00000000..f061e4cd --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Models/TASKS.completed.md @@ -0,0 +1,34 @@ +# Completed Tasks + +|Canonical JSON serializer with stable ordering|BE-Merge|Models|DONE – `CanonicalJsonSerializer` ensures deterministic property ordering.| + +|Equality/comparison helpers for ranges|BE-Merge|Models|DONE – added `AffectedVersionRangeComparer` & equality comparer.| + +|Type enums/constants for AffectedPackage.Type|BE-Merge|Models|DONE – introduced `AffectedPackageTypes`.| + +|Validation helpers (lightweight)|BE-Merge|Models|DONE – added `Validation` static helpers and URL guard.| + +|Snapshot serializer for tests|QA|Models|DONE – `SnapshotSerializer` emits canonical JSON.| + +|Docs: field provenance guidelines|BE-Merge|Models|DONE – see `PROVENANCE_GUIDELINES.md`.| + +|Canonical record definitions kept in sync|BE-Merge|Models|DONE – documented in `CANONICAL_RECORDS.md`; update alongside model changes.| + +|Alias scheme registry and validation helpers|BE-Merge|Models|DONE – see `AliasSchemes` & `AliasSchemeRegistry` plus validation integration/tests.| + +|Range primitives for SemVer/EVR/NEVRA metadata|BE-Merge|Models|DONE – SemVer/Evr/Nevra primitives now project canonical normalized rules; range helpers emit fallback rules for legacy inputs and tests cover canonical string generation so connectors can populate `normalizedVersions` deterministically.| + +|Provenance envelope field masks|BE-Merge|Models|DONE – `AdvisoryProvenance.fieldMask` added with diagnostics/tests/docs refreshed; connectors can now emit canonical masks for QA dashboards.| + +|Backward-compatibility playbook|BE-Merge, QA|Models|DONE – see `BACKWARD_COMPATIBILITY.md` for evolution policy/test checklist.| + +|Golden canonical examples|QA|Models|DONE – added `/p:UpdateGoldens=true` test hook wiring `UPDATE_GOLDENS=1` so canonical fixtures regenerate via `dotnet test`; docs/tests unchanged.| + +|Serialization determinism regression tests|QA|Models|DONE – locale-stability tests hash canonical serializer output across multiple cultures and runs.| + +|Severity normalization helpers|BE-Merge|Models|DONE – helper now normalizes compound vendor labels/priority tiers with expanded synonym coverage and regression tests.| + +|AffectedPackage status glossary & guardrails|BE-Merge|Models|DONE – catalog now exposes deterministic listing, TryNormalize helpers, and synonym coverage for vendor phrases (not vulnerable, workaround available, etc.).| + +|Advisory schema parity (description, CWE collection, canonical metric id)|BE-Merge, BE-Core|Core, Exporters|DONE (2025-10-15) – extended `Advisory`/related records with description/CWEs/canonical metric id plus serializer/tests updated; exporters validated via new coverage.| + diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Models/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Models/TASKS.md index 96325245..2265b2fc 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Models/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Models/TASKS.md @@ -1,19 +1,3 @@ -# TASKS -| Task | Owner(s) | Depends on | Notes | -|---|---|---|---| -|Canonical JSON serializer with stable ordering|BE-Merge|Models|DONE – `CanonicalJsonSerializer` ensures deterministic property ordering.| -|Equality/comparison helpers for ranges|BE-Merge|Models|DONE – added `AffectedVersionRangeComparer` & equality comparer.| -|Type enums/constants for AffectedPackage.Type|BE-Merge|Models|DONE – introduced `AffectedPackageTypes`.| -|Validation helpers (lightweight)|BE-Merge|Models|DONE – added `Validation` static helpers and URL guard.| -|Snapshot serializer for tests|QA|Models|DONE – `SnapshotSerializer` emits canonical JSON.| -|Docs: field provenance guidelines|BE-Merge|Models|DONE – see `PROVENANCE_GUIDELINES.md`.| -|Canonical record definitions kept in sync|BE-Merge|Models|DONE – documented in `CANONICAL_RECORDS.md`; update alongside model changes.| -|Alias scheme registry and validation helpers|BE-Merge|Models|DONE – see `AliasSchemes` & `AliasSchemeRegistry` plus validation integration/tests.| -|Range primitives for SemVer/EVR/NEVRA metadata|BE-Merge|Models|DONE – SemVer/Evr/Nevra primitives now project canonical normalized rules; range helpers emit fallback rules for legacy inputs and tests cover canonical string generation so connectors can populate `normalizedVersions` deterministically.| -|Provenance envelope field masks|BE-Merge|Models|DONE – `AdvisoryProvenance.fieldMask` added with diagnostics/tests/docs refreshed; connectors can now emit canonical masks for QA dashboards.| -|Backward-compatibility playbook|BE-Merge, QA|Models|DONE – see `BACKWARD_COMPATIBILITY.md` for evolution policy/test checklist.| -|Golden canonical examples|QA|Models|DONE – added `/p:UpdateGoldens=true` test hook wiring `UPDATE_GOLDENS=1` so canonical fixtures regenerate via `dotnet test`; docs/tests unchanged.| -|Serialization determinism regression tests|QA|Models|DONE – locale-stability tests hash canonical serializer output across multiple cultures and runs.| -|Severity normalization helpers|BE-Merge|Models|DONE – helper now normalizes compound vendor labels/priority tiers with expanded synonym coverage and regression tests.| -|AffectedPackage status glossary & guardrails|BE-Merge|Models|DONE – catalog now exposes deterministic listing, TryNormalize helpers, and synonym coverage for vendor phrases (not vulnerable, workaround available, etc.).| -|Advisory schema parity (description, CWE collection, canonical metric id)|BE-Merge, BE-Core|Core, Exporters|DONE (2025-10-15) – extended `Advisory`/related records with description/CWEs/canonical metric id plus serializer/tests updated; exporters validated via new coverage.| +# TASKS +| Task | Owner(s) | Depends on | Notes | +|---|---|---|---| diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Normalization/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Normalization/TASKS.completed.md new file mode 100644 index 00000000..168482ef --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Normalization/TASKS.completed.md @@ -0,0 +1,16 @@ +# Completed Tasks + +|Canonical NEVRA/EVR parsing helpers|BE-Norm (Distro WG)|Models|DONE – `Normalization.Distro` exposes parsers + canonical formatters consumed by Merge comparers/tests.| + +|PURL/CPE identifier normalization|BE-Norm (OSS WG)|Models|DONE – canonical PURL/CPE helpers feed connectors and exporter tooling.| + +|CPE normalization escape handling|BE-Norm (OSS WG)|Normalization identifiers|DONE – percent-decoding, edition sub-field expansion, and deterministic escaping landed in `Cpe23` with new tests covering boundary cases.| + +|CVSS metric normalization & severity bands|BE-Norm (Risk WG)|Models|DONE – `CvssMetricNormalizer` unifies vectors, recomputes scores/severities, and is wired through NVD/RedHat/JVN mappers with unit coverage.| + +|Description and locale normalization pipeline|BE-Norm (I18N)|Source connectors|DONE – `DescriptionNormalizer` strips markup, collapses whitespace, and provides locale fallback used by core mappers.| + +|SemVer normalized rule emitter (FEEDNORM-NORM-02-001)|BE-Norm (SemVer WG)|Models, `FASTER_MODELING_AND_NORMALIZATION.md`|**DONE (2025-10-12)** – `SemVerRangeRuleBuilder` now parses comparator chains without comma delimiters, supports multi-segment `||` ranges, pushes exact-value metadata, and new tests document the contract for connector teams.| + +|SemVer normalized rule convenience API|BE-Norm (SemVer WG)|SemVer normalized rule emitter|**DONE (2025-10-15)** – added `SemVerRangeRuleBuilder.BuildNormalizedRules` projection helper and unit coverage for empty/standard ranges so callers can access normalized rules without materializing primitives.| + diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Normalization/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Normalization/TASKS.md index cb3b00e7..2265b2fc 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Normalization/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Normalization/TASKS.md @@ -1,10 +1,3 @@ -# TASKS -| Task | Owner(s) | Depends on | Notes | -|---|---|---|---| -|Canonical NEVRA/EVR parsing helpers|BE-Norm (Distro WG)|Models|DONE – `Normalization.Distro` exposes parsers + canonical formatters consumed by Merge comparers/tests.| -|PURL/CPE identifier normalization|BE-Norm (OSS WG)|Models|DONE – canonical PURL/CPE helpers feed connectors and exporter tooling.| -|CPE normalization escape handling|BE-Norm (OSS WG)|Normalization identifiers|DONE – percent-decoding, edition sub-field expansion, and deterministic escaping landed in `Cpe23` with new tests covering boundary cases.| -|CVSS metric normalization & severity bands|BE-Norm (Risk WG)|Models|DONE – `CvssMetricNormalizer` unifies vectors, recomputes scores/severities, and is wired through NVD/RedHat/JVN mappers with unit coverage.| -|Description and locale normalization pipeline|BE-Norm (I18N)|Source connectors|DONE – `DescriptionNormalizer` strips markup, collapses whitespace, and provides locale fallback used by core mappers.| -|SemVer normalized rule emitter (FEEDNORM-NORM-02-001)|BE-Norm (SemVer WG)|Models, `FASTER_MODELING_AND_NORMALIZATION.md`|**DONE (2025-10-12)** – `SemVerRangeRuleBuilder` now parses comparator chains without comma delimiters, supports multi-segment `||` ranges, pushes exact-value metadata, and new tests document the contract for connector teams.| -|SemVer normalized rule convenience API|BE-Norm (SemVer WG)|SemVer normalized rule emitter|**DONE (2025-10-15)** – added `SemVerRangeRuleBuilder.BuildNormalizedRules` projection helper and unit coverage for empty/standard ranges so callers can access normalized rules without materializing primitives.| +# TASKS +| Task | Owner(s) | Depends on | Notes | +|---|---|---|---| diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.completed.md b/src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.completed.md new file mode 100644 index 00000000..ecc62888 --- /dev/null +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.completed.md @@ -0,0 +1,8 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Notes | +|---|---|---|---|---| +| CONCELIER-STORE-AOC-19-001 `advisory_raw schema validator` | DONE (2025-10-28) | Concelier Storage Guild | Mongo cluster ops sign-off | Author MongoDB JSON schema enforcing required fields (`source`, `upstream`, `content`, `linkset`, `tenant`) and forbidding normalized/severity fields. Include migration toggles for staged rollout. | +| CONCELIER-STORE-AOC-19-002 `idempotency unique index` | DONE (2025-10-28) | Concelier Storage Guild | CONCELIER-STORE-AOC-19-001 | Create compound unique index on `(source.vendor, upstream.upstream_id, upstream.content_hash, tenant)` with backfill script verifying existing data, and document offline validator bootstrap. | +| CONCELIER-STORE-AOC-19-003 `append-only supersedes migration` | DONE (2025-10-28) | Concelier Storage Guild | CONCELIER-STORE-AOC-19-002 | Introduce migration that freezes legacy `advisories` writes, copies data into `_backup_*`, and backfills supersedes pointers for raw revisions. Provide rollback plan. | +| CONCELIER-STORE-AOC-19-004 `validator deployment playbook` | DONE (2025-10-28) | Concelier Storage Guild, DevOps Guild | CONCELIER-STORE-AOC-19-001 | Update `MIGRATIONS.md` and Offline Kit docs to cover enabling validators, rolling restarts, and validator smoke tests for air-gapped installs. | diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md index c07389df..4ed3c0a0 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md @@ -1,30 +1,26 @@ -# TASKS — Epic 1: Aggregation-Only Contract -> **AOC Reminder:** storage enforces append-only raw documents; no precedence/severity/normalization in ingestion collections. -| ID | Status | Owner(s) | Depends on | Notes | -|---|---|---|---|---| -| CONCELIER-STORE-AOC-19-001 `advisory_raw schema validator` | DONE (2025-10-28) | Concelier Storage Guild | Mongo cluster ops sign-off | Author MongoDB JSON schema enforcing required fields (`source`, `upstream`, `content`, `linkset`, `tenant`) and forbidding normalized/severity fields. Include migration toggles for staged rollout. | -> 2025-10-28: Added configurable validator migration (`20251028_advisory_raw_validator`), bootstrapper collection registration, storage options toggle, and Mongo migration tests covering schema + enforcement levels. -> Docs alignment (2025-10-26): Validator expectations + deployment steps documented in `docs/deploy/containers.md` §1. -| CONCELIER-STORE-AOC-19-002 `idempotency unique index` | DONE (2025-10-28) | Concelier Storage Guild | CONCELIER-STORE-AOC-19-001 | Create compound unique index on `(source.vendor, upstream.upstream_id, upstream.content_hash, tenant)` with backfill script verifying existing data, and document offline validator bootstrap. | -> 2025-10-28: Added `20251028_advisory_raw_idempotency_index` migration that detects duplicate raw advisories before creating the unique compound index, wired into DI, and extended migration tests to cover index shape + duplicate handling with supporting package updates. -> Docs alignment (2025-10-26): Idempotency contract + supersedes metrics in `docs/ingestion/aggregation-only-contract.md` §7 and observability guide. -| CONCELIER-STORE-AOC-19-003 `append-only supersedes migration` | DONE (2025-10-28) | Concelier Storage Guild | CONCELIER-STORE-AOC-19-002 | Introduce migration that freezes legacy `advisories` writes, copies data into `_backup_*`, and backfills supersedes pointers for raw revisions. Provide rollback plan. | -> 2025-10-28: Added supersedes backfill migration (`20251028_advisory_supersedes_backfill`) that renames `advisory` to a read-only view, snapshots data into `_backup_20251028`, and walks raw revisions to populate deterministic supersedes chains with integration coverage and operator scripts. -> Docs alignment (2025-10-26): Rollback guidance added to `docs/deploy/containers.md` §6. -| CONCELIER-STORE-AOC-19-004 `validator deployment playbook` | DONE (2025-10-28) | Concelier Storage Guild, DevOps Guild | CONCELIER-STORE-AOC-19-001 | Update `MIGRATIONS.md` and Offline Kit docs to cover enabling validators, rolling restarts, and validator smoke tests for air-gapped installs. | -> 2025-10-28: Documented duplicate audit + migration workflow in `docs/deploy/containers.md`, Offline Kit guide, and `MIGRATIONS.md`; published `ops/devops/scripts/check-advisory-raw-duplicates.js` for staging/offline clusters. -> Docs alignment (2025-10-26): Offline kit requirements documented in `docs/deploy/containers.md` §5. - -## Policy Engine v2 - -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| CONCELIER-POLICY-20-003 `Selection cursors` | TODO | Concelier Storage Guild | CONCELIER-STORE-AOC-19-002, POLICY-ENGINE-20-003 | Add advisory/vex selection cursors (per policy run) with change stream checkpoints, indexes, and offline migration scripts to support incremental evaluations. | - -## Link-Not-Merge v1 - -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| CONCELIER-LNM-21-101 `Observations collections` | TODO | Concelier Storage Guild | CONCELIER-LNM-21-001 | Provision `advisory_observations` and `advisory_linksets` collections with hashed shard keys, TTL for ingest metadata, and required indexes (`aliases`, `purls`, `observation_ids`). | -| CONCELIER-LNM-21-102 `Migration tooling` | TODO | Concelier Storage Guild, DevOps Guild | CONCELIER-LNM-21-101 | Backfill legacy merged advisories into observation/linkset collections, create tombstones for merged docs, and supply rollback scripts. | -| CONCELIER-LNM-21-103 `Blob/store wiring` | TODO | Concelier Storage Guild | CONCELIER-LNM-21-101 | Store large raw payloads in object storage with pointers from observations; update bootstrapper/offline kit to seed sample blobs. | +# TASKS — Epic 1: Aggregation-Only Contract +> **AOC Reminder:** storage enforces append-only raw documents; no precedence/severity/normalization in ingestion collections. +| ID | Status | Owner(s) | Depends on | Notes | +|---|---|---|---|---| +> 2025-10-28: Added configurable validator migration (`20251028_advisory_raw_validator`), bootstrapper collection registration, storage options toggle, and Mongo migration tests covering schema + enforcement levels. +> Docs alignment (2025-10-26): Validator expectations + deployment steps documented in `docs/deploy/containers.md` §1. +> 2025-10-28: Added `20251028_advisory_raw_idempotency_index` migration that detects duplicate raw advisories before creating the unique compound index, wired into DI, and extended migration tests to cover index shape + duplicate handling with supporting package updates. +> Docs alignment (2025-10-26): Idempotency contract + supersedes metrics in `docs/ingestion/aggregation-only-contract.md` §7 and observability guide. +> 2025-10-28: Added supersedes backfill migration (`20251028_advisory_supersedes_backfill`) that renames `advisory` to a read-only view, snapshots data into `_backup_20251028`, and walks raw revisions to populate deterministic supersedes chains with integration coverage and operator scripts. +> Docs alignment (2025-10-26): Rollback guidance added to `docs/deploy/containers.md` §6. +> 2025-10-28: Documented duplicate audit + migration workflow in `docs/deploy/containers.md`, Offline Kit guide, and `MIGRATIONS.md`; published `ops/devops/scripts/check-advisory-raw-duplicates.js` for staging/offline clusters. +> Docs alignment (2025-10-26): Offline kit requirements documented in `docs/deploy/containers.md` §5. + +## Policy Engine v2 + +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| CONCELIER-POLICY-20-003 `Selection cursors` | TODO | Concelier Storage Guild | CONCELIER-STORE-AOC-19-002, POLICY-ENGINE-20-003 | Add advisory/vex selection cursors (per policy run) with change stream checkpoints, indexes, and offline migration scripts to support incremental evaluations. | + +## Link-Not-Merge v1 + +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| CONCELIER-LNM-21-101 `Observations collections` | TODO | Concelier Storage Guild | CONCELIER-LNM-21-001 | Provision `advisory_observations` and `advisory_linksets` collections with hashed shard keys, TTL for ingest metadata, and required indexes (`aliases`, `purls`, `observation_ids`). | +| CONCELIER-LNM-21-102 `Migration tooling` | TODO | Concelier Storage Guild, DevOps Guild | CONCELIER-LNM-21-101 | Backfill legacy merged advisories into observation/linkset collections, create tombstones for merged docs, and supply rollback scripts. | +| CONCELIER-LNM-21-103 `Blob/store wiring` | TODO | Concelier Storage Guild | CONCELIER-LNM-21-101 | Store large raw payloads in object storage with pointers from observations; update bootstrapper/offline kit to seed sample blobs. | diff --git a/src/Excititor/StellaOps.Excititor.Connectors.StellaOpsMirror/TASKS.completed.md b/src/Excititor/StellaOps.Excititor.Connectors.StellaOpsMirror/TASKS.completed.md new file mode 100644 index 00000000..370bce58 --- /dev/null +++ b/src/Excititor/StellaOps.Excititor.Connectors.StellaOpsMirror/TASKS.completed.md @@ -0,0 +1,5 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| EXCITITOR-CONN-STELLA-07-001 | DONE (2025-10-21) | Excititor Connectors – Stella | EXCITITOR-EXPORT-01-007 | **DONE (2025-10-21)** – Implemented `StellaOpsMirrorConnector` with `MirrorManifestClient` + `MirrorSignatureVerifier`, digest validation, signature enforcement, raw document + DTO persistence, and resume cursor updates. Added fixture-backed tests covering happy path and tampered manifest rejection. | Fetch job downloads mirror manifest, verifies DSSE/signature, stores raw documents + provenance; unit tests cover happy path and tampered manifest failure. | diff --git a/src/Excititor/StellaOps.Excititor.Connectors.StellaOpsMirror/TASKS.md b/src/Excititor/StellaOps.Excititor.Connectors.StellaOpsMirror/TASKS.md index 3830dd17..453420fa 100644 --- a/src/Excititor/StellaOps.Excititor.Connectors.StellaOpsMirror/TASKS.md +++ b/src/Excititor/StellaOps.Excititor.Connectors.StellaOpsMirror/TASKS.md @@ -1,7 +1,6 @@ -# StellaOps Mirror VEX Connector Task Board (Sprint 7) - -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| EXCITITOR-CONN-STELLA-07-001 | DONE (2025-10-21) | Excititor Connectors – Stella | EXCITITOR-EXPORT-01-007 | **DONE (2025-10-21)** – Implemented `StellaOpsMirrorConnector` with `MirrorManifestClient` + `MirrorSignatureVerifier`, digest validation, signature enforcement, raw document + DTO persistence, and resume cursor updates. Added fixture-backed tests covering happy path and tampered manifest rejection. | Fetch job downloads mirror manifest, verifies DSSE/signature, stores raw documents + provenance; unit tests cover happy path and tampered manifest failure. | -| EXCITITOR-CONN-STELLA-07-002 | TODO | Excititor Connectors – Stella | EXCITITOR-CONN-STELLA-07-001 | Normalize mirror bundles into VexClaim sets referencing original provider metadata and mirror provenance. | Normalizer emits VexClaims with mirror provenance + policy metadata, fixtures assert deterministic output parity vs local exports. | -| EXCITITOR-CONN-STELLA-07-003 | TODO | Excititor Connectors – Stella | EXCITITOR-CONN-STELLA-07-002 | Implement incremental cursor handling per-export digest, support resume, and document configuration for downstream Excititor mirrors. | Connector resumes from last export digest, handles delta/export rotation, docs show configuration; integration test covers resume + new export ingest. | +# StellaOps Mirror VEX Connector Task Board (Sprint 7) + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| EXCITITOR-CONN-STELLA-07-002 | TODO | Excititor Connectors – Stella | EXCITITOR-CONN-STELLA-07-001 | Normalize mirror bundles into VexClaim sets referencing original provider metadata and mirror provenance. | Normalizer emits VexClaims with mirror provenance + policy metadata, fixtures assert deterministic output parity vs local exports. | +| EXCITITOR-CONN-STELLA-07-003 | TODO | Excititor Connectors – Stella | EXCITITOR-CONN-STELLA-07-002 | Implement incremental cursor handling per-export digest, support resume, and document configuration for downstream Excititor mirrors. | Connector resumes from last export digest, handles delta/export rotation, docs show configuration; integration test covers resume + new export ingest. | diff --git a/src/Excititor/StellaOps.Excititor.Worker/TASKS.completed.md b/src/Excititor/StellaOps.Excititor.Worker/TASKS.completed.md new file mode 100644 index 00000000..6dafa630 --- /dev/null +++ b/src/Excititor/StellaOps.Excititor.Worker/TASKS.completed.md @@ -0,0 +1,7 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Notes | +|---|---|---|---|---| +| EXCITITOR-WORKER-AOC-19-001 `Raw pipeline rewiring` | DONE (2025-10-31) | Excititor Worker Guild | EXCITITOR-CORE-AOC-19-001 | Update ingest pipelines to persist upstream documents directly into `vex_raw` via the new repository guard. Remove consensus/folding hooks and ensure retries respect append-only semantics. | +| EXCITITOR-WORKER-AOC-19-002 `Signature & checksum enforcement` | DONE (2025-10-28) | Excititor Worker Guild | EXCITITOR-WORKER-AOC-19-001 | Add signature verification + checksum computation before writes, capturing failure reasons mapped to `ERR_AOC_005`, with structured logs/metrics for verification results. | +| EXCITITOR-WORKER-AOC-19-003 `Deterministic batching tests` | DONE (2025-10-28) | QA Guild | EXCITITOR-WORKER-AOC-19-001 | Extend worker integration tests to replay large VEX batches ensuring idempotent upserts, supersedes chaining, and guard enforcement across restart scenarios. | diff --git a/src/Excititor/StellaOps.Excititor.Worker/TASKS.md b/src/Excititor/StellaOps.Excititor.Worker/TASKS.md index 3083ac17..fa4032da 100644 --- a/src/Excititor/StellaOps.Excititor.Worker/TASKS.md +++ b/src/Excititor/StellaOps.Excititor.Worker/TASKS.md @@ -1,19 +1,16 @@ -# TASKS — Epic 1: Aggregation-Only Contract -| ID | Status | Owner(s) | Depends on | Notes | -|---|---|---|---|---| -| EXCITITOR-WORKER-AOC-19-001 `Raw pipeline rewiring` | DONE (2025-10-31) | Excititor Worker Guild | EXCITITOR-CORE-AOC-19-001 | Update ingest pipelines to persist upstream documents directly into `vex_raw` via the new repository guard. Remove consensus/folding hooks and ensure retries respect append-only semantics. | -> 2025-10-31: Worker now runs in raw-only mode; `DefaultVexProviderRunner` no longer normalizes or schedules consensus refresh and logs document counts only. Tests updated to assert the normalizer is not invoked. -| EXCITITOR-WORKER-AOC-19-002 `Signature & checksum enforcement` | DONE (2025-10-28) | Excititor Worker Guild | EXCITITOR-WORKER-AOC-19-001 | Add signature verification + checksum computation before writes, capturing failure reasons mapped to `ERR_AOC_005`, with structured logs/metrics for verification results. | -> 2025-10-28: Resuming implementation to finish attestation metadata plumbing, wiring into runner, and tests (`WorkerSignatureVerifier`, `DefaultVexProviderRunner`). -> 2025-10-28: Attestation verification now enriches signature metadata & runner tests cover DSSE path; metrics unchanged. -> 2025-10-31: Worker wraps raw sink with checksum enforcement. Digest mismatches raise `ERR_AOC_005`, signature metadata is captured when present, and `ingestion_signature_verified_total` is emitted (`result=ok|fail|skipped`). -| EXCITITOR-WORKER-AOC-19-003 `Deterministic batching tests` | DONE (2025-10-28) | QA Guild | EXCITITOR-WORKER-AOC-19-001 | Extend worker integration tests to replay large VEX batches ensuring idempotent upserts, supersedes chaining, and guard enforcement across restart scenarios. | -> 2025-10-28: Added Mongo-backed integration suite validating large batch replay, guard-triggered failures, and restart idempotency (`DefaultVexProviderRunnerIntegrationTests`). Worker unit tests now exercise the verifying sink path, and `dotnet test` passes after attestation envelope fixes. - -## Orchestrator Dashboard - -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| EXCITITOR-ORCH-32-001 `Worker SDK adoption` | TODO | Excititor Worker Guild | ORCH-SVC-32-005, WORKER-GO-32-001, WORKER-PY-32-001 | Integrate orchestrator worker SDK in Excititor ingestion jobs, emit heartbeats/progress/artifact hashes, and register source metadata. | -| EXCITITOR-ORCH-33-001 `Control compliance` | TODO | Excititor Worker Guild | EXCITITOR-ORCH-32-001, ORCH-SVC-33-001, ORCH-SVC-33-002 | Honor orchestrator pause/throttle/retry actions, classify error outputs, and persist restart checkpoints. | -| EXCITITOR-ORCH-34-001 `Backfill & circuit breaker` | TODO | Excititor Worker Guild | EXCITITOR-ORCH-33-001, ORCH-SVC-33-003, ORCH-SVC-34-001 | Implement orchestrator-driven backfills, apply circuit breaker reset rules, and ensure artifact dedupe alignment. | +# TASKS — Epic 1: Aggregation-Only Contract +| ID | Status | Owner(s) | Depends on | Notes | +|---|---|---|---|---| +> 2025-10-31: Worker now runs in raw-only mode; `DefaultVexProviderRunner` no longer normalizes or schedules consensus refresh and logs document counts only. Tests updated to assert the normalizer is not invoked. +> 2025-10-28: Resuming implementation to finish attestation metadata plumbing, wiring into runner, and tests (`WorkerSignatureVerifier`, `DefaultVexProviderRunner`). +> 2025-10-28: Attestation verification now enriches signature metadata & runner tests cover DSSE path; metrics unchanged. +> 2025-10-31: Worker wraps raw sink with checksum enforcement. Digest mismatches raise `ERR_AOC_005`, signature metadata is captured when present, and `ingestion_signature_verified_total` is emitted (`result=ok|fail|skipped`). +> 2025-10-28: Added Mongo-backed integration suite validating large batch replay, guard-triggered failures, and restart idempotency (`DefaultVexProviderRunnerIntegrationTests`). Worker unit tests now exercise the verifying sink path, and `dotnet test` passes after attestation envelope fixes. + +## Orchestrator Dashboard + +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| EXCITITOR-ORCH-32-001 `Worker SDK adoption` | TODO | Excititor Worker Guild | ORCH-SVC-32-005, WORKER-GO-32-001, WORKER-PY-32-001 | Integrate orchestrator worker SDK in Excititor ingestion jobs, emit heartbeats/progress/artifact hashes, and register source metadata. | +| EXCITITOR-ORCH-33-001 `Control compliance` | TODO | Excititor Worker Guild | EXCITITOR-ORCH-32-001, ORCH-SVC-33-001, ORCH-SVC-33-002 | Honor orchestrator pause/throttle/retry actions, classify error outputs, and persist restart checkpoints. | +| EXCITITOR-ORCH-34-001 `Backfill & circuit breaker` | TODO | Excititor Worker Guild | EXCITITOR-ORCH-33-001, ORCH-SVC-33-003, ORCH-SVC-34-001 | Implement orchestrator-driven backfills, apply circuit breaker reset rules, and ensure artifact dedupe alignment. | diff --git a/src/Excititor/__Libraries/StellaOps.Excititor.Attestation/TASKS.completed.md b/src/Excititor/__Libraries/StellaOps.Excititor.Attestation/TASKS.completed.md new file mode 100644 index 00000000..66798eae --- /dev/null +++ b/src/Excititor/__Libraries/StellaOps.Excititor.Attestation/TASKS.completed.md @@ -0,0 +1,6 @@ +# Completed Tasks + +|EXCITITOR-ATTEST-01-001 – In-toto predicate & DSSE builder|Team Excititor Attestation|EXCITITOR-CORE-01-001|**DONE (2025-10-16)** – Added deterministic in-toto predicate/statement models, DSSE envelope builder wired to signer abstraction, and attestation client producing metadata + diagnostics.| + +|EXCITITOR-ATTEST-01-002 – Rekor v2 client integration|Team Excititor Attestation|EXCITITOR-ATTEST-01-001|**DONE (2025-10-16)** – Implemented Rekor HTTP client with retry/backoff, transparency log abstraction, DI helpers, and attestation client integration capturing Rekor metadata + diagnostics.| + diff --git a/src/Excititor/__Libraries/StellaOps.Excititor.Attestation/TASKS.md b/src/Excititor/__Libraries/StellaOps.Excititor.Attestation/TASKS.md index 07d8a1bc..2b8b0196 100644 --- a/src/Excititor/__Libraries/StellaOps.Excititor.Attestation/TASKS.md +++ b/src/Excititor/__Libraries/StellaOps.Excititor.Attestation/TASKS.md @@ -2,8 +2,6 @@ If you are working on this file you need to read docs/modules/excititor/ARCHITEC # TASKS | Task | Owner(s) | Depends on | Notes | |---|---|---|---| -|EXCITITOR-ATTEST-01-001 – In-toto predicate & DSSE builder|Team Excititor Attestation|EXCITITOR-CORE-01-001|**DONE (2025-10-16)** – Added deterministic in-toto predicate/statement models, DSSE envelope builder wired to signer abstraction, and attestation client producing metadata + diagnostics.| -|EXCITITOR-ATTEST-01-002 – Rekor v2 client integration|Team Excititor Attestation|EXCITITOR-ATTEST-01-001|**DONE (2025-10-16)** – Implemented Rekor HTTP client with retry/backoff, transparency log abstraction, DI helpers, and attestation client integration capturing Rekor metadata + diagnostics.| |EXCITITOR-ATTEST-01-003 – Verification suite & observability|Team Excititor Attestation|EXCITITOR-ATTEST-01-002|DOING (2025-10-22) – Continuing implementation: build `IVexAttestationVerifier`, wire metrics/logging, and add regression tests. Draft plan in `EXCITITOR-ATTEST-01-003-plan.md` (2025-10-19) guides scope; updating with worknotes as progress lands.| > Remark (2025-10-22): Added verifier implementation + metrics/tests; next steps include wiring into WebService/Worker flows and expanding negative-path coverage. diff --git a/src/Excititor/__Libraries/StellaOps.Excititor.Attestation/Verification/VexAttestationVerificationOptions.cs b/src/Excititor/__Libraries/StellaOps.Excititor.Attestation/Verification/VexAttestationVerificationOptions.cs index f0181bc3..7d54b7a3 100644 --- a/src/Excititor/__Libraries/StellaOps.Excititor.Attestation/Verification/VexAttestationVerificationOptions.cs +++ b/src/Excititor/__Libraries/StellaOps.Excititor.Attestation/Verification/VexAttestationVerificationOptions.cs @@ -1,4 +1,5 @@ using System; +using System.Collections.Immutable; namespace StellaOps.Excititor.Attestation.Verification; @@ -25,4 +26,30 @@ public sealed class VexAttestationVerificationOptions get => _maxClockSkew; set => _maxClockSkew = value < TimeSpan.Zero ? TimeSpan.Zero : value; } + + /// + /// When true, DSSE signatures must verify successfully against configured trusted signers. + /// + public bool RequireSignatureVerification { get; init; } + + /// + /// Mapping of trusted signer key identifiers to verification configuration. + /// + public ImmutableDictionary TrustedSigners { get; init; } = + ImmutableDictionary.Empty; + + public sealed record TrustedSignerOptions + { + public string Algorithm { get; init; } = StellaOps.Cryptography.SignatureAlgorithms.Ed25519; + + /// + /// Key identifier to resolve from the crypto provider registry. + /// + public string KeyReference { get; init; } = string.Empty; + + /// + /// Optional provider hint to bias registry resolution. + /// + public string? ProviderHint { get; init; } + } } diff --git a/src/Excititor/__Libraries/StellaOps.Excititor.Attestation/Verification/VexAttestationVerifier.cs b/src/Excititor/__Libraries/StellaOps.Excititor.Attestation/Verification/VexAttestationVerifier.cs index c138ab81..9b6f9b07 100644 --- a/src/Excititor/__Libraries/StellaOps.Excititor.Attestation/Verification/VexAttestationVerifier.cs +++ b/src/Excititor/__Libraries/StellaOps.Excititor.Attestation/Verification/VexAttestationVerifier.cs @@ -3,17 +3,18 @@ using System.Collections.Generic; using System.Collections.Immutable; using System.Diagnostics; using System.Linq; -using System.Text; -using System.Text.Json; -using System.Text.Json.Serialization; -using System.Threading; -using System.Threading.Tasks; -using Microsoft.Extensions.Logging; -using Microsoft.Extensions.Options; -using StellaOps.Excititor.Attestation.Dsse; -using StellaOps.Excititor.Attestation.Models; -using StellaOps.Excititor.Attestation.Transparency; -using StellaOps.Excititor.Core; +using System.Text; +using System.Text.Json; +using System.Text.Json.Serialization; +using System.Threading; +using System.Threading.Tasks; +using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Options; +using StellaOps.Excititor.Attestation.Dsse; +using StellaOps.Excititor.Attestation.Models; +using StellaOps.Excititor.Attestation.Transparency; +using StellaOps.Excititor.Core; +using StellaOps.Cryptography; namespace StellaOps.Excititor.Attestation.Verification; @@ -32,23 +33,28 @@ internal sealed class VexAttestationVerifier : IVexAttestationVerifier Converters = { new JsonStringEnumConverter(JsonNamingPolicy.CamelCase) }, }; - private readonly ILogger _logger; - private readonly ITransparencyLogClient? _transparencyLogClient; - private readonly VexAttestationVerificationOptions _options; - private readonly VexAttestationMetrics _metrics; - - public VexAttestationVerifier( - ILogger logger, - ITransparencyLogClient? transparencyLogClient, - IOptions options, - VexAttestationMetrics metrics) - { - _logger = logger ?? throw new ArgumentNullException(nameof(logger)); - ArgumentNullException.ThrowIfNull(options); - _transparencyLogClient = transparencyLogClient; - _options = options.Value; - _metrics = metrics ?? throw new ArgumentNullException(nameof(metrics)); - } + private readonly ILogger _logger; + private readonly ITransparencyLogClient? _transparencyLogClient; + private readonly VexAttestationVerificationOptions _options; + private readonly VexAttestationMetrics _metrics; + private readonly ICryptoProviderRegistry? _cryptoRegistry; + private readonly ImmutableDictionary _trustedSigners; + + public VexAttestationVerifier( + ILogger logger, + ITransparencyLogClient? transparencyLogClient, + IOptions options, + VexAttestationMetrics metrics, + ICryptoProviderRegistry? cryptoRegistry = null) + { + _logger = logger ?? throw new ArgumentNullException(nameof(logger)); + ArgumentNullException.ThrowIfNull(options); + _transparencyLogClient = transparencyLogClient; + _options = options.Value; + _metrics = metrics ?? throw new ArgumentNullException(nameof(metrics)); + _cryptoRegistry = cryptoRegistry; + _trustedSigners = _options.TrustedSigners; + } public async ValueTask VerifyAsync( VexAttestationVerificationRequest request, @@ -148,15 +154,27 @@ internal sealed class VexAttestationVerifier : IVexAttestationVerifier return BuildResult(false); } - rekorState = await VerifyTransparencyAsync(request.Metadata, diagnostics, cancellationToken).ConfigureAwait(false); - if (rekorState is "missing" or "unverified" or "client_unavailable") - { - resultLabel = "invalid"; - return BuildResult(false); - } - - diagnostics["signature.state"] = "present"; - return BuildResult(true); + rekorState = await VerifyTransparencyAsync(request.Metadata, diagnostics, cancellationToken).ConfigureAwait(false); + if (rekorState is "missing" or "unverified" or "client_unavailable") + { + resultLabel = "invalid"; + return BuildResult(false); + } + + var signaturesVerified = await VerifySignaturesAsync(payloadBytes, envelope.Signatures, diagnostics, cancellationToken).ConfigureAwait(false); + if (!signaturesVerified) + { + if (_options.RequireSignatureVerification) + { + resultLabel = "invalid"; + return BuildResult(false); + } + + resultLabel = "degraded"; + } + + diagnostics["signature.state"] = "present"; + return BuildResult(true); } catch (Exception ex) { diff --git a/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Abstractions/TASKS.completed.md b/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Abstractions/TASKS.completed.md new file mode 100644 index 00000000..80f8a61a --- /dev/null +++ b/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Abstractions/TASKS.completed.md @@ -0,0 +1,8 @@ +# Completed Tasks + +|EXCITITOR-CONN-ABS-01-001 – Connector context & base classes|Team Excititor Connectors|EXCITITOR-CORE-01-003|**DONE (2025-10-17)** – Added `StellaOps.Excititor.Connectors.Abstractions` project with `VexConnectorBase`, deterministic logging scopes, metadata builder helpers, and connector descriptors; docs updated to highlight the shared abstractions.| + +|EXCITITOR-CONN-ABS-01-002 – YAML options & validation|Team Excititor Connectors|EXCITITOR-CONN-ABS-01-001|**DONE (2025-10-17)** – Delivered `VexConnectorOptionsBinder` + binder options/validators, environment-variable expansion, data-annotation checks, and custom validation hooks with documentation updates covering the workflow.| + +|EXCITITOR-CONN-ABS-01-003 – Plugin packaging & docs|Team Excititor Connectors|EXCITITOR-CONN-ABS-01-001|**DONE (2025-10-17)** – Authored `docs/dev/30_EXCITITOR_CONNECTOR_GUIDE.md`, added quick-start template under `docs/dev/templates/excititor-connector/`, and updated module docs to reference the packaging workflow.| + diff --git a/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Abstractions/TASKS.md b/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Abstractions/TASKS.md index 726652a5..5a0de659 100644 --- a/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Abstractions/TASKS.md +++ b/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Abstractions/TASKS.md @@ -2,6 +2,3 @@ If you are working on this file you need to read docs/modules/excititor/ARCHITEC # TASKS | Task | Owner(s) | Depends on | Notes | |---|---|---|---| -|EXCITITOR-CONN-ABS-01-001 – Connector context & base classes|Team Excititor Connectors|EXCITITOR-CORE-01-003|**DONE (2025-10-17)** – Added `StellaOps.Excititor.Connectors.Abstractions` project with `VexConnectorBase`, deterministic logging scopes, metadata builder helpers, and connector descriptors; docs updated to highlight the shared abstractions.| -|EXCITITOR-CONN-ABS-01-002 – YAML options & validation|Team Excititor Connectors|EXCITITOR-CONN-ABS-01-001|**DONE (2025-10-17)** – Delivered `VexConnectorOptionsBinder` + binder options/validators, environment-variable expansion, data-annotation checks, and custom validation hooks with documentation updates covering the workflow.| -|EXCITITOR-CONN-ABS-01-003 – Plugin packaging & docs|Team Excititor Connectors|EXCITITOR-CONN-ABS-01-001|**DONE (2025-10-17)** – Authored `docs/dev/30_EXCITITOR_CONNECTOR_GUIDE.md`, added quick-start template under `docs/dev/templates/excititor-connector/`, and updated module docs to reference the packaging workflow.| diff --git a/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Cisco.CSAF/TASKS.completed.md b/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Cisco.CSAF/TASKS.completed.md new file mode 100644 index 00000000..5441c362 --- /dev/null +++ b/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Cisco.CSAF/TASKS.completed.md @@ -0,0 +1,8 @@ +# Completed Tasks + +|EXCITITOR-CONN-CISCO-01-001 – Endpoint discovery & auth plumbing|Team Excititor Connectors – Cisco|EXCITITOR-CONN-ABS-01-001|**DONE (2025-10-17)** – Added `CiscoProviderMetadataLoader` with bearer token support, offline snapshot fallback, DI helpers, and tests covering network/offline discovery to unblock subsequent fetch work.| + +|EXCITITOR-CONN-CISCO-01-002 – CSAF pull loop & pagination|Team Excititor Connectors – Cisco|EXCITITOR-CONN-CISCO-01-001, EXCITITOR-STORAGE-01-003|**DONE (2025-10-17)** – Implemented paginated advisory fetch using provider directories, raw document persistence with dedupe/state tracking, offline resiliency, and unit coverage.| + +|EXCITITOR-CONN-CISCO-01-003 – Provider trust metadata|Team Excititor Connectors – Cisco|EXCITITOR-CONN-CISCO-01-002, EXCITITOR-POLICY-01-001|**DONE (2025-10-29)** – Connector now annotates raw documents with provider trust + cosign/PGP provenance and upserts `VexProvider` entries; new unit coverage asserts metadata emission and provider-store invocation.| + diff --git a/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Cisco.CSAF/TASKS.md b/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Cisco.CSAF/TASKS.md index f4f02071..5a0de659 100644 --- a/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Cisco.CSAF/TASKS.md +++ b/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Cisco.CSAF/TASKS.md @@ -2,6 +2,3 @@ If you are working on this file you need to read docs/modules/excititor/ARCHITEC # TASKS | Task | Owner(s) | Depends on | Notes | |---|---|---|---| -|EXCITITOR-CONN-CISCO-01-001 – Endpoint discovery & auth plumbing|Team Excititor Connectors – Cisco|EXCITITOR-CONN-ABS-01-001|**DONE (2025-10-17)** – Added `CiscoProviderMetadataLoader` with bearer token support, offline snapshot fallback, DI helpers, and tests covering network/offline discovery to unblock subsequent fetch work.| -|EXCITITOR-CONN-CISCO-01-002 – CSAF pull loop & pagination|Team Excititor Connectors – Cisco|EXCITITOR-CONN-CISCO-01-001, EXCITITOR-STORAGE-01-003|**DONE (2025-10-17)** – Implemented paginated advisory fetch using provider directories, raw document persistence with dedupe/state tracking, offline resiliency, and unit coverage.| -|EXCITITOR-CONN-CISCO-01-003 – Provider trust metadata|Team Excititor Connectors – Cisco|EXCITITOR-CONN-CISCO-01-002, EXCITITOR-POLICY-01-001|**DONE (2025-10-29)** – Connector now annotates raw documents with provider trust + cosign/PGP provenance and upserts `VexProvider` entries; new unit coverage asserts metadata emission and provider-store invocation.| diff --git a/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.MSRC.CSAF/TASKS.completed.md b/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.MSRC.CSAF/TASKS.completed.md new file mode 100644 index 00000000..86ac7db8 --- /dev/null +++ b/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.MSRC.CSAF/TASKS.completed.md @@ -0,0 +1,6 @@ +# Completed Tasks + +|EXCITITOR-CONN-MS-01-001 – AAD onboarding & token cache|Team Excititor Connectors – MSRC|EXCITITOR-CONN-ABS-01-001|**DONE (2025-10-17)** – Added MSRC connector project with configurable AAD options, token provider (offline/online modes), DI wiring, and unit tests covering caching and fallback scenarios.| + +|EXCITITOR-CONN-MS-01-002 – CSAF download pipeline|Team Excititor Connectors – MSRC|EXCITITOR-CONN-MS-01-001, EXCITITOR-STORAGE-01-003|**DONE (2025-10-29)** – Implemented authenticated CSAF retrieval with retry/backoff, checksum enforcement, quarantine for invalid archives, and regression tests covering dedupe + idempotent state updates.| + diff --git a/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.MSRC.CSAF/TASKS.md b/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.MSRC.CSAF/TASKS.md index 9e2d5d9d..3edff2ec 100644 --- a/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.MSRC.CSAF/TASKS.md +++ b/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.MSRC.CSAF/TASKS.md @@ -2,6 +2,4 @@ If you are working on this file you need to read docs/modules/excititor/ARCHITEC # TASKS | Task | Owner(s) | Depends on | Notes | |---|---|---|---| -|EXCITITOR-CONN-MS-01-001 – AAD onboarding & token cache|Team Excititor Connectors – MSRC|EXCITITOR-CONN-ABS-01-001|**DONE (2025-10-17)** – Added MSRC connector project with configurable AAD options, token provider (offline/online modes), DI wiring, and unit tests covering caching and fallback scenarios.| -|EXCITITOR-CONN-MS-01-002 – CSAF download pipeline|Team Excititor Connectors – MSRC|EXCITITOR-CONN-MS-01-001, EXCITITOR-STORAGE-01-003|**DONE (2025-10-29)** – Implemented authenticated CSAF retrieval with retry/backoff, checksum enforcement, quarantine for invalid archives, and regression tests covering dedupe + idempotent state updates.| |EXCITITOR-CONN-MS-01-003 – Trust metadata & provenance hints|Team Excititor Connectors – MSRC|EXCITITOR-CONN-MS-01-002, EXCITITOR-POLICY-01-001|TODO – Emit cosign/AAD issuer metadata, attach provenance details, and document policy integration.| diff --git a/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.OCI.OpenVEX.Attest/TASKS.completed.md b/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.OCI.OpenVEX.Attest/TASKS.completed.md new file mode 100644 index 00000000..67d21644 --- /dev/null +++ b/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.OCI.OpenVEX.Attest/TASKS.completed.md @@ -0,0 +1,8 @@ +# Completed Tasks + +|EXCITITOR-CONN-OCI-01-001 – OCI discovery & auth plumbing|Team Excititor Connectors – OCI|EXCITITOR-CONN-ABS-01-001|DONE (2025-10-18) – Added connector skeleton, options/validators, discovery caching, cosign/auth descriptors, offline bundle resolution, DI wiring, and regression tests.| + +|EXCITITOR-CONN-OCI-01-002 – Attestation fetch & verify loop|Team Excititor Connectors – OCI|EXCITITOR-CONN-OCI-01-001, EXCITITOR-ATTEST-01-002|DONE (2025-10-18) – Added offline/registry fetch services, DSSE retrieval with retries, signature verification callout, and raw persistence coverage.| + +|EXCITITOR-CONN-OCI-01-003 – Provenance metadata & policy hooks|Team Excititor Connectors – OCI|EXCITITOR-CONN-OCI-01-002, EXCITITOR-POLICY-01-001|DONE (2025-10-18) – Enriched attestation metadata with provenance hints, cosign expectations, registry auth context, and signature diagnostics for policy consumption.| + diff --git a/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.OCI.OpenVEX.Attest/TASKS.md b/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.OCI.OpenVEX.Attest/TASKS.md index 1d818756..5a0de659 100644 --- a/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.OCI.OpenVEX.Attest/TASKS.md +++ b/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.OCI.OpenVEX.Attest/TASKS.md @@ -2,6 +2,3 @@ If you are working on this file you need to read docs/modules/excititor/ARCHITEC # TASKS | Task | Owner(s) | Depends on | Notes | |---|---|---|---| -|EXCITITOR-CONN-OCI-01-001 – OCI discovery & auth plumbing|Team Excititor Connectors – OCI|EXCITITOR-CONN-ABS-01-001|DONE (2025-10-18) – Added connector skeleton, options/validators, discovery caching, cosign/auth descriptors, offline bundle resolution, DI wiring, and regression tests.| -|EXCITITOR-CONN-OCI-01-002 – Attestation fetch & verify loop|Team Excititor Connectors – OCI|EXCITITOR-CONN-OCI-01-001, EXCITITOR-ATTEST-01-002|DONE (2025-10-18) – Added offline/registry fetch services, DSSE retrieval with retries, signature verification callout, and raw persistence coverage.| -|EXCITITOR-CONN-OCI-01-003 – Provenance metadata & policy hooks|Team Excititor Connectors – OCI|EXCITITOR-CONN-OCI-01-002, EXCITITOR-POLICY-01-001|DONE (2025-10-18) – Enriched attestation metadata with provenance hints, cosign expectations, registry auth context, and signature diagnostics for policy consumption.| diff --git a/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Oracle.CSAF/TASKS.completed.md b/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Oracle.CSAF/TASKS.completed.md new file mode 100644 index 00000000..65037188 --- /dev/null +++ b/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Oracle.CSAF/TASKS.completed.md @@ -0,0 +1,6 @@ +# Completed Tasks + +|EXCITITOR-CONN-ORACLE-01-001 – Oracle CSAF catalogue discovery|Team Excititor Connectors – Oracle|EXCITITOR-CONN-ABS-01-001|**DONE (2025-10-19)** – Implemented cached Oracle CSAF catalog loader with CPU calendar merge, offline snapshot ingest/persist, options validation + DI wiring, and regression tests; prerequisite EXCITITOR-CONN-ABS-01-001 verified DONE per Sprint 5 log (2025-10-19).| + +|EXCITITOR-CONN-ORACLE-01-002 – CSAF download & dedupe pipeline|Team Excititor Connectors – Oracle|EXCITITOR-CONN-ORACLE-01-001, EXCITITOR-STORAGE-01-003|**DONE (2025-10-19)** – Added Oracle CSAF fetch loop with retry/backoff, checksum validation, resume-aware state persistence, digest dedupe, configurable throttling, and raw storage wiring; regression tests cover new ingestion and mismatch handling.| + diff --git a/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Oracle.CSAF/TASKS.md b/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Oracle.CSAF/TASKS.md index a23205b3..68d94c87 100644 --- a/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Oracle.CSAF/TASKS.md +++ b/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Oracle.CSAF/TASKS.md @@ -2,6 +2,4 @@ If you are working on this file you need to read docs/modules/excititor/ARCHITEC # TASKS | Task | Owner(s) | Depends on | Notes | |---|---|---|---| -|EXCITITOR-CONN-ORACLE-01-001 – Oracle CSAF catalogue discovery|Team Excititor Connectors – Oracle|EXCITITOR-CONN-ABS-01-001|**DONE (2025-10-19)** – Implemented cached Oracle CSAF catalog loader with CPU calendar merge, offline snapshot ingest/persist, options validation + DI wiring, and regression tests; prerequisite EXCITITOR-CONN-ABS-01-001 verified DONE per Sprint 5 log (2025-10-19).| -|EXCITITOR-CONN-ORACLE-01-002 – CSAF download & dedupe pipeline|Team Excititor Connectors – Oracle|EXCITITOR-CONN-ORACLE-01-001, EXCITITOR-STORAGE-01-003|**DONE (2025-10-19)** – Added Oracle CSAF fetch loop with retry/backoff, checksum validation, resume-aware state persistence, digest dedupe, configurable throttling, and raw storage wiring; regression tests cover new ingestion and mismatch handling.| |EXCITITOR-CONN-ORACLE-01-003 – Trust metadata + provenance|Team Excititor Connectors – Oracle|EXCITITOR-CONN-ORACLE-01-002, EXCITITOR-POLICY-01-001|TODO – Emit Oracle signing metadata (PGP/cosign) and provenance hints for consensus weighting.| diff --git a/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.RedHat.CSAF/TASKS.completed.md b/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.RedHat.CSAF/TASKS.completed.md new file mode 100644 index 00000000..493aca8f --- /dev/null +++ b/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.RedHat.CSAF/TASKS.completed.md @@ -0,0 +1,14 @@ +# Completed Tasks + +|EXCITITOR-CONN-RH-01-001 – Provider metadata discovery|Team Excititor Connectors – Red Hat|EXCITITOR-CONN-ABS-01-001|**DONE (2025-10-17)** – Added `RedHatProviderMetadataLoader` with HTTP/ETag caching, offline snapshot handling, and validation; exposed DI helper + tests covering live, cached, and offline scenarios.| + +|EXCITITOR-CONN-RH-01-002 – Incremental CSAF pulls|Team Excititor Connectors – Red Hat|EXCITITOR-CONN-RH-01-001, EXCITITOR-STORAGE-01-003|**DONE (2025-10-17)** – Implemented `RedHatCsafConnector` with ROLIE feed parsing, incremental filtering via `context.Since`, CSAF document download + metadata capture, and persistence through `IVexRawDocumentSink`; tests cover live fetch/cache/offline scenarios with ETag handling.| + +|EXCITITOR-CONN-RH-01-003 – Trust metadata emission|Team Excititor Connectors – Red Hat|EXCITITOR-CONN-RH-01-002, EXCITITOR-POLICY-01-001|**DONE (2025-10-17)** – Provider metadata loader now emits trust overrides (weight, cosign issuer/pattern, PGP fingerprints) and the connector surfaces provenance hints for policy/consensus layers.| + +|EXCITITOR-CONN-RH-01-004 – Resume state persistence|Team Excititor Connectors – Red Hat|EXCITITOR-CONN-RH-01-002, EXCITITOR-STORAGE-01-003|**DONE (2025-10-17)** – Connector now loads/saves resume state via `IVexConnectorStateRepository`, tracking last update timestamp and recent document digests to avoid duplicate CSAF ingestion; regression covers state persistence and duplicate skips.| + +|EXCITITOR-CONN-RH-01-005 – Worker/WebService integration|Team Excititor Connectors – Red Hat|EXCITITOR-CONN-RH-01-002|**DONE (2025-10-17)** – Worker/WebService now call `AddRedHatCsafConnector`, register the connector + state repo, and default worker scheduling adds the `excititor:redhat` provider so background jobs and orchestration can activate the connector without extra wiring.| + +|EXCITITOR-CONN-RH-01-006 – CSAF normalization parity tests|Team Excititor Connectors – Red Hat|EXCITITOR-CONN-RH-01-002, EXCITITOR-FMT-CSAF-01-001|**DONE (2025-10-17)** – Added RHSA fixture-driven regression verifying CSAF normalizer retains Red Hat product metadata, tracking fields, and timestamps (`rhsa-sample.json` + `CsafNormalizerTests.NormalizeAsync_PreservesRedHatSpecificMetadata`).| + diff --git a/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.RedHat.CSAF/TASKS.md b/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.RedHat.CSAF/TASKS.md index e04572ce..5a0de659 100644 --- a/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.RedHat.CSAF/TASKS.md +++ b/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.RedHat.CSAF/TASKS.md @@ -2,9 +2,3 @@ If you are working on this file you need to read docs/modules/excititor/ARCHITEC # TASKS | Task | Owner(s) | Depends on | Notes | |---|---|---|---| -|EXCITITOR-CONN-RH-01-001 – Provider metadata discovery|Team Excititor Connectors – Red Hat|EXCITITOR-CONN-ABS-01-001|**DONE (2025-10-17)** – Added `RedHatProviderMetadataLoader` with HTTP/ETag caching, offline snapshot handling, and validation; exposed DI helper + tests covering live, cached, and offline scenarios.| -|EXCITITOR-CONN-RH-01-002 – Incremental CSAF pulls|Team Excititor Connectors – Red Hat|EXCITITOR-CONN-RH-01-001, EXCITITOR-STORAGE-01-003|**DONE (2025-10-17)** – Implemented `RedHatCsafConnector` with ROLIE feed parsing, incremental filtering via `context.Since`, CSAF document download + metadata capture, and persistence through `IVexRawDocumentSink`; tests cover live fetch/cache/offline scenarios with ETag handling.| -|EXCITITOR-CONN-RH-01-003 – Trust metadata emission|Team Excititor Connectors – Red Hat|EXCITITOR-CONN-RH-01-002, EXCITITOR-POLICY-01-001|**DONE (2025-10-17)** – Provider metadata loader now emits trust overrides (weight, cosign issuer/pattern, PGP fingerprints) and the connector surfaces provenance hints for policy/consensus layers.| -|EXCITITOR-CONN-RH-01-004 – Resume state persistence|Team Excititor Connectors – Red Hat|EXCITITOR-CONN-RH-01-002, EXCITITOR-STORAGE-01-003|**DONE (2025-10-17)** – Connector now loads/saves resume state via `IVexConnectorStateRepository`, tracking last update timestamp and recent document digests to avoid duplicate CSAF ingestion; regression covers state persistence and duplicate skips.| -|EXCITITOR-CONN-RH-01-005 – Worker/WebService integration|Team Excititor Connectors – Red Hat|EXCITITOR-CONN-RH-01-002|**DONE (2025-10-17)** – Worker/WebService now call `AddRedHatCsafConnector`, register the connector + state repo, and default worker scheduling adds the `excititor:redhat` provider so background jobs and orchestration can activate the connector without extra wiring.| -|EXCITITOR-CONN-RH-01-006 – CSAF normalization parity tests|Team Excititor Connectors – Red Hat|EXCITITOR-CONN-RH-01-002, EXCITITOR-FMT-CSAF-01-001|**DONE (2025-10-17)** – Added RHSA fixture-driven regression verifying CSAF normalizer retains Red Hat product metadata, tracking fields, and timestamps (`rhsa-sample.json` + `CsafNormalizerTests.NormalizeAsync_PreservesRedHatSpecificMetadata`).| diff --git a/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.SUSE.RancherVEXHub/TASKS.completed.md b/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.SUSE.RancherVEXHub/TASKS.completed.md new file mode 100644 index 00000000..16790142 --- /dev/null +++ b/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.SUSE.RancherVEXHub/TASKS.completed.md @@ -0,0 +1,6 @@ +# Completed Tasks + +|EXCITITOR-CONN-SUSE-01-001 – Rancher hub discovery & auth|Team Excititor Connectors – SUSE|EXCITITOR-CONN-ABS-01-001|**DONE (2025-10-17)** – Added Rancher hub options/token provider, discovery metadata loader with offline snapshots + caching, connector shell, DI wiring, and unit tests covering network/offline paths.| + +|EXCITITOR-CONN-SUSE-01-002 – Checkpointed event ingestion|Team Excititor Connectors – SUSE|EXCITITOR-CONN-SUSE-01-001, EXCITITOR-STORAGE-01-003|**DONE (2025-10-29)** – Wired checkpoint manager/event client into DI, bounded digest history, and exercised offline snapshot/dedup/quarantine flows with new connector tests ensuring state persistence and replay determinism.| + diff --git a/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.SUSE.RancherVEXHub/TASKS.md b/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.SUSE.RancherVEXHub/TASKS.md index 04377fdc..e3192043 100644 --- a/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.SUSE.RancherVEXHub/TASKS.md +++ b/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.SUSE.RancherVEXHub/TASKS.md @@ -2,6 +2,4 @@ If you are working on this file you need to read docs/modules/excititor/ARCHITEC # TASKS | Task | Owner(s) | Depends on | Notes | |---|---|---|---| -|EXCITITOR-CONN-SUSE-01-001 – Rancher hub discovery & auth|Team Excititor Connectors – SUSE|EXCITITOR-CONN-ABS-01-001|**DONE (2025-10-17)** – Added Rancher hub options/token provider, discovery metadata loader with offline snapshots + caching, connector shell, DI wiring, and unit tests covering network/offline paths.| -|EXCITITOR-CONN-SUSE-01-002 – Checkpointed event ingestion|Team Excititor Connectors – SUSE|EXCITITOR-CONN-SUSE-01-001, EXCITITOR-STORAGE-01-003|**DONE (2025-10-29)** – Wired checkpoint manager/event client into DI, bounded digest history, and exercised offline snapshot/dedup/quarantine flows with new connector tests ensuring state persistence and replay determinism.| |EXCITITOR-CONN-SUSE-01-003 – Trust metadata & policy hints|Team Excititor Connectors – SUSE|EXCITITOR-CONN-SUSE-01-002, EXCITITOR-POLICY-01-001|TODO – Emit provider trust configuration (signers, weight overrides) and attach provenance hints for consensus engine.| diff --git a/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Ubuntu.CSAF/TASKS.completed.md b/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Ubuntu.CSAF/TASKS.completed.md new file mode 100644 index 00000000..09d3f7e0 --- /dev/null +++ b/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Ubuntu.CSAF/TASKS.completed.md @@ -0,0 +1,6 @@ +# Completed Tasks + +|EXCITITOR-CONN-UBUNTU-01-001 – Ubuntu CSAF discovery & channels|Team Excititor Connectors – Ubuntu|EXCITITOR-CONN-ABS-01-001|**DONE (2025-10-17)** – Added Ubuntu connector project with configurable channel options, catalog loader (network/offline), DI wiring, and discovery unit tests.| + +|EXCITITOR-CONN-UBUNTU-01-002 – Incremental fetch & deduplication|Team Excititor Connectors – Ubuntu|EXCITITOR-CONN-UBUNTU-01-001, EXCITITOR-STORAGE-01-003|**DONE (2025-10-29)** – Incremental pull loop now enforces ETag/sha validation, resumes from persisted state, and includes regression tests covering checksum mismatch quarantine and If-None-Match replay.| + diff --git a/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Ubuntu.CSAF/TASKS.md b/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Ubuntu.CSAF/TASKS.md index 35584fb1..083833c4 100644 --- a/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Ubuntu.CSAF/TASKS.md +++ b/src/Excititor/__Libraries/StellaOps.Excititor.Connectors.Ubuntu.CSAF/TASKS.md @@ -2,7 +2,5 @@ If you are working on this file you need to read docs/modules/excititor/ARCHITEC # TASKS | Task | Owner(s) | Depends on | Notes | |---|---|---|---| -|EXCITITOR-CONN-UBUNTU-01-001 – Ubuntu CSAF discovery & channels|Team Excititor Connectors – Ubuntu|EXCITITOR-CONN-ABS-01-001|**DONE (2025-10-17)** – Added Ubuntu connector project with configurable channel options, catalog loader (network/offline), DI wiring, and discovery unit tests.| -|EXCITITOR-CONN-UBUNTU-01-002 – Incremental fetch & deduplication|Team Excititor Connectors – Ubuntu|EXCITITOR-CONN-UBUNTU-01-001, EXCITITOR-STORAGE-01-003|**DONE (2025-10-29)** – Incremental pull loop now enforces ETag/sha validation, resumes from persisted state, and includes regression tests covering checksum mismatch quarantine and If-None-Match replay.| |EXCITITOR-CONN-UBUNTU-01-003 – Trust metadata & provenance|Team Excititor Connectors – Ubuntu|EXCITITOR-CONN-UBUNTU-01-002, EXCITITOR-POLICY-01-001|TODO – Emit Ubuntu signing metadata (GPG fingerprints) plus provenance hints for policy weighting and diagnostics.| > Remark (2025-10-29, EXCITITOR-CONN-UBUNTU-01-002): Offline + network regression pass validated resume tokens, dedupe skips, checksum enforcement, and ETag handling before closing the task. diff --git a/src/Excititor/__Libraries/StellaOps.Excititor.Core/TASKS.completed.md b/src/Excititor/__Libraries/StellaOps.Excititor.Core/TASKS.completed.md new file mode 100644 index 00000000..9ac8b575 --- /dev/null +++ b/src/Excititor/__Libraries/StellaOps.Excititor.Core/TASKS.completed.md @@ -0,0 +1,9 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Notes | +|---|---|---|---|---| +| EXCITITOR-CORE-AOC-19-001 `AOC guard & provenance enforcement` | DONE (2025-10-29) | Excititor Core Guild | WEB-AOC-19-001 | Introduce repository interceptor validating provenance/signatures, rejecting forbidden fields (`severity`, `consensus`, etc.), and surfacing `ERR_AOC_00x` codes. | + +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| EXCITITOR-GRAPH-24-001 `VEX overlay inputs` | DONE (2025-10-29) | Excititor Core Guild | EXCITITOR-POLICY-23-001 | Expose raw VEX statements/linksets scoped for overlay services; no suppression/precedence logic in ingestion. | diff --git a/src/Excititor/__Libraries/StellaOps.Excititor.Core/TASKS.md b/src/Excititor/__Libraries/StellaOps.Excititor.Core/TASKS.md index 06331821..c7699cf3 100644 --- a/src/Excititor/__Libraries/StellaOps.Excititor.Core/TASKS.md +++ b/src/Excititor/__Libraries/StellaOps.Excititor.Core/TASKS.md @@ -1,103 +1,101 @@ -# TASKS — Epic 1: Aggregation-Only Contract -> **AOC Reminder:** ingestion captures raw VEX statements/linksets only—no precedence, suppression, or severity derivation within Excititor. -| ID | Status | Owner(s) | Depends on | Notes | -|---|---|---|---|---| -| EXCITITOR-CORE-AOC-19-001 `AOC guard & provenance enforcement` | DONE (2025-10-29) | Excititor Core Guild | WEB-AOC-19-001 | Introduce repository interceptor validating provenance/signatures, rejecting forbidden fields (`severity`, `consensus`, etc.), and surfacing `ERR_AOC_00x` codes. | -> 2025-10-31: Raw guard now enforced by `MongoVexRawStore` and worker DI via `AddExcititorAocGuards`; repository + backfill tests cover guard pass/fail and storage rollback. -> 2025-10-29: Added `VexRawWriteGuard` + DI hooks consuming `AocWriteGuard`; unit coverage validates minimal and invalid signature cases. Integration with raw sinks remains outstanding. -| EXCITITOR-CORE-AOC-19-002 `VEX linkset extraction` | TODO | Excititor Core Guild | EXCITITOR-CORE-AOC-19-001 | Implement deterministic extraction of advisory IDs, component PURLs, and references into `linkset`, capturing reconciled-from metadata for traceability. | -| EXCITITOR-CORE-AOC-19-003 `Idempotent VEX raw upsert` | TODO | Excititor Core Guild | EXCITITOR-STORE-AOC-19-002 | Enforce `(vendor, upstreamId, contentHash, tenant)` uniqueness, generate supersedes chains, and ensure append-only versioning of raw VEX documents. | -| EXCITITOR-CORE-AOC-19-004 `Remove ingestion consensus` | TODO | Excititor Core Guild | EXCITITOR-CORE-AOC-19-002, POLICY-AOC-19-003 | Excise consensus/merge/severity logic from Excititor ingestion paths, updating exports/tests to rely on Policy Engine materializations instead. | -| EXCITITOR-CORE-AOC-19-013 `Authority tenant scope smoke coverage` | TODO | Excititor Core Guild | AUTH-AOC-19-002 | Update Excititor smoke/e2e suites to seed tenant-aware Authority clients and ensure cross-tenant VEX ingestion is rejected. | Required for Authority docs (`AUTH-AOC-19-003`) sign-off; share results with Authority Core. | - -## Policy Engine v2 - -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| EXCITITOR-POLICY-20-002 `Scope-aware linksets` | TODO | Excititor Core Guild, Policy Guild | EXCITITOR-CORE-AOC-19-002, POLICY-ENGINE-20-001 | Enhance VEX linkset extraction with scope resolution (product/component) + version range matching to boost policy join accuracy; refresh fixtures/tests. | - -## Graph Explorer v1 - -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| +# TASKS — Epic 1: Aggregation-Only Contract +> **AOC Reminder:** ingestion captures raw VEX statements/linksets only—no precedence, suppression, or severity derivation within Excititor. +| ID | Status | Owner(s) | Depends on | Notes | +|---|---|---|---|---| +> 2025-10-31: Raw guard now enforced by `MongoVexRawStore` and worker DI via `AddExcititorAocGuards`; repository + backfill tests cover guard pass/fail and storage rollback. +> 2025-10-29: Added `VexRawWriteGuard` + DI hooks consuming `AocWriteGuard`; unit coverage validates minimal and invalid signature cases. Integration with raw sinks remains outstanding. +| EXCITITOR-CORE-AOC-19-002 `VEX linkset extraction` | TODO | Excititor Core Guild | EXCITITOR-CORE-AOC-19-001 | Implement deterministic extraction of advisory IDs, component PURLs, and references into `linkset`, capturing reconciled-from metadata for traceability. | +| EXCITITOR-CORE-AOC-19-003 `Idempotent VEX raw upsert` | TODO | Excititor Core Guild | EXCITITOR-STORE-AOC-19-002 | Enforce `(vendor, upstreamId, contentHash, tenant)` uniqueness, generate supersedes chains, and ensure append-only versioning of raw VEX documents. | +| EXCITITOR-CORE-AOC-19-004 `Remove ingestion consensus` | TODO | Excititor Core Guild | EXCITITOR-CORE-AOC-19-002, POLICY-AOC-19-003 | Excise consensus/merge/severity logic from Excititor ingestion paths, updating exports/tests to rely on Policy Engine materializations instead. | +| EXCITITOR-CORE-AOC-19-013 `Authority tenant scope smoke coverage` | TODO | Excititor Core Guild | AUTH-AOC-19-002 | Update Excititor smoke/e2e suites to seed tenant-aware Authority clients and ensure cross-tenant VEX ingestion is rejected. | Required for Authority docs (`AUTH-AOC-19-003`) sign-off; share results with Authority Core. | + +## Policy Engine v2 + +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| EXCITITOR-POLICY-20-002 `Scope-aware linksets` | TODO | Excititor Core Guild, Policy Guild | EXCITITOR-CORE-AOC-19-002, POLICY-ENGINE-20-001 | Enhance VEX linkset extraction with scope resolution (product/component) + version range matching to boost policy join accuracy; refresh fixtures/tests. | + +## Graph Explorer v1 + +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| | EXCITITOR-GRAPH-21-001 `Inspector linkouts` | BLOCKED (2025-10-27) | Excititor Core Guild, Cartographer Guild | EXCITITOR-POLICY-20-002, CARTO-GRAPH-21-005 | Provide batched VEX/advisory reference fetches keyed by graph node PURLs so UI inspector can display raw documents and justification metadata. | > 2025-10-27: Pending policy-driven linkset enrichment (`EXCITITOR-POLICY-20-002`) and Cartographer inspector contract (`CARTO-GRAPH-21-005`). No stable payload to target. > 2025-10-29: Handshake actions in `docs/dev/cartographer-graph-handshake.md` — draft batch linkout API skeleton + fixture plan once Cartographer delivers query patterns. | EXCITITOR-GRAPH-21-002 `Overlay enrichment` | BLOCKED (2025-10-27) | Excititor Core Guild | EXCITITOR-GRAPH-21-001, POLICY-ENGINE-30-001 | Ensure overlay metadata includes VEX justification summaries and document versions for Cartographer overlays; update fixtures/tests. | > 2025-10-27: Requires inspector linkouts (`EXCITITOR-GRAPH-21-001`) and Policy Engine overlay schema (`POLICY-ENGINE-30-001`) before enrichment can be implemented. > 2025-10-29: Align overlay schema work with the handshake doc once Policy Guild publishes the overlay additions; collect sample payloads for review. - -## Link-Not-Merge v1 - -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| EXCITITOR-LNM-21-001 `VEX observation model` | TODO | Excititor Core Guild | EXCITITOR-CORE-AOC-19-001 | Define immutable `vex_observations` schema capturing raw statements, product PURLs, justification, and AOC metadata. `DOCS-LNM-22-002` blocked pending this schema. | -| EXCITITOR-LNM-21-002 `Linkset correlator` | TODO | Excititor Core Guild | EXCITITOR-LNM-21-001 | Build correlation pipeline combining alias + product PURL signals to form `vex_linksets` with confidence metrics. Docs waiting to finalize VEX aggregation guide. | -| EXCITITOR-LNM-21-003 `Conflict annotator` | TODO | Excititor Core Guild | EXCITITOR-LNM-21-002 | Record status/justification disagreements within linksets and expose structured conflicts. Provide structured payloads for `DOCS-LNM-22-002`. | -| EXCITITOR-LNM-21-004 `Merge removal` | TODO | Excititor Core Guild | EXCITITOR-LNM-21-002 | Remove legacy VEX merge logic, enforce immutability, and add guards/tests to prevent future merges. | -| EXCITITOR-LNM-21-005 `Event emission` | TODO | Excititor Core Guild, Platform Events Guild | EXCITITOR-LNM-21-002 | Emit `vex.linkset.updated` events for downstream consumers with delta descriptions and tenant context. | - -## Policy Engine + Editor v1 - -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| EXCITITOR-POLICY-23-001 `Evidence indexes` | TODO | Excititor Core Guild | EXCITITOR-LNM-21-002 | Provide indexes/materialized views for policy runtime (status, justification, product PURL) to accelerate queries; document contract. | -| EXCITITOR-POLICY-23-002 `Event guarantees` | TODO | Excititor Core Guild, Platform Events Guild | EXCITITOR-LNM-21-005 | Ensure `vex.linkset.updated` events include correlation confidence, conflict summaries, and idempotent ids for evaluator consumption. | - -## Graph & Vuln Explorer v1 - -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| EXCITITOR-GRAPH-24-001 `VEX overlay inputs` | DONE (2025-10-29) | Excititor Core Guild | EXCITITOR-POLICY-23-001 | Expose raw VEX statements/linksets scoped for overlay services; no suppression/precedence logic in ingestion. | - -## Reachability v1 - -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| EXCITITOR-SIG-26-001 `Vendor exploitability hints` | TODO | Excititor Core Guild, Signals Guild | SIGNALS-24-004 | Surface vendor-provided exploitability indicators and affected symbol lists to Signals service via projection endpoints. | - -## Authority-Backed Scopes & Tenancy (Epic 14) -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| EXCITITOR-TEN-48-001 `Tenant-aware VEX linking` | TODO | Excititor Core Guild | AUTH-TEN-47-001 | Apply tenant context to VEX linkers, enable RLS, and expose capability endpoint confirming aggregation-only behavior. | - -## Observability & Forensics (Epic 15) -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| EXCITITOR-OBS-50-001 `Telemetry adoption` | TODO | Excititor Core Guild, Observability Guild | TELEMETRY-OBS-50-001, TELEMETRY-OBS-50-002 | Integrate telemetry core across VEX ingestion/linking, ensuring spans/logs capture tenant, product scope, upstream id, justification hash, and trace IDs. | -| EXCITITOR-OBS-51-001 `Metrics & SLOs` | TODO | Excititor Core Guild, DevOps Guild | EXCITITOR-OBS-50-001, TELEMETRY-OBS-51-001 | Publish metrics for VEX ingest latency, scope resolution success, conflict rate, signature verification failures. Define SLOs (link latency P95 <30s) and configure burn-rate alerts. | -| EXCITITOR-OBS-52-001 `Timeline events` | TODO | Excititor Core Guild | EXCITITOR-OBS-50-001, TIMELINE-OBS-52-002 | Emit `timeline_event` entries for VEX ingest/linking/outcome changes with trace IDs, justification summaries, and evidence placeholders. | -| EXCITITOR-OBS-53-001 `Evidence snapshots` | TODO | Excititor Core Guild, Evidence Locker Guild | EXCITITOR-OBS-52-001, EVID-OBS-53-002 | Build evidence payloads for VEX statements (raw doc, normalization diff, precedence notes) and push to evidence locker with Merkle manifests. | -| EXCITITOR-OBS-54-001 `Attestation & verification` | TODO | Excititor Core Guild, Provenance Guild | EXCITITOR-OBS-53-001, PROV-OBS-54-001 | Attach DSSE attestations to VEX batch processing, verify chain-of-custody via Provenance library, and link attestation IDs to timeline + ledger. | -| EXCITITOR-OBS-55-001 `Incident mode` | TODO | Excititor Core Guild, DevOps Guild | EXCITITOR-OBS-51-001, DEVOPS-OBS-55-001 | Implement incident sampling bump, additional raw payload retention, and activation events for VEX pipelines with redaction guard rails. | - -## Air-Gapped Mode (Epic 16) -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| EXCITITOR-AIRGAP-56-001 `Mirror ingestion adapters` | TODO | Excititor Core Guild | AIRGAP-IMP-57-002, MIRROR-CRT-56-001 | Add mirror-based VEX ingestion, preserving statement digests and bundle IDs. | -| EXCITITOR-AIRGAP-56-002 `Bundle provenance` | TODO | Excititor Core Guild, AirGap Importer Guild | EXCITITOR-AIRGAP-56-001, AIRGAP-IMP-57-001 | Persist bundle metadata on VEX observations/linksets with provenance references. | -| EXCITITOR-AIRGAP-57-001 `Sealed-mode enforcement` | TODO | Excititor Core Guild, AirGap Policy Guild | EXCITITOR-AIRGAP-56-001, AIRGAP-POL-56-001 | Block non-mirror connectors in sealed mode and surface remediation errors. | -| EXCITITOR-AIRGAP-57-002 `Staleness annotations` | TODO | Excititor Core Guild, AirGap Time Guild | EXCITITOR-AIRGAP-56-002, AIRGAP-TIME-58-001 | Annotate VEX statements with staleness metrics and expose via API. | -| EXCITITOR-AIRGAP-58-001 `Portable VEX evidence` | TODO | Excititor Core Guild, Evidence Locker Guild | EXCITITOR-OBS-53-001, EVID-OBS-54-001 | Package VEX evidence segments into portable evidence bundles linked to timeline. | - -## SDKs & OpenAPI (Epic 17) -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| EXCITITOR-OAS-61-001 `Spec coverage` | TODO | Excititor Core Guild, API Contracts Guild | OAS-61-001 | Update VEX OAS to include observation/linkset endpoints with provenance fields and examples. | -| EXCITITOR-OAS-61-002 `Example catalog` | TODO | Excititor Core Guild | EXCITITOR-OAS-61-001 | Provide examples for VEX justifications, statuses, conflicts; ensure SDK docs reference them. | -| EXCITITOR-OAS-62-001 `SDK smoke tests` | TODO | Excititor Core Guild, SDK Generator Guild | EXCITITOR-OAS-61-001, SDKGEN-63-001 | Add SDK scenarios for VEX observation queries and conflict handling to language smoke suites. | -| EXCITITOR-OAS-63-001 `Deprecation headers` | TODO | Excititor Core Guild, API Governance Guild | APIGOV-63-001 | Add deprecation metadata and notifications for legacy VEX routes. | - -## Risk Profiles (Epic 18) -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| EXCITITOR-RISK-66-001 `VEX gate provider` | TODO | Excititor Core Guild, Risk Engine Guild | RISK-ENGINE-67-002 | Supply VEX status and justification data for risk engine gating with full source provenance. | -| EXCITITOR-RISK-66-002 `Reachability inputs` | TODO | Excititor Core Guild | EXCITITOR-RISK-66-001 | Provide component/product scoping metadata enabling reachability and runtime factor mapping. | -| EXCITITOR-RISK-67-001 `Explainability metadata` | TODO | Excititor Core Guild | EXCITITOR-RISK-66-001 | Include VEX justification, status reasoning, and source digests in explainability artifacts. | -| EXCITITOR-RISK-68-001 `Policy Studio integration` | TODO | Excititor Core Guild, Policy Studio Guild | POLICY-RISK-68-001 | Surface VEX-specific gates/weights within profile editor UI and validation messages. | - -## Attestor Console (Epic 19) -| ID | Status | Owner(s) | Depends on | Notes | -|----|--------|----------|------------|-------| -| EXCITITOR-ATTEST-73-001 `VEX attestation payloads` | TODO | Excititor Core Guild, Attestation Payloads Guild | ATTEST-TYPES-72-001 | Provide VEX statement metadata (supplier identity, justification, scope) required for VEXAttestation payloads. | -| EXCITITOR-ATTEST-73-002 `Chain provenance` | TODO | Excititor Core Guild | EXCITITOR-ATTEST-73-001 | Expose linkage from VEX statements to subject/product for chain of custody graph. | + +## Link-Not-Merge v1 + +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| EXCITITOR-LNM-21-001 `VEX observation model` | TODO | Excititor Core Guild | EXCITITOR-CORE-AOC-19-001 | Define immutable `vex_observations` schema capturing raw statements, product PURLs, justification, and AOC metadata. `DOCS-LNM-22-002` blocked pending this schema. | +| EXCITITOR-LNM-21-002 `Linkset correlator` | TODO | Excititor Core Guild | EXCITITOR-LNM-21-001 | Build correlation pipeline combining alias + product PURL signals to form `vex_linksets` with confidence metrics. Docs waiting to finalize VEX aggregation guide. | +| EXCITITOR-LNM-21-003 `Conflict annotator` | TODO | Excititor Core Guild | EXCITITOR-LNM-21-002 | Record status/justification disagreements within linksets and expose structured conflicts. Provide structured payloads for `DOCS-LNM-22-002`. | +| EXCITITOR-LNM-21-004 `Merge removal` | TODO | Excititor Core Guild | EXCITITOR-LNM-21-002 | Remove legacy VEX merge logic, enforce immutability, and add guards/tests to prevent future merges. | +| EXCITITOR-LNM-21-005 `Event emission` | TODO | Excititor Core Guild, Platform Events Guild | EXCITITOR-LNM-21-002 | Emit `vex.linkset.updated` events for downstream consumers with delta descriptions and tenant context. | + +## Policy Engine + Editor v1 + +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| EXCITITOR-POLICY-23-001 `Evidence indexes` | TODO | Excititor Core Guild | EXCITITOR-LNM-21-002 | Provide indexes/materialized views for policy runtime (status, justification, product PURL) to accelerate queries; document contract. | +| EXCITITOR-POLICY-23-002 `Event guarantees` | TODO | Excititor Core Guild, Platform Events Guild | EXCITITOR-LNM-21-005 | Ensure `vex.linkset.updated` events include correlation confidence, conflict summaries, and idempotent ids for evaluator consumption. | + +## Graph & Vuln Explorer v1 + +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| + +## Reachability v1 + +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| EXCITITOR-SIG-26-001 `Vendor exploitability hints` | TODO | Excititor Core Guild, Signals Guild | SIGNALS-24-004 | Surface vendor-provided exploitability indicators and affected symbol lists to Signals service via projection endpoints. | + +## Authority-Backed Scopes & Tenancy (Epic 14) +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| EXCITITOR-TEN-48-001 `Tenant-aware VEX linking` | TODO | Excititor Core Guild | AUTH-TEN-47-001 | Apply tenant context to VEX linkers, enable RLS, and expose capability endpoint confirming aggregation-only behavior. | + +## Observability & Forensics (Epic 15) +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| EXCITITOR-OBS-50-001 `Telemetry adoption` | TODO | Excititor Core Guild, Observability Guild | TELEMETRY-OBS-50-001, TELEMETRY-OBS-50-002 | Integrate telemetry core across VEX ingestion/linking, ensuring spans/logs capture tenant, product scope, upstream id, justification hash, and trace IDs. | +| EXCITITOR-OBS-51-001 `Metrics & SLOs` | TODO | Excititor Core Guild, DevOps Guild | EXCITITOR-OBS-50-001, TELEMETRY-OBS-51-001 | Publish metrics for VEX ingest latency, scope resolution success, conflict rate, signature verification failures. Define SLOs (link latency P95 <30s) and configure burn-rate alerts. | +| EXCITITOR-OBS-52-001 `Timeline events` | TODO | Excititor Core Guild | EXCITITOR-OBS-50-001, TIMELINE-OBS-52-002 | Emit `timeline_event` entries for VEX ingest/linking/outcome changes with trace IDs, justification summaries, and evidence placeholders. | +| EXCITITOR-OBS-53-001 `Evidence snapshots` | TODO | Excititor Core Guild, Evidence Locker Guild | EXCITITOR-OBS-52-001, EVID-OBS-53-002 | Build evidence payloads for VEX statements (raw doc, normalization diff, precedence notes) and push to evidence locker with Merkle manifests. | +| EXCITITOR-OBS-54-001 `Attestation & verification` | TODO | Excititor Core Guild, Provenance Guild | EXCITITOR-OBS-53-001, PROV-OBS-54-001 | Attach DSSE attestations to VEX batch processing, verify chain-of-custody via Provenance library, and link attestation IDs to timeline + ledger. | +| EXCITITOR-OBS-55-001 `Incident mode` | TODO | Excititor Core Guild, DevOps Guild | EXCITITOR-OBS-51-001, DEVOPS-OBS-55-001 | Implement incident sampling bump, additional raw payload retention, and activation events for VEX pipelines with redaction guard rails. | + +## Air-Gapped Mode (Epic 16) +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| EXCITITOR-AIRGAP-56-001 `Mirror ingestion adapters` | TODO | Excititor Core Guild | AIRGAP-IMP-57-002, MIRROR-CRT-56-001 | Add mirror-based VEX ingestion, preserving statement digests and bundle IDs. | +| EXCITITOR-AIRGAP-56-002 `Bundle provenance` | TODO | Excititor Core Guild, AirGap Importer Guild | EXCITITOR-AIRGAP-56-001, AIRGAP-IMP-57-001 | Persist bundle metadata on VEX observations/linksets with provenance references. | +| EXCITITOR-AIRGAP-57-001 `Sealed-mode enforcement` | TODO | Excititor Core Guild, AirGap Policy Guild | EXCITITOR-AIRGAP-56-001, AIRGAP-POL-56-001 | Block non-mirror connectors in sealed mode and surface remediation errors. | +| EXCITITOR-AIRGAP-57-002 `Staleness annotations` | TODO | Excititor Core Guild, AirGap Time Guild | EXCITITOR-AIRGAP-56-002, AIRGAP-TIME-58-001 | Annotate VEX statements with staleness metrics and expose via API. | +| EXCITITOR-AIRGAP-58-001 `Portable VEX evidence` | TODO | Excititor Core Guild, Evidence Locker Guild | EXCITITOR-OBS-53-001, EVID-OBS-54-001 | Package VEX evidence segments into portable evidence bundles linked to timeline. | + +## SDKs & OpenAPI (Epic 17) +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| EXCITITOR-OAS-61-001 `Spec coverage` | TODO | Excititor Core Guild, API Contracts Guild | OAS-61-001 | Update VEX OAS to include observation/linkset endpoints with provenance fields and examples. | +| EXCITITOR-OAS-61-002 `Example catalog` | TODO | Excititor Core Guild | EXCITITOR-OAS-61-001 | Provide examples for VEX justifications, statuses, conflicts; ensure SDK docs reference them. | +| EXCITITOR-OAS-62-001 `SDK smoke tests` | TODO | Excititor Core Guild, SDK Generator Guild | EXCITITOR-OAS-61-001, SDKGEN-63-001 | Add SDK scenarios for VEX observation queries and conflict handling to language smoke suites. | +| EXCITITOR-OAS-63-001 `Deprecation headers` | TODO | Excititor Core Guild, API Governance Guild | APIGOV-63-001 | Add deprecation metadata and notifications for legacy VEX routes. | + +## Risk Profiles (Epic 18) +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| EXCITITOR-RISK-66-001 `VEX gate provider` | TODO | Excititor Core Guild, Risk Engine Guild | RISK-ENGINE-67-002 | Supply VEX status and justification data for risk engine gating with full source provenance. | +| EXCITITOR-RISK-66-002 `Reachability inputs` | TODO | Excititor Core Guild | EXCITITOR-RISK-66-001 | Provide component/product scoping metadata enabling reachability and runtime factor mapping. | +| EXCITITOR-RISK-67-001 `Explainability metadata` | TODO | Excititor Core Guild | EXCITITOR-RISK-66-001 | Include VEX justification, status reasoning, and source digests in explainability artifacts. | +| EXCITITOR-RISK-68-001 `Policy Studio integration` | TODO | Excititor Core Guild, Policy Studio Guild | POLICY-RISK-68-001 | Surface VEX-specific gates/weights within profile editor UI and validation messages. | + +## Attestor Console (Epic 19) +| ID | Status | Owner(s) | Depends on | Notes | +|----|--------|----------|------------|-------| +| EXCITITOR-ATTEST-73-001 `VEX attestation payloads` | TODO | Excititor Core Guild, Attestation Payloads Guild | ATTEST-TYPES-72-001 | Provide VEX statement metadata (supplier identity, justification, scope) required for VEXAttestation payloads. | +| EXCITITOR-ATTEST-73-002 `Chain provenance` | TODO | Excititor Core Guild | EXCITITOR-ATTEST-73-001 | Expose linkage from VEX statements to subject/product for chain of custody graph. | diff --git a/src/Excititor/__Libraries/StellaOps.Excititor.Export/TASKS.completed.md b/src/Excititor/__Libraries/StellaOps.Excititor.Export/TASKS.completed.md new file mode 100644 index 00000000..df5a70ec --- /dev/null +++ b/src/Excititor/__Libraries/StellaOps.Excititor.Export/TASKS.completed.md @@ -0,0 +1,16 @@ +# Completed Tasks + +|EXCITITOR-EXPORT-01-001 – Export engine orchestration|Team Excititor Export|EXCITITOR-CORE-01-003|DONE (2025-10-15) – Export engine scaffolding with cache lookup, data source hooks, and deterministic manifest emission.| + +|EXCITITOR-EXPORT-01-002 – Cache index & eviction hooks|Team Excititor Export|EXCITITOR-EXPORT-01-001, EXCITITOR-STORAGE-01-003|**DONE (2025-10-16)** – Export engine now invalidates cache entries on force refresh, cache services expose prune/invalidate APIs, and storage maintenance trims expired/dangling records with Mongo2Go coverage.| + +|EXCITITOR-EXPORT-01-003 – Artifact store adapters|Team Excititor Export|EXCITITOR-EXPORT-01-001|**DONE (2025-10-16)** – Implemented multi-store pipeline with filesystem, S3-compatible, and offline bundle adapters (hash verification + manifest/zip output) plus unit coverage and DI hooks.| + +|EXCITITOR-EXPORT-01-004 – Attestation handoff integration|Team Excititor Export|EXCITITOR-EXPORT-01-001, EXCITITOR-ATTEST-01-001|**DONE (2025-10-17)** – Export engine now invokes attestation client, logs diagnostics, and persists Rekor/envelope metadata on manifests; regression coverage added in `ExportEngineTests.ExportAsync_AttachesAttestationMetadata`.| + +|EXCITITOR-EXPORT-01-005 – Score & resolve envelope surfaces|Team Excititor Export|EXCITITOR-EXPORT-01-004, EXCITITOR-CORE-02-001|**DONE (2025-10-21)** – Export engine now canonicalizes consensus/score envelopes, persists their SHA-256 digests into manifests/attestation metadata, and regression tests validate metadata wiring via `ExportEngineTests`.| + +|EXCITITOR-EXPORT-01-006 – Quiet provenance packaging|Team Excititor Export|EXCITITOR-EXPORT-01-005, POLICY-CORE-09-005|**DONE (2025-10-21)** – Export manifests now carry quiet-provenance entries (statement digests, signers, justification codes); metadata flows into offline bundles & attestations with regression coverage in `ExportEngineTests`.| + +|EXCITITOR-EXPORT-01-007 – Mirror bundle + domain manifest|Team Excititor Export|EXCITITOR-EXPORT-01-006|**DONE (2025-10-21)** – Created per-domain mirror bundles with consensus/score artefacts, published signed-ready manifests/index for downstream Excititor sync, and added regression coverage.| + diff --git a/src/Excititor/__Libraries/StellaOps.Excititor.Export/TASKS.md b/src/Excititor/__Libraries/StellaOps.Excititor.Export/TASKS.md index 8070135c..5a0de659 100644 --- a/src/Excititor/__Libraries/StellaOps.Excititor.Export/TASKS.md +++ b/src/Excititor/__Libraries/StellaOps.Excititor.Export/TASKS.md @@ -2,10 +2,3 @@ If you are working on this file you need to read docs/modules/excititor/ARCHITEC # TASKS | Task | Owner(s) | Depends on | Notes | |---|---|---|---| -|EXCITITOR-EXPORT-01-001 – Export engine orchestration|Team Excititor Export|EXCITITOR-CORE-01-003|DONE (2025-10-15) – Export engine scaffolding with cache lookup, data source hooks, and deterministic manifest emission.| -|EXCITITOR-EXPORT-01-002 – Cache index & eviction hooks|Team Excititor Export|EXCITITOR-EXPORT-01-001, EXCITITOR-STORAGE-01-003|**DONE (2025-10-16)** – Export engine now invalidates cache entries on force refresh, cache services expose prune/invalidate APIs, and storage maintenance trims expired/dangling records with Mongo2Go coverage.| -|EXCITITOR-EXPORT-01-003 – Artifact store adapters|Team Excititor Export|EXCITITOR-EXPORT-01-001|**DONE (2025-10-16)** – Implemented multi-store pipeline with filesystem, S3-compatible, and offline bundle adapters (hash verification + manifest/zip output) plus unit coverage and DI hooks.| -|EXCITITOR-EXPORT-01-004 – Attestation handoff integration|Team Excititor Export|EXCITITOR-EXPORT-01-001, EXCITITOR-ATTEST-01-001|**DONE (2025-10-17)** – Export engine now invokes attestation client, logs diagnostics, and persists Rekor/envelope metadata on manifests; regression coverage added in `ExportEngineTests.ExportAsync_AttachesAttestationMetadata`.| -|EXCITITOR-EXPORT-01-005 – Score & resolve envelope surfaces|Team Excititor Export|EXCITITOR-EXPORT-01-004, EXCITITOR-CORE-02-001|**DONE (2025-10-21)** – Export engine now canonicalizes consensus/score envelopes, persists their SHA-256 digests into manifests/attestation metadata, and regression tests validate metadata wiring via `ExportEngineTests`.| -|EXCITITOR-EXPORT-01-006 – Quiet provenance packaging|Team Excititor Export|EXCITITOR-EXPORT-01-005, POLICY-CORE-09-005|**DONE (2025-10-21)** – Export manifests now carry quiet-provenance entries (statement digests, signers, justification codes); metadata flows into offline bundles & attestations with regression coverage in `ExportEngineTests`.| -|EXCITITOR-EXPORT-01-007 – Mirror bundle + domain manifest|Team Excititor Export|EXCITITOR-EXPORT-01-006|**DONE (2025-10-21)** – Created per-domain mirror bundles with consensus/score artefacts, published signed-ready manifests/index for downstream Excititor sync, and added regression coverage.| diff --git a/src/Excititor/__Libraries/StellaOps.Excititor.Formats.CSAF/TASKS.completed.md b/src/Excititor/__Libraries/StellaOps.Excititor.Formats.CSAF/TASKS.completed.md new file mode 100644 index 00000000..6b8954e3 --- /dev/null +++ b/src/Excititor/__Libraries/StellaOps.Excititor.Formats.CSAF/TASKS.completed.md @@ -0,0 +1,8 @@ +# Completed Tasks + +|EXCITITOR-FMT-CSAF-01-001 – CSAF normalizer foundation|Team Excititor Formats|EXCITITOR-CORE-01-001|**DONE (2025-10-17)** – Implemented CSAF normalizer + DI hook, parsing tracking metadata, product tree branches/full names, and mapping product statuses into canonical `VexClaim`s with baseline precedence. Regression added in `CsafNormalizerTests`.| + +|EXCITITOR-FMT-CSAF-01-002 – Status/justification mapping|Team Excititor Formats|EXCITITOR-FMT-CSAF-01-001, EXCITITOR-POLICY-01-001|**DONE (2025-10-29)** – Added policy-aligned diagnostics for unsupported statuses/justifications and flagged missing not_affected evidence inside normalizer outputs.| + +|EXCITITOR-FMT-CSAF-01-003 – CSAF export adapter|Team Excititor Formats|EXCITITOR-EXPORT-01-001, EXCITITOR-FMT-CSAF-01-001|**DONE (2025-10-29)** – Implemented deterministic CSAF exporter with product tree reconciliation, vulnerability status mapping, and metadata for downstream attestation.| + diff --git a/src/Excititor/__Libraries/StellaOps.Excititor.Formats.CSAF/TASKS.md b/src/Excititor/__Libraries/StellaOps.Excititor.Formats.CSAF/TASKS.md index bbb8d361..5a0de659 100644 --- a/src/Excititor/__Libraries/StellaOps.Excititor.Formats.CSAF/TASKS.md +++ b/src/Excititor/__Libraries/StellaOps.Excititor.Formats.CSAF/TASKS.md @@ -2,6 +2,3 @@ If you are working on this file you need to read docs/modules/excititor/ARCHITEC # TASKS | Task | Owner(s) | Depends on | Notes | |---|---|---|---| -|EXCITITOR-FMT-CSAF-01-001 – CSAF normalizer foundation|Team Excititor Formats|EXCITITOR-CORE-01-001|**DONE (2025-10-17)** – Implemented CSAF normalizer + DI hook, parsing tracking metadata, product tree branches/full names, and mapping product statuses into canonical `VexClaim`s with baseline precedence. Regression added in `CsafNormalizerTests`.| -|EXCITITOR-FMT-CSAF-01-002 – Status/justification mapping|Team Excititor Formats|EXCITITOR-FMT-CSAF-01-001, EXCITITOR-POLICY-01-001|**DONE (2025-10-29)** – Added policy-aligned diagnostics for unsupported statuses/justifications and flagged missing not_affected evidence inside normalizer outputs.| -|EXCITITOR-FMT-CSAF-01-003 – CSAF export adapter|Team Excititor Formats|EXCITITOR-EXPORT-01-001, EXCITITOR-FMT-CSAF-01-001|**DONE (2025-10-29)** – Implemented deterministic CSAF exporter with product tree reconciliation, vulnerability status mapping, and metadata for downstream attestation.| diff --git a/src/Excititor/__Libraries/StellaOps.Excititor.Formats.CycloneDX/TASKS.completed.md b/src/Excititor/__Libraries/StellaOps.Excititor.Formats.CycloneDX/TASKS.completed.md new file mode 100644 index 00000000..0e657108 --- /dev/null +++ b/src/Excititor/__Libraries/StellaOps.Excititor.Formats.CycloneDX/TASKS.completed.md @@ -0,0 +1,8 @@ +# Completed Tasks + +|EXCITITOR-FMT-CYCLONE-01-001 – CycloneDX VEX normalizer|Team Excititor Formats|EXCITITOR-CORE-01-001|**DONE (2025-10-17)** – CycloneDX normalizer parses `analysis` data, resolves component references, and emits canonical `VexClaim`s; regression lives in `CycloneDxNormalizerTests`.| + +|EXCITITOR-FMT-CYCLONE-01-002 – Component reference reconciliation|Team Excititor Formats|EXCITITOR-FMT-CYCLONE-01-001|**DONE (2025-10-29)** – Added reconciler producing stable bom-refs, aggregating component metadata, and reporting missing PURL diagnostics for policy gating.| + +|EXCITITOR-FMT-CYCLONE-01-003 – CycloneDX export serializer|Team Excititor Formats|EXCITITOR-EXPORT-01-001, EXCITITOR-FMT-CYCLONE-01-001|**DONE (2025-10-29)** – Implemented CycloneDX VEX exporter emitting reconciled components, vulnerability analysis blocks, and canonical metadata.| + diff --git a/src/Excititor/__Libraries/StellaOps.Excititor.Formats.CycloneDX/TASKS.md b/src/Excititor/__Libraries/StellaOps.Excititor.Formats.CycloneDX/TASKS.md index 7b148289..5a0de659 100644 --- a/src/Excititor/__Libraries/StellaOps.Excititor.Formats.CycloneDX/TASKS.md +++ b/src/Excititor/__Libraries/StellaOps.Excititor.Formats.CycloneDX/TASKS.md @@ -2,6 +2,3 @@ If you are working on this file you need to read docs/modules/excititor/ARCHITEC # TASKS | Task | Owner(s) | Depends on | Notes | |---|---|---|---| -|EXCITITOR-FMT-CYCLONE-01-001 – CycloneDX VEX normalizer|Team Excititor Formats|EXCITITOR-CORE-01-001|**DONE (2025-10-17)** – CycloneDX normalizer parses `analysis` data, resolves component references, and emits canonical `VexClaim`s; regression lives in `CycloneDxNormalizerTests`.| -|EXCITITOR-FMT-CYCLONE-01-002 – Component reference reconciliation|Team Excititor Formats|EXCITITOR-FMT-CYCLONE-01-001|**DONE (2025-10-29)** – Added reconciler producing stable bom-refs, aggregating component metadata, and reporting missing PURL diagnostics for policy gating.| -|EXCITITOR-FMT-CYCLONE-01-003 – CycloneDX export serializer|Team Excititor Formats|EXCITITOR-EXPORT-01-001, EXCITITOR-FMT-CYCLONE-01-001|**DONE (2025-10-29)** – Implemented CycloneDX VEX exporter emitting reconciled components, vulnerability analysis blocks, and canonical metadata.| diff --git a/src/Excititor/__Libraries/StellaOps.Excititor.Formats.OpenVEX/TASKS.completed.md b/src/Excititor/__Libraries/StellaOps.Excititor.Formats.OpenVEX/TASKS.completed.md new file mode 100644 index 00000000..7e4955f2 --- /dev/null +++ b/src/Excititor/__Libraries/StellaOps.Excititor.Formats.OpenVEX/TASKS.completed.md @@ -0,0 +1,8 @@ +# Completed Tasks + +|EXCITITOR-FMT-OPENVEX-01-001 – OpenVEX normalizer|Team Excititor Formats|EXCITITOR-CORE-01-001|**DONE (2025-10-17)** – OpenVEX normalizer parses statements/products, maps status/justification, and surfaces provenance metadata; coverage in `OpenVexNormalizerTests`.| + +|EXCITITOR-FMT-OPENVEX-01-002 – Statement merge utilities|Team Excititor Formats|EXCITITOR-FMT-OPENVEX-01-001|**DONE (2025-10-29)** – Delivered deterministic statement merger prioritising risk status, preserving source provenance, and surfacing conflict diagnostics.| + +|EXCITITOR-FMT-OPENVEX-01-003 – OpenVEX export writer|Team Excititor Formats|EXCITITOR-EXPORT-01-001, EXCITITOR-FMT-OPENVEX-01-001|**DONE (2025-10-29)** – Shipped canonical OpenVEX exporter emitting merged statements, metadata, and stable digests for attested distribution.| + diff --git a/src/Excititor/__Libraries/StellaOps.Excititor.Formats.OpenVEX/TASKS.md b/src/Excititor/__Libraries/StellaOps.Excititor.Formats.OpenVEX/TASKS.md index 6a84726a..5a0de659 100644 --- a/src/Excititor/__Libraries/StellaOps.Excititor.Formats.OpenVEX/TASKS.md +++ b/src/Excititor/__Libraries/StellaOps.Excititor.Formats.OpenVEX/TASKS.md @@ -2,6 +2,3 @@ If you are working on this file you need to read docs/modules/excititor/ARCHITEC # TASKS | Task | Owner(s) | Depends on | Notes | |---|---|---|---| -|EXCITITOR-FMT-OPENVEX-01-001 – OpenVEX normalizer|Team Excititor Formats|EXCITITOR-CORE-01-001|**DONE (2025-10-17)** – OpenVEX normalizer parses statements/products, maps status/justification, and surfaces provenance metadata; coverage in `OpenVexNormalizerTests`.| -|EXCITITOR-FMT-OPENVEX-01-002 – Statement merge utilities|Team Excititor Formats|EXCITITOR-FMT-OPENVEX-01-001|**DONE (2025-10-29)** – Delivered deterministic statement merger prioritising risk status, preserving source provenance, and surfacing conflict diagnostics.| -|EXCITITOR-FMT-OPENVEX-01-003 – OpenVEX export writer|Team Excititor Formats|EXCITITOR-EXPORT-01-001, EXCITITOR-FMT-OPENVEX-01-001|**DONE (2025-10-29)** – Shipped canonical OpenVEX exporter emitting merged statements, metadata, and stable digests for attested distribution.| diff --git a/src/Excititor/__Libraries/StellaOps.Excititor.Policy/TASKS.completed.md b/src/Excititor/__Libraries/StellaOps.Excititor.Policy/TASKS.completed.md new file mode 100644 index 00000000..b5c2eba1 --- /dev/null +++ b/src/Excititor/__Libraries/StellaOps.Excititor.Policy/TASKS.completed.md @@ -0,0 +1,14 @@ +# Completed Tasks + +|EXCITITOR-POLICY-01-001 – Policy schema & binding|Team Excititor Policy|EXCITITOR-CORE-01-001|DONE (2025-10-15) – Established `VexPolicyOptions`, options binding, and snapshot provider covering baseline weights/overrides.| + +|EXCITITOR-POLICY-01-002 – Policy evaluator service|Team Excititor Policy|EXCITITOR-POLICY-01-001|DONE (2025-10-15) – `VexPolicyEvaluator` exposes immutable snapshots to consensus and normalizes rejection reasons.| + +|EXCITITOR-POLICY-01-003 – Operator diagnostics & docs|Team Excititor Policy|EXCITITOR-POLICY-01-001|**DONE (2025-10-16)** – Surface structured diagnostics (CLI/WebService) and author policy upgrade guidance in docs/modules/excititor/ARCHITECTURE.md appendix.
2025-10-16: Added `IVexPolicyDiagnostics`/`VexPolicyDiagnosticsReport`, sorted issue ordering, recommendations, and appendix guidance. Tests: `dotnet test src/Excititor/__Tests/StellaOps.Excititor.Core.Tests/StellaOps.Excititor.Core.Tests.csproj`.| + +|EXCITITOR-POLICY-01-004 – Policy schema validation & YAML binding|Team Excititor Policy|EXCITITOR-POLICY-01-001|**DONE (2025-10-16)** – Added strongly-typed YAML/JSON binding, schema validation, and deterministic diagnostics for operator-supplied policy bundles.| + +|EXCITITOR-POLICY-01-005 – Policy change tracking & telemetry|Team Excititor Policy|EXCITITOR-POLICY-01-002|**DONE (2025-10-16)** – Emit revision history, expose snapshot digests via CLI/WebService, and add structured logging/metrics for policy reloads.
2025-10-16: `VexPolicySnapshot` now carries revision/digest, provider logs reloads, `vex.policy.reloads` metric emitted, binder/diagnostics expose digest metadata. Tests: `dotnet test src/Excititor/__Tests/StellaOps.Excititor.Core.Tests/StellaOps.Excititor.Core.Tests.csproj`.| + +|EXCITITOR-POLICY-02-001 – Scoring coefficients & weight ceilings|Team Excititor Policy|EXCITITOR-POLICY-01-004|DONE (2025-10-19) – Added `weights.ceiling` + `scoring.{alpha,beta}` options with normalization warnings, extended consensus policy/digest, refreshed docs (`docs/modules/excititor/ARCHITECTURE.md`, `docs/modules/excititor/scoring.md`), and validated via `dotnet test` for core/policy suites.| + diff --git a/src/Excititor/__Libraries/StellaOps.Excititor.Policy/TASKS.md b/src/Excititor/__Libraries/StellaOps.Excititor.Policy/TASKS.md index 438f68fe..6422d4e3 100644 --- a/src/Excititor/__Libraries/StellaOps.Excititor.Policy/TASKS.md +++ b/src/Excititor/__Libraries/StellaOps.Excititor.Policy/TASKS.md @@ -2,10 +2,4 @@ If you are working on this file you need to read docs/modules/excititor/ARCHITEC # TASKS | Task | Owner(s) | Depends on | Notes | |---|---|---|---| -|EXCITITOR-POLICY-01-001 – Policy schema & binding|Team Excititor Policy|EXCITITOR-CORE-01-001|DONE (2025-10-15) – Established `VexPolicyOptions`, options binding, and snapshot provider covering baseline weights/overrides.| -|EXCITITOR-POLICY-01-002 – Policy evaluator service|Team Excititor Policy|EXCITITOR-POLICY-01-001|DONE (2025-10-15) – `VexPolicyEvaluator` exposes immutable snapshots to consensus and normalizes rejection reasons.| -|EXCITITOR-POLICY-01-003 – Operator diagnostics & docs|Team Excititor Policy|EXCITITOR-POLICY-01-001|**DONE (2025-10-16)** – Surface structured diagnostics (CLI/WebService) and author policy upgrade guidance in docs/modules/excititor/ARCHITECTURE.md appendix.
2025-10-16: Added `IVexPolicyDiagnostics`/`VexPolicyDiagnosticsReport`, sorted issue ordering, recommendations, and appendix guidance. Tests: `dotnet test src/Excititor/__Tests/StellaOps.Excititor.Core.Tests/StellaOps.Excititor.Core.Tests.csproj`.| -|EXCITITOR-POLICY-01-004 – Policy schema validation & YAML binding|Team Excititor Policy|EXCITITOR-POLICY-01-001|**DONE (2025-10-16)** – Added strongly-typed YAML/JSON binding, schema validation, and deterministic diagnostics for operator-supplied policy bundles.| -|EXCITITOR-POLICY-01-005 – Policy change tracking & telemetry|Team Excititor Policy|EXCITITOR-POLICY-01-002|**DONE (2025-10-16)** – Emit revision history, expose snapshot digests via CLI/WebService, and add structured logging/metrics for policy reloads.
2025-10-16: `VexPolicySnapshot` now carries revision/digest, provider logs reloads, `vex.policy.reloads` metric emitted, binder/diagnostics expose digest metadata. Tests: `dotnet test src/Excititor/__Tests/StellaOps.Excititor.Core.Tests/StellaOps.Excititor.Core.Tests.csproj`.| -|EXCITITOR-POLICY-02-001 – Scoring coefficients & weight ceilings|Team Excititor Policy|EXCITITOR-POLICY-01-004|DONE (2025-10-19) – Added `weights.ceiling` + `scoring.{alpha,beta}` options with normalization warnings, extended consensus policy/digest, refreshed docs (`docs/modules/excititor/ARCHITECTURE.md`, `docs/modules/excititor/scoring.md`), and validated via `dotnet test` for core/policy suites.| |EXCITITOR-POLICY-02-002 – Diagnostics for scoring signals|Team Excititor Policy|EXCITITOR-POLICY-02-001|BACKLOG – Update diagnostics reports to surface missing severity/KEV/EPSS mappings, coefficient overrides, and provide actionable recommendations for policy tuning.| diff --git a/src/Notifier/StellaOps.Notifier/TASKS.completed.md b/src/Notifier/StellaOps.Notifier/TASKS.completed.md new file mode 100644 index 00000000..f8160cfc --- /dev/null +++ b/src/Notifier/StellaOps.Notifier/TASKS.completed.md @@ -0,0 +1,5 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| NOTIFY-SVC-38-001 | DONE (2025-10-29) | Notifications Service Guild | ORCH-SVC-38-101, AUTH-NOTIFY-38-001 | Bootstrap notifier service, DB migrations (`notif_*` tables), event ingestion consumer with idempotency, and baseline rule/routing engine for policy violations + job failures. | Service builds/tests; migrations scripted; ingestion handles orchestrator events; initial rules evaluated deterministically; compliance checklist recorded. | diff --git a/src/Notifier/StellaOps.Notifier/TASKS.md b/src/Notifier/StellaOps.Notifier/TASKS.md index d62a55bb..040313fd 100644 --- a/src/Notifier/StellaOps.Notifier/TASKS.md +++ b/src/Notifier/StellaOps.Notifier/TASKS.md @@ -1,74 +1,73 @@ -# Notifier Service Task Board — Epic 11: Notifications Studio - -# Sprint 37 – Pack Approval Bridge (Task Runner integration) -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| NOTIFY-SVC-37-001 | TODO | Notifications Service Guild | TASKRUN-43-001 | Define pack approval & policy notification contract, including OpenAPI schema, event payloads, resume token mechanics, and security guidance. | Requirements doc published (`docs/notifications/pack-approvals-integration.md`), OpenAPI fragment merged, reviewers sign off from Task Runner & Authority guilds. | -| NOTIFY-SVC-37-002 | TODO | Notifications Service Guild | NOTIFY-SVC-37-001 | Implement secure ingestion endpoint, Mongo persistence (`pack_approvals`), idempotent writes, and audit trail for approval events. | Endpoint authenticated/authorized, persistence migrations merged, integration tests cover happy/error paths, audit log samples recorded. | -| NOTIFY-SVC-37-003 | TODO | Notifications Service Guild | NOTIFY-SVC-37-001 | Deliver approval/policy templates, routing predicates, and channel dispatch (email + webhook) with localization + redaction. | Templates rendered, routing rules active, localization fallback tested, sample notifications archived. | -| NOTIFY-SVC-37-004 | TODO | Notifications Service Guild | NOTIFY-SVC-37-002 | Provide acknowledgement API, Task Runner callback client, metrics for outstanding approvals, and runbook updates. | Ack endpoint live, resume callback validated with Task Runner simulator, metrics/dashboards in place, runbook entry updated. | - -## Sprint 38 – Foundations (Immediate notifications) -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| NOTIFY-SVC-38-001 | DONE (2025-10-29) | Notifications Service Guild | ORCH-SVC-38-101, AUTH-NOTIFY-38-001 | Bootstrap notifier service, DB migrations (`notif_*` tables), event ingestion consumer with idempotency, and baseline rule/routing engine for policy violations + job failures. | Service builds/tests; migrations scripted; ingestion handles orchestrator events; initial rules evaluated deterministically; compliance checklist recorded. | -> 2025-10-29: Worker/WebService now compose `StellaOps.Notify.Storage.Mongo` + `StellaOps.Notify.Queue`, with a default rule evaluator and idempotent delivery ledger. See `docs/NOTIFY-SVC-38-001-FOUNDATIONS.md` for implementation notes and follow-ups. -| NOTIFY-SVC-38-002 | TODO | Notifications Service Guild | NOTIFY-SVC-38-001 | Implement channel adapters (email, chat webhook, generic webhook) with retry policies, health checks, and audit logging. | Adapters send test notifications; retries/backoff validated; health endpoints available; audit logs captured. | -| NOTIFY-SVC-38-003 | TODO | Notifications Service Guild | NOTIFY-SVC-38-001 | Deliver template service (versioned templates, localization scaffolding) and renderer with redaction allowlists, Markdown/HTML/JSON outputs, and provenance links. | Templates versioned; preview API works; rendered content includes provenance; redaction tests pass. | -| NOTIFY-SVC-38-004 | TODO | Notifications Service Guild | NOTIFY-SVC-38-001..003 | Expose REST + WS APIs (rules CRUD, templates preview, incidents list, ack) with audit logging, RBAC checks, and live feed stream. | OpenAPI published; WS feed delivers events; ack endpoint updates state; tests cover RBAC and audit logs. | - -## Sprint 39 – Correlation, Digests, Simulation -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| NOTIFY-SVC-39-001 | TODO | Notifications Service Guild | NOTIFY-SVC-38-004 | Implement correlation engine with pluggable key expressions/windows, throttler (token buckets), quiet hours/maintenance evaluator, and incident lifecycle. | Correlation merges duplicates; throttling enforced; quiet hours respect tenant schedules; incident state transitions tested. | -| NOTIFY-SVC-39-002 | TODO | Notifications Service Guild | NOTIFY-SVC-39-001, LEDGER-NOTIFY-39-001 | Build digest generator (queries, formatting) with schedule runner and distribution via existing channels. | Digests generated on schedule; content accurate; provenance linked; metrics emitted. | -| NOTIFY-SVC-39-003 | TODO | Notifications Service Guild | NOTIFY-SVC-39-001 | Provide simulation engine/API to dry-run rules against historical events, returning matched actions with explanations. | Simulation endpoint returns deterministic results; explanation includes rule/field matches; integration tests pass. | -| NOTIFY-SVC-39-004 | TODO | Notifications Service Guild | NOTIFY-SVC-39-001 | Integrate quiet hour calendars and default throttles with audit logging and operator overrides. | Quiet schedules stored; overrides audited; preview API shows suppression windows; tests cover timezone handling. | - -## Sprint 40 – Escalations, Localization, Hardening -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| NOTIFY-SVC-40-001 | TODO | Notifications Service Guild | NOTIFY-SVC-39-001 | Implement escalations + on-call schedules, ack bridge, PagerDuty/OpsGenie adapters, and CLI/in-app inbox channels. | Escalation workflow operational; ack tokens flow; external adapters tested; inbox channel live. | -| NOTIFY-SVC-40-002 | TODO | Notifications Service Guild | NOTIFY-SVC-39-002 | Add summary storm breaker notifications, localization bundles, and localization fallback handling. | Storm breaker emits summaries; localization catalogs loaded; fallback behavior tested. | -| NOTIFY-SVC-40-003 | TODO | Notifications Service Guild | NOTIFY-SVC-38-004 | Harden security: signed ack links (KMS), webhook HMAC/IP allowlists, tenant isolation fuzz tests, HTML sanitization. | Ack tokens verified; webhook security enforced; fuzz tests green; sanitization validated. | -| NOTIFY-SVC-40-004 | TODO | Notifications Service Guild | NOTIFY-SVC-40-001..003 | Finalize observability (metrics/traces for escalations, latency), dead-letter handling, chaos tests for channel outages, and retention policies. | Metrics dashboards live; chaos run documented; DLQ drains; retention job operational. | - -## Authority-Backed Scopes & Tenancy (Epic 14) -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| NOTIFY-TEN-48-001 | TODO | Notifications Service Guild | WEB-TEN-48-001 | Tenant-scope rules/templates/incidents, RLS on storage, tenant-prefixed channels, and inclusion of tenant context in notifications. | Notifications isolated per tenant; RLS enabled; tests cover cross-tenant leakage. | - -## Observability & Forensics (Epic 15) -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| NOTIFY-OBS-51-001 | TODO | Notifications Service Guild, Observability Guild | DEVOPS-OBS-51-001, WEB-OBS-51-001 | Integrate SLO evaluator webhooks into Notifier rules (burn-rate breaches, health degradations) with templates, routing, and suppression logic. Provide sample policies and ensure imposed rule propagation. | Webhooks ingested; notifications delivered across channels; suppression guardrails tested; docs updated. | -| NOTIFY-OBS-55-001 | TODO | Notifications Service Guild, Ops Guild | DEVOPS-OBS-55-001, WEB-OBS-55-001 | Publish incident mode start/stop notifications with trace/evidence quick links, retention notes, and automatic escalation paths. Include quiet-hour overrides + legal compliance logging. | Incident notifications triggered in staging; CLI/Console deep links validated; audit logs capture scope usage. | - -## Air-Gapped Mode (Epic 16) -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| NOTIFY-AIRGAP-56-001 | TODO | Notifications Service Guild | AIRGAP-CTL-56-002, AIRGAP-POL-56-001 | Disable external webhook targets in sealed mode, default to enclave-safe channels (SMTP relay, syslog, file sink), and surface remediation guidance. | Sealed mode blocks external channels; configuration validation raises errors; tests cover allowances. | -| NOTIFY-AIRGAP-56-002 | TODO | Notifications Service Guild, DevOps Guild | NOTIFY-AIRGAP-56-001, DEVOPS-AIRGAP-56-001 | Provide local notifier configurations bundled within Bootstrap Pack with deterministic secrets handling. | Offline config templates published; bootstrap script validated; docs updated. | -| NOTIFY-AIRGAP-57-001 | TODO | Notifications Service Guild, AirGap Time Guild | NOTIFY-AIRGAP-56-001, AIRGAP-TIME-58-001 | Send staleness drift and bundle import notifications with remediation steps. | Notifications emitted on thresholds; tests cover suppression/resend. | -| NOTIFY-AIRGAP-58-001 | TODO | Notifications Service Guild, Evidence Locker Guild | NOTIFY-AIRGAP-56-001, EVID-OBS-54-002 | Add portable evidence export completion notifications including checksum + location metadata. | Notification payload includes bundle details; audit logs recorded; CLI integration validated. | - -## SDKs & OpenAPI (Epic 17) -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| NOTIFY-OAS-61-001 | TODO | Notifications Service Guild, API Contracts Guild | OAS-61-001 | Update notifier OAS with rules, templates, incidents, quiet hours endpoints using standard error envelope and examples. | Spec covers notifier APIs; lint passes; examples validated. | -| NOTIFY-OAS-61-002 | TODO | Notifications Service Guild | NOTIFY-OAS-61-001 | Implement `/.well-known/openapi` discovery endpoint with scope metadata. | Discovery endpoint live; contract tests cover response. | -| NOTIFY-OAS-62-001 | TODO | Notifications Service Guild, SDK Generator Guild | NOTIFY-OAS-61-001, SDKGEN-63-001 | Provide SDK usage examples for rule CRUD, incident ack, and quiet hours; ensure SDK smoke tests. | SDK tests cover notifier flows; docs embed snippets. | -| NOTIFY-OAS-63-001 | TODO | Notifications Service Guild, API Governance Guild | APIGOV-63-001 | Emit deprecation headers and Notifications templates for retiring notifier APIs. | Headers + notifications verified; documentation updated. | - -## Risk Profiles (Epic 18) -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| NOTIFY-RISK-66-001 | TODO | Notifications Service Guild, Risk Engine Guild | RISK-ENGINE-68-001 | Add notification triggers for risk severity escalation/downgrade events with profile metadata in payload. | Trigger processed in staging; payload shows profile and explainability link; docs updated. | -| NOTIFY-RISK-67-001 | TODO | Notifications Service Guild, Policy Guild | POLICY-RISK-67-002 | Notify stakeholders when risk profiles are published, deprecated, or thresholds change. | Notifications delivered via email/chat; audit logs captured. | -| NOTIFY-RISK-68-001 | TODO | Notifications Service Guild | NOTIFY-RISK-66-001 | Support per-profile routing rules, quiet hours, and dedupe for risk alerts; integrate with CLI/Console preferences. | Routing/quiet-hour logic tested; UI exposes settings; metrics reflect dedupe. | - -## Attestor Console (Epic 19) -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| NOTIFY-ATTEST-74-001 | TODO | Notifications Service Guild, Attestor Service Guild | ATTESTOR-73-002 | Create notification templates for verification failures, expiring attestations, key revocations, and transparency anomalies. | Templates deployed; staging verification failure triggers alert; documentation updated. | -| NOTIFY-ATTEST-74-002 | TODO | Notifications Service Guild, KMS Guild | KMS-73-001 | Wire notifications to key rotation/revocation events and transparency witness failures. | Rotation/revocation emits alerts; audit logs recorded; tests cover scenarios. | +# Notifier Service Task Board — Epic 11: Notifications Studio + +# Sprint 37 – Pack Approval Bridge (Task Runner integration) +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| NOTIFY-SVC-37-001 | TODO | Notifications Service Guild | TASKRUN-43-001 | Define pack approval & policy notification contract, including OpenAPI schema, event payloads, resume token mechanics, and security guidance. | Requirements doc published (`docs/notifications/pack-approvals-integration.md`), OpenAPI fragment merged, reviewers sign off from Task Runner & Authority guilds. | +| NOTIFY-SVC-37-002 | TODO | Notifications Service Guild | NOTIFY-SVC-37-001 | Implement secure ingestion endpoint, Mongo persistence (`pack_approvals`), idempotent writes, and audit trail for approval events. | Endpoint authenticated/authorized, persistence migrations merged, integration tests cover happy/error paths, audit log samples recorded. | +| NOTIFY-SVC-37-003 | TODO | Notifications Service Guild | NOTIFY-SVC-37-001 | Deliver approval/policy templates, routing predicates, and channel dispatch (email + webhook) with localization + redaction. | Templates rendered, routing rules active, localization fallback tested, sample notifications archived. | +| NOTIFY-SVC-37-004 | TODO | Notifications Service Guild | NOTIFY-SVC-37-002 | Provide acknowledgement API, Task Runner callback client, metrics for outstanding approvals, and runbook updates. | Ack endpoint live, resume callback validated with Task Runner simulator, metrics/dashboards in place, runbook entry updated. | + +## Sprint 38 – Foundations (Immediate notifications) +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +> 2025-10-29: Worker/WebService now compose `StellaOps.Notify.Storage.Mongo` + `StellaOps.Notify.Queue`, with a default rule evaluator and idempotent delivery ledger. See `docs/NOTIFY-SVC-38-001-FOUNDATIONS.md` for implementation notes and follow-ups. +| NOTIFY-SVC-38-002 | TODO | Notifications Service Guild | NOTIFY-SVC-38-001 | Implement channel adapters (email, chat webhook, generic webhook) with retry policies, health checks, and audit logging. | Adapters send test notifications; retries/backoff validated; health endpoints available; audit logs captured. | +| NOTIFY-SVC-38-003 | TODO | Notifications Service Guild | NOTIFY-SVC-38-001 | Deliver template service (versioned templates, localization scaffolding) and renderer with redaction allowlists, Markdown/HTML/JSON outputs, and provenance links. | Templates versioned; preview API works; rendered content includes provenance; redaction tests pass. | +| NOTIFY-SVC-38-004 | TODO | Notifications Service Guild | NOTIFY-SVC-38-001..003 | Expose REST + WS APIs (rules CRUD, templates preview, incidents list, ack) with audit logging, RBAC checks, and live feed stream. | OpenAPI published; WS feed delivers events; ack endpoint updates state; tests cover RBAC and audit logs. | + +## Sprint 39 – Correlation, Digests, Simulation +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| NOTIFY-SVC-39-001 | TODO | Notifications Service Guild | NOTIFY-SVC-38-004 | Implement correlation engine with pluggable key expressions/windows, throttler (token buckets), quiet hours/maintenance evaluator, and incident lifecycle. | Correlation merges duplicates; throttling enforced; quiet hours respect tenant schedules; incident state transitions tested. | +| NOTIFY-SVC-39-002 | TODO | Notifications Service Guild | NOTIFY-SVC-39-001, LEDGER-NOTIFY-39-001 | Build digest generator (queries, formatting) with schedule runner and distribution via existing channels. | Digests generated on schedule; content accurate; provenance linked; metrics emitted. | +| NOTIFY-SVC-39-003 | TODO | Notifications Service Guild | NOTIFY-SVC-39-001 | Provide simulation engine/API to dry-run rules against historical events, returning matched actions with explanations. | Simulation endpoint returns deterministic results; explanation includes rule/field matches; integration tests pass. | +| NOTIFY-SVC-39-004 | TODO | Notifications Service Guild | NOTIFY-SVC-39-001 | Integrate quiet hour calendars and default throttles with audit logging and operator overrides. | Quiet schedules stored; overrides audited; preview API shows suppression windows; tests cover timezone handling. | + +## Sprint 40 – Escalations, Localization, Hardening +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| NOTIFY-SVC-40-001 | TODO | Notifications Service Guild | NOTIFY-SVC-39-001 | Implement escalations + on-call schedules, ack bridge, PagerDuty/OpsGenie adapters, and CLI/in-app inbox channels. | Escalation workflow operational; ack tokens flow; external adapters tested; inbox channel live. | +| NOTIFY-SVC-40-002 | TODO | Notifications Service Guild | NOTIFY-SVC-39-002 | Add summary storm breaker notifications, localization bundles, and localization fallback handling. | Storm breaker emits summaries; localization catalogs loaded; fallback behavior tested. | +| NOTIFY-SVC-40-003 | TODO | Notifications Service Guild | NOTIFY-SVC-38-004 | Harden security: signed ack links (KMS), webhook HMAC/IP allowlists, tenant isolation fuzz tests, HTML sanitization. | Ack tokens verified; webhook security enforced; fuzz tests green; sanitization validated. | +| NOTIFY-SVC-40-004 | TODO | Notifications Service Guild | NOTIFY-SVC-40-001..003 | Finalize observability (metrics/traces for escalations, latency), dead-letter handling, chaos tests for channel outages, and retention policies. | Metrics dashboards live; chaos run documented; DLQ drains; retention job operational. | + +## Authority-Backed Scopes & Tenancy (Epic 14) +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| NOTIFY-TEN-48-001 | TODO | Notifications Service Guild | WEB-TEN-48-001 | Tenant-scope rules/templates/incidents, RLS on storage, tenant-prefixed channels, and inclusion of tenant context in notifications. | Notifications isolated per tenant; RLS enabled; tests cover cross-tenant leakage. | + +## Observability & Forensics (Epic 15) +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| NOTIFY-OBS-51-001 | TODO | Notifications Service Guild, Observability Guild | DEVOPS-OBS-51-001, WEB-OBS-51-001 | Integrate SLO evaluator webhooks into Notifier rules (burn-rate breaches, health degradations) with templates, routing, and suppression logic. Provide sample policies and ensure imposed rule propagation. | Webhooks ingested; notifications delivered across channels; suppression guardrails tested; docs updated. | +| NOTIFY-OBS-55-001 | TODO | Notifications Service Guild, Ops Guild | DEVOPS-OBS-55-001, WEB-OBS-55-001 | Publish incident mode start/stop notifications with trace/evidence quick links, retention notes, and automatic escalation paths. Include quiet-hour overrides + legal compliance logging. | Incident notifications triggered in staging; CLI/Console deep links validated; audit logs capture scope usage. | + +## Air-Gapped Mode (Epic 16) +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| NOTIFY-AIRGAP-56-001 | TODO | Notifications Service Guild | AIRGAP-CTL-56-002, AIRGAP-POL-56-001 | Disable external webhook targets in sealed mode, default to enclave-safe channels (SMTP relay, syslog, file sink), and surface remediation guidance. | Sealed mode blocks external channels; configuration validation raises errors; tests cover allowances. | +| NOTIFY-AIRGAP-56-002 | TODO | Notifications Service Guild, DevOps Guild | NOTIFY-AIRGAP-56-001, DEVOPS-AIRGAP-56-001 | Provide local notifier configurations bundled within Bootstrap Pack with deterministic secrets handling. | Offline config templates published; bootstrap script validated; docs updated. | +| NOTIFY-AIRGAP-57-001 | TODO | Notifications Service Guild, AirGap Time Guild | NOTIFY-AIRGAP-56-001, AIRGAP-TIME-58-001 | Send staleness drift and bundle import notifications with remediation steps. | Notifications emitted on thresholds; tests cover suppression/resend. | +| NOTIFY-AIRGAP-58-001 | TODO | Notifications Service Guild, Evidence Locker Guild | NOTIFY-AIRGAP-56-001, EVID-OBS-54-002 | Add portable evidence export completion notifications including checksum + location metadata. | Notification payload includes bundle details; audit logs recorded; CLI integration validated. | + +## SDKs & OpenAPI (Epic 17) +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| NOTIFY-OAS-61-001 | TODO | Notifications Service Guild, API Contracts Guild | OAS-61-001 | Update notifier OAS with rules, templates, incidents, quiet hours endpoints using standard error envelope and examples. | Spec covers notifier APIs; lint passes; examples validated. | +| NOTIFY-OAS-61-002 | TODO | Notifications Service Guild | NOTIFY-OAS-61-001 | Implement `/.well-known/openapi` discovery endpoint with scope metadata. | Discovery endpoint live; contract tests cover response. | +| NOTIFY-OAS-62-001 | TODO | Notifications Service Guild, SDK Generator Guild | NOTIFY-OAS-61-001, SDKGEN-63-001 | Provide SDK usage examples for rule CRUD, incident ack, and quiet hours; ensure SDK smoke tests. | SDK tests cover notifier flows; docs embed snippets. | +| NOTIFY-OAS-63-001 | TODO | Notifications Service Guild, API Governance Guild | APIGOV-63-001 | Emit deprecation headers and Notifications templates for retiring notifier APIs. | Headers + notifications verified; documentation updated. | + +## Risk Profiles (Epic 18) +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| NOTIFY-RISK-66-001 | TODO | Notifications Service Guild, Risk Engine Guild | RISK-ENGINE-68-001 | Add notification triggers for risk severity escalation/downgrade events with profile metadata in payload. | Trigger processed in staging; payload shows profile and explainability link; docs updated. | +| NOTIFY-RISK-67-001 | TODO | Notifications Service Guild, Policy Guild | POLICY-RISK-67-002 | Notify stakeholders when risk profiles are published, deprecated, or thresholds change. | Notifications delivered via email/chat; audit logs captured. | +| NOTIFY-RISK-68-001 | TODO | Notifications Service Guild | NOTIFY-RISK-66-001 | Support per-profile routing rules, quiet hours, and dedupe for risk alerts; integrate with CLI/Console preferences. | Routing/quiet-hour logic tested; UI exposes settings; metrics reflect dedupe. | + +## Attestor Console (Epic 19) +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| NOTIFY-ATTEST-74-001 | TODO | Notifications Service Guild, Attestor Service Guild | ATTESTOR-73-002 | Create notification templates for verification failures, expiring attestations, key revocations, and transparency anomalies. | Templates deployed; staging verification failure triggers alert; documentation updated. | +| NOTIFY-ATTEST-74-002 | TODO | Notifications Service Guild, KMS Guild | KMS-73-001 | Wire notifications to key rotation/revocation events and transparency witness failures. | Rotation/revocation emits alerts; audit logs recorded; tests cover scenarios. | diff --git a/src/Policy/StellaOps.Policy.Engine/TASKS.completed.md b/src/Policy/StellaOps.Policy.Engine/TASKS.completed.md new file mode 100644 index 00000000..e38f6ef9 --- /dev/null +++ b/src/Policy/StellaOps.Policy.Engine/TASKS.completed.md @@ -0,0 +1,13 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| POLICY-ENGINE-20-000 | DONE (2025-10-26) | Policy Guild, BE-Base Platform Guild | POLICY-AOC-19-001 | Spin up new `StellaOps.Policy.Engine` service project (minimal API host + worker), wire DI composition root, configuration binding, and Authority client scaffolding. | New project builds/tests; registered in solution; bootstrap validates configuration; host template committed with compliance checklist. | +| POLICY-ENGINE-27-001 | DONE (2025-10-31) | Policy Guild, Security Guild | AUTH-POLICY-27-001, POLICY-ENGINE-20-004 | Replace legacy `policy:write/submit` scope usage across Policy Engine API/worker/scheduler clients with the new Policy Studio scope family (`policy:author/review/approve/operate/audit/simulate`), update bootstrap configuration and tests, and ensure RBAC denials surface deterministic errors. | All configs/tests reference new scope set, integration tests cover missing-scope failures, CLI/docs samples updated, and CI guard prevents reintroduction of legacy scope names. | +| POLICY-GATEWAY-18-001 | DONE (2025-10-27) | Policy Gateway Strike Team | POLICY-ENGINE-20-000 | Bootstrap Policy Gateway host (`StellaOps.Policy.Gateway`) with configuration bootstrap, Authority resource-server auth, structured logging, health endpoints, and solution registration. | Gateway project builds/tests, configuration validation wired, `/healthz` + `/readyz` exposed, logging uses standard format. | +| POLICY-ENGINE-70-001 | DONE (2025-10-27) | Policy Guild, Governance Guild | POLICY-EXC-25-001 | Implement exception evaluation layer: specificity resolution, effect application (suppress/defer/downgrade/require control), and integration with explain traces. | Engine applies exceptions deterministically; unit/property tests cover precedence; explainer includes exception metadata. | +| POLICY-ENGINE-20-001 | DONE (2025-10-26) | Policy Guild, Language Infrastructure Guild | POLICY-ENGINE-20-000 | Implement `stella-dsl@1` parser + IR compiler with grammar validation, syntax diagnostics, and checksum outputs for caching. | DSL parser handles full grammar + error reporting; IR checksum stored with policy version; unit tests cover success/error paths. | +| POLICY-GATEWAY-18-002 | DONE (2025-10-27) | Policy Gateway Strike Team | POLICY-GATEWAY-18-001 | Implement proxy routes for policy packs/revisions (`GET/POST /api/policy/packs`, `/revisions`) with scope enforcement (`policy:read`, `policy:edit`) and deterministic DTOs. | Endpoints proxy to Policy Engine, unit tests cover happy/error paths, unauthorized requests rejected correctly. | +| POLICY-GATEWAY-18-003 | DONE (2025-10-27) | Policy Gateway Strike Team | POLICY-GATEWAY-18-002 | Implement activation proxy (`POST /api/policy/packs/{packId}/revisions/{version}:activate`) supporting single/two-person flows, returning 202 when awaiting second approval, and emitting structured logs/metrics. | Activation responses match Policy Engine contract, logs include tenant/actor/pack info, metrics published for outcomes. | +| POLICY-GATEWAY-18-004 | DONE (2025-10-27) | Policy Gateway Strike Team | POLICY-GATEWAY-18-001 | Add typed HttpClient for Policy Engine with DPoP client credentials, retry/backoff, and consistent error mapping to ProblemDetails. | HttpClient registered with resilient pipeline, integration tests verify error translation and token usage. | +| POLICY-GATEWAY-18-005 | DONE (2025-10-27) | Policy Gateway Strike Team | POLICY-GATEWAY-18-002, POLICY-GATEWAY-18-003 | Update docs/offline kit configs with new gateway service, sample curl commands, and CLI/UI integration guidance. | Docs merged, Offline Kit includes gateway config, verification script updated, release notes prepared. | diff --git a/src/Policy/StellaOps.Policy.Engine/TASKS.md b/src/Policy/StellaOps.Policy.Engine/TASKS.md index e458f45f..34f8a17c 100644 --- a/src/Policy/StellaOps.Policy.Engine/TASKS.md +++ b/src/Policy/StellaOps.Policy.Engine/TASKS.md @@ -2,9 +2,7 @@ | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| POLICY-ENGINE-20-000 | DONE (2025-10-26) | Policy Guild, BE-Base Platform Guild | POLICY-AOC-19-001 | Spin up new `StellaOps.Policy.Engine` service project (minimal API host + worker), wire DI composition root, configuration binding, and Authority client scaffolding. | New project builds/tests; registered in solution; bootstrap validates configuration; host template committed with compliance checklist. | > 2025-10-26: Added policy-engine host bootstrap (config, auth client, resource server auth, readiness probe) + sample YAML and compliance readme. -| POLICY-ENGINE-20-001 | DONE (2025-10-26) | Policy Guild, Language Infrastructure Guild | POLICY-ENGINE-20-000 | Implement `stella-dsl@1` parser + IR compiler with grammar validation, syntax diagnostics, and checksum outputs for caching. | DSL parser handles full grammar + error reporting; IR checksum stored with policy version; unit tests cover success/error paths. | | POLICY-ENGINE-20-002 | BLOCKED (2025-10-26) | Policy Guild | POLICY-ENGINE-20-001 | Build deterministic evaluator honoring lexical/priority order, first-match semantics, and safe value types (no wall-clock/network access). | Evaluator executes policies deterministically in unit/property tests; guard rejects forbidden intrinsics; perf baseline recorded. | > 2025-10-26: Blocked while bootstrapping DSL parser/evaluator; remaining grammar coverage (profile keywords, condition parsing) and rule evaluation semantics still pending to satisfy acceptance tests. | POLICY-ENGINE-20-003 | TODO | Policy Guild, Concelier Core Guild, Excititor Core Guild | POLICY-ENGINE-20-001, CONCELIER-POLICY-20-002, EXCITITOR-POLICY-20-002 | Implement selection joiners resolving SBOM↔advisory↔VEX tuples using linksets and PURL equivalence tables, with deterministic batching. | Joiners fetch correct candidate sets in integration tests; batching meets memory targets; explain traces list input provenance. | @@ -21,21 +19,15 @@ | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| POLICY-ENGINE-27-001 | DONE (2025-10-31) | Policy Guild, Security Guild | AUTH-POLICY-27-001, POLICY-ENGINE-20-004 | Replace legacy `policy:write/submit` scope usage across Policy Engine API/worker/scheduler clients with the new Policy Studio scope family (`policy:author/review/approve/operate/audit/simulate`), update bootstrap configuration and tests, and ensure RBAC denials surface deterministic errors. | All configs/tests reference new scope set, integration tests cover missing-scope failures, CLI/docs samples updated, and CI guard prevents reintroduction of legacy scope names. | > 2025-10-31: Policy Gateway now enforces `policy:author/review/operate` scopes, configuration defaults and Offline Kit samples updated, Authority clients seeded with new bundles, and scope verification script adjusted for the refreshed set. ## Gateway Implementation (Sprint 18.5) | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| POLICY-GATEWAY-18-001 | DONE (2025-10-27) | Policy Gateway Strike Team | POLICY-ENGINE-20-000 | Bootstrap Policy Gateway host (`StellaOps.Policy.Gateway`) with configuration bootstrap, Authority resource-server auth, structured logging, health endpoints, and solution registration. | Gateway project builds/tests, configuration validation wired, `/healthz` + `/readyz` exposed, logging uses standard format. | > 2025-10-27: Added the `StellaOps.Policy.Gateway` project with configuration bootstrapper, JSON logging, Authority resource server auth, and health/readiness endpoints plus sample config and solution wiring. -| POLICY-GATEWAY-18-002 | DONE (2025-10-27) | Policy Gateway Strike Team | POLICY-GATEWAY-18-001 | Implement proxy routes for policy packs/revisions (`GET/POST /api/policy/packs`, `/revisions`) with scope enforcement (`policy:read`, `policy:edit`) and deterministic DTOs. | Endpoints proxy to Policy Engine, unit tests cover happy/error paths, unauthorized requests rejected correctly. | > 2025-10-27: Implemented `/api/policy/packs` gateway routes with per-scope authorisation, forwarded bearer/DPoP/tenant headers, typed Policy Engine client, and deterministic DTO/ProblemDetails mapping. -| POLICY-GATEWAY-18-003 | DONE (2025-10-27) | Policy Gateway Strike Team | POLICY-GATEWAY-18-002 | Implement activation proxy (`POST /api/policy/packs/{packId}/revisions/{version}:activate`) supporting single/two-person flows, returning 202 when awaiting second approval, and emitting structured logs/metrics. | Activation responses match Policy Engine contract, logs include tenant/actor/pack info, metrics published for outcomes. | > 2025-10-27: Gateway proxy annotates activation outcomes (`activated`, `pending_second_approval`, etc.), emits `policy_gateway_activation_*` metrics, and logs PackId/Version/Tenant for auditability. -| POLICY-GATEWAY-18-004 | DONE (2025-10-27) | Policy Gateway Strike Team | POLICY-GATEWAY-18-001 | Add typed HttpClient for Policy Engine with DPoP client credentials, retry/backoff, and consistent error mapping to ProblemDetails. | HttpClient registered with resilient pipeline, integration tests verify error translation and token usage. | > 2025-10-27: Added client-credential fallback with ES256 DPoP proofs, Polly retry policy, and uniform ProblemDetails mapping for upstream failures. -| POLICY-GATEWAY-18-005 | DONE (2025-10-27) | Policy Gateway Strike Team | POLICY-GATEWAY-18-002, POLICY-GATEWAY-18-003 | Update docs/offline kit configs with new gateway service, sample curl commands, and CLI/UI integration guidance. | Docs merged, Offline Kit includes gateway config, verification script updated, release notes prepared. | > 2025-10-27: Published `/docs/policy/gateway.md`, Offline Kit instructions for bundling configs/keys, and curl workflows for Console/CLI verification. ## StellaOps Console (Sprint 23) @@ -111,7 +103,6 @@ | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| POLICY-ENGINE-70-001 | DONE (2025-10-27) | Policy Guild, Governance Guild | POLICY-EXC-25-001 | Implement exception evaluation layer: specificity resolution, effect application (suppress/defer/downgrade/require control), and integration with explain traces. | Engine applies exceptions deterministically; unit/property tests cover precedence; explainer includes exception metadata. | | POLICY-ENGINE-70-002 | TODO | Policy Guild, Storage Guild | POLICY-ENGINE-70-001 | Design and create Mongo collections (`exceptions`, `exception_reviews`, `exception_bindings`) with indexes and migrations; expose repository APIs. | Collections created; migrations documented; tests cover CRUD and binding lookups. | | POLICY-ENGINE-70-003 | TODO | Policy Guild, Runtime Guild | POLICY-ENGINE-70-001 | Build Redis exception decision cache (`exceptions_effective_map`) with warm/invalidation logic reacting to `exception.*` events. | Cache layer operational; metrics track hit/miss; fallback path tested. | | POLICY-ENGINE-70-004 | TODO | Policy Guild, Observability Guild | POLICY-ENGINE-70-001 | Extend metrics/tracing/logging for exception application (latency, counts, expiring events) and include AOC references in logs. | Metrics emitted (`policy_exception_applied_total` etc.); traces updated; log schema documented. | diff --git a/src/Policy/__Libraries/StellaOps.Policy/TASKS.completed.md b/src/Policy/__Libraries/StellaOps.Policy/TASKS.completed.md new file mode 100644 index 00000000..02de8402 --- /dev/null +++ b/src/Policy/__Libraries/StellaOps.Policy/TASKS.completed.md @@ -0,0 +1,5 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| POLICY-EXC-25-001 | DONE (2025-10-27) | Policy Guild, Governance Guild | POLICY-SPL-23-001 | Extend SPL schema/spec to reference exception effects and routing templates; publish updated docs and validation fixtures. | Schema updated with exception references; validation tests cover effect types; docs draft ready. | diff --git a/src/Policy/__Libraries/StellaOps.Policy/TASKS.md b/src/Policy/__Libraries/StellaOps.Policy/TASKS.md index ef272f25..c873f000 100644 --- a/src/Policy/__Libraries/StellaOps.Policy/TASKS.md +++ b/src/Policy/__Libraries/StellaOps.Policy/TASKS.md @@ -22,7 +22,6 @@ | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| POLICY-EXC-25-001 | DONE (2025-10-27) | Policy Guild, Governance Guild | POLICY-SPL-23-001 | Extend SPL schema/spec to reference exception effects and routing templates; publish updated docs and validation fixtures. | Schema updated with exception references; validation tests cover effect types; docs draft ready. | ## Reachability v1 (Epic 8) diff --git a/src/Scanner/StellaOps.Scanner.Sbomer.BuildXPlugin/TASKS.completed.md b/src/Scanner/StellaOps.Scanner.Sbomer.BuildXPlugin/TASKS.completed.md new file mode 100644 index 00000000..1a02b47e --- /dev/null +++ b/src/Scanner/StellaOps.Scanner.Sbomer.BuildXPlugin/TASKS.completed.md @@ -0,0 +1,9 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| SP9-BLDX-09-001 | DONE | BuildX Guild | SCANNER-EMIT-10-601 (awareness) | Scaffold buildx driver, manifest, local CAS handshake; ensure plugin loads from `plugins/scanner/buildx/`. | Plugin manifest + loader tests; local CAS writes succeed; restart required to activate. | +| SP9-BLDX-09-002 | DONE | BuildX Guild | SP9-BLDX-09-001 | Emit OCI annotations + provenance metadata for Attestor handoff (image + SBOM). | OCI descriptors include DSSE/provenance placeholders; Attestor mock accepts payload. | +| SP9-BLDX-09-003 | DONE | BuildX Guild | SP9-BLDX-09-002 | CI demo pipeline: build sample image, produce SBOM, verify backend report wiring. | GitHub/CI job runs sample build within 5 s overhead; artifacts saved; documentation updated. | +| SP9-BLDX-09-004 | DONE (2025-10-19) | BuildX Guild | SP9-BLDX-09-002 | Stabilize descriptor nonce derivation so repeated builds emit deterministic placeholders. | Repeated descriptor runs with fixed inputs yield identical JSON; regression tests cover nonce determinism. | +| SP9-BLDX-09-005 | DONE (2025-10-19) | BuildX Guild | SP9-BLDX-09-004 | Integrate determinism check in GitHub/Gitea workflows and capture sample artifacts. | Determinism step runs in `.gitea/workflows/build-test-deploy.yml` and `samples/ci/buildx-demo`, producing matching descriptors + archived artifacts. | diff --git a/src/Scanner/StellaOps.Scanner.Sbomer.BuildXPlugin/TASKS.md b/src/Scanner/StellaOps.Scanner.Sbomer.BuildXPlugin/TASKS.md index 5d93f607..9ae89c62 100644 --- a/src/Scanner/StellaOps.Scanner.Sbomer.BuildXPlugin/TASKS.md +++ b/src/Scanner/StellaOps.Scanner.Sbomer.BuildXPlugin/TASKS.md @@ -1,9 +1,4 @@ -# BuildX Plugin Task Board (Sprint 9) - -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| SP9-BLDX-09-001 | DONE | BuildX Guild | SCANNER-EMIT-10-601 (awareness) | Scaffold buildx driver, manifest, local CAS handshake; ensure plugin loads from `plugins/scanner/buildx/`. | Plugin manifest + loader tests; local CAS writes succeed; restart required to activate. | -| SP9-BLDX-09-002 | DONE | BuildX Guild | SP9-BLDX-09-001 | Emit OCI annotations + provenance metadata for Attestor handoff (image + SBOM). | OCI descriptors include DSSE/provenance placeholders; Attestor mock accepts payload. | -| SP9-BLDX-09-003 | DONE | BuildX Guild | SP9-BLDX-09-002 | CI demo pipeline: build sample image, produce SBOM, verify backend report wiring. | GitHub/CI job runs sample build within 5 s overhead; artifacts saved; documentation updated. | -| SP9-BLDX-09-004 | DONE (2025-10-19) | BuildX Guild | SP9-BLDX-09-002 | Stabilize descriptor nonce derivation so repeated builds emit deterministic placeholders. | Repeated descriptor runs with fixed inputs yield identical JSON; regression tests cover nonce determinism. | -| SP9-BLDX-09-005 | DONE (2025-10-19) | BuildX Guild | SP9-BLDX-09-004 | Integrate determinism check in GitHub/Gitea workflows and capture sample artifacts. | Determinism step runs in `.gitea/workflows/build-test-deploy.yml` and `samples/ci/buildx-demo`, producing matching descriptors + archived artifacts. | +# BuildX Plugin Task Board (Sprint 9) + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| diff --git a/src/Scanner/StellaOps.Scanner.WebService/Contracts/OrchestratorEventContracts.cs b/src/Scanner/StellaOps.Scanner.WebService/Contracts/OrchestratorEventContracts.cs index f61bcd77..db656938 100644 --- a/src/Scanner/StellaOps.Scanner.WebService/Contracts/OrchestratorEventContracts.cs +++ b/src/Scanner/StellaOps.Scanner.WebService/Contracts/OrchestratorEventContracts.cs @@ -226,28 +226,40 @@ internal sealed record ReportDeltaPayload public IReadOnlyList? Kev { get; init; } } -internal sealed record ReportLinksPayload -{ - [JsonPropertyName("ui")] - [JsonPropertyOrder(0)] - [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] - public string? Ui { get; init; } - - [JsonPropertyName("report")] - [JsonPropertyOrder(1)] - [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] - public string? Report { get; init; } - - [JsonPropertyName("policy")] - [JsonPropertyOrder(2)] - [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] - public string? Policy { get; init; } - - [JsonPropertyName("attestation")] - [JsonPropertyOrder(3)] - [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] - public string? Attestation { get; init; } -} +internal sealed record ReportLinksPayload +{ + [JsonPropertyName("report")] + [JsonPropertyOrder(0)] + [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] + public LinkTarget? Report { get; init; } + + [JsonPropertyName("policy")] + [JsonPropertyOrder(1)] + [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] + public LinkTarget? Policy { get; init; } + + [JsonPropertyName("attestation")] + [JsonPropertyOrder(2)] + [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] + public LinkTarget? Attestation { get; init; } +} + +internal sealed record LinkTarget( + [property: JsonPropertyName("ui"), JsonPropertyOrder(0), JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] string? Ui, + [property: JsonPropertyName("api"), JsonPropertyOrder(1), JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] string? Api) +{ + public static LinkTarget? Create(string? ui, string? api) + { + if (string.IsNullOrWhiteSpace(ui) && string.IsNullOrWhiteSpace(api)) + { + return null; + } + + return new LinkTarget( + string.IsNullOrWhiteSpace(ui) ? null : ui, + string.IsNullOrWhiteSpace(api) ? null : api); + } +} internal sealed record FindingSummaryPayload { diff --git a/src/Scanner/StellaOps.Scanner.WebService/Endpoints/PolicyEndpoints.cs b/src/Scanner/StellaOps.Scanner.WebService/Endpoints/PolicyEndpoints.cs index 9a137a89..8bc499f7 100644 --- a/src/Scanner/StellaOps.Scanner.WebService/Endpoints/PolicyEndpoints.cs +++ b/src/Scanner/StellaOps.Scanner.WebService/Endpoints/PolicyEndpoints.cs @@ -16,9 +16,11 @@ using StellaOps.Scanner.WebService.Services; using StellaOps.Zastava.Core.Contracts; using RuntimePolicyVerdict = StellaOps.Zastava.Core.Contracts.PolicyVerdict; -namespace StellaOps.Scanner.WebService.Endpoints; - -internal static class PolicyEndpoints +namespace StellaOps.Scanner.WebService.Endpoints; + +#pragma warning disable ASPDEPR002 + +internal static class PolicyEndpoints { private static readonly JsonSerializerOptions SerializerOptions = new(JsonSerializerDefaults.Web) { @@ -335,3 +337,5 @@ internal static class PolicyEndpoints _ => "unknown" }; } + +#pragma warning restore ASPDEPR002 diff --git a/src/Scanner/StellaOps.Scanner.WebService/Endpoints/ReportEndpoints.cs b/src/Scanner/StellaOps.Scanner.WebService/Endpoints/ReportEndpoints.cs index fcc84e1b..4c4fef6f 100644 --- a/src/Scanner/StellaOps.Scanner.WebService/Endpoints/ReportEndpoints.cs +++ b/src/Scanner/StellaOps.Scanner.WebService/Endpoints/ReportEndpoints.cs @@ -13,9 +13,11 @@ using StellaOps.Scanner.WebService.Infrastructure; using StellaOps.Scanner.WebService.Security; using StellaOps.Scanner.WebService.Services; -namespace StellaOps.Scanner.WebService.Endpoints; - -internal static class ReportEndpoints +namespace StellaOps.Scanner.WebService.Endpoints; + +#pragma warning disable ASPDEPR002 + +internal static class ReportEndpoints { private const string PayloadType = "application/vnd.stellaops.report+json"; @@ -25,7 +27,7 @@ internal static class ReportEndpoints Converters = { new JsonStringEnumConverter() } }; - public static void MapReportEndpoints(this RouteGroupBuilder apiGroup, string reportsSegment) + public static void MapReportEndpoints(this RouteGroupBuilder apiGroup, string reportsSegment) { ArgumentNullException.ThrowIfNull(apiGroup); @@ -33,11 +35,11 @@ internal static class ReportEndpoints .MapGroup(NormalizeSegment(reportsSegment)) .WithTags("Reports"); - reports.MapPost("/", HandleCreateReportAsync) - .WithName("scanner.reports.create") - .Produces(StatusCodes.Status200OK) - .Produces(StatusCodes.Status400BadRequest) - .Produces(StatusCodes.Status503ServiceUnavailable) + reports.MapPost("/", HandleCreateReportAsync) + .WithName("scanner.reports.create") + .Produces(StatusCodes.Status200OK) + .Produces(StatusCodes.Status400BadRequest) + .Produces(StatusCodes.Status503ServiceUnavailable) .RequireAuthorization(ScannerPolicies.Reports) .WithOpenApi(operation => { @@ -263,4 +265,6 @@ internal static class ReportEndpoints var payload = JsonSerializer.Serialize(value, SerializerOptions); return Results.Content(payload, "application/json", Encoding.UTF8); } -} +} + +#pragma warning restore ASPDEPR002 diff --git a/src/Scanner/StellaOps.Scanner.WebService/Serialization/OrchestratorEventSerializer.cs b/src/Scanner/StellaOps.Scanner.WebService/Serialization/OrchestratorEventSerializer.cs index 48b13f1c..f52aa71f 100644 --- a/src/Scanner/StellaOps.Scanner.WebService/Serialization/OrchestratorEventSerializer.cs +++ b/src/Scanner/StellaOps.Scanner.WebService/Serialization/OrchestratorEventSerializer.cs @@ -98,13 +98,17 @@ internal static class OrchestratorEventSerializer "newHigh", "kev" }, - [typeof(ReportLinksPayload)] = new[] - { - "ui", - "report", - "policy", - "attestation" - }, + [typeof(ReportLinksPayload)] = new[] + { + "report", + "policy", + "attestation" + }, + [typeof(LinkTarget)] = new[] + { + "ui", + "api" + }, [typeof(FindingSummaryPayload)] = new[] { "id", @@ -158,12 +162,12 @@ internal static class OrchestratorEventSerializer _inner = inner ?? throw new ArgumentNullException(nameof(inner)); } - public JsonTypeInfo GetTypeInfo(Type type, JsonSerializerOptions options) - { - var info = _inner.GetTypeInfo(type, options) - ?? throw new InvalidOperationException($"Unable to resolve JsonTypeInfo for '{type}'."); - - if (info.Kind is JsonTypeInfoKind.Object && info.Properties is { Count: > 1 }) + public JsonTypeInfo GetTypeInfo(Type type, JsonSerializerOptions options) + { + var info = _inner.GetTypeInfo(type, options) + ?? throw new InvalidOperationException($"Unable to resolve JsonTypeInfo for '{type}'."); + + if (info.Kind is JsonTypeInfoKind.Object && info.Properties is { Count: > 1 }) { var ordered = info.Properties .OrderBy(property => GetOrder(type, property.Name)) @@ -174,25 +178,52 @@ internal static class OrchestratorEventSerializer foreach (var property in ordered) { info.Properties.Add(property); - } - } - - return info; - } - - private static int GetOrder(Type type, string propertyName) - { + } + } + + ConfigurePolymorphism(info); + return info; + } + + private static int GetOrder(Type type, string propertyName) + { if (PropertyOrder.TryGetValue(type, out var order) && Array.IndexOf(order, propertyName) is { } index and >= 0) { return index; } - - if (type.BaseType is not null) - { - return GetOrder(type.BaseType, propertyName); - } - - return int.MaxValue; - } - } -} + + if (type.BaseType is not null) + { + return GetOrder(type.BaseType, propertyName); + } + + return int.MaxValue; + } + + private static void ConfigurePolymorphism(JsonTypeInfo info) + { + if (info.Type != typeof(OrchestratorEventPayload)) + { + return; + } + + info.PolymorphismOptions ??= new JsonPolymorphismOptions + { + UnknownDerivedTypeHandling = JsonUnknownDerivedTypeHandling.Fail + }; + + AddDerivedType(info.PolymorphismOptions, typeof(ReportReadyEventPayload)); + AddDerivedType(info.PolymorphismOptions, typeof(ScanCompletedEventPayload)); + } + + private static void AddDerivedType(JsonPolymorphismOptions options, Type derivedType) + { + if (options.DerivedTypes.Any(d => d.DerivedType == derivedType)) + { + return; + } + + options.DerivedTypes.Add(new JsonDerivedType(derivedType)); + } + } +} diff --git a/src/Scanner/StellaOps.Scanner.WebService/Services/ReportEventDispatcher.cs b/src/Scanner/StellaOps.Scanner.WebService/Services/ReportEventDispatcher.cs index 50a9ed2b..41fd0925 100644 --- a/src/Scanner/StellaOps.Scanner.WebService/Services/ReportEventDispatcher.cs +++ b/src/Scanner/StellaOps.Scanner.WebService/Services/ReportEventDispatcher.cs @@ -247,23 +247,33 @@ internal sealed class ReportEventDispatcher : IReportEventDispatcher return new ReportLinksPayload(); } - var uiLink = BuildAbsoluteUri(context, "ui", "reports", document.ReportId); - var reportLink = BuildAbsoluteUri(context, ConcatSegments(_apiBaseSegments, _reportsSegment, document.ReportId)); - var policyLink = string.IsNullOrWhiteSpace(document.Policy.RevisionId) - ? null - : BuildAbsoluteUri(context, ConcatSegments(_apiBaseSegments, _policySegment, "revisions", document.Policy.RevisionId)); - var attestationLink = envelope is null - ? null - : BuildAbsoluteUri(context, "ui", "attestations", document.ReportId); - - return new ReportLinksPayload - { - Ui = uiLink, - Report = reportLink, - Policy = policyLink, - Attestation = attestationLink - }; - } + var reportUi = BuildAbsoluteUri(context, "ui", "reports", document.ReportId); + var reportApi = BuildAbsoluteUri(context, ConcatSegments(_apiBaseSegments, _reportsSegment, document.ReportId)); + + LinkTarget? policyLink = null; + if (!string.IsNullOrWhiteSpace(document.Policy.RevisionId)) + { + var policyRevision = document.Policy.RevisionId!; + var policyUi = BuildAbsoluteUri(context, "ui", "policy", "revisions", policyRevision); + var policyApi = BuildAbsoluteUri(context, ConcatSegments(_apiBaseSegments, _policySegment, "revisions", policyRevision)); + policyLink = LinkTarget.Create(policyUi, policyApi); + } + + LinkTarget? attestationLink = null; + if (envelope is not null) + { + var attestationUi = BuildAbsoluteUri(context, "ui", "attestations", document.ReportId); + var attestationApi = BuildAbsoluteUri(context, ConcatSegments(_apiBaseSegments, _reportsSegment, document.ReportId, "attestation")); + attestationLink = LinkTarget.Create(attestationUi, attestationApi); + } + + return new ReportLinksPayload + { + Report = LinkTarget.Create(reportUi, reportApi), + Policy = policyLink, + Attestation = attestationLink + }; + } private static ReportDeltaPayload? BuildDelta(PolicyPreviewResponse preview, ReportRequestDto request) { diff --git a/src/Scanner/StellaOps.Scanner.WebService/TASKS.completed.md b/src/Scanner/StellaOps.Scanner.WebService/TASKS.completed.md new file mode 100644 index 00000000..6c314a96 --- /dev/null +++ b/src/Scanner/StellaOps.Scanner.WebService/TASKS.completed.md @@ -0,0 +1,19 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| SCANNER-WEB-09-101 | DONE (2025-10-18) | Scanner WebService Guild | SCANNER-CORE-09-501 | Stand up minimal API host with Authority OpTok + DPoP enforcement, health/ready endpoints, and restart-time plug-in loader per architecture §1, §4. | Host boots with configuration validation, `/healthz` and `/readyz` return 200, Authority middleware enforced in integration tests. | +| SCANNER-WEB-09-102 | DONE (2025-10-18) | Scanner WebService Guild | SCANNER-WEB-09-101, SCANNER-QUEUE-09-401 | Implement `/api/v1/scans` submission/status endpoints with deterministic IDs, validation, and cancellation tokens. | Contract documented, e2e test posts scan request and retrieves status, cancellation token honoured. | +| SCANNER-WEB-09-103 | DONE (2025-10-19) | Scanner WebService Guild | SCANNER-WEB-09-102, SCANNER-CORE-09-502 | Emit scan progress via SSE/JSONL with correlation IDs and deterministic timestamps; document API reference. | Streaming endpoint verified in tests, timestamps formatted ISO-8601 UTC, docs updated in `docs/09_API_CLI_REFERENCE.md`. | +| SCANNER-WEB-09-104 | DONE (2025-10-19) | Scanner WebService Guild | SCANNER-STORAGE-09-301, SCANNER-QUEUE-09-401 | Bind configuration for Mongo, MinIO, queue, feature flags; add startup diagnostics and fail-fast policy for missing deps. | Misconfiguration fails fast with actionable errors, configuration bound tests pass, diagnostics logged with correlation IDs. | +| SCANNER-POLICY-09-105 | DONE (2025-10-19) | Scanner WebService Guild | POLICY-CORE-09-001 | Integrate policy schema loader + diagnostics + OpenAPI (YAML ignore rules, VEX include/exclude, vendor precedence). | Policy endpoints documented; validation surfaces actionable errors; OpenAPI schema published. | +| SCANNER-POLICY-09-106 | DONE (2025-10-19) | Scanner WebService Guild | POLICY-CORE-09-002, SCANNER-POLICY-09-105 | `/reports` verdict assembly (Feedser/Vexer/Policy merge) + signed response envelope. | Aggregated report includes policy metadata; integration test verifies signed response; docs updated. | +| SCANNER-POLICY-09-107 | DONE (2025-10-19) | Scanner WebService Guild | POLICY-CORE-09-005, SCANNER-POLICY-09-106 | Surface score inputs, config version, and `quietedBy` provenance in `/reports` response and signed payload; document schema changes. | `/reports` JSON + DSSE contain score, reachability, sourceTrust, confidenceBand, quiet provenance; contract tests updated; docs refreshed. | +| SCANNER-WEB-10-201 | DONE (2025-10-19) | Scanner WebService Guild | SCANNER-CACHE-10-101 | Register scanner cache services and maintenance loop within WebService host. | `AddScannerCache` wired for configuration binding; maintenance service skips when disabled; project references updated. | +| SCANNER-RUNTIME-12-301 | DONE (2025-10-20) | Scanner WebService Guild | ZASTAVA-CORE-12-201 | Implement `/runtime/events` ingestion endpoint with validation, batching, and storage hooks per Zastava contract. | Observer fixtures POST events, data persisted and acked; invalid payloads rejected with deterministic errors. | +| SCANNER-RUNTIME-12-302 | DONE (2025-10-24) | Scanner WebService Guild | SCANNER-RUNTIME-12-301, ZASTAVA-CORE-12-201 | Implement `/policy/runtime` endpoint joining SBOM baseline + policy verdict, returning admission guidance. Coordinate with CLI (`CLI-RUNTIME-13-008`) before GA to lock response field names/metadata. | Webhook integration test passes; responses include verdict, TTL, reasons; metrics/logging added; CLI contract review signed off. | +| SCANNER-RUNTIME-12-303 | DONE (2025-10-24) | Scanner WebService Guild | SCANNER-RUNTIME-12-302 | Replace `/policy/runtime` heuristic with canonical policy evaluation (Feedser/Vexer inputs, PolicyPreviewService) so results align with `/reports`. | Runtime policy endpoint now pipes findings through `PolicyPreviewService`, emits canonical verdicts/confidence/quiet metadata, and updated tests cover pass/warn/fail paths + CLI contract fixtures. | +| SCANNER-RUNTIME-12-304 | DONE (2025-10-24) | Scanner WebService Guild | SCANNER-RUNTIME-12-302 | Surface attestation verification status by integrating Authority/Attestor Rekor validation (beyond presence-only). | `/policy/runtime` maps Rekor UUIDs through the runtime attestation verifier so `rekor.verified` reflects attestor outcomes; webhook/CLI coverage added. | +| SCANNER-RUNTIME-12-305 | DONE (2025-10-24) | Scanner WebService Guild | SCANNER-RUNTIME-12-301, SCANNER-RUNTIME-12-302 | Promote shared fixtures with Zastava/CLI and add end-to-end automation for `/runtime/events` + `/policy/runtime`. | Runtime policy integration test + CLI-aligned fixture assert confidence, metadata JSON, and Rekor verification; docs note shared contract. | +| SCANNER-EVENTS-15-201 | DONE (2025-10-20) | Scanner WebService Guild | NOTIFY-QUEUE-15-401 | Emit `scanner.report.ready` and `scanner.scan.completed` events (bus adapters + tests). | Event envelopes published to queue with schemas; fixtures committed; Notify consumption test passes. | +| SCANNER-RUNTIME-17-401 | DONE (2025-10-25) | Scanner WebService Guild | SCANNER-RUNTIME-12-301, ZASTAVA-OBS-17-005, SCANNER-EMIT-17-701, POLICY-RUNTIME-17-201 | Persist runtime build-id observations and expose them via `/runtime/events` + policy joins for debug-symbol correlation. | Runtime events store normalized digests + build IDs with supporting indexes, runtime policy responses surface `buildIds`, tests/docs updated, and CLI/API consumers can derive debug-store paths deterministically. | diff --git a/src/Scanner/StellaOps.Scanner.WebService/TASKS.md b/src/Scanner/StellaOps.Scanner.WebService/TASKS.md index 55afa383..31164219 100644 --- a/src/Scanner/StellaOps.Scanner.WebService/TASKS.md +++ b/src/Scanner/StellaOps.Scanner.WebService/TASKS.md @@ -2,23 +2,8 @@ | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| SCANNER-WEB-09-101 | DONE (2025-10-18) | Scanner WebService Guild | SCANNER-CORE-09-501 | Stand up minimal API host with Authority OpTok + DPoP enforcement, health/ready endpoints, and restart-time plug-in loader per architecture §1, §4. | Host boots with configuration validation, `/healthz` and `/readyz` return 200, Authority middleware enforced in integration tests. | -| SCANNER-WEB-09-102 | DONE (2025-10-18) | Scanner WebService Guild | SCANNER-WEB-09-101, SCANNER-QUEUE-09-401 | Implement `/api/v1/scans` submission/status endpoints with deterministic IDs, validation, and cancellation tokens. | Contract documented, e2e test posts scan request and retrieves status, cancellation token honoured. | -| SCANNER-WEB-09-103 | DONE (2025-10-19) | Scanner WebService Guild | SCANNER-WEB-09-102, SCANNER-CORE-09-502 | Emit scan progress via SSE/JSONL with correlation IDs and deterministic timestamps; document API reference. | Streaming endpoint verified in tests, timestamps formatted ISO-8601 UTC, docs updated in `docs/09_API_CLI_REFERENCE.md`. | -| SCANNER-WEB-09-104 | DONE (2025-10-19) | Scanner WebService Guild | SCANNER-STORAGE-09-301, SCANNER-QUEUE-09-401 | Bind configuration for Mongo, MinIO, queue, feature flags; add startup diagnostics and fail-fast policy for missing deps. | Misconfiguration fails fast with actionable errors, configuration bound tests pass, diagnostics logged with correlation IDs. | -| SCANNER-POLICY-09-105 | DONE (2025-10-19) | Scanner WebService Guild | POLICY-CORE-09-001 | Integrate policy schema loader + diagnostics + OpenAPI (YAML ignore rules, VEX include/exclude, vendor precedence). | Policy endpoints documented; validation surfaces actionable errors; OpenAPI schema published. | -| SCANNER-POLICY-09-106 | DONE (2025-10-19) | Scanner WebService Guild | POLICY-CORE-09-002, SCANNER-POLICY-09-105 | `/reports` verdict assembly (Feedser/Vexer/Policy merge) + signed response envelope. | Aggregated report includes policy metadata; integration test verifies signed response; docs updated. | -| SCANNER-POLICY-09-107 | DONE (2025-10-19) | Scanner WebService Guild | POLICY-CORE-09-005, SCANNER-POLICY-09-106 | Surface score inputs, config version, and `quietedBy` provenance in `/reports` response and signed payload; document schema changes. | `/reports` JSON + DSSE contain score, reachability, sourceTrust, confidenceBand, quiet provenance; contract tests updated; docs refreshed. | -| SCANNER-WEB-10-201 | DONE (2025-10-19) | Scanner WebService Guild | SCANNER-CACHE-10-101 | Register scanner cache services and maintenance loop within WebService host. | `AddScannerCache` wired for configuration binding; maintenance service skips when disabled; project references updated. | -| SCANNER-RUNTIME-12-301 | DONE (2025-10-20) | Scanner WebService Guild | ZASTAVA-CORE-12-201 | Implement `/runtime/events` ingestion endpoint with validation, batching, and storage hooks per Zastava contract. | Observer fixtures POST events, data persisted and acked; invalid payloads rejected with deterministic errors. | -| SCANNER-RUNTIME-12-302 | DONE (2025-10-24) | Scanner WebService Guild | SCANNER-RUNTIME-12-301, ZASTAVA-CORE-12-201 | Implement `/policy/runtime` endpoint joining SBOM baseline + policy verdict, returning admission guidance. Coordinate with CLI (`CLI-RUNTIME-13-008`) before GA to lock response field names/metadata. | Webhook integration test passes; responses include verdict, TTL, reasons; metrics/logging added; CLI contract review signed off. | -| SCANNER-RUNTIME-12-303 | DONE (2025-10-24) | Scanner WebService Guild | SCANNER-RUNTIME-12-302 | Replace `/policy/runtime` heuristic with canonical policy evaluation (Feedser/Vexer inputs, PolicyPreviewService) so results align with `/reports`. | Runtime policy endpoint now pipes findings through `PolicyPreviewService`, emits canonical verdicts/confidence/quiet metadata, and updated tests cover pass/warn/fail paths + CLI contract fixtures. | -| SCANNER-RUNTIME-12-304 | DONE (2025-10-24) | Scanner WebService Guild | SCANNER-RUNTIME-12-302 | Surface attestation verification status by integrating Authority/Attestor Rekor validation (beyond presence-only). | `/policy/runtime` maps Rekor UUIDs through the runtime attestation verifier so `rekor.verified` reflects attestor outcomes; webhook/CLI coverage added. | -| SCANNER-RUNTIME-12-305 | DONE (2025-10-24) | Scanner WebService Guild | SCANNER-RUNTIME-12-301, SCANNER-RUNTIME-12-302 | Promote shared fixtures with Zastava/CLI and add end-to-end automation for `/runtime/events` + `/policy/runtime`. | Runtime policy integration test + CLI-aligned fixture assert confidence, metadata JSON, and Rekor verification; docs note shared contract. | -| SCANNER-EVENTS-15-201 | DONE (2025-10-20) | Scanner WebService Guild | NOTIFY-QUEUE-15-401 | Emit `scanner.report.ready` and `scanner.scan.completed` events (bus adapters + tests). | Event envelopes published to queue with schemas; fixtures committed; Notify consumption test passes. | | SCANNER-EVENTS-16-301 | BLOCKED (2025-10-26) | Scanner WebService Guild | ORCH-SVC-38-101, NOTIFY-SVC-38-001 | Emit orchestrator-compatible envelopes (`scanner.event.*`) and update integration tests to verify Notifier ingestion (no Redis queue coupling). | Tests assert envelope schema + orchestrator publish; Notifier consumer harness passes; docs updated with new event contract. Blocked by .NET 10 preview OpenAPI/Auth dependency drift preventing `dotnet test` completion. | | SCANNER-EVENTS-16-302 | DOING (2025-10-26) | Scanner WebService Guild | SCANNER-EVENTS-16-301 | Extend orchestrator event links (report/policy/attestation) once endpoints are finalised across gateway + console. | Links section covers UI/API targets; downstream consumers validated; docs/samples updated. | -| SCANNER-RUNTIME-17-401 | DONE (2025-10-25) | Scanner WebService Guild | SCANNER-RUNTIME-12-301, ZASTAVA-OBS-17-005, SCANNER-EMIT-17-701, POLICY-RUNTIME-17-201 | Persist runtime build-id observations and expose them via `/runtime/events` + policy joins for debug-symbol correlation. | Runtime events store normalized digests + build IDs with supporting indexes, runtime policy responses surface `buildIds`, tests/docs updated, and CLI/API consumers can derive debug-store paths deterministically. | ## Graph Explorer v1 (Sprint 21) @@ -41,5 +26,6 @@ - 2025-10-21: Hardened progress streaming determinism by sorting `data` payload keys within `ScanProgressStream`; added regression `ProgressStreamDataKeysAreSortedDeterministically` ensuring JSONL ordering. - 2025-10-24: `/policy/runtime` now streams through PolicyPreviewService + attestation verifier; CLI and webhook fixtures updated alongside Zastava observer batching completion. - 2025-10-26: SCANNER-EVENTS-16-302 populates orchestrator link payloads (UI, API report lookup, policy revision, attestation) pending cross-service integration; samples/tests updated. +- 2025-10-30: SCANNER-EVENTS-16-302 upgraded link payloads to nested UI/API targets, refreshed JSON schemas/docs/samples, and updated dispatcher/tests; downstream consumers to validate before closure. - 2025-10-26: Coordinate with Gateway + Console owners to confirm final API/UX paths for report, policy revision, and attestation links before promoting SCANNER-EVENTS-16-301 out of BLOCKED. - 2025-10-26: SCANNER-EVENTS-16-301 emitting new orchestrator envelopes; solution-wide `dotnet test` currently blocked by preview `Microsoft.AspNetCore.OpenApi` APIs and missing `StellaOps.Auth` dependency wiring. JSON Schemas validated via `ajv`; service-level verification pending SDK alignment. diff --git a/src/Scanner/StellaOps.Scanner.Worker/TASKS.completed.md b/src/Scanner/StellaOps.Scanner.Worker/TASKS.completed.md new file mode 100644 index 00000000..edb888e4 --- /dev/null +++ b/src/Scanner/StellaOps.Scanner.Worker/TASKS.completed.md @@ -0,0 +1,10 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| SCANNER-WORKER-09-201 | DONE (2025-10-19) | Scanner Worker Guild | SCANNER-CORE-09-501 | Worker host bootstrap with Authority auth, hosted services, and graceful shutdown semantics. | `Program.cs` binds `Scanner:Worker` options, registers delay scheduler, configures telemetry + Authority client, and enforces shutdown timeout. | +| SCANNER-WORKER-09-202 | DONE (2025-10-19) | Scanner Worker Guild | SCANNER-WORKER-09-201, SCANNER-QUEUE-09-401 | Lease/heartbeat loop with retry+jitter, poison-job quarantine, structured logging. | `ScannerWorkerHostedService` + `LeaseHeartbeatService` manage concurrency, renewal margins, poison handling, and structured logs exercised by integration fixture. | +| SCANNER-WORKER-09-203 | DONE (2025-10-19) | Scanner Worker Guild | SCANNER-WORKER-09-202, SCANNER-STORAGE-09-301 | Analyzer dispatch skeleton emitting deterministic stage progress and honoring cancellation tokens. | Deterministic stage list + `ScanProgressReporter`; `WorkerBasicScanScenario` validates ordering and cancellation propagation. | +| SCANNER-WORKER-09-204 | DONE (2025-10-19) | Scanner Worker Guild | SCANNER-WORKER-09-203 | Worker metrics (queue latency, stage duration, failure counts) with OpenTelemetry resource wiring. | `ScannerWorkerMetrics` records queue/job/stage metrics; integration test asserts analyzer stage histogram entries. | +| SCANNER-WORKER-09-205 | DONE (2025-10-19) | Scanner Worker Guild | SCANNER-WORKER-09-202 | Harden heartbeat jitter so lease safety margin stays ≥3× and cover with regression tests. | `LeaseHeartbeatService` clamps jitter to safety window, validator enforces ≥3 safety factor, regression tests cover heartbeat scheduling and metrics. | +| SCANNER-WORKER-10-201 | DONE (2025-10-19) | Scanner Worker Guild | SCANNER-CACHE-10-101 | Wire scanner cache services and maintenance into worker host. | `AddScannerCache` registered with worker configuration; cache maintenance hosted service runs respecting enabled/auto-evict flags. | diff --git a/src/Scanner/StellaOps.Scanner.Worker/TASKS.md b/src/Scanner/StellaOps.Scanner.Worker/TASKS.md index a21d474d..0daf77b6 100644 --- a/src/Scanner/StellaOps.Scanner.Worker/TASKS.md +++ b/src/Scanner/StellaOps.Scanner.Worker/TASKS.md @@ -1,10 +1,4 @@ -# Scanner Worker Task Board - -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| SCANNER-WORKER-09-201 | DONE (2025-10-19) | Scanner Worker Guild | SCANNER-CORE-09-501 | Worker host bootstrap with Authority auth, hosted services, and graceful shutdown semantics. | `Program.cs` binds `Scanner:Worker` options, registers delay scheduler, configures telemetry + Authority client, and enforces shutdown timeout. | -| SCANNER-WORKER-09-202 | DONE (2025-10-19) | Scanner Worker Guild | SCANNER-WORKER-09-201, SCANNER-QUEUE-09-401 | Lease/heartbeat loop with retry+jitter, poison-job quarantine, structured logging. | `ScannerWorkerHostedService` + `LeaseHeartbeatService` manage concurrency, renewal margins, poison handling, and structured logs exercised by integration fixture. | -| SCANNER-WORKER-09-203 | DONE (2025-10-19) | Scanner Worker Guild | SCANNER-WORKER-09-202, SCANNER-STORAGE-09-301 | Analyzer dispatch skeleton emitting deterministic stage progress and honoring cancellation tokens. | Deterministic stage list + `ScanProgressReporter`; `WorkerBasicScanScenario` validates ordering and cancellation propagation. | -| SCANNER-WORKER-09-204 | DONE (2025-10-19) | Scanner Worker Guild | SCANNER-WORKER-09-203 | Worker metrics (queue latency, stage duration, failure counts) with OpenTelemetry resource wiring. | `ScannerWorkerMetrics` records queue/job/stage metrics; integration test asserts analyzer stage histogram entries. | -| SCANNER-WORKER-09-205 | DONE (2025-10-19) | Scanner Worker Guild | SCANNER-WORKER-09-202 | Harden heartbeat jitter so lease safety margin stays ≥3× and cover with regression tests. | `LeaseHeartbeatService` clamps jitter to safety window, validator enforces ≥3 safety factor, regression tests cover heartbeat scheduling and metrics. | -| SCANNER-WORKER-10-201 | DONE (2025-10-19) | Scanner Worker Guild | SCANNER-CACHE-10-101 | Wire scanner cache services and maintenance into worker host. | `AddScannerCache` registered with worker configuration; cache maintenance hosted service runs respecting enabled/auto-evict flags. | +# Scanner Worker Task Board + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| diff --git a/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.DotNet/TASKS.completed.md b/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.DotNet/TASKS.completed.md new file mode 100644 index 00000000..8e31cdfa --- /dev/null +++ b/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.DotNet/TASKS.completed.md @@ -0,0 +1,14 @@ +# Completed Tasks + +| 1 | SCANNER-ANALYZERS-LANG-10-305A | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-307 | Parse `*.deps.json` + `runtimeconfig.json`, build RID graph, and normalize to `pkg:nuget` components. | RID graph deterministic; fixtures confirm consistent component ordering; fallback to `bin:{sha256}` documented. | + +| 2 | SCANNER-ANALYZERS-LANG-10-305B | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-305A | Extract assembly metadata (strong name, file/product info) and optional Authenticode details when offline cert bundle provided. | Signing metadata captured for signed assemblies; offline trust store documented; hash validations deterministic. | + +| 3 | SCANNER-ANALYZERS-LANG-10-305C | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-305B | Handle self-contained apps and native assets; merge with EntryTrace usage hints. | Self-contained fixtures map to components with RID flags; usage hints propagate; tests cover linux/win variants. | + +| 4 | SCANNER-ANALYZERS-LANG-10-307D | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-305C | Integrate shared helpers (license mapping, quiet provenance) and concurrency-safe caches. | Shared helpers reused; concurrency tests for parallel layer scans pass; no redundant allocations. | + +| 5 | SCANNER-ANALYZERS-LANG-10-308D | DONE (2025-10-23) | SCANNER-ANALYZERS-LANG-10-307D | Determinism fixtures + benchmark harness; compare to competitor scanners for accuracy/perf. | Fixtures in `Fixtures/lang/dotnet/`; determinism CI guard; benchmark demonstrates lower duplication + faster runtime. | + +| 6 | SCANNER-ANALYZERS-LANG-10-309D | DONE (2025-10-23) | SCANNER-ANALYZERS-LANG-10-308D | Package plug-in (manifest, DI registration) and update Offline Kit instructions. | Manifest copied to `plugins/scanner/analyzers/lang/`; Worker loads analyzer; Offline Kit doc updated. | + diff --git a/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.DotNet/TASKS.md b/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.DotNet/TASKS.md index 94e5945c..cd329a68 100644 --- a/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.DotNet/TASKS.md +++ b/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.DotNet/TASKS.md @@ -1,19 +1,13 @@ -# .NET Analyzer Task Flow - -| Seq | ID | Status | Depends on | Description | Exit Criteria | -|-----|----|--------|------------|-------------|---------------| -| 1 | SCANNER-ANALYZERS-LANG-10-305A | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-307 | Parse `*.deps.json` + `runtimeconfig.json`, build RID graph, and normalize to `pkg:nuget` components. | RID graph deterministic; fixtures confirm consistent component ordering; fallback to `bin:{sha256}` documented. | -| 2 | SCANNER-ANALYZERS-LANG-10-305B | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-305A | Extract assembly metadata (strong name, file/product info) and optional Authenticode details when offline cert bundle provided. | Signing metadata captured for signed assemblies; offline trust store documented; hash validations deterministic. | -| 3 | SCANNER-ANALYZERS-LANG-10-305C | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-305B | Handle self-contained apps and native assets; merge with EntryTrace usage hints. | Self-contained fixtures map to components with RID flags; usage hints propagate; tests cover linux/win variants. | -| 4 | SCANNER-ANALYZERS-LANG-10-307D | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-305C | Integrate shared helpers (license mapping, quiet provenance) and concurrency-safe caches. | Shared helpers reused; concurrency tests for parallel layer scans pass; no redundant allocations. | -| 5 | SCANNER-ANALYZERS-LANG-10-308D | DONE (2025-10-23) | SCANNER-ANALYZERS-LANG-10-307D | Determinism fixtures + benchmark harness; compare to competitor scanners for accuracy/perf. | Fixtures in `Fixtures/lang/dotnet/`; determinism CI guard; benchmark demonstrates lower duplication + faster runtime. | -| 6 | SCANNER-ANALYZERS-LANG-10-309D | DONE (2025-10-23) | SCANNER-ANALYZERS-LANG-10-308D | Package plug-in (manifest, DI registration) and update Offline Kit instructions. | Manifest copied to `plugins/scanner/analyzers/lang/`; Worker loads analyzer; Offline Kit doc updated. | - -## .NET Entry-Point & Dependency Resolver (Sprint 11) -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| SCANNER-ANALYZERS-LANG-11-001 | TODO | StellaOps.Scanner EPDR Guild, Language Analyzer Guild | - | Build entrypoint resolver that maps project/publish artifacts to entrypoint identities (assembly name, MVID, TFM, RID) and environment profiles (publish mode, host kind, probing paths). Output normalized `entrypoints[]` records with deterministic IDs. | Entrypoint records produced for fixtures (framework-dependent, self-contained, single-file, multi-TFM/RID); determinism check passes; docs updated. | -| SCANNER-ANALYZERS-LANG-11-002 | TODO | StellaOps.Scanner EPDR Guild | SCANNER-ANALYZERS-LANG-11-001 | Implement static analyzer (IL + reflection heuristics) capturing AssemblyRef, ModuleRef/PInvoke, DynamicDependency, reflection literals, DI patterns, and custom AssemblyLoadContext probing hints. Emit dependency edges with reason codes and confidence. | Static analysis coverage demonstrated on fixtures; edges carry reason codes (`il-assemblyref`, `il-moduleref`, `reflection-literal`, `alc-probing`); tests cover trimmed/single-file cases. | -| SCANNER-ANALYZERS-LANG-11-003 | TODO | StellaOps.Scanner EPDR Guild, Signals Guild | SCANNER-ANALYZERS-LANG-11-002 | Ingest optional runtime evidence (AssemblyLoad, Resolving, P/Invoke) via event listener harness; merge runtime edges with static/declared ones and attach reason codes/confidence. | Runtime listener service pluggable; fixtures record runtime edges; merged output shows combined reason set with confidence per edge. | -| SCANNER-ANALYZERS-LANG-11-004 | TODO | StellaOps.Scanner EPDR Guild, SBOM Service Guild | SCANNER-ANALYZERS-LANG-11-002 | Produce normalized observation export to Scanner writer: entrypoints + dependency edges + environment profiles (AOC compliant). Wire to SBOM service entrypoint tagging. | Analyzer writes observation records consumed by SBOM service tests; AOC compliance docs updated; determinism checked. | -| SCANNER-ANALYZERS-LANG-11-005 | TODO | StellaOps.Scanner EPDR Guild, QA Guild | SCANNER-ANALYZERS-LANG-11-004 | Add comprehensive fixtures/benchmarks covering framework-dependent, self-contained, single-file, trimmed, NativeAOT, multi-RID scenarios; include explain traces and perf benchmarks vs previous analyzer. | Fixtures stored under `fixtures/lang/dotnet/epdr`; determinism + perf thresholds validated; benchmark results documented. | +# .NET Analyzer Task Flow + +| Seq | ID | Status | Depends on | Description | Exit Criteria | +|-----|----|--------|------------|-------------|---------------| + +## .NET Entry-Point & Dependency Resolver (Sprint 11) +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| SCANNER-ANALYZERS-LANG-11-001 | TODO | StellaOps.Scanner EPDR Guild, Language Analyzer Guild | - | Build entrypoint resolver that maps project/publish artifacts to entrypoint identities (assembly name, MVID, TFM, RID) and environment profiles (publish mode, host kind, probing paths). Output normalized `entrypoints[]` records with deterministic IDs. | Entrypoint records produced for fixtures (framework-dependent, self-contained, single-file, multi-TFM/RID); determinism check passes; docs updated. | +| SCANNER-ANALYZERS-LANG-11-002 | TODO | StellaOps.Scanner EPDR Guild | SCANNER-ANALYZERS-LANG-11-001 | Implement static analyzer (IL + reflection heuristics) capturing AssemblyRef, ModuleRef/PInvoke, DynamicDependency, reflection literals, DI patterns, and custom AssemblyLoadContext probing hints. Emit dependency edges with reason codes and confidence. | Static analysis coverage demonstrated on fixtures; edges carry reason codes (`il-assemblyref`, `il-moduleref`, `reflection-literal`, `alc-probing`); tests cover trimmed/single-file cases. | +| SCANNER-ANALYZERS-LANG-11-003 | TODO | StellaOps.Scanner EPDR Guild, Signals Guild | SCANNER-ANALYZERS-LANG-11-002 | Ingest optional runtime evidence (AssemblyLoad, Resolving, P/Invoke) via event listener harness; merge runtime edges with static/declared ones and attach reason codes/confidence. | Runtime listener service pluggable; fixtures record runtime edges; merged output shows combined reason set with confidence per edge. | +| SCANNER-ANALYZERS-LANG-11-004 | TODO | StellaOps.Scanner EPDR Guild, SBOM Service Guild | SCANNER-ANALYZERS-LANG-11-002 | Produce normalized observation export to Scanner writer: entrypoints + dependency edges + environment profiles (AOC compliant). Wire to SBOM service entrypoint tagging. | Analyzer writes observation records consumed by SBOM service tests; AOC compliance docs updated; determinism checked. | +| SCANNER-ANALYZERS-LANG-11-005 | TODO | StellaOps.Scanner EPDR Guild, QA Guild | SCANNER-ANALYZERS-LANG-11-004 | Add comprehensive fixtures/benchmarks covering framework-dependent, self-contained, single-file, trimmed, NativeAOT, multi-RID scenarios; include explain traces and perf benchmarks vs previous analyzer. | Fixtures stored under `fixtures/lang/dotnet/epdr`; determinism + perf thresholds validated; benchmark results documented. | diff --git a/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Go/TASKS.completed.md b/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Go/TASKS.completed.md new file mode 100644 index 00000000..f15ab2bb --- /dev/null +++ b/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Go/TASKS.completed.md @@ -0,0 +1,18 @@ +# Completed Tasks + +| 1 | SCANNER-ANALYZERS-LANG-10-304A | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-307 | Parse Go build info blob (`runtime/debug` format) and `.note.go.buildid`; map to module/version and evidence. | Build info extracted across Go 1.18–1.23 fixtures; evidence includes VCS, module path, and build settings. | + +| 2 | SCANNER-ANALYZERS-LANG-10-304B | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-304A | Implement DWARF-lite reader for VCS metadata + dirty flag; add cache to avoid re-reading identical binaries. | DWARF reader supplies commit hash for ≥95 % fixtures; cache reduces duplicated IO by ≥70 %. | + +| 3 | SCANNER-ANALYZERS-LANG-10-304C | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-304B | Fallback heuristics for stripped binaries with deterministic `bin:{sha256}` labeling and quiet provenance. | Heuristic labels clearly separated; tests ensure no false “observed” provenance; documentation updated. | + +| 4 | SCANNER-ANALYZERS-LANG-10-307G | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-304C | Wire shared helpers (license mapping, usage flags) and ensure concurrency-safe buffer reuse. | Analyzer reuses shared infrastructure; concurrency tests with parallel scans pass; no data races. | + +| 5 | SCANNER-ANALYZERS-LANG-10-308G | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-307G | Determinism fixtures + benchmark harness (Vs competitor). | Fixtures under `Fixtures/lang/go/`; CI determinism check; benchmark runs showing ≥20 % speed advantage. | + +| 6 | SCANNER-ANALYZERS-LANG-10-309G | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-308G | Package plug-in manifest + Offline Kit notes; ensure Worker DI registration. | Manifest copied; Worker loads analyzer; Offline Kit docs updated with Go analyzer presence. | + +| 7 | SCANNER-ANALYZERS-LANG-10-304D | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-304C | Emit telemetry counters for stripped-binary heuristics and document metrics wiring. | New `scanner_analyzer_golang_heuristic_total` counter recorded; docs updated with offline aggregation notes. | + +| 8 | SCANNER-ANALYZERS-LANG-10-304E | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-304D | Plumb Go heuristic counter into Scanner metrics pipeline and alerting. | Counter emitted through Worker telemetry/export pipeline; dashboard & alert rule documented; smoke test proves metric visibility. | + diff --git a/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Go/TASKS.md b/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Go/TASKS.md index 8e1a38e3..8c163aeb 100644 --- a/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Go/TASKS.md +++ b/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Go/TASKS.md @@ -1,12 +1,4 @@ -# Go Analyzer Task Flow - -| Seq | ID | Status | Depends on | Description | Exit Criteria | -|-----|----|--------|------------|-------------|---------------| -| 1 | SCANNER-ANALYZERS-LANG-10-304A | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-307 | Parse Go build info blob (`runtime/debug` format) and `.note.go.buildid`; map to module/version and evidence. | Build info extracted across Go 1.18–1.23 fixtures; evidence includes VCS, module path, and build settings. | -| 2 | SCANNER-ANALYZERS-LANG-10-304B | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-304A | Implement DWARF-lite reader for VCS metadata + dirty flag; add cache to avoid re-reading identical binaries. | DWARF reader supplies commit hash for ≥95 % fixtures; cache reduces duplicated IO by ≥70 %. | -| 3 | SCANNER-ANALYZERS-LANG-10-304C | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-304B | Fallback heuristics for stripped binaries with deterministic `bin:{sha256}` labeling and quiet provenance. | Heuristic labels clearly separated; tests ensure no false “observed” provenance; documentation updated. | -| 4 | SCANNER-ANALYZERS-LANG-10-307G | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-304C | Wire shared helpers (license mapping, usage flags) and ensure concurrency-safe buffer reuse. | Analyzer reuses shared infrastructure; concurrency tests with parallel scans pass; no data races. | -| 5 | SCANNER-ANALYZERS-LANG-10-308G | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-307G | Determinism fixtures + benchmark harness (Vs competitor). | Fixtures under `Fixtures/lang/go/`; CI determinism check; benchmark runs showing ≥20 % speed advantage. | -| 6 | SCANNER-ANALYZERS-LANG-10-309G | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-308G | Package plug-in manifest + Offline Kit notes; ensure Worker DI registration. | Manifest copied; Worker loads analyzer; Offline Kit docs updated with Go analyzer presence. | -| 7 | SCANNER-ANALYZERS-LANG-10-304D | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-304C | Emit telemetry counters for stripped-binary heuristics and document metrics wiring. | New `scanner_analyzer_golang_heuristic_total` counter recorded; docs updated with offline aggregation notes. | -| 8 | SCANNER-ANALYZERS-LANG-10-304E | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-304D | Plumb Go heuristic counter into Scanner metrics pipeline and alerting. | Counter emitted through Worker telemetry/export pipeline; dashboard & alert rule documented; smoke test proves metric visibility. | +# Go Analyzer Task Flow + +| Seq | ID | Status | Depends on | Description | Exit Criteria | +|-----|----|--------|------------|-------------|---------------| diff --git a/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Java/TASKS.completed.md b/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Java/TASKS.completed.md new file mode 100644 index 00000000..20869eb1 --- /dev/null +++ b/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Java/TASKS.completed.md @@ -0,0 +1,8 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| SCANNER-ANALYZERS-JAVA-21-001 | DONE (2025-10-27) | Java Analyzer Guild | SCANNER-CORE-09-501 | Build input normalizer and virtual file system for JAR/WAR/EAR/fat-jar/JMOD/jimage/container roots. Detect packaging type, layered dirs (BOOT-INF/WEB-INF), multi-release overlays, and jlink runtime metadata. | Normalizer walks fixtures without extraction, classifies packaging, selects MR overlays deterministically, records java version + vendor from runtime images. | +| SCANNER-ANALYZERS-JAVA-21-002 | DONE (2025-10-27) | Java Analyzer Guild | SCANNER-ANALYZERS-JAVA-21-001 | Implement module/classpath builder: JPMS graph parser (`module-info.class`), classpath order rules (fat jar, war, ear), duplicate & split-package detection, package fingerprinting. | Classpath order reproduced for fixtures; module graph serialized; duplicate provider + split-package warnings emitted deterministically. | +| SCANNER-ANALYZERS-JAVA-21-003 | DONE (2025-10-27) | Java Analyzer Guild | SCANNER-ANALYZERS-JAVA-21-002 | SPI scanner covering META-INF/services, provider selection, and warning generation. Include configurable SPI corpus (JDK, Spring, logging, Jackson, MicroProfile). | SPI tables produced with selected provider + candidates; fixtures show first-wins behaviour; warnings recorded for duplicate providers. | +| SCANNER-ANALYZERS-JAVA-21-004 | DONE (2025-10-29) | Java Analyzer Guild | SCANNER-ANALYZERS-JAVA-21-002 | Reflection/dynamic loader heuristics: scan constant pools, bytecode sites (Class.forName, loadClass, TCCL usage), resource-based plugin hints, manifest loader hints. Emit edges with reason codes + confidence. | Reflection edges generated for fixtures (classpath, boot, war); includes call site metadata and confidence scoring; TCCL warning emitted where detected. | diff --git a/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Java/TASKS.md b/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Java/TASKS.md index d81a25ea..ef148371 100644 --- a/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Java/TASKS.md +++ b/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Java/TASKS.md @@ -1,31 +1,27 @@ -# Java Analyzer Task Board -> **Imposed rule:** work of this type or tasks of this type on this component — and everywhere else it should be applied. - -## Java Static Core (Sprint 39) -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| SCANNER-ANALYZERS-JAVA-21-001 | DONE (2025-10-27) | Java Analyzer Guild | SCANNER-CORE-09-501 | Build input normalizer and virtual file system for JAR/WAR/EAR/fat-jar/JMOD/jimage/container roots. Detect packaging type, layered dirs (BOOT-INF/WEB-INF), multi-release overlays, and jlink runtime metadata. | Normalizer walks fixtures without extraction, classifies packaging, selects MR overlays deterministically, records java version + vendor from runtime images. | -| SCANNER-ANALYZERS-JAVA-21-002 | DONE (2025-10-27) | Java Analyzer Guild | SCANNER-ANALYZERS-JAVA-21-001 | Implement module/classpath builder: JPMS graph parser (`module-info.class`), classpath order rules (fat jar, war, ear), duplicate & split-package detection, package fingerprinting. | Classpath order reproduced for fixtures; module graph serialized; duplicate provider + split-package warnings emitted deterministically. | -| SCANNER-ANALYZERS-JAVA-21-003 | DONE (2025-10-27) | Java Analyzer Guild | SCANNER-ANALYZERS-JAVA-21-002 | SPI scanner covering META-INF/services, provider selection, and warning generation. Include configurable SPI corpus (JDK, Spring, logging, Jackson, MicroProfile). | SPI tables produced with selected provider + candidates; fixtures show first-wins behaviour; warnings recorded for duplicate providers. | -| SCANNER-ANALYZERS-JAVA-21-004 | DONE (2025-10-29) | Java Analyzer Guild | SCANNER-ANALYZERS-JAVA-21-002 | Reflection/dynamic loader heuristics: scan constant pools, bytecode sites (Class.forName, loadClass, TCCL usage), resource-based plugin hints, manifest loader hints. Emit edges with reason codes + confidence. | Reflection edges generated for fixtures (classpath, boot, war); includes call site metadata and confidence scoring; TCCL warning emitted where detected. | -| SCANNER-ANALYZERS-JAVA-21-005 | TODO | Java Analyzer Guild | SCANNER-ANALYZERS-JAVA-21-002 | Framework config extraction: Spring Boot imports, spring.factories, application properties/yaml, Jakarta web.xml & fragments, JAX-RS/JPA/CDI/JAXB configs, logging files, Graal native-image configs. | Framework fixtures parsed; relevant class FQCNs surfaced with reasons (`config-spring`, `config-jaxrs`, etc.); non-class config ignored; determinism guard passes. | -| SCANNER-ANALYZERS-JAVA-21-006 | TODO | Java Analyzer Guild | SCANNER-ANALYZERS-JAVA-21-002 | JNI/native hint scanner: detect native methods, System.load/Library literals, bundled native libs, Graal JNI configs; emit `jni-load` edges for native analyzer correlation. | JNI fixtures produce hint edges pointing at embedded libs; metadata includes candidate paths and reason `jni`. | -| SCANNER-ANALYZERS-JAVA-21-007 | TODO | Java Analyzer Guild | SCANNER-ANALYZERS-JAVA-21-003 | Signature and manifest metadata collector: verify JAR signature structure, capture signers, manifest loader attributes (Main-Class, Agent-Class, Start-Class, Class-Path). | Signed jar fixture reports signer info and structural validation result; manifest metadata attached to entrypoints. | - -> 2025-10-27 — SCANNER-ANALYZERS-JAVA-21-001 implemented `JavaWorkspaceNormalizer` + fixtures covering packaging, layered directories, multi-release overlays, and runtime image metadata. -> -> 2025-10-27 — SCANNER-ANALYZERS-JAVA-21-002 delivered `JavaClassPathBuilder` producing ordered segments (jar/war/boot fat, embedded libs), JPMS descriptors via `JavaModuleInfoParser`, and duplicate/split-package detection with package fingerprints + unit tests. -> -> 2025-10-27 — SCANNER-ANALYZERS-JAVA-21-004 in progress: added bytecode-driven `JavaReflectionAnalyzer` covering `Class.forName`, `ClassLoader.loadClass`, `ServiceLoader.load`, resource lookups, and TCCL warnings with unit fixtures (boot jar, embedded jar, synthetic classes). -> -> 2025-10-27 — SCANNER-ANALYZERS-JAVA-21-003 added SPI catalog + `JavaServiceProviderScanner`, capturing META-INF/services across layered jars, selecting first-wins providers, and emitting duplicate warnings with coverage tests (fat-jar, duplicates, simple jars). - -## Java Observation & Runtime (Sprint 40) -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| SCANNER-ANALYZERS-JAVA-21-008 | BLOCKED (2025-10-27) | Java Analyzer Guild | SCANNER-ANALYZERS-JAVA-21-003, SCANNER-ANALYZERS-JAVA-21-004, SCANNER-ANALYZERS-JAVA-21-005 | Implement resolver + AOC writer: produce entrypoints (env profiles, warnings), components (jar_id + semantic ids), edges (jpms, cp, spi, reflect, jni) with reason codes/confidence. | Observation JSON for fixtures deterministic; includes entrypoints, edges, warnings; passes AOC compliance lint. | -| SCANNER-ANALYZERS-JAVA-21-009 | TODO | Java Analyzer Guild, QA Guild | SCANNER-ANALYZERS-JAVA-21-008 | Author comprehensive fixtures (modular app, boot fat jar, war, ear, MR-jar, jlink image, JNI, reflection heavy, signed jar, microprofile) with golden outputs and perf benchmarks. | Fixture suite committed under `fixtures/lang/java/ep`; determinism + benchmark gates (<300ms fat jar) configured in CI. | -| SCANNER-ANALYZERS-JAVA-21-010 | TODO | Java Analyzer Guild, Signals Guild | SCANNER-ANALYZERS-JAVA-21-008 | Optional runtime ingestion: Java agent + JFR reader capturing class load, ServiceLoader, and System.load events with path scrubbing. Emit append-only runtime edges `runtime-class`/`runtime-spi`/`runtime-load`. | Runtime harness produces scrubbed events for sample app; edges merge with static output; docs describe sandbox & privacy. | +# Java Analyzer Task Board +> **Imposed rule:** work of this type or tasks of this type on this component — and everywhere else it should be applied. + +## Java Static Core (Sprint 39) +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| SCANNER-ANALYZERS-JAVA-21-005 | TODO | Java Analyzer Guild | SCANNER-ANALYZERS-JAVA-21-002 | Framework config extraction: Spring Boot imports, spring.factories, application properties/yaml, Jakarta web.xml & fragments, JAX-RS/JPA/CDI/JAXB configs, logging files, Graal native-image configs. | Framework fixtures parsed; relevant class FQCNs surfaced with reasons (`config-spring`, `config-jaxrs`, etc.); non-class config ignored; determinism guard passes. | +| SCANNER-ANALYZERS-JAVA-21-006 | TODO | Java Analyzer Guild | SCANNER-ANALYZERS-JAVA-21-002 | JNI/native hint scanner: detect native methods, System.load/Library literals, bundled native libs, Graal JNI configs; emit `jni-load` edges for native analyzer correlation. | JNI fixtures produce hint edges pointing at embedded libs; metadata includes candidate paths and reason `jni`. | +| SCANNER-ANALYZERS-JAVA-21-007 | TODO | Java Analyzer Guild | SCANNER-ANALYZERS-JAVA-21-003 | Signature and manifest metadata collector: verify JAR signature structure, capture signers, manifest loader attributes (Main-Class, Agent-Class, Start-Class, Class-Path). | Signed jar fixture reports signer info and structural validation result; manifest metadata attached to entrypoints. | + +> 2025-10-27 — SCANNER-ANALYZERS-JAVA-21-001 implemented `JavaWorkspaceNormalizer` + fixtures covering packaging, layered directories, multi-release overlays, and runtime image metadata. +> +> 2025-10-27 — SCANNER-ANALYZERS-JAVA-21-002 delivered `JavaClassPathBuilder` producing ordered segments (jar/war/boot fat, embedded libs), JPMS descriptors via `JavaModuleInfoParser`, and duplicate/split-package detection with package fingerprints + unit tests. +> +> 2025-10-27 — SCANNER-ANALYZERS-JAVA-21-004 in progress: added bytecode-driven `JavaReflectionAnalyzer` covering `Class.forName`, `ClassLoader.loadClass`, `ServiceLoader.load`, resource lookups, and TCCL warnings with unit fixtures (boot jar, embedded jar, synthetic classes). +> +> 2025-10-27 — SCANNER-ANALYZERS-JAVA-21-003 added SPI catalog + `JavaServiceProviderScanner`, capturing META-INF/services across layered jars, selecting first-wins providers, and emitting duplicate warnings with coverage tests (fat-jar, duplicates, simple jars). + +## Java Observation & Runtime (Sprint 40) +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| SCANNER-ANALYZERS-JAVA-21-008 | BLOCKED (2025-10-27) | Java Analyzer Guild | SCANNER-ANALYZERS-JAVA-21-003, SCANNER-ANALYZERS-JAVA-21-004, SCANNER-ANALYZERS-JAVA-21-005 | Implement resolver + AOC writer: produce entrypoints (env profiles, warnings), components (jar_id + semantic ids), edges (jpms, cp, spi, reflect, jni) with reason codes/confidence. | Observation JSON for fixtures deterministic; includes entrypoints, edges, warnings; passes AOC compliance lint. | +| SCANNER-ANALYZERS-JAVA-21-009 | TODO | Java Analyzer Guild, QA Guild | SCANNER-ANALYZERS-JAVA-21-008 | Author comprehensive fixtures (modular app, boot fat jar, war, ear, MR-jar, jlink image, JNI, reflection heavy, signed jar, microprofile) with golden outputs and perf benchmarks. | Fixture suite committed under `fixtures/lang/java/ep`; determinism + benchmark gates (<300ms fat jar) configured in CI. | +| SCANNER-ANALYZERS-JAVA-21-010 | TODO | Java Analyzer Guild, Signals Guild | SCANNER-ANALYZERS-JAVA-21-008 | Optional runtime ingestion: Java agent + JFR reader capturing class load, ServiceLoader, and System.load events with path scrubbing. Emit append-only runtime edges `runtime-class`/`runtime-spi`/`runtime-load`. | Runtime harness produces scrubbed events for sample app; edges merge with static output; docs describe sandbox & privacy. | | SCANNER-ANALYZERS-JAVA-21-011 | TODO | Java Analyzer Guild, DevOps Guild | SCANNER-ANALYZERS-JAVA-21-008 | Package analyzer as restart-time plug-in (manifest/DI), update Offline Kit docs, add CLI/worker hooks for Java inspection commands. | Plugin manifest deployed to `plugins/scanner/analyzers/lang/`; Worker loads new analyzer; Offline Kit + CLI instructions updated; smoke test verifies packaging. | > 2025-10-27 — SCANNER-ANALYZERS-JAVA-21-008 blocked pending upstream completion of tasks 003–005 (module/classpath resolver, SPI scanner, reflection/config extraction). Observation writer needs their outputs for components/edges/warnings per exit criteria. diff --git a/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Node/TASKS.completed.md b/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Node/TASKS.completed.md new file mode 100644 index 00000000..a5f9de2b --- /dev/null +++ b/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Node/TASKS.completed.md @@ -0,0 +1,14 @@ +# Completed Tasks + +| 1 | SCANNER-ANALYZERS-LANG-10-302A | DONE (2025-10-19) | SCANNER-ANALYZERS-LANG-10-307 | Build deterministic module graph walker covering npm, Yarn, and PNPM; capture package.json provenance and integrity metadata. | Walker indexes >100 k modules in <1.5 s (hot cache); golden fixtures verify deterministic ordering and path normalization. | + +| 2 | SCANNER-ANALYZERS-LANG-10-302B | DONE (2025-10-19) | SCANNER-ANALYZERS-LANG-10-302A | Resolve workspaces/symlinks and attribute components to originating package with usage hints; guard against directory traversal. | Workspace attribution accurate on multi-workspace fixture; symlink resolver proves canonical path; security tests ensure no traversal. | + +| 3 | SCANNER-ANALYZERS-LANG-10-302C | DONE (2025-10-19) | SCANNER-ANALYZERS-LANG-10-302B | Surface script metadata (postinstall/preinstall) and policy hints; emit telemetry counters and evidence records. | Analyzer output includes script metadata + evidence; metrics `scanner_analyzer_node_scripts_total` recorded; policy hints documented. | + +| 4 | SCANNER-ANALYZERS-LANG-10-307N | DONE (2025-10-21) | SCANNER-ANALYZERS-LANG-10-302C | Integrate shared helpers for license/licence evidence, canonical JSON serialization, and usage flag propagation. | Reuse shared helpers without duplication; unit tests confirm stable metadata merge; no analyzer-specific serializer drift. | + +| 5 | SCANNER-ANALYZERS-LANG-10-308N | DONE (2025-10-21) | SCANNER-ANALYZERS-LANG-10-307N | Author determinism harness + fixtures for Node analyzer; add benchmark suite. | Fixtures committed under `Fixtures/lang/node/`; determinism CI job compares JSON snapshots; benchmark CSV published. | + +| 6 | SCANNER-ANALYZERS-LANG-10-309N | DONE (2025-10-21) | SCANNER-ANALYZERS-LANG-10-308N | Package Node analyzer as restart-time plug-in (manifest, DI registration, Offline Kit notes). | Manifest copied to `plugins/scanner/analyzers/lang/`; Worker loads analyzer after restart; Offline Kit docs updated. | + diff --git a/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Node/TASKS.md b/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Node/TASKS.md index 43f48e8c..f5d7684a 100644 --- a/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Node/TASKS.md +++ b/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Node/TASKS.md @@ -1,31 +1,25 @@ -# Node Analyzer Task Flow - -| Seq | ID | Status | Depends on | Description | Exit Criteria | -|-----|----|--------|------------|-------------|---------------| -| 1 | SCANNER-ANALYZERS-LANG-10-302A | DONE (2025-10-19) | SCANNER-ANALYZERS-LANG-10-307 | Build deterministic module graph walker covering npm, Yarn, and PNPM; capture package.json provenance and integrity metadata. | Walker indexes >100 k modules in <1.5 s (hot cache); golden fixtures verify deterministic ordering and path normalization. | -| 2 | SCANNER-ANALYZERS-LANG-10-302B | DONE (2025-10-19) | SCANNER-ANALYZERS-LANG-10-302A | Resolve workspaces/symlinks and attribute components to originating package with usage hints; guard against directory traversal. | Workspace attribution accurate on multi-workspace fixture; symlink resolver proves canonical path; security tests ensure no traversal. | -| 3 | SCANNER-ANALYZERS-LANG-10-302C | DONE (2025-10-19) | SCANNER-ANALYZERS-LANG-10-302B | Surface script metadata (postinstall/preinstall) and policy hints; emit telemetry counters and evidence records. | Analyzer output includes script metadata + evidence; metrics `scanner_analyzer_node_scripts_total` recorded; policy hints documented. | -| 4 | SCANNER-ANALYZERS-LANG-10-307N | DONE (2025-10-21) | SCANNER-ANALYZERS-LANG-10-302C | Integrate shared helpers for license/licence evidence, canonical JSON serialization, and usage flag propagation. | Reuse shared helpers without duplication; unit tests confirm stable metadata merge; no analyzer-specific serializer drift. | -| 5 | SCANNER-ANALYZERS-LANG-10-308N | DONE (2025-10-21) | SCANNER-ANALYZERS-LANG-10-307N | Author determinism harness + fixtures for Node analyzer; add benchmark suite. | Fixtures committed under `Fixtures/lang/node/`; determinism CI job compares JSON snapshots; benchmark CSV published. | -| 6 | SCANNER-ANALYZERS-LANG-10-309N | DONE (2025-10-21) | SCANNER-ANALYZERS-LANG-10-308N | Package Node analyzer as restart-time plug-in (manifest, DI registration, Offline Kit notes). | Manifest copied to `plugins/scanner/analyzers/lang/`; Worker loads analyzer after restart; Offline Kit docs updated. | - -## Node Entry-Point Analyzer (Sprint 41) -> **Imposed rule:** work of this type or tasks of this type on this component — and everywhere else it should be applied. -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| SCANNER-ANALYZERS-NODE-22-001 | TODO | Node Analyzer Guild | SCANNER-ANALYZERS-LANG-10-309N | Build input normalizer + VFS for Node projects: dirs, tgz, container layers, pnpm store, Yarn PnP zips; detect Node version targets (`.nvmrc`, `.node-version`, Dockerfile) and workspace roots deterministically. | Normalizer handles fixtures (npm, pnpm, Yarn classic/PnP, container) without extraction; records node_version, workspace list, and symlink mode with golden outputs. | -| SCANNER-ANALYZERS-NODE-22-002 | TODO | Node Analyzer Guild | SCANNER-ANALYZERS-NODE-22-001 | Implement entrypoint discovery (bin/main/module/exports/imports, workers, electron, shebang scripts) and condition set builder per entrypoint. | Entrypoint inventory generated for fixtures (library, CLI, electron, worker); each entrypoint includes kind, start file, conditions; determinism harness updated. | -| SCANNER-ANALYZERS-NODE-22-003 | TODO | Node Analyzer Guild | SCANNER-ANALYZERS-NODE-22-001 | Parse JS/TS sources for static `import`, `require`, `import()` and string concat cases; flag dynamic patterns with confidence levels; support source map de-bundling. | Static edges + dynamic-specifier warnings emitted for fixtures (ESM, CJS, bundles); source map fixture rewrites concatenated modules with `confidence` metadata. | -| SCANNER-ANALYZERS-NODE-22-004 | TODO | Node Analyzer Guild | SCANNER-ANALYZERS-NODE-22-002 | Implement Node resolver engine for CJS + ESM (core modules, exports/imports maps, conditions, extension priorities, self-references) parameterised by node_version. | Resolver reproduces Node 18/20 semantics across fixture matrix; includes explain trace per edge; unit tests cover exports/conditions/extension ordering. | -| SCANNER-ANALYZERS-NODE-22-005 | TODO | Node Analyzer Guild | SCANNER-ANALYZERS-NODE-22-004 | Add package manager adapters: Yarn PnP (.pnp.data/.pnp.cjs), pnpm virtual store, npm/Yarn classic hoists; operate entirely in virtual FS. | PnP fixture resolves via `.pnp.data.json`/cache zips; pnpm fixture follows `.pnpm` symlinks; classic hoist fixture maintains first-wins ordering; warnings emitted for unreadable PnP. | -| SCANNER-ANALYZERS-NODE-22-006 | TODO | Node Analyzer Guild | SCANNER-ANALYZERS-NODE-22-004 | Detect bundles + source maps, reconstruct module specifiers, and correlate to original paths; support dual CJS/ESM graphs with conditions. | Bundle fixture using source maps produces `bundle-map` edges with medium confidence and original source paths; dual package fixture yields separate import/require graphs. | -| SCANNER-ANALYZERS-NODE-22-007 | TODO | Node Analyzer Guild | SCANNER-ANALYZERS-NODE-22-002 | Scan for native addons (.node), WASM modules, and core capability signals (child_process, vm, worker_threads); emit hint edges and native metadata. | Fixtures with native addon/WASM produce `native-addon`/`wasm` edges plus capability hints; metadata captures ABI, N-API, OS/arch where available. | - -## Node Observation & Runtime (Sprint 42) -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| SCANNER-ANALYZERS-NODE-22-008 | TODO | Node Analyzer Guild | SCANNER-ANALYZERS-NODE-22-004 | Produce AOC-compliant observations: entrypoints, components (pkg/native/wasm), edges (esm-import, cjs-require, exports, json, native-addon, wasm, worker) with reason codes/confidence and resolver traces. | Observation JSON for fixtures deterministic; edges include reason codes + confidence; resolves attach trace data; passes AOC lint. | -| SCANNER-ANALYZERS-NODE-22-009 | TODO | Node Analyzer Guild, QA Guild | SCANNER-ANALYZERS-NODE-22-008 | Author fixture suite + performance benchmarks (npm, pnpm, PnP, bundle, electron, worker) with golden outputs and latency budgets. | Fixtures stored under `fixtures/lang/node/ep`; determinism + perf (<350ms npm, <900ms PnP) enforced via CI benchmarks. | -| SCANNER-ANALYZERS-NODE-22-010 | TODO | Node Analyzer Guild, Signals Guild | SCANNER-ANALYZERS-NODE-22-008 | Implement optional runtime evidence hooks (ESM loader, CJS require hook) with path scrubbing and loader ID hashing; emit runtime-* edges. | Sandbox harness records runtime resolve events for sample app; paths hashed; runtime edges merge with static graph without altering first-wins selection. | -| SCANNER-ANALYZERS-NODE-22-011 | TODO | Node Analyzer Guild, DevOps Guild | SCANNER-ANALYZERS-NODE-22-008 | Package updated analyzer as restart-time plug-in, expose Scanner CLI (`stella node *`) commands, refresh Offline Kit documentation. | Plugins folder updated; worker loads analyzer; CLI commands documented and smoke-tested; Offline Kit instructions include Node analyzer usage. | -| SCANNER-ANALYZERS-NODE-22-012 | TODO | Node Analyzer Guild | SCANNER-ANALYZERS-NODE-22-001 | Integrate container filesystem adapter (OCI layers, Dockerfile hints) and record NODE_OPTIONS/env warnings. | Container fixture parsed: Node base image + NODE_OPTIONS captured; entrypoints resolved relative to image root; warnings emitted for loader flags. | +# Node Analyzer Task Flow + +| Seq | ID | Status | Depends on | Description | Exit Criteria | +|-----|----|--------|------------|-------------|---------------| + +## Node Entry-Point Analyzer (Sprint 41) +> **Imposed rule:** work of this type or tasks of this type on this component — and everywhere else it should be applied. +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| SCANNER-ANALYZERS-NODE-22-001 | TODO | Node Analyzer Guild | SCANNER-ANALYZERS-LANG-10-309N | Build input normalizer + VFS for Node projects: dirs, tgz, container layers, pnpm store, Yarn PnP zips; detect Node version targets (`.nvmrc`, `.node-version`, Dockerfile) and workspace roots deterministically. | Normalizer handles fixtures (npm, pnpm, Yarn classic/PnP, container) without extraction; records node_version, workspace list, and symlink mode with golden outputs. | +| SCANNER-ANALYZERS-NODE-22-002 | TODO | Node Analyzer Guild | SCANNER-ANALYZERS-NODE-22-001 | Implement entrypoint discovery (bin/main/module/exports/imports, workers, electron, shebang scripts) and condition set builder per entrypoint. | Entrypoint inventory generated for fixtures (library, CLI, electron, worker); each entrypoint includes kind, start file, conditions; determinism harness updated. | +| SCANNER-ANALYZERS-NODE-22-003 | TODO | Node Analyzer Guild | SCANNER-ANALYZERS-NODE-22-001 | Parse JS/TS sources for static `import`, `require`, `import()` and string concat cases; flag dynamic patterns with confidence levels; support source map de-bundling. | Static edges + dynamic-specifier warnings emitted for fixtures (ESM, CJS, bundles); source map fixture rewrites concatenated modules with `confidence` metadata. | +| SCANNER-ANALYZERS-NODE-22-004 | TODO | Node Analyzer Guild | SCANNER-ANALYZERS-NODE-22-002 | Implement Node resolver engine for CJS + ESM (core modules, exports/imports maps, conditions, extension priorities, self-references) parameterised by node_version. | Resolver reproduces Node 18/20 semantics across fixture matrix; includes explain trace per edge; unit tests cover exports/conditions/extension ordering. | +| SCANNER-ANALYZERS-NODE-22-005 | TODO | Node Analyzer Guild | SCANNER-ANALYZERS-NODE-22-004 | Add package manager adapters: Yarn PnP (.pnp.data/.pnp.cjs), pnpm virtual store, npm/Yarn classic hoists; operate entirely in virtual FS. | PnP fixture resolves via `.pnp.data.json`/cache zips; pnpm fixture follows `.pnpm` symlinks; classic hoist fixture maintains first-wins ordering; warnings emitted for unreadable PnP. | +| SCANNER-ANALYZERS-NODE-22-006 | TODO | Node Analyzer Guild | SCANNER-ANALYZERS-NODE-22-004 | Detect bundles + source maps, reconstruct module specifiers, and correlate to original paths; support dual CJS/ESM graphs with conditions. | Bundle fixture using source maps produces `bundle-map` edges with medium confidence and original source paths; dual package fixture yields separate import/require graphs. | +| SCANNER-ANALYZERS-NODE-22-007 | TODO | Node Analyzer Guild | SCANNER-ANALYZERS-NODE-22-002 | Scan for native addons (.node), WASM modules, and core capability signals (child_process, vm, worker_threads); emit hint edges and native metadata. | Fixtures with native addon/WASM produce `native-addon`/`wasm` edges plus capability hints; metadata captures ABI, N-API, OS/arch where available. | + +## Node Observation & Runtime (Sprint 42) +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| SCANNER-ANALYZERS-NODE-22-008 | TODO | Node Analyzer Guild | SCANNER-ANALYZERS-NODE-22-004 | Produce AOC-compliant observations: entrypoints, components (pkg/native/wasm), edges (esm-import, cjs-require, exports, json, native-addon, wasm, worker) with reason codes/confidence and resolver traces. | Observation JSON for fixtures deterministic; edges include reason codes + confidence; resolves attach trace data; passes AOC lint. | +| SCANNER-ANALYZERS-NODE-22-009 | TODO | Node Analyzer Guild, QA Guild | SCANNER-ANALYZERS-NODE-22-008 | Author fixture suite + performance benchmarks (npm, pnpm, PnP, bundle, electron, worker) with golden outputs and latency budgets. | Fixtures stored under `fixtures/lang/node/ep`; determinism + perf (<350ms npm, <900ms PnP) enforced via CI benchmarks. | +| SCANNER-ANALYZERS-NODE-22-010 | TODO | Node Analyzer Guild, Signals Guild | SCANNER-ANALYZERS-NODE-22-008 | Implement optional runtime evidence hooks (ESM loader, CJS require hook) with path scrubbing and loader ID hashing; emit runtime-* edges. | Sandbox harness records runtime resolve events for sample app; paths hashed; runtime edges merge with static graph without altering first-wins selection. | +| SCANNER-ANALYZERS-NODE-22-011 | TODO | Node Analyzer Guild, DevOps Guild | SCANNER-ANALYZERS-NODE-22-008 | Package updated analyzer as restart-time plug-in, expose Scanner CLI (`stella node *`) commands, refresh Offline Kit documentation. | Plugins folder updated; worker loads analyzer; CLI commands documented and smoke-tested; Offline Kit instructions include Node analyzer usage. | +| SCANNER-ANALYZERS-NODE-22-012 | TODO | Node Analyzer Guild | SCANNER-ANALYZERS-NODE-22-001 | Integrate container filesystem adapter (OCI layers, Dockerfile hints) and record NODE_OPTIONS/env warnings. | Container fixture parsed: Node base image + NODE_OPTIONS captured; entrypoints resolved relative to image root; warnings emitted for loader flags. | diff --git a/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Python/TASKS.completed.md b/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Python/TASKS.completed.md new file mode 100644 index 00000000..4db729af --- /dev/null +++ b/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Python/TASKS.completed.md @@ -0,0 +1,14 @@ +# Completed Tasks + +| 1 | SCANNER-ANALYZERS-LANG-10-303A | DONE (2025-10-21) | SCANNER-ANALYZERS-LANG-10-307 | STREAM-based parser for `*.dist-info` (`METADATA`, `WHEEL`, `entry_points.txt`) with normalization + evidence capture. | Parser handles CPython 3.8–3.12 metadata variations; fixtures confirm canonical ordering and UTF-8 handling. | + +| 2 | SCANNER-ANALYZERS-LANG-10-303B | DONE (2025-10-21) | SCANNER-ANALYZERS-LANG-10-303A | RECORD hash verifier with chunked hashing, Zip64 support, and mismatch diagnostics. | Verifier processes 5 GB RECORD fixture without allocations >2 MB; mismatches produce deterministic evidence records. | + +| 3 | SCANNER-ANALYZERS-LANG-10-303C | DONE (2025-10-21) | SCANNER-ANALYZERS-LANG-10-303B | Editable install + pip cache detection; integrate EntryTrace hints for runtime usage flags. | Editable installs resolved to source path; usage flags propagated; regression tests cover mixed editable + wheel installs. | + +| 4 | SCANNER-ANALYZERS-LANG-10-307P | DONE (2025-10-23) | SCANNER-ANALYZERS-LANG-10-303C | Shared helper integration (license metadata, quiet provenance, component merging). | Shared helpers reused; analyzer-specific metadata minimal; deterministic merge tests pass. | + +| 5 | SCANNER-ANALYZERS-LANG-10-308P | DONE (2025-10-23) | SCANNER-ANALYZERS-LANG-10-307P | Golden fixtures + determinism harness for Python analyzer; add benchmark and hash throughput reporting. | Fixtures under `Fixtures/lang/python/`; determinism CI guard; benchmark CSV added with threshold alerts. | + +| 6 | SCANNER-ANALYZERS-LANG-10-309P | DONE (2025-10-23) | SCANNER-ANALYZERS-LANG-10-308P | Package plug-in (manifest, DI registration) and document Offline Kit bundling of Python stdlib metadata if needed. | Manifest copied to `plugins/scanner/analyzers/lang/`; Worker loads analyzer; Offline Kit doc updated. | + diff --git a/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Python/TASKS.md b/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Python/TASKS.md index 0714d00b..10b2876e 100644 --- a/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Python/TASKS.md +++ b/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Python/TASKS.md @@ -1,31 +1,25 @@ -# Python Analyzer Task Flow - -| Seq | ID | Status | Depends on | Description | Exit Criteria | -|-----|----|--------|------------|-------------|---------------| -| 1 | SCANNER-ANALYZERS-LANG-10-303A | DONE (2025-10-21) | SCANNER-ANALYZERS-LANG-10-307 | STREAM-based parser for `*.dist-info` (`METADATA`, `WHEEL`, `entry_points.txt`) with normalization + evidence capture. | Parser handles CPython 3.8–3.12 metadata variations; fixtures confirm canonical ordering and UTF-8 handling. | -| 2 | SCANNER-ANALYZERS-LANG-10-303B | DONE (2025-10-21) | SCANNER-ANALYZERS-LANG-10-303A | RECORD hash verifier with chunked hashing, Zip64 support, and mismatch diagnostics. | Verifier processes 5 GB RECORD fixture without allocations >2 MB; mismatches produce deterministic evidence records. | -| 3 | SCANNER-ANALYZERS-LANG-10-303C | DONE (2025-10-21) | SCANNER-ANALYZERS-LANG-10-303B | Editable install + pip cache detection; integrate EntryTrace hints for runtime usage flags. | Editable installs resolved to source path; usage flags propagated; regression tests cover mixed editable + wheel installs. | -| 4 | SCANNER-ANALYZERS-LANG-10-307P | DONE (2025-10-23) | SCANNER-ANALYZERS-LANG-10-303C | Shared helper integration (license metadata, quiet provenance, component merging). | Shared helpers reused; analyzer-specific metadata minimal; deterministic merge tests pass. | -| 5 | SCANNER-ANALYZERS-LANG-10-308P | DONE (2025-10-23) | SCANNER-ANALYZERS-LANG-10-307P | Golden fixtures + determinism harness for Python analyzer; add benchmark and hash throughput reporting. | Fixtures under `Fixtures/lang/python/`; determinism CI guard; benchmark CSV added with threshold alerts. | -| 6 | SCANNER-ANALYZERS-LANG-10-309P | DONE (2025-10-23) | SCANNER-ANALYZERS-LANG-10-308P | Package plug-in (manifest, DI registration) and document Offline Kit bundling of Python stdlib metadata if needed. | Manifest copied to `plugins/scanner/analyzers/lang/`; Worker loads analyzer; Offline Kit doc updated. | - -## Python Entry-Point Analyzer (Sprint 43) -> **Imposed rule:** work of this type or tasks of this type on this component — and everywhere else it should be applied. -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| SCANNER-ANALYZERS-PYTHON-23-001 | TODO | Python Analyzer Guild | SCANNER-ANALYZERS-LANG-10-309P | Build input normalizer & virtual filesystem for wheels, sdists, editable installs, zipapps, site-packages trees, and container roots. Detect Python version targets (`pyproject.toml`, `runtime.txt`, Dockerfile) + virtualenv layout deterministically. | Normalizer ingests fixtures (venv, wheel, sdist, zipapp, container layer) without extraction; records python_version, root metadata, and namespace resolution hints; determinism harness updated. | -| SCANNER-ANALYZERS-PYTHON-23-002 | TODO | Python Analyzer Guild | SCANNER-ANALYZERS-PYTHON-23-001 | Entrypoint discovery: module `__main__`, console_scripts entry points, `scripts`, zipapp main, `manage.py`/gunicorn/celery patterns. Capture invocation context (module vs package, argv wrappers). | Fixtures produce entrypoint list with kind (console, module, package, zipapp, framework) and deterministic ordering; warnings for missing targets recorded. | -| SCANNER-ANALYZERS-PYTHON-23-003 | TODO | Python Analyzer Guild | SCANNER-ANALYZERS-PYTHON-23-001 | Static import graph builder using AST and bytecode fallback. Support `import`, `from ... import`, relative imports, `importlib.import_module`, `__import__` with literal args, `pkgutil.extend_path`. | AST scanner emits edges for explicit imports; literal importlib calls covered; unresolved/dynamic patterns yield `dynamic-import` warnings with candidate prefixes; regression fixtures pass. | -| SCANNER-ANALYZERS-PYTHON-23-004 | TODO | Python Analyzer Guild | SCANNER-ANALYZERS-PYTHON-23-003 | Python resolver engine (importlib semantics) handling namespace packages (PEP 420), package discovery order, `.pth` files, `sys.path` composition, zipimport, and site-packages precedence across virtualenv/container roots. | Resolver reproduces importlib behaviour on fixture matrix (namespace pkg, zipimport, multi-site-dir); includes explain traces; determinism tests for path ordering succeed. | -| SCANNER-ANALYZERS-PYTHON-23-005 | TODO | Python Analyzer Guild | SCANNER-ANALYZERS-PYTHON-23-004 | Packaging adapters: pip editable (`.egg-link`), Poetry/Flit layout, Conda prefix, `.dist-info/RECORD` cross-check, container layer overlays. | Adapters resolve editable links, conda pkgs, layered site-packages; edges capture provider path + metadata; warnings emitted for missing RECORD entries. | -| SCANNER-ANALYZERS-PYTHON-23-006 | TODO | Python Analyzer Guild | SCANNER-ANALYZERS-PYTHON-23-003 | Detect native extensions (`*.so`, `*.pyd`), CFFI modules, ctypes loaders, embedded WASM, and runtime capability signals (subprocess, multiprocessing, ctypes, eval). | Fixtures with native/CFFI/ctypes emit `native-extension`, `cffi`, `ctypes` hints; capability flags recorded; metadata captures ABI/platform info. | -| SCANNER-ANALYZERS-PYTHON-23-007 | TODO | Python Analyzer Guild | SCANNER-ANALYZERS-PYTHON-23-002 | Framework/config heuristics: Django, Flask, FastAPI, Celery, AWS Lambda handlers, Gunicorn, Click/Typer CLIs, logging configs, pyproject optional dependencies. Tagged as hints only. | Framework fixtures produce hint records with source files (settings.py, pyproject extras, celery app); no resolver impact; determinism maintained. | - -## Python Observation & Runtime (Sprint 44) -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| SCANNER-ANALYZERS-PYTHON-23-008 | TODO | Python Analyzer Guild | SCANNER-ANALYZERS-PYTHON-23-004 | Produce AOC-compliant observations: entrypoints, components (modules/packages/native), edges (import, namespace, dynamic-hint, native-extension) with reason codes/confidence and resolver traces. | Observation JSON for fixtures deterministic; includes explain trace per edge and namespace resolution metadata; passes AOC compliance lint. | -| SCANNER-ANALYZERS-PYTHON-23-009 | TODO | Python Analyzer Guild, QA Guild | SCANNER-ANALYZERS-PYTHON-23-008 | Fixture suite + perf benchmarks covering virtualenv, namespace packages, zipapp, editable installs, containers, lambda handler. | Fixture set committed under `fixtures/lang/python/ep`; determinism CI and perf (<250ms medium project) gates enabled. | -| SCANNER-ANALYZERS-PYTHON-23-010 | TODO | Python Analyzer Guild, Signals Guild | SCANNER-ANALYZERS-PYTHON-23-008 | Optional runtime evidence: import hook capturing module load events with path scrubbing, optional bytecode instrumentation for `importlib` hooks, multiprocessing tracer. | Runtime harness records module loads for sample app; paths hashed; runtime edges merge without altering resolver precedence; privacy doc updated. | -| SCANNER-ANALYZERS-PYTHON-23-011 | TODO | Python Analyzer Guild, DevOps Guild | SCANNER-ANALYZERS-PYTHON-23-008 | Package analyzer plug-in, add CLI commands (`stella python inspect|resolve|trace`), update Offline Kit guidance. | Plugin manifest deployed; CLI commands documented & smoke tested; Offline Kit instructions cover Python analyzer usage; worker restart verified. | -| SCANNER-ANALYZERS-PYTHON-23-012 | TODO | Python Analyzer Guild | SCANNER-ANALYZERS-PYTHON-23-001 | Container/zipapp adapter enhancements: parse OCI layers for Python runtime, detect `PYTHONPATH`/`PYTHONHOME` env, record warnings for sitecustomize/startup hooks. | Container fixtures output runtime metadata (python binary, env vars) and warnings for startup hooks; zipapp fixture resolves internal modules; determinism retained. | +# Python Analyzer Task Flow + +| Seq | ID | Status | Depends on | Description | Exit Criteria | +|-----|----|--------|------------|-------------|---------------| + +## Python Entry-Point Analyzer (Sprint 43) +> **Imposed rule:** work of this type or tasks of this type on this component — and everywhere else it should be applied. +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| SCANNER-ANALYZERS-PYTHON-23-001 | TODO | Python Analyzer Guild | SCANNER-ANALYZERS-LANG-10-309P | Build input normalizer & virtual filesystem for wheels, sdists, editable installs, zipapps, site-packages trees, and container roots. Detect Python version targets (`pyproject.toml`, `runtime.txt`, Dockerfile) + virtualenv layout deterministically. | Normalizer ingests fixtures (venv, wheel, sdist, zipapp, container layer) without extraction; records python_version, root metadata, and namespace resolution hints; determinism harness updated. | +| SCANNER-ANALYZERS-PYTHON-23-002 | TODO | Python Analyzer Guild | SCANNER-ANALYZERS-PYTHON-23-001 | Entrypoint discovery: module `__main__`, console_scripts entry points, `scripts`, zipapp main, `manage.py`/gunicorn/celery patterns. Capture invocation context (module vs package, argv wrappers). | Fixtures produce entrypoint list with kind (console, module, package, zipapp, framework) and deterministic ordering; warnings for missing targets recorded. | +| SCANNER-ANALYZERS-PYTHON-23-003 | TODO | Python Analyzer Guild | SCANNER-ANALYZERS-PYTHON-23-001 | Static import graph builder using AST and bytecode fallback. Support `import`, `from ... import`, relative imports, `importlib.import_module`, `__import__` with literal args, `pkgutil.extend_path`. | AST scanner emits edges for explicit imports; literal importlib calls covered; unresolved/dynamic patterns yield `dynamic-import` warnings with candidate prefixes; regression fixtures pass. | +| SCANNER-ANALYZERS-PYTHON-23-004 | TODO | Python Analyzer Guild | SCANNER-ANALYZERS-PYTHON-23-003 | Python resolver engine (importlib semantics) handling namespace packages (PEP 420), package discovery order, `.pth` files, `sys.path` composition, zipimport, and site-packages precedence across virtualenv/container roots. | Resolver reproduces importlib behaviour on fixture matrix (namespace pkg, zipimport, multi-site-dir); includes explain traces; determinism tests for path ordering succeed. | +| SCANNER-ANALYZERS-PYTHON-23-005 | TODO | Python Analyzer Guild | SCANNER-ANALYZERS-PYTHON-23-004 | Packaging adapters: pip editable (`.egg-link`), Poetry/Flit layout, Conda prefix, `.dist-info/RECORD` cross-check, container layer overlays. | Adapters resolve editable links, conda pkgs, layered site-packages; edges capture provider path + metadata; warnings emitted for missing RECORD entries. | +| SCANNER-ANALYZERS-PYTHON-23-006 | TODO | Python Analyzer Guild | SCANNER-ANALYZERS-PYTHON-23-003 | Detect native extensions (`*.so`, `*.pyd`), CFFI modules, ctypes loaders, embedded WASM, and runtime capability signals (subprocess, multiprocessing, ctypes, eval). | Fixtures with native/CFFI/ctypes emit `native-extension`, `cffi`, `ctypes` hints; capability flags recorded; metadata captures ABI/platform info. | +| SCANNER-ANALYZERS-PYTHON-23-007 | TODO | Python Analyzer Guild | SCANNER-ANALYZERS-PYTHON-23-002 | Framework/config heuristics: Django, Flask, FastAPI, Celery, AWS Lambda handlers, Gunicorn, Click/Typer CLIs, logging configs, pyproject optional dependencies. Tagged as hints only. | Framework fixtures produce hint records with source files (settings.py, pyproject extras, celery app); no resolver impact; determinism maintained. | + +## Python Observation & Runtime (Sprint 44) +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| SCANNER-ANALYZERS-PYTHON-23-008 | TODO | Python Analyzer Guild | SCANNER-ANALYZERS-PYTHON-23-004 | Produce AOC-compliant observations: entrypoints, components (modules/packages/native), edges (import, namespace, dynamic-hint, native-extension) with reason codes/confidence and resolver traces. | Observation JSON for fixtures deterministic; includes explain trace per edge and namespace resolution metadata; passes AOC compliance lint. | +| SCANNER-ANALYZERS-PYTHON-23-009 | TODO | Python Analyzer Guild, QA Guild | SCANNER-ANALYZERS-PYTHON-23-008 | Fixture suite + perf benchmarks covering virtualenv, namespace packages, zipapp, editable installs, containers, lambda handler. | Fixture set committed under `fixtures/lang/python/ep`; determinism CI and perf (<250ms medium project) gates enabled. | +| SCANNER-ANALYZERS-PYTHON-23-010 | TODO | Python Analyzer Guild, Signals Guild | SCANNER-ANALYZERS-PYTHON-23-008 | Optional runtime evidence: import hook capturing module load events with path scrubbing, optional bytecode instrumentation for `importlib` hooks, multiprocessing tracer. | Runtime harness records module loads for sample app; paths hashed; runtime edges merge without altering resolver precedence; privacy doc updated. | +| SCANNER-ANALYZERS-PYTHON-23-011 | TODO | Python Analyzer Guild, DevOps Guild | SCANNER-ANALYZERS-PYTHON-23-008 | Package analyzer plug-in, add CLI commands (`stella python inspect|resolve|trace`), update Offline Kit guidance. | Plugin manifest deployed; CLI commands documented & smoke tested; Offline Kit instructions cover Python analyzer usage; worker restart verified. | +| SCANNER-ANALYZERS-PYTHON-23-012 | TODO | Python Analyzer Guild | SCANNER-ANALYZERS-PYTHON-23-001 | Container/zipapp adapter enhancements: parse OCI layers for Python runtime, detect `PYTHONPATH`/`PYTHONHOME` env, record warnings for sitecustomize/startup hooks. | Container fixtures output runtime metadata (python binary, env vars) and warnings for startup hooks; zipapp fixture resolves internal modules; determinism retained. | diff --git a/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Rust/TASKS.completed.md b/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Rust/TASKS.completed.md new file mode 100644 index 00000000..10301261 --- /dev/null +++ b/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Rust/TASKS.completed.md @@ -0,0 +1,10 @@ +# Completed Tasks + +| 1 | SCANNER-ANALYZERS-LANG-10-306A | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-307 | Parse Cargo metadata (`Cargo.lock`, `.fingerprint`, `.metadata`) and map crates to components with evidence. | Fixtures confirm crate attribution ≥85 % coverage; metadata normalized; evidence includes path + hash. | + +| 2 | SCANNER-ANALYZERS-LANG-10-306B | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-306A | Implement heuristic classifier using ELF section names, symbol mangling, and `.comment` data for stripped binaries. | Heuristic output flagged as `heuristic`; regression tests ensure no false “observed” classifications. | + +| 3 | SCANNER-ANALYZERS-LANG-10-306C | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-306B | Integrate binary hash fallback (`bin:{sha256}`) and tie into shared quiet provenance helpers. | Fallback path deterministic; shared helpers reused; tests verify consistent hashing. | + +| 4 | SCANNER-ANALYZERS-LANG-10-307R | DONE (2025-10-29) | SCANNER-ANALYZERS-LANG-10-306C | Finalize shared helper usage (license, usage flags) and concurrency-safe caches. | Analyzer uses shared utilities; concurrency tests pass; no race conditions. | + diff --git a/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Rust/TASKS.md b/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Rust/TASKS.md index 655b043c..970d7c49 100644 --- a/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Rust/TASKS.md +++ b/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Rust/TASKS.md @@ -1,10 +1,6 @@ -# Rust Analyzer Task Flow - -| Seq | ID | Status | Depends on | Description | Exit Criteria | -|-----|----|--------|------------|-------------|---------------| -| 1 | SCANNER-ANALYZERS-LANG-10-306A | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-307 | Parse Cargo metadata (`Cargo.lock`, `.fingerprint`, `.metadata`) and map crates to components with evidence. | Fixtures confirm crate attribution ≥85 % coverage; metadata normalized; evidence includes path + hash. | -| 2 | SCANNER-ANALYZERS-LANG-10-306B | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-306A | Implement heuristic classifier using ELF section names, symbol mangling, and `.comment` data for stripped binaries. | Heuristic output flagged as `heuristic`; regression tests ensure no false “observed” classifications. | -| 3 | SCANNER-ANALYZERS-LANG-10-306C | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-306B | Integrate binary hash fallback (`bin:{sha256}`) and tie into shared quiet provenance helpers. | Fallback path deterministic; shared helpers reused; tests verify consistent hashing. | -| 4 | SCANNER-ANALYZERS-LANG-10-307R | DONE (2025-10-29) | SCANNER-ANALYZERS-LANG-10-306C | Finalize shared helper usage (license, usage flags) and concurrency-safe caches. | Analyzer uses shared utilities; concurrency tests pass; no race conditions. | -| 5 | SCANNER-ANALYZERS-LANG-10-308R | TODO | SCANNER-ANALYZERS-LANG-10-307R | Determinism fixtures + performance benchmarks; compare against competitor heuristic coverage. | Fixtures `Fixtures/lang/rust/` committed; determinism guard; benchmark shows ≥15 % better coverage vs competitor. | -| 6 | SCANNER-ANALYZERS-LANG-10-309R | TODO | SCANNER-ANALYZERS-LANG-10-308R | Package plug-in manifest + Offline Kit documentation; ensure Worker integration. | Manifest copied; Worker loads analyzer; Offline Kit doc updated. | +# Rust Analyzer Task Flow + +| Seq | ID | Status | Depends on | Description | Exit Criteria | +|-----|----|--------|------------|-------------|---------------| +| 5 | SCANNER-ANALYZERS-LANG-10-308R | TODO | SCANNER-ANALYZERS-LANG-10-307R | Determinism fixtures + performance benchmarks; compare against competitor heuristic coverage. | Fixtures `Fixtures/lang/rust/` committed; determinism guard; benchmark shows ≥15 % better coverage vs competitor. | +| 6 | SCANNER-ANALYZERS-LANG-10-309R | TODO | SCANNER-ANALYZERS-LANG-10-308R | Package plug-in manifest + Offline Kit documentation; ensure Worker integration. | Manifest copied; Worker loads analyzer; Offline Kit doc updated. | diff --git a/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang/TASKS.completed.md b/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang/TASKS.completed.md new file mode 100644 index 00000000..548efd06 --- /dev/null +++ b/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang/TASKS.completed.md @@ -0,0 +1,13 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| SCANNER-ANALYZERS-LANG-10-301 | DONE (2025-10-19) | Language Analyzer Guild | SCANNER-CORE-09-501, SCANNER-WORKER-09-203 | Java analyzer emitting deterministic `pkg:maven` components using pom.properties / MANIFEST evidence. | Java analyzer extracts coordinates+version+licenses with provenance; golden fixtures deterministic; microbenchmark meets target. | +| SCANNER-ANALYZERS-LANG-10-302 | DONE (2025-10-21) | Language Analyzer Guild | SCANNER-ANALYZERS-LANG-10-307 | Node analyzer resolving workspaces/symlinks into `pkg:npm` identities. | Node analyzer handles symlinks/workspaces; outputs sorted components; determinism harness covers hoisted deps. | +| SCANNER-ANALYZERS-LANG-10-303 | DONE (2025-10-21) | Language Analyzer Guild | SCANNER-ANALYZERS-LANG-10-307 | Python analyzer consuming `*.dist-info` metadata and RECORD hashes. | Analyzer binds METADATA + RECORD evidence, includes entry points, determinism fixtures stable. | +| SCANNER-ANALYZERS-LANG-10-304 | DONE (2025-10-22) | Language Analyzer Guild | SCANNER-ANALYZERS-LANG-10-307 | Go analyzer leveraging buildinfo for `pkg:golang` components. | Buildinfo parser emits module path/version + vcs metadata; binaries without buildinfo downgraded gracefully. | +| SCANNER-ANALYZERS-LANG-10-305 | DONE (2025-10-22) | Language Analyzer Guild | SCANNER-ANALYZERS-LANG-10-307 | .NET analyzer parsing `*.deps.json`, assembly metadata, and RID variants. | Analyzer merges deps.json + assembly info; dedupes per RID; determinism verified. | +| SCANNER-ANALYZERS-LANG-10-306 | DONE (2025-10-22) | Language Analyzer Guild | SCANNER-ANALYZERS-LANG-10-307 | Rust analyzer detecting crate provenance or falling back to `bin:{sha256}`. | Analyzer emits `pkg:cargo` when metadata present; falls back to binary hash; fixtures cover both paths. | +| SCANNER-ANALYZERS-LANG-10-307 | DONE (2025-10-19) | Language Analyzer Guild | SCANNER-CORE-09-501 | Shared language evidence helpers + usage flag propagation. | Shared abstractions implemented; analyzers reuse helpers; evidence includes usage hints; unit tests cover canonical ordering. | +| SCANNER-ANALYZERS-LANG-10-308 | DONE (2025-10-19) | Language Analyzer Guild | SCANNER-ANALYZERS-LANG-10-307 | Determinism + fixture harness for language analyzers. | Harness executes analyzers against fixtures; golden JSON stored; CI helper ensures stable hashes. | +| SCANNER-ANALYZERS-LANG-10-309 | DONE (2025-10-21) | Language Analyzer Guild | SCANNER-ANALYZERS-LANG-10-301..308 | Package language analyzers as restart-time plug-ins (manifest + host registration). | Plugin manifests authored under `plugins/scanner/analyzers/lang`; Worker loads via DI; restart required flag enforced; tests confirm manifest integrity. | diff --git a/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang/TASKS.md b/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang/TASKS.md index e53f4d52..f16b869a 100644 --- a/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang/TASKS.md +++ b/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang/TASKS.md @@ -1,13 +1,4 @@ -# Language Analyzer Task Board - -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| SCANNER-ANALYZERS-LANG-10-301 | DONE (2025-10-19) | Language Analyzer Guild | SCANNER-CORE-09-501, SCANNER-WORKER-09-203 | Java analyzer emitting deterministic `pkg:maven` components using pom.properties / MANIFEST evidence. | Java analyzer extracts coordinates+version+licenses with provenance; golden fixtures deterministic; microbenchmark meets target. | -| SCANNER-ANALYZERS-LANG-10-302 | DONE (2025-10-21) | Language Analyzer Guild | SCANNER-ANALYZERS-LANG-10-307 | Node analyzer resolving workspaces/symlinks into `pkg:npm` identities. | Node analyzer handles symlinks/workspaces; outputs sorted components; determinism harness covers hoisted deps. | -| SCANNER-ANALYZERS-LANG-10-303 | DONE (2025-10-21) | Language Analyzer Guild | SCANNER-ANALYZERS-LANG-10-307 | Python analyzer consuming `*.dist-info` metadata and RECORD hashes. | Analyzer binds METADATA + RECORD evidence, includes entry points, determinism fixtures stable. | -| SCANNER-ANALYZERS-LANG-10-304 | DONE (2025-10-22) | Language Analyzer Guild | SCANNER-ANALYZERS-LANG-10-307 | Go analyzer leveraging buildinfo for `pkg:golang` components. | Buildinfo parser emits module path/version + vcs metadata; binaries without buildinfo downgraded gracefully. | -| SCANNER-ANALYZERS-LANG-10-305 | DONE (2025-10-22) | Language Analyzer Guild | SCANNER-ANALYZERS-LANG-10-307 | .NET analyzer parsing `*.deps.json`, assembly metadata, and RID variants. | Analyzer merges deps.json + assembly info; dedupes per RID; determinism verified. | -| SCANNER-ANALYZERS-LANG-10-306 | DONE (2025-10-22) | Language Analyzer Guild | SCANNER-ANALYZERS-LANG-10-307 | Rust analyzer detecting crate provenance or falling back to `bin:{sha256}`. | Analyzer emits `pkg:cargo` when metadata present; falls back to binary hash; fixtures cover both paths. | -| SCANNER-ANALYZERS-LANG-10-307 | DONE (2025-10-19) | Language Analyzer Guild | SCANNER-CORE-09-501 | Shared language evidence helpers + usage flag propagation. | Shared abstractions implemented; analyzers reuse helpers; evidence includes usage hints; unit tests cover canonical ordering. | -| SCANNER-ANALYZERS-LANG-10-308 | DONE (2025-10-19) | Language Analyzer Guild | SCANNER-ANALYZERS-LANG-10-307 | Determinism + fixture harness for language analyzers. | Harness executes analyzers against fixtures; golden JSON stored; CI helper ensures stable hashes. | -| SCANNER-ANALYZERS-LANG-10-309 | DONE (2025-10-21) | Language Analyzer Guild | SCANNER-ANALYZERS-LANG-10-301..308 | Package language analyzers as restart-time plug-ins (manifest + host registration). | Plugin manifests authored under `plugins/scanner/analyzers/lang`; Worker loads via DI; restart required flag enforced; tests confirm manifest integrity. | +# Language Analyzer Task Board + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| diff --git a/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.OS/TASKS.completed.md b/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.OS/TASKS.completed.md new file mode 100644 index 00000000..35048366 --- /dev/null +++ b/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.OS/TASKS.completed.md @@ -0,0 +1,11 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| SCANNER-ANALYZERS-OS-10-201 | DONE (2025-10-19) | OS Analyzer Guild | Scanner Core contracts | Alpine/apk analyzer emitting deterministic package components with provenance evidence. | Analyzer reads `/lib/apk/db/installed`, emits deterministic `pkg:alpine` components with provenance, license, and file evidence; snapshot tests cover fixture. | +| SCANNER-ANALYZERS-OS-10-202 | DONE (2025-10-19) | OS Analyzer Guild | Shared helpers (204) | Debian/dpkg analyzer mapping packages to canonical `pkg:deb` identities with evidence and normalized metadata. | Analyzer parses `status` + `info/*.list`/`md5sums`, outputs normalized packages with config flags and provenance evidence. | +| SCANNER-ANALYZERS-OS-10-203 | DONE (2025-10-19) | OS Analyzer Guild | Shared helpers (204) | RPM analyzer capturing EVR/NEVRA, declared file lists, provenance metadata. | SQLite rpmdb reader parses headers, reconstructs NEVRA, provides/requires, file evidence, and vendor metadata for fixtures. | +| SCANNER-ANALYZERS-OS-10-204 | DONE (2025-10-19) | OS Analyzer Guild | — | Build shared OS evidence helpers for package identity normalization, file attribution, and metadata enrichment used by analyzers. | Shared helpers deliver analyzer base context, PURL builders, CVE hint extraction, and file evidence model reused across plugins. | +| SCANNER-ANALYZERS-OS-10-205 | DONE (2025-10-19) | OS Analyzer Guild | Shared helpers (204) | Vendor metadata enrichment (source packages, declared licenses, CVE hints). | Apk/dpkg/rpm analyzers populate source, license, maintainer, URLs, and CVE hints; metadata stored deterministically. | +| SCANNER-ANALYZERS-OS-10-206 | DONE (2025-10-19) | QA + OS Analyzer Guild | 201–205 | Determinism harness + fixtures for OS analyzers (warm/cold runs). | xUnit snapshot harness with fixtures + goldens ensures byte-stable JSON; helper normalizes newlines and supports env-based regen. | +| SCANNER-ANALYZERS-OS-10-207 | DONE (2025-10-19) | OS Analyzer Guild + DevOps | 201–206 | Package OS analyzers as restart-time plug-ins (manifest + host registration). | Build targets copy analyzer DLLs/manifests to `plugins/scanner/analyzers/os/`; Worker dispatcher loads via restart-only plugin guard. | diff --git a/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.OS/TASKS.md b/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.OS/TASKS.md index 6d5532c7..08027cfa 100644 --- a/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.OS/TASKS.md +++ b/src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.OS/TASKS.md @@ -1,11 +1,4 @@ -# OS Analyzer Task Board - -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| SCANNER-ANALYZERS-OS-10-201 | DONE (2025-10-19) | OS Analyzer Guild | Scanner Core contracts | Alpine/apk analyzer emitting deterministic package components with provenance evidence. | Analyzer reads `/lib/apk/db/installed`, emits deterministic `pkg:alpine` components with provenance, license, and file evidence; snapshot tests cover fixture. | -| SCANNER-ANALYZERS-OS-10-202 | DONE (2025-10-19) | OS Analyzer Guild | Shared helpers (204) | Debian/dpkg analyzer mapping packages to canonical `pkg:deb` identities with evidence and normalized metadata. | Analyzer parses `status` + `info/*.list`/`md5sums`, outputs normalized packages with config flags and provenance evidence. | -| SCANNER-ANALYZERS-OS-10-203 | DONE (2025-10-19) | OS Analyzer Guild | Shared helpers (204) | RPM analyzer capturing EVR/NEVRA, declared file lists, provenance metadata. | SQLite rpmdb reader parses headers, reconstructs NEVRA, provides/requires, file evidence, and vendor metadata for fixtures. | -| SCANNER-ANALYZERS-OS-10-204 | DONE (2025-10-19) | OS Analyzer Guild | — | Build shared OS evidence helpers for package identity normalization, file attribution, and metadata enrichment used by analyzers. | Shared helpers deliver analyzer base context, PURL builders, CVE hint extraction, and file evidence model reused across plugins. | -| SCANNER-ANALYZERS-OS-10-205 | DONE (2025-10-19) | OS Analyzer Guild | Shared helpers (204) | Vendor metadata enrichment (source packages, declared licenses, CVE hints). | Apk/dpkg/rpm analyzers populate source, license, maintainer, URLs, and CVE hints; metadata stored deterministically. | -| SCANNER-ANALYZERS-OS-10-206 | DONE (2025-10-19) | QA + OS Analyzer Guild | 201–205 | Determinism harness + fixtures for OS analyzers (warm/cold runs). | xUnit snapshot harness with fixtures + goldens ensures byte-stable JSON; helper normalizes newlines and supports env-based regen. | -| SCANNER-ANALYZERS-OS-10-207 | DONE (2025-10-19) | OS Analyzer Guild + DevOps | 201–206 | Package OS analyzers as restart-time plug-ins (manifest + host registration). | Build targets copy analyzer DLLs/manifests to `plugins/scanner/analyzers/os/`; Worker dispatcher loads via restart-only plugin guard. | +# OS Analyzer Task Board + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| diff --git a/src/Scanner/__Libraries/StellaOps.Scanner.Cache/TASKS.completed.md b/src/Scanner/__Libraries/StellaOps.Scanner.Cache/TASKS.completed.md new file mode 100644 index 00000000..12596ade --- /dev/null +++ b/src/Scanner/__Libraries/StellaOps.Scanner.Cache/TASKS.completed.md @@ -0,0 +1,8 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| SCANNER-CACHE-10-101 | DONE (2025-10-19) | Scanner Cache Guild | SCANNER-WORKER-09-201 | Implement layer cache store keyed by layer digest with metadata retention aligned with architecture §3.3 object layout. | Layer cache API supports get/put/delete by digest; metadata persisted with deterministic serialization; warm lookup covered by tests. | +| SCANNER-CACHE-10-102 | DONE (2025-10-19) | Scanner Cache Guild | SCANNER-CACHE-10-101 | Build file CAS with dedupe, TTL enforcement, and offline import/export hooks for offline kit workflows. | CAS stores content by SHA-256, enforces TTL policy, import/export commands documented and exercised in tests. | +| SCANNER-CACHE-10-103 | DONE (2025-10-19) | Scanner Cache Guild | SCANNER-CACHE-10-101 | Expose cache metrics/logging and configuration toggles for warm/cold thresholds. | Metrics counters/gauges emitted; options validated; logs include correlation IDs; configuration doc references settings. | +| SCANNER-CACHE-10-104 | DONE (2025-10-19) | Scanner Cache Guild | SCANNER-CACHE-10-101 | Implement cache invalidation workflows (layer delete, TTL expiry, diff invalidation). | Invalidation API implemented with deterministic eviction; tests cover TTL expiry + explicit delete; logs instrumented. | diff --git a/src/Scanner/__Libraries/StellaOps.Scanner.Cache/TASKS.md b/src/Scanner/__Libraries/StellaOps.Scanner.Cache/TASKS.md index 7821781b..8cd1796f 100644 --- a/src/Scanner/__Libraries/StellaOps.Scanner.Cache/TASKS.md +++ b/src/Scanner/__Libraries/StellaOps.Scanner.Cache/TASKS.md @@ -1,10 +1,6 @@ -# Scanner Cache Task Board (Sprint 10) - -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| SCANNER-CACHE-10-101 | DONE (2025-10-19) | Scanner Cache Guild | SCANNER-WORKER-09-201 | Implement layer cache store keyed by layer digest with metadata retention aligned with architecture §3.3 object layout. | Layer cache API supports get/put/delete by digest; metadata persisted with deterministic serialization; warm lookup covered by tests. | -| SCANNER-CACHE-10-102 | DONE (2025-10-19) | Scanner Cache Guild | SCANNER-CACHE-10-101 | Build file CAS with dedupe, TTL enforcement, and offline import/export hooks for offline kit workflows. | CAS stores content by SHA-256, enforces TTL policy, import/export commands documented and exercised in tests. | -| SCANNER-CACHE-10-103 | DONE (2025-10-19) | Scanner Cache Guild | SCANNER-CACHE-10-101 | Expose cache metrics/logging and configuration toggles for warm/cold thresholds. | Metrics counters/gauges emitted; options validated; logs include correlation IDs; configuration doc references settings. | -| SCANNER-CACHE-10-104 | DONE (2025-10-19) | Scanner Cache Guild | SCANNER-CACHE-10-101 | Implement cache invalidation workflows (layer delete, TTL expiry, diff invalidation). | Invalidation API implemented with deterministic eviction; tests cover TTL expiry + explicit delete; logs instrumented. | - -> Update statuses to DONE once acceptance criteria and tests/documentation are delivered. +# Scanner Cache Task Board (Sprint 10) + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| + +> Update statuses to DONE once acceptance criteria and tests/documentation are delivered. diff --git a/src/Scanner/__Libraries/StellaOps.Scanner.Core/TASKS.completed.md b/src/Scanner/__Libraries/StellaOps.Scanner.Core/TASKS.completed.md new file mode 100644 index 00000000..5cf211af --- /dev/null +++ b/src/Scanner/__Libraries/StellaOps.Scanner.Core/TASKS.completed.md @@ -0,0 +1,7 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| SCANNER-CORE-09-501 | DONE (2025-10-19) | Scanner Core Guild | — | Define shared DTOs (ScanJob, ProgressEvent), error taxonomy, and deterministic ID/timestamp helpers aligning with `modules/scanner/ARCHITECTURE.md` §3–§4.
2025-10-19: Added golden fixtures + `ScannerCoreContractsTests` to lock canonical JSON.
2025-10-19: Published canonical JSON snippet + acceptance notes in `docs/scanner-core-contracts.md`. | DTOs serialize deterministically, helpers produce reproducible IDs/timestamps, tests cover round-trips and hash derivation. | +| SCANNER-CORE-09-502 | DONE (2025-10-19) | Scanner Core Guild | SCANNER-CORE-09-501 | Observability helpers (correlation IDs, logging scopes, metric namespacing, deterministic hashes) consumed by WebService/Worker.
2025-10-19: Verified progress scope serialisation via new fixtures/tests.
2025-10-19: Added `ScannerLogExtensionsPerformanceTests` to enforce ≤ 5 µs scope overhead + documented micro-bench results. | Logging/metrics helpers allocate minimally, correlation IDs stable, ActivitySource emitted; tests assert determinism. | +| SCANNER-CORE-09-503 | DONE (2025-10-18) | Scanner Core Guild | SCANNER-CORE-09-501, SCANNER-CORE-09-502 | Security utilities: Authority client factory, OpTok caching, DPoP verifier, restart-time plug-in guardrails for scanner components. | Authority helpers cache tokens, DPoP validator rejects invalid proofs, plug-in guard prevents runtime additions; tests cover happy/error paths. | diff --git a/src/Scanner/__Libraries/StellaOps.Scanner.Core/TASKS.md b/src/Scanner/__Libraries/StellaOps.Scanner.Core/TASKS.md index d004ae29..9d97da7f 100644 --- a/src/Scanner/__Libraries/StellaOps.Scanner.Core/TASKS.md +++ b/src/Scanner/__Libraries/StellaOps.Scanner.Core/TASKS.md @@ -2,6 +2,3 @@ | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| SCANNER-CORE-09-501 | DONE (2025-10-19) | Scanner Core Guild | — | Define shared DTOs (ScanJob, ProgressEvent), error taxonomy, and deterministic ID/timestamp helpers aligning with `modules/scanner/ARCHITECTURE.md` §3–§4.
2025-10-19: Added golden fixtures + `ScannerCoreContractsTests` to lock canonical JSON.
2025-10-19: Published canonical JSON snippet + acceptance notes in `docs/scanner-core-contracts.md`. | DTOs serialize deterministically, helpers produce reproducible IDs/timestamps, tests cover round-trips and hash derivation. | -| SCANNER-CORE-09-502 | DONE (2025-10-19) | Scanner Core Guild | SCANNER-CORE-09-501 | Observability helpers (correlation IDs, logging scopes, metric namespacing, deterministic hashes) consumed by WebService/Worker.
2025-10-19: Verified progress scope serialisation via new fixtures/tests.
2025-10-19: Added `ScannerLogExtensionsPerformanceTests` to enforce ≤ 5 µs scope overhead + documented micro-bench results. | Logging/metrics helpers allocate minimally, correlation IDs stable, ActivitySource emitted; tests assert determinism. | -| SCANNER-CORE-09-503 | DONE (2025-10-18) | Scanner Core Guild | SCANNER-CORE-09-501, SCANNER-CORE-09-502 | Security utilities: Authority client factory, OpTok caching, DPoP verifier, restart-time plug-in guardrails for scanner components. | Authority helpers cache tokens, DPoP validator rejects invalid proofs, plug-in guard prevents runtime additions; tests cover happy/error paths. | diff --git a/src/Scanner/__Libraries/StellaOps.Scanner.Diff/TASKS.completed.md b/src/Scanner/__Libraries/StellaOps.Scanner.Diff/TASKS.completed.md new file mode 100644 index 00000000..411628f2 --- /dev/null +++ b/src/Scanner/__Libraries/StellaOps.Scanner.Diff/TASKS.completed.md @@ -0,0 +1,7 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| SCANNER-DIFF-10-501 | DONE (2025-10-19) | Diff Guild | SCANNER-CORE-09-501 | Build component differ tracking add/remove/version changes with deterministic ordering. | Diff engine produces deterministic results across runs; unit tests cover add/remove/version scenarios. | +| SCANNER-DIFF-10-502 | DONE (2025-10-19) | Diff Guild | SCANNER-DIFF-10-501 | Attribute diffs to introducing/removing layers including provenance evidence. | Layer attribution stored on every change; tests validate provenance with synthetic layer stacks. | +| SCANNER-DIFF-10-503 | DONE (2025-10-19) | Diff Guild | SCANNER-DIFF-10-502 | Produce JSON diff output for inventory vs usage views aligned with API contract. | JSON serializer emits stable ordering; golden fixture captured; API contract documented. | diff --git a/src/Scanner/__Libraries/StellaOps.Scanner.Diff/TASKS.md b/src/Scanner/__Libraries/StellaOps.Scanner.Diff/TASKS.md index 987cc228..2f8ec76e 100644 --- a/src/Scanner/__Libraries/StellaOps.Scanner.Diff/TASKS.md +++ b/src/Scanner/__Libraries/StellaOps.Scanner.Diff/TASKS.md @@ -1,7 +1,4 @@ -# Scanner Diff Task Board (Sprint 10) - -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| SCANNER-DIFF-10-501 | DONE (2025-10-19) | Diff Guild | SCANNER-CORE-09-501 | Build component differ tracking add/remove/version changes with deterministic ordering. | Diff engine produces deterministic results across runs; unit tests cover add/remove/version scenarios. | -| SCANNER-DIFF-10-502 | DONE (2025-10-19) | Diff Guild | SCANNER-DIFF-10-501 | Attribute diffs to introducing/removing layers including provenance evidence. | Layer attribution stored on every change; tests validate provenance with synthetic layer stacks. | -| SCANNER-DIFF-10-503 | DONE (2025-10-19) | Diff Guild | SCANNER-DIFF-10-502 | Produce JSON diff output for inventory vs usage views aligned with API contract. | JSON serializer emits stable ordering; golden fixture captured; API contract documented. | +# Scanner Diff Task Board (Sprint 10) + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| diff --git a/src/Scanner/__Libraries/StellaOps.Scanner.Emit/TASKS.completed.md b/src/Scanner/__Libraries/StellaOps.Scanner.Emit/TASKS.completed.md new file mode 100644 index 00000000..36c5227b --- /dev/null +++ b/src/Scanner/__Libraries/StellaOps.Scanner.Emit/TASKS.completed.md @@ -0,0 +1,12 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| SCANNER-EMIT-10-601 | DONE (2025-10-22) | Emit Guild | SCANNER-CACHE-10-101 | Compose inventory SBOM (CycloneDX JSON/Protobuf) from layer fragments with deterministic ordering. | Inventory SBOM validated against schema; fixtures confirm deterministic output. | +| SCANNER-EMIT-10-602 | DONE (2025-10-22) | Emit Guild | SCANNER-EMIT-10-601 | Compose usage SBOM leveraging EntryTrace to flag actual usage; ensure separate view toggles. | Usage SBOM tests confirm correct subset; API contract documented. | +| SCANNER-EMIT-10-603 | DONE (2025-10-22) | Emit Guild | SCANNER-EMIT-10-601 | Generate BOM index sidecar (purl table + roaring bitmap + usedByEntrypoint flag). | Index format validated; query helpers proven; stored artifacts hashed deterministically. | +| SCANNER-EMIT-10-604 | DONE (2025-10-22) | Emit Guild | SCANNER-EMIT-10-602 | Package artifacts for export + attestation (naming, compression, manifests). | Export pipeline produces deterministic file paths/hashes; integration test with storage passes. | +| SCANNER-EMIT-10-605 | DONE (2025-10-22) | Emit Guild | SCANNER-EMIT-10-603 | Emit BOM-Index sidecar schema/fixtures (`bom-index@1`) and note CRITICAL PATH for Scheduler. | Schema + fixtures in docs/artifacts/bom-index; tests `BOMIndexGoldenIsStable` green. | +| SCANNER-EMIT-10-606 | DONE (2025-10-22) | Emit Guild | SCANNER-EMIT-10-605 | Integrate EntryTrace usage flags into BOM-Index; document semantics. | Usage bits present in sidecar; integration tests with EntryTrace fixtures pass. | +| SCANNER-EMIT-17-701 | DONE (2025-10-26) | Emit Guild, Native Analyzer Guild | SCANNER-EMIT-10-602 | Record GNU build-id for ELF components and surface it in inventory/usage SBOM plus diff payloads with deterministic ordering. | Native analyzer emits buildId for every ELF executable/library, SBOM/diff fixtures updated with canonical `buildId` field, regression tests prove stability, docs call out debug-symbol lookup flow. | +| SCANNER-EMIT-10-607 | DONE (2025-10-22) | Emit Guild | SCANNER-EMIT-10-604, POLICY-CORE-09-005 | Embed scoring inputs, confidence band, and `quietedBy` provenance into CycloneDX 1.6 and DSSE predicates; verify deterministic serialization. | SBOM/attestation fixtures include score, inputs, configVersion, quiet metadata; golden tests confirm canonical output. | diff --git a/src/Scanner/__Libraries/StellaOps.Scanner.Emit/TASKS.md b/src/Scanner/__Libraries/StellaOps.Scanner.Emit/TASKS.md index 3dfbb050..b42f655d 100644 --- a/src/Scanner/__Libraries/StellaOps.Scanner.Emit/TASKS.md +++ b/src/Scanner/__Libraries/StellaOps.Scanner.Emit/TASKS.md @@ -1,12 +1,4 @@ -# Scanner Emit Task Board (Sprint 10) - -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| SCANNER-EMIT-10-601 | DONE (2025-10-22) | Emit Guild | SCANNER-CACHE-10-101 | Compose inventory SBOM (CycloneDX JSON/Protobuf) from layer fragments with deterministic ordering. | Inventory SBOM validated against schema; fixtures confirm deterministic output. | -| SCANNER-EMIT-10-602 | DONE (2025-10-22) | Emit Guild | SCANNER-EMIT-10-601 | Compose usage SBOM leveraging EntryTrace to flag actual usage; ensure separate view toggles. | Usage SBOM tests confirm correct subset; API contract documented. | -| SCANNER-EMIT-10-603 | DONE (2025-10-22) | Emit Guild | SCANNER-EMIT-10-601 | Generate BOM index sidecar (purl table + roaring bitmap + usedByEntrypoint flag). | Index format validated; query helpers proven; stored artifacts hashed deterministically. | -| SCANNER-EMIT-10-604 | DONE (2025-10-22) | Emit Guild | SCANNER-EMIT-10-602 | Package artifacts for export + attestation (naming, compression, manifests). | Export pipeline produces deterministic file paths/hashes; integration test with storage passes. | -| SCANNER-EMIT-10-605 | DONE (2025-10-22) | Emit Guild | SCANNER-EMIT-10-603 | Emit BOM-Index sidecar schema/fixtures (`bom-index@1`) and note CRITICAL PATH for Scheduler. | Schema + fixtures in docs/artifacts/bom-index; tests `BOMIndexGoldenIsStable` green. | -| SCANNER-EMIT-10-606 | DONE (2025-10-22) | Emit Guild | SCANNER-EMIT-10-605 | Integrate EntryTrace usage flags into BOM-Index; document semantics. | Usage bits present in sidecar; integration tests with EntryTrace fixtures pass. | -| SCANNER-EMIT-17-701 | DONE (2025-10-26) | Emit Guild, Native Analyzer Guild | SCANNER-EMIT-10-602 | Record GNU build-id for ELF components and surface it in inventory/usage SBOM plus diff payloads with deterministic ordering. | Native analyzer emits buildId for every ELF executable/library, SBOM/diff fixtures updated with canonical `buildId` field, regression tests prove stability, docs call out debug-symbol lookup flow. | -| SCANNER-EMIT-10-607 | DONE (2025-10-22) | Emit Guild | SCANNER-EMIT-10-604, POLICY-CORE-09-005 | Embed scoring inputs, confidence band, and `quietedBy` provenance into CycloneDX 1.6 and DSSE predicates; verify deterministic serialization. | SBOM/attestation fixtures include score, inputs, configVersion, quiet metadata; golden tests confirm canonical output. | +# Scanner Emit Task Board (Sprint 10) + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| diff --git a/src/Scanner/__Libraries/StellaOps.Scanner.EntryTrace/TASKS.completed.md b/src/Scanner/__Libraries/StellaOps.Scanner.EntryTrace/TASKS.completed.md new file mode 100644 index 00000000..f0b3676d --- /dev/null +++ b/src/Scanner/__Libraries/StellaOps.Scanner.EntryTrace/TASKS.completed.md @@ -0,0 +1,12 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| SCANNER-ENTRYTRACE-10-401 | DONE (2025-10-19) | EntryTrace Guild | Scanner Core contracts | Implement deterministic POSIX shell AST parser covering exec/command/source/run-parts/case/if used by ENTRYPOINT scripts. | Parser emits stable AST and serialization tests prove determinism for representative fixtures; see `ShellParserTests`. | +| SCANNER-ENTRYTRACE-10-402 | DONE (2025-10-19) | EntryTrace Guild | SCANNER-ENTRYTRACE-10-401 | Resolve commands across layered rootfs, tracking evidence per hop (PATH hit, layer origin, shebang). | Resolver returns terminal program path with layer attribution for fixtures; deterministic traversal asserted in `EntryTraceAnalyzerTests.ResolveAsync_IsDeterministic`. | +| SCANNER-ENTRYTRACE-10-403 | DONE (2025-10-19) | EntryTrace Guild | SCANNER-ENTRYTRACE-10-402 | Follow interpreter wrappers (shell → Python/Node/Java launchers) to terminal target, including module/jar detection. | Interpreter tracer reports correct module/script for language launchers; tests cover Python/Node/Java wrappers. | +| SCANNER-ENTRYTRACE-10-404 | DONE (2025-10-19) | EntryTrace Guild | SCANNER-ENTRYTRACE-10-403 | Build Python entry analyzer detecting venv shebangs, module invocations, `-m` usage and record usage flag. | Python fixtures produce expected module metadata (`python-module` edge) and diagnostics for missing scripts. | +| SCANNER-ENTRYTRACE-10-405 | DONE (2025-10-19) | EntryTrace Guild | SCANNER-ENTRYTRACE-10-403 | Implement Node/Java launcher analyzer capturing script/jar targets including npm lifecycle wrappers. | Node/Java fixtures resolved with evidence chain; `RunParts` coverage ensures child scripts traced. | +| SCANNER-ENTRYTRACE-10-406 | DONE (2025-10-19) | EntryTrace Guild | SCANNER-ENTRYTRACE-10-402 | Surface explainability + diagnostics for unresolved constructs and emit metrics counters. | Diagnostics catalog enumerates unknown reasons; metrics wired via `EntryTraceMetrics`; explainability doc updated. | +| SCANNER-ENTRYTRACE-10-407 | DONE (2025-10-19) | EntryTrace Guild | SCANNER-ENTRYTRACE-10-401..406 | Package EntryTrace analyzers as restart-time plug-ins with manifest + host registration. | Plug-in manifest under `plugins/scanner/entrytrace/`; restart-only policy documented; DI extension exposes `AddEntryTraceAnalyzer`. | +| SCANNER-ENTRYTRACE-18-501 | DONE (2025-10-29) | EntryTrace Guild | SCANNER-ENTRYTRACE-10-407 | Build OCI config reader and layered rootfs adapter so EntryTrace can hydrate PATH, WorkingDir, User, and provenance from real images. | Fixtures covering tar/dir inputs produce deterministic `IRootFileSystem` descriptors (whiteouts, symlinks, shebangs) and `EntrypointSpecification` derived from config merges with default PATH fallbacks. | diff --git a/src/Scanner/__Libraries/StellaOps.Scanner.EntryTrace/TASKS.md b/src/Scanner/__Libraries/StellaOps.Scanner.EntryTrace/TASKS.md index c41d9f3e..cd90dd03 100644 --- a/src/Scanner/__Libraries/StellaOps.Scanner.EntryTrace/TASKS.md +++ b/src/Scanner/__Libraries/StellaOps.Scanner.EntryTrace/TASKS.md @@ -1,15 +1,7 @@ -# EntryTrace Analyzer Task Board (Sprint 10) - -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| SCANNER-ENTRYTRACE-10-401 | DONE (2025-10-19) | EntryTrace Guild | Scanner Core contracts | Implement deterministic POSIX shell AST parser covering exec/command/source/run-parts/case/if used by ENTRYPOINT scripts. | Parser emits stable AST and serialization tests prove determinism for representative fixtures; see `ShellParserTests`. | -| SCANNER-ENTRYTRACE-10-402 | DONE (2025-10-19) | EntryTrace Guild | SCANNER-ENTRYTRACE-10-401 | Resolve commands across layered rootfs, tracking evidence per hop (PATH hit, layer origin, shebang). | Resolver returns terminal program path with layer attribution for fixtures; deterministic traversal asserted in `EntryTraceAnalyzerTests.ResolveAsync_IsDeterministic`. | -| SCANNER-ENTRYTRACE-10-403 | DONE (2025-10-19) | EntryTrace Guild | SCANNER-ENTRYTRACE-10-402 | Follow interpreter wrappers (shell → Python/Node/Java launchers) to terminal target, including module/jar detection. | Interpreter tracer reports correct module/script for language launchers; tests cover Python/Node/Java wrappers. | -| SCANNER-ENTRYTRACE-10-404 | DONE (2025-10-19) | EntryTrace Guild | SCANNER-ENTRYTRACE-10-403 | Build Python entry analyzer detecting venv shebangs, module invocations, `-m` usage and record usage flag. | Python fixtures produce expected module metadata (`python-module` edge) and diagnostics for missing scripts. | -| SCANNER-ENTRYTRACE-10-405 | DONE (2025-10-19) | EntryTrace Guild | SCANNER-ENTRYTRACE-10-403 | Implement Node/Java launcher analyzer capturing script/jar targets including npm lifecycle wrappers. | Node/Java fixtures resolved with evidence chain; `RunParts` coverage ensures child scripts traced. | -| SCANNER-ENTRYTRACE-10-406 | DONE (2025-10-19) | EntryTrace Guild | SCANNER-ENTRYTRACE-10-402 | Surface explainability + diagnostics for unresolved constructs and emit metrics counters. | Diagnostics catalog enumerates unknown reasons; metrics wired via `EntryTraceMetrics`; explainability doc updated. | -| SCANNER-ENTRYTRACE-10-407 | DONE (2025-10-19) | EntryTrace Guild | SCANNER-ENTRYTRACE-10-401..406 | Package EntryTrace analyzers as restart-time plug-ins with manifest + host registration. | Plug-in manifest under `plugins/scanner/entrytrace/`; restart-only policy documented; DI extension exposes `AddEntryTraceAnalyzer`. | -| SCANNER-ENTRYTRACE-18-501 | DONE (2025-10-29) | EntryTrace Guild | SCANNER-ENTRYTRACE-10-407 | Build OCI config reader and layered rootfs adapter so EntryTrace can hydrate PATH, WorkingDir, User, and provenance from real images. | Fixtures covering tar/dir inputs produce deterministic `IRootFileSystem` descriptors (whiteouts, symlinks, shebangs) and `EntrypointSpecification` derived from config merges with default PATH fallbacks. | +# EntryTrace Analyzer Task Board (Sprint 10) + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| | SCANNER-ENTRYTRACE-18-502 | TODO | EntryTrace Guild | SCANNER-ENTRYTRACE-18-501 | Expand chain walker with init shim/user-switch/supervisor recognition plus env/workdir accumulation and guarded edges. | Graph nodes annotate tini/dumb-init/gosu/su-exec/s6/supervisord/runit branches with capability tags, environment deltas, and guard metadata validated against fixture scripts. | | SCANNER-ENTRYTRACE-18-503 | TODO | EntryTrace Guild | SCANNER-ENTRYTRACE-18-502 | Introduce target classifier + EntryPlan handoff with confidence scoring for ELF/Java/.NET/Node/Python and user/workdir context. | Analyzer returns typed targets with confidence metrics and per-branch EntryPlans exercised via golden fixtures and language analyzer stubs. | | SCANNER-ENTRYTRACE-18-504 | TODO | EntryTrace Guild | SCANNER-ENTRYTRACE-18-503 | Emit EntryTrace AOC NDJSON (`entrytrace.entry/node/edge/target/warning/capability`) and wire CLI/service streaming outputs. | NDJSON writer passes determinism tests, CLI/service endpoints stream ordered observations, and diagnostics integrate new warning codes for dynamic eval/glob limits/windows shims. | diff --git a/src/Scanner/__Libraries/StellaOps.Scanner.Queue/TASKS.completed.md b/src/Scanner/__Libraries/StellaOps.Scanner.Queue/TASKS.completed.md new file mode 100644 index 00000000..22b4cbee --- /dev/null +++ b/src/Scanner/__Libraries/StellaOps.Scanner.Queue/TASKS.completed.md @@ -0,0 +1,7 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| SCANNER-QUEUE-09-401 | DONE (2025-10-19) | Scanner Queue Guild | — | Implement queue abstraction + Redis Streams adapter with ack/lease semantics, idempotency tokens, and deterministic job IDs. | Interfaces finalized; Redis adapter passes enqueue/dequeue/ack/claim lease tests; structured logs exercised. | +| SCANNER-QUEUE-09-402 | DONE (2025-10-19) | Scanner Queue Guild | SCANNER-QUEUE-09-401 | Add pluggable backend support (Redis, NATS) with configuration binding, health probes, failover documentation. | NATS adapter + DI bindings delivered; health checks documented; configuration tests green. | +| SCANNER-QUEUE-09-403 | DONE (2025-10-19) | Scanner Queue Guild | SCANNER-QUEUE-09-401 | Implement retry and dead-letter flow with structured metrics/logs for offline deployments. | Retry policy configurable; dead-letter queue persisted; metrics counters validated in integration tests. | diff --git a/src/Scanner/__Libraries/StellaOps.Scanner.Queue/TASKS.md b/src/Scanner/__Libraries/StellaOps.Scanner.Queue/TASKS.md index c675440b..42c73e7e 100644 --- a/src/Scanner/__Libraries/StellaOps.Scanner.Queue/TASKS.md +++ b/src/Scanner/__Libraries/StellaOps.Scanner.Queue/TASKS.md @@ -1,7 +1,4 @@ -# Scanner Queue Task Board (Sprint 9) - -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| SCANNER-QUEUE-09-401 | DONE (2025-10-19) | Scanner Queue Guild | — | Implement queue abstraction + Redis Streams adapter with ack/lease semantics, idempotency tokens, and deterministic job IDs. | Interfaces finalized; Redis adapter passes enqueue/dequeue/ack/claim lease tests; structured logs exercised. | -| SCANNER-QUEUE-09-402 | DONE (2025-10-19) | Scanner Queue Guild | SCANNER-QUEUE-09-401 | Add pluggable backend support (Redis, NATS) with configuration binding, health probes, failover documentation. | NATS adapter + DI bindings delivered; health checks documented; configuration tests green. | -| SCANNER-QUEUE-09-403 | DONE (2025-10-19) | Scanner Queue Guild | SCANNER-QUEUE-09-401 | Implement retry and dead-letter flow with structured metrics/logs for offline deployments. | Retry policy configurable; dead-letter queue persisted; metrics counters validated in integration tests. | +# Scanner Queue Task Board (Sprint 9) + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| diff --git a/src/Scanner/__Libraries/StellaOps.Scanner.Storage/TASKS.completed.md b/src/Scanner/__Libraries/StellaOps.Scanner.Storage/TASKS.completed.md new file mode 100644 index 00000000..ebc3da60 --- /dev/null +++ b/src/Scanner/__Libraries/StellaOps.Scanner.Storage/TASKS.completed.md @@ -0,0 +1,9 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| SCANNER-STORAGE-09-301 | DONE (2025-10-18) | Scanner Storage Guild | SCANNER-CORE-09-501 | Mongo catalog schemas/indexes for images, layers, artifacts, jobs, lifecycle rules plus migrations. | Collections created via bootstrapper; migrations recorded; indexes enforce uniqueness + TTL; majority read/write configured. | +| SCANNER-STORAGE-09-302 | DONE (2025-10-18) | Scanner Storage Guild | SCANNER-STORAGE-09-301 | MinIO layout, immutability policies, client abstraction, and configuration binding. | S3 client abstraction configurable via options; bucket/prefix defaults documented; immutability flags enforced with tests; config binding validated. | +| SCANNER-STORAGE-09-303 | DONE (2025-10-18) | Scanner Storage Guild | SCANNER-STORAGE-09-301, SCANNER-STORAGE-09-302 | Repositories/services with dual-write feature flag, deterministic digests, TTL enforcement tests. | Dual-write service writes metadata + objects atomically; digest determinism covered by tests; TTL enforcement fixture passing. | +| SCANNER-STORAGE-09-304 | DONE (2025-10-19) | Scanner Storage Guild | SCANNER-STORAGE-09-303 | Adopt `TimeProvider` across storage timestamps for determinism. | Storage services/repositories use injected `TimeProvider`; tests cover timestamp determinism. | +| SCANNER-STORAGE-11-401 | DONE (2025-10-23) | Scanner Storage Guild | SCANNER-STORAGE-09-302 | Replace MinIO artifact store with RustFS driver, including migration tooling and configuration updates. | RustFS provider registered across Worker/WebService; data migration plan/tooling validated on staging; Helm/offline kit configs updated; regression tests cover RustFS paths with deterministic results. | diff --git a/src/Scanner/__Libraries/StellaOps.Scanner.Storage/TASKS.md b/src/Scanner/__Libraries/StellaOps.Scanner.Storage/TASKS.md index 8baf8227..197ff394 100644 --- a/src/Scanner/__Libraries/StellaOps.Scanner.Storage/TASKS.md +++ b/src/Scanner/__Libraries/StellaOps.Scanner.Storage/TASKS.md @@ -1,9 +1,4 @@ -# Scanner Storage Task Board - -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| SCANNER-STORAGE-09-301 | DONE (2025-10-18) | Scanner Storage Guild | SCANNER-CORE-09-501 | Mongo catalog schemas/indexes for images, layers, artifacts, jobs, lifecycle rules plus migrations. | Collections created via bootstrapper; migrations recorded; indexes enforce uniqueness + TTL; majority read/write configured. | -| SCANNER-STORAGE-09-302 | DONE (2025-10-18) | Scanner Storage Guild | SCANNER-STORAGE-09-301 | MinIO layout, immutability policies, client abstraction, and configuration binding. | S3 client abstraction configurable via options; bucket/prefix defaults documented; immutability flags enforced with tests; config binding validated. | -| SCANNER-STORAGE-09-303 | DONE (2025-10-18) | Scanner Storage Guild | SCANNER-STORAGE-09-301, SCANNER-STORAGE-09-302 | Repositories/services with dual-write feature flag, deterministic digests, TTL enforcement tests. | Dual-write service writes metadata + objects atomically; digest determinism covered by tests; TTL enforcement fixture passing. | -| SCANNER-STORAGE-09-304 | DONE (2025-10-19) | Scanner Storage Guild | SCANNER-STORAGE-09-303 | Adopt `TimeProvider` across storage timestamps for determinism. | Storage services/repositories use injected `TimeProvider`; tests cover timestamp determinism. | -| SCANNER-STORAGE-11-401 | DONE (2025-10-23) | Scanner Storage Guild | SCANNER-STORAGE-09-302 | Replace MinIO artifact store with RustFS driver, including migration tooling and configuration updates. | RustFS provider registered across Worker/WebService; data migration plan/tooling validated on staging; Helm/offline kit configs updated; regression tests cover RustFS paths with deterministic results. | +# Scanner Storage Task Board + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| diff --git a/src/Scanner/__Tests/StellaOps.Scanner.WebService.Tests/PlatformEventSamplesTests.cs b/src/Scanner/__Tests/StellaOps.Scanner.WebService.Tests/PlatformEventSamplesTests.cs index df06acaa..04253d6b 100644 --- a/src/Scanner/__Tests/StellaOps.Scanner.WebService.Tests/PlatformEventSamplesTests.cs +++ b/src/Scanner/__Tests/StellaOps.Scanner.WebService.Tests/PlatformEventSamplesTests.cs @@ -34,8 +34,8 @@ public sealed class PlatformEventSamplesTests Assert.NotEqual(Guid.Empty, orchestratorEvent.EventId); Assert.NotNull(orchestratorEvent.Payload); - AssertCanonical(json, orchestratorEvent); AssertReportConsistency(orchestratorEvent); + AssertCanonical(json, orchestratorEvent); } private static void AssertCanonical(string originalJson, OrchestratorEvent orchestratorEvent) @@ -58,18 +58,38 @@ public sealed class PlatformEventSamplesTests Assert.Equal(ready.ReportId, ready.Report.ReportId); Assert.Equal(ready.ScanId, ready.Report.ReportId); AssertDsseMatchesReport(ready.Dsse, ready.Report); - Assert.False(string.IsNullOrWhiteSpace(ready.Links.Ui)); - Assert.False(string.IsNullOrWhiteSpace(ready.Links.Report)); - Assert.False(string.IsNullOrWhiteSpace(ready.Links.Attestation)); + Assert.NotNull(ready.Links.Report); + Assert.False(string.IsNullOrWhiteSpace(ready.Links.Report!.Ui)); + Assert.False(string.IsNullOrWhiteSpace(ready.Links.Report!.Api)); + if (ready.Links.Policy is not null) + { + Assert.False(string.IsNullOrWhiteSpace(ready.Links.Policy.Ui)); + Assert.False(string.IsNullOrWhiteSpace(ready.Links.Policy.Api)); + } + if (ready.Links.Attestation is not null) + { + Assert.False(string.IsNullOrWhiteSpace(ready.Links.Attestation.Ui)); + Assert.False(string.IsNullOrWhiteSpace(ready.Links.Attestation.Api)); + } break; case ScanCompletedEventPayload completed: Assert.Equal(completed.ReportId, completed.Report.ReportId); Assert.Equal(completed.ScanId, completed.Report.ReportId); AssertDsseMatchesReport(completed.Dsse, completed.Report); Assert.NotEmpty(completed.Findings); - Assert.False(string.IsNullOrWhiteSpace(completed.Links.Ui)); - Assert.False(string.IsNullOrWhiteSpace(completed.Links.Report)); - Assert.False(string.IsNullOrWhiteSpace(completed.Links.Attestation)); + Assert.NotNull(completed.Links.Report); + Assert.False(string.IsNullOrWhiteSpace(completed.Links.Report!.Ui)); + Assert.False(string.IsNullOrWhiteSpace(completed.Links.Report!.Api)); + if (completed.Links.Policy is not null) + { + Assert.False(string.IsNullOrWhiteSpace(completed.Links.Policy.Ui)); + Assert.False(string.IsNullOrWhiteSpace(completed.Links.Policy.Api)); + } + if (completed.Links.Attestation is not null) + { + Assert.False(string.IsNullOrWhiteSpace(completed.Links.Attestation.Ui)); + Assert.False(string.IsNullOrWhiteSpace(completed.Links.Attestation.Api)); + } break; default: throw new InvalidOperationException($"Unexpected payload type {orchestratorEvent.Payload.GetType().Name}."); @@ -118,6 +138,16 @@ public sealed class PlatformEventSamplesTests _ => throw new InvalidOperationException("Unexpected event kind.") }; + if (payload is ReportReadyEventPayload readyPayload && string.IsNullOrEmpty(readyPayload.ReportId)) + { + throw new InvalidOperationException("ReportId was not parsed from sample payload."); + } + + if (payload is ScanCompletedEventPayload completedPayload && string.IsNullOrEmpty(completedPayload.ReportId)) + { + throw new InvalidOperationException("ReportId was not parsed from scan completed payload."); + } + return new OrchestratorEvent { EventId = Guid.Parse(root["eventId"]!.GetValue()), diff --git a/src/Scanner/__Tests/StellaOps.Scanner.WebService.Tests/ReportEventDispatcherTests.cs b/src/Scanner/__Tests/StellaOps.Scanner.WebService.Tests/ReportEventDispatcherTests.cs index 2f52392a..aed11d85 100644 --- a/src/Scanner/__Tests/StellaOps.Scanner.WebService.Tests/ReportEventDispatcherTests.cs +++ b/src/Scanner/__Tests/StellaOps.Scanner.WebService.Tests/ReportEventDispatcherTests.cs @@ -10,9 +10,11 @@ using System.Threading.Tasks; using Microsoft.AspNetCore.Http; using Microsoft.Extensions.Logging; using Microsoft.Extensions.Logging.Abstractions; +using Microsoft.Extensions.Options; using StellaOps.Auth.Abstractions; using StellaOps.Policy; using StellaOps.Scanner.WebService.Contracts; +using StellaOps.Scanner.WebService.Options; using StellaOps.Scanner.WebService.Services; namespace StellaOps.Scanner.WebService.Tests; @@ -28,7 +30,7 @@ public sealed class ReportEventDispatcherTests public async Task PublishAsync_EmitsReportReadyAndScanCompleted() { var publisher = new RecordingEventPublisher(); - var dispatcher = new ReportEventDispatcher(publisher, TimeProvider.System, NullLogger.Instance); + var dispatcher = new ReportEventDispatcher(publisher, Microsoft.Extensions.Options.Options.Create(new ScannerWebServiceOptions()), TimeProvider.System, NullLogger.Instance); var cancellationToken = CancellationToken.None; var request = new ReportRequestDto @@ -132,10 +134,12 @@ public sealed class ReportEventDispatcherTests Assert.NotNull(readyPayload.Delta); Assert.Equal(1, readyPayload.Delta?.NewCritical); Assert.Contains("CVE-2024-9999", readyPayload.Delta?.Kev ?? Array.Empty()); - Assert.Equal("https://scanner.example/ui/reports/report-abc", readyPayload.Links.Ui); - Assert.Equal("https://scanner.example/api/v1/reports/report-abc", readyPayload.Links.Report); - Assert.Equal("https://scanner.example/api/v1/policy/revisions/rev-42", readyPayload.Links.Policy); - Assert.Equal("https://scanner.example/ui/attestations/report-abc", readyPayload.Links.Attestation); + Assert.Equal("https://scanner.example/ui/reports/report-abc", readyPayload.Links.Report?.Ui); + Assert.Equal("https://scanner.example/api/v1/reports/report-abc", readyPayload.Links.Report?.Api); + Assert.Equal("https://scanner.example/ui/policy/revisions/rev-42", readyPayload.Links.Policy?.Ui); + Assert.Equal("https://scanner.example/api/v1/policy/revisions/rev-42", readyPayload.Links.Policy?.Api); + Assert.Equal("https://scanner.example/ui/attestations/report-abc", readyPayload.Links.Attestation?.Ui); + Assert.Equal("https://scanner.example/api/v1/reports/report-abc/attestation", readyPayload.Links.Attestation?.Api); Assert.Equal(envelope.Payload, readyPayload.Dsse?.Payload); Assert.Equal("blocked", readyPayload.Report.Verdict); @@ -151,9 +155,12 @@ public sealed class ReportEventDispatcherTests Assert.Equal("finding-1", finding.Id); Assert.Equal("runtime", finding.Reachability); Assert.Equal("CVE-2024-9999", finding.Cve); - Assert.Equal("https://scanner.example/api/v1/reports/report-abc", scanPayload.Links.Report); - Assert.Equal("https://scanner.example/api/v1/policy/revisions/rev-42", scanPayload.Links.Policy); - Assert.Equal("https://scanner.example/ui/attestations/report-abc", scanPayload.Links.Attestation); + Assert.Equal("https://scanner.example/api/v1/reports/report-abc", scanPayload.Links.Report?.Api); + Assert.Equal("https://scanner.example/ui/reports/report-abc", scanPayload.Links.Report?.Ui); + Assert.Equal("https://scanner.example/ui/policy/revisions/rev-42", scanPayload.Links.Policy?.Ui); + Assert.Equal("https://scanner.example/api/v1/policy/revisions/rev-42", scanPayload.Links.Policy?.Api); + Assert.Equal("https://scanner.example/ui/attestations/report-abc", scanPayload.Links.Attestation?.Ui); + Assert.Equal("https://scanner.example/api/v1/reports/report-abc/attestation", scanPayload.Links.Attestation?.Api); Assert.Equal(envelope.Payload, scanPayload.Dsse?.Payload); Assert.Equal("blocked", scanPayload.Report.Verdict); } diff --git a/src/Scanner/__Tests/StellaOps.Scanner.WebService.Tests/ReportsEndpointsTests.cs b/src/Scanner/__Tests/StellaOps.Scanner.WebService.Tests/ReportsEndpointsTests.cs index ee922c9e..55bfc6de 100644 --- a/src/Scanner/__Tests/StellaOps.Scanner.WebService.Tests/ReportsEndpointsTests.cs +++ b/src/Scanner/__Tests/StellaOps.Scanner.WebService.Tests/ReportsEndpointsTests.cs @@ -218,23 +218,28 @@ rules: Assert.Equal("sha256:cafebabe", ready.Scope?.Digest); Assert.NotNull(readyPayload.Dsse); Assert.Equal(readyPayload.ReportId, readyPayload.Report.ReportId); - Assert.Equal("http://localhost/ui/reports/" + readyPayload.ReportId, readyPayload.Links.Ui); - Assert.Equal("http://localhost/api/v1/reports/" + readyPayload.ReportId, readyPayload.Links.Report); + Assert.Equal("http://localhost/ui/reports/" + readyPayload.ReportId, readyPayload.Links.Report?.Ui); + Assert.Equal("http://localhost/api/v1/reports/" + readyPayload.ReportId, readyPayload.Links.Report?.Api); if (!string.IsNullOrWhiteSpace(revisionId)) { - Assert.Equal("http://localhost/api/v1/policy/revisions/" + revisionId, readyPayload.Links.Policy); + Assert.Equal("http://localhost/ui/policy/revisions/" + revisionId, readyPayload.Links.Policy?.Ui); + Assert.Equal("http://localhost/api/v1/policy/revisions/" + revisionId, readyPayload.Links.Policy?.Api); } - Assert.Equal("http://localhost/ui/attestations/" + readyPayload.ReportId, readyPayload.Links.Attestation); + Assert.Equal("http://localhost/ui/attestations/" + readyPayload.ReportId, readyPayload.Links.Attestation?.Ui); + Assert.Equal("http://localhost/api/v1/reports/" + readyPayload.ReportId + "/attestation", readyPayload.Links.Attestation?.Api); Assert.Equal("fail", completedPayload.Verdict); Assert.NotEmpty(completedPayload.Findings); Assert.Equal("finding-42", completedPayload.Findings[0].Id); - Assert.Equal("http://localhost/api/v1/reports/" + completedPayload.ReportId, completedPayload.Links.Report); + Assert.Equal("http://localhost/api/v1/reports/" + completedPayload.ReportId, completedPayload.Links.Report?.Api); + Assert.Equal("http://localhost/ui/reports/" + completedPayload.ReportId, completedPayload.Links.Report?.Ui); if (!string.IsNullOrWhiteSpace(revisionId)) { - Assert.Equal("http://localhost/api/v1/policy/revisions/" + revisionId, completedPayload.Links.Policy); + Assert.Equal("http://localhost/ui/policy/revisions/" + revisionId, completedPayload.Links.Policy?.Ui); + Assert.Equal("http://localhost/api/v1/policy/revisions/" + revisionId, completedPayload.Links.Policy?.Api); } - Assert.Equal("http://localhost/ui/attestations/" + completedPayload.ReportId, completedPayload.Links.Attestation); + Assert.Equal("http://localhost/ui/attestations/" + completedPayload.ReportId, completedPayload.Links.Attestation?.Ui); + Assert.Equal("http://localhost/api/v1/reports/" + completedPayload.ReportId + "/attestation", completedPayload.Links.Attestation?.Api); } private sealed class RecordingPlatformEventPublisher : IPlatformEventPublisher diff --git a/src/Scanner/docs/events/samples/scanner.event.report.ready@1.sample.json b/src/Scanner/docs/events/samples/scanner.event.report.ready@1.sample.json new file mode 100644 index 00000000..e986e2d1 --- /dev/null +++ b/src/Scanner/docs/events/samples/scanner.event.report.ready@1.sample.json @@ -0,0 +1,101 @@ +{ + "eventId": "6d2d1b77-f3c3-4f70-8a9d-6f2d0c8801ab", + "kind": "scanner.event.report.ready", + "version": 1, + "tenant": "tenant-alpha", + "occurredAt": "2025-10-19T12:34:56Z", + "recordedAt": "2025-10-19T12:34:57Z", + "source": "scanner.webservice", + "idempotencyKey": "scanner.event.report.ready:tenant-alpha:report-abc", + "correlationId": "report-abc", + "traceId": "0af7651916cd43dd8448eb211c80319c", + "spanId": "b7ad6b7169203331", + "scope": { + "namespace": "acme/edge", + "repo": "api", + "digest": "sha256:feedface" + }, + "attributes": { + "reportId": "report-abc", + "policyRevisionId": "rev-42", + "policyDigest": "digest-123", + "verdict": "blocked" + }, + "payload": { + "reportId": "report-abc", + "scanId": "report-abc", + "imageDigest": "sha256:feedface", + "generatedAt": "2025-10-19T12:34:56Z", + "verdict": "fail", + "summary": { + "total": 1, + "blocked": 1, + "warned": 0, + "ignored": 0, + "quieted": 0 + }, + "delta": { + "newCritical": 1, + "kev": [ + "CVE-2024-9999" + ] + }, + "quietedFindingCount": 0, + "policy": { + "digest": "digest-123", + "revisionId": "rev-42" + }, + "links": { + "report": { + "ui": "https://scanner.example/ui/reports/report-abc", + "api": "https://scanner.example/api/v1/reports/report-abc" + }, + "policy": { + "ui": "https://scanner.example/ui/policy/revisions/rev-42", + "api": "https://scanner.example/api/v1/policy/revisions/rev-42" + }, + "attestation": { + "ui": "https://scanner.example/ui/attestations/report-abc", + "api": "https://scanner.example/api/v1/reports/report-abc/attestation" + } + }, + "dsse": { + "payloadType": "application/vnd.stellaops.report+json", + "payload": "eyJyZXBvcnRJZCI6InJlcG9ydC1hYmMiLCJpbWFnZURpZ2VzdCI6InNoYTI1NjpmZWVkZmFjZSIsImdlbmVyYXRlZEF0IjoiMjAyNS0xMC0xOVQxMjozNDo1NiswMDowMCIsInZlcmRpY3QiOiJibG9ja2VkIiwicG9saWN5Ijp7InJldmlzaW9uSWQiOiJyZXYtNDIiLCJkaWdlc3QiOiJkaWdlc3QtMTIzIn0sInN1bW1hcnkiOnsidG90YWwiOjEsImJsb2NrZWQiOjEsIndhcm5lZCI6MCwiaWdub3JlZCI6MCwicXVpZXRlZCI6MH0sInZlcmRpY3RzIjpbeyJmaW5kaW5nSWQiOiJmaW5kaW5nLTEiLCJzdGF0dXMiOiJCbG9ja2VkIiwic2NvcmUiOjQ3LjUsInNvdXJjZVRydXN0IjoiTlZEIiwicmVhY2hhYmlsaXR5IjoicnVudGltZSJ9XSwiaXNzdWVzIjpbXX0=", + "signatures": [ + { + "keyId": "test-key", + "algorithm": "hs256", + "signature": "signature-value" + } + ] + }, + "report": { + "reportId": "report-abc", + "generatedAt": "2025-10-19T12:34:56Z", + "imageDigest": "sha256:feedface", + "policy": { + "digest": "digest-123", + "revisionId": "rev-42" + }, + "summary": { + "total": 1, + "blocked": 1, + "warned": 0, + "ignored": 0, + "quieted": 0 + }, + "verdict": "blocked", + "verdicts": [ + { + "findingId": "finding-1", + "status": "Blocked", + "score": 47.5, + "sourceTrust": "NVD", + "reachability": "runtime" + } + ], + "issues": [] + } + } +} diff --git a/src/Scanner/docs/events/samples/scanner.event.scan.completed@1.sample.json b/src/Scanner/docs/events/samples/scanner.event.scan.completed@1.sample.json new file mode 100644 index 00000000..ab4a3db4 --- /dev/null +++ b/src/Scanner/docs/events/samples/scanner.event.scan.completed@1.sample.json @@ -0,0 +1,107 @@ +{ + "eventId": "08a6de24-4a94-4d14-8432-9d14f36f6da3", + "kind": "scanner.event.scan.completed", + "version": 1, + "tenant": "tenant-alpha", + "occurredAt": "2025-10-19T12:34:56Z", + "recordedAt": "2025-10-19T12:34:57Z", + "source": "scanner.webservice", + "idempotencyKey": "scanner.event.scan.completed:tenant-alpha:report-abc", + "correlationId": "report-abc", + "traceId": "4bf92f3577b34da6a3ce929d0e0e4736", + "scope": { + "namespace": "acme/edge", + "repo": "api", + "digest": "sha256:feedface" + }, + "attributes": { + "reportId": "report-abc", + "policyRevisionId": "rev-42", + "policyDigest": "digest-123", + "verdict": "blocked" + }, + "payload": { + "reportId": "report-abc", + "scanId": "report-abc", + "imageDigest": "sha256:feedface", + "verdict": "fail", + "summary": { + "total": 1, + "blocked": 1, + "warned": 0, + "ignored": 0, + "quieted": 0 + }, + "delta": { + "newCritical": 1, + "kev": [ + "CVE-2024-9999" + ] + }, + "policy": { + "digest": "digest-123", + "revisionId": "rev-42" + }, + "findings": [ + { + "id": "finding-1", + "severity": "Critical", + "cve": "CVE-2024-9999", + "purl": "pkg:docker/acme/edge-api@sha256-feedface", + "reachability": "runtime" + } + ], + "links": { + "report": { + "ui": "https://scanner.example/ui/reports/report-abc", + "api": "https://scanner.example/api/v1/reports/report-abc" + }, + "policy": { + "ui": "https://scanner.example/ui/policy/revisions/rev-42", + "api": "https://scanner.example/api/v1/policy/revisions/rev-42" + }, + "attestation": { + "ui": "https://scanner.example/ui/attestations/report-abc", + "api": "https://scanner.example/api/v1/reports/report-abc/attestation" + } + }, + "dsse": { + "payloadType": "application/vnd.stellaops.report+json", + "payload": "eyJyZXBvcnRJZCI6InJlcG9ydC1hYmMiLCJpbWFnZURpZ2VzdCI6InNoYTI1NjpmZWVkZmFjZSIsImdlbmVyYXRlZEF0IjoiMjAyNS0xMC0xOVQxMjozNDo1NiswMDowMCIsInZlcmRpY3QiOiJibG9ja2VkIiwicG9saWN5Ijp7InJldmlzaW9uSWQiOiJyZXYtNDIiLCJkaWdlc3QiOiJkaWdlc3QtMTIzIn0sInN1bW1hcnkiOnsidG90YWwiOjEsImJsb2NrZWQiOjEsIndhcm5lZCI6MCwiaWdub3JlZCI6MCwicXVpZXRlZCI6MH0sInZlcmRpY3RzIjpbeyJmaW5kaW5nSWQiOiJmaW5kaW5nLTEiLCJzdGF0dXMiOiJCbG9ja2VkIiwic2NvcmUiOjQ3LjUsInNvdXJjZVRydXN0IjoiTlZEIiwicmVhY2hhYmlsaXR5IjoicnVudGltZSJ9XSwiaXNzdWVzIjpbXX0=", + "signatures": [ + { + "keyId": "test-key", + "algorithm": "hs256", + "signature": "signature-value" + } + ] + }, + "report": { + "reportId": "report-abc", + "generatedAt": "2025-10-19T12:34:56Z", + "imageDigest": "sha256:feedface", + "policy": { + "digest": "digest-123", + "revisionId": "rev-42" + }, + "summary": { + "total": 1, + "blocked": 1, + "warned": 0, + "ignored": 0, + "quieted": 0 + }, + "verdict": "blocked", + "verdicts": [ + { + "findingId": "finding-1", + "status": "Blocked", + "score": 47.5, + "sourceTrust": "NVD", + "reachability": "runtime" + } + ], + "issues": [] + } + } +} diff --git a/src/Scheduler/StellaOps.Scheduler.WebService/GraphJobs/Events/GraphJobEventPublisher.cs b/src/Scheduler/StellaOps.Scheduler.WebService/GraphJobs/Events/GraphJobEventPublisher.cs index df6b17cc..55fb23a2 100644 --- a/src/Scheduler/StellaOps.Scheduler.WebService/GraphJobs/Events/GraphJobEventPublisher.cs +++ b/src/Scheduler/StellaOps.Scheduler.WebService/GraphJobs/Events/GraphJobEventPublisher.cs @@ -1,41 +1,185 @@ -using System.Text.Json; -using System.Text.Json.Serialization; -using Microsoft.Extensions.Logging; -using Microsoft.Extensions.Options; -using StellaOps.Scheduler.WebService.Options; - -namespace StellaOps.Scheduler.WebService.GraphJobs.Events; - -internal sealed class GraphJobEventPublisher : IGraphJobCompletionPublisher -{ - private static readonly JsonSerializerOptions SerializerOptions = new(JsonSerializerDefaults.Web) - { - DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull - }; - - private readonly IOptionsMonitor _options; - private readonly ILogger _logger; - - public GraphJobEventPublisher( - IOptionsMonitor options, - ILogger logger) - { - _options = options ?? throw new ArgumentNullException(nameof(options)); - _logger = logger ?? throw new ArgumentNullException(nameof(logger)); - } - - public Task PublishAsync(GraphJobCompletionNotification notification, CancellationToken cancellationToken) - { - var options = _options.CurrentValue; - if (!options.GraphJobs.Enabled) - { - _logger.LogDebug("Graph job events disabled; skipping emission for {JobId}.", notification.Job.Id); - return Task.CompletedTask; - } - - var envelope = GraphJobEventFactory.Create(notification); - var json = JsonSerializer.Serialize(envelope, SerializerOptions); - _logger.LogInformation("{EventJson}", json); - return Task.CompletedTask; - } -} +using System; +using System.Text.Json; +using System.Text.Json.Serialization; +using System.Threading; +using System.Threading.Tasks; +using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Options; +using StackExchange.Redis; +using StellaOps.Scheduler.WebService.Options; + +namespace StellaOps.Scheduler.WebService.GraphJobs.Events; + +internal sealed class GraphJobEventPublisher : IGraphJobCompletionPublisher, IAsyncDisposable +{ + private static readonly JsonSerializerOptions SerializerOptions = new(JsonSerializerDefaults.Web) + { + DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull + }; + + private readonly IOptionsMonitor _options; + private readonly IRedisConnectionFactory _connectionFactory; + private readonly ILogger _logger; + private readonly SemaphoreSlim _connectionGate = new(1, 1); + + private IConnectionMultiplexer? _connection; + private bool _disposed; + + public GraphJobEventPublisher( + IOptionsMonitor options, + IRedisConnectionFactory connectionFactory, + ILogger logger) + { + _options = options ?? throw new ArgumentNullException(nameof(options)); + _connectionFactory = connectionFactory ?? throw new ArgumentNullException(nameof(connectionFactory)); + _logger = logger ?? throw new ArgumentNullException(nameof(logger)); + } + + public async Task PublishAsync(GraphJobCompletionNotification notification, CancellationToken cancellationToken) + { + if (notification is null) + { + throw new ArgumentNullException(nameof(notification)); + } + + var options = _options.CurrentValue?.GraphJobs ?? new GraphJobEventsOptions(); + if (!options.Enabled) + { + _logger.LogDebug("Graph job events disabled; skipping emission for {JobId}.", notification.Job.Id); + return; + } + + if (!string.Equals(options.Driver, "redis", StringComparison.OrdinalIgnoreCase)) + { + _logger.LogWarning( + "Graph job events configured with unsupported driver '{Driver}'. Falling back to logging.", + options.Driver); + LogEnvelope(notification); + return; + } + + try + { + var database = await GetDatabaseAsync(options, cancellationToken).ConfigureAwait(false); + var envelope = GraphJobEventFactory.Create(notification); + var payload = JsonSerializer.Serialize(envelope, SerializerOptions); + var entries = new[] + { + new NameValueEntry("event", payload), + new NameValueEntry("kind", envelope.Kind), + new NameValueEntry("tenant", envelope.Tenant), + new NameValueEntry("occurredAt", envelope.Timestamp.ToString("O")), + new NameValueEntry("jobId", notification.Job.Id), + new NameValueEntry("status", notification.Status.ToString()) + }; + + var streamKey = string.IsNullOrWhiteSpace(options.Stream) ? "stella.events" : options.Stream; + var publishTask = CreatePublishTask(database, streamKey, entries, options.MaxStreamLength); + + if (options.PublishTimeoutSeconds > 0) + { + var timeout = TimeSpan.FromSeconds(options.PublishTimeoutSeconds); + await publishTask.WaitAsync(timeout, cancellationToken).ConfigureAwait(false); + } + else + { + await publishTask.ConfigureAwait(false); + } + + _logger.LogDebug("Published graph job event {JobId} to stream {Stream}.", notification.Job.Id, streamKey); + } + catch (Exception ex) + { + _logger.LogError(ex, "Failed to publish graph job completion for {JobId}; logging payload instead.", notification.Job.Id); + LogEnvelope(notification); + } + } + + private Task CreatePublishTask(IDatabase database, string streamKey, NameValueEntry[] entries, long maxStreamLength) + { + if (maxStreamLength > 0) + { + var clamped = (int)Math.Min(maxStreamLength, int.MaxValue); + return database.StreamAddAsync(streamKey, entries, maxLength: clamped, useApproximateMaxLength: true); + } + + return database.StreamAddAsync(streamKey, entries); + } + + private async Task GetDatabaseAsync(GraphJobEventsOptions options, CancellationToken cancellationToken) + { + cancellationToken.ThrowIfCancellationRequested(); + + if (_connection is { IsConnected: true }) + { + return _connection.GetDatabase(); + } + + await _connectionGate.WaitAsync(cancellationToken).ConfigureAwait(false); + try + { + if (_connection is null || !_connection.IsConnected) + { + var configuration = ConfigurationOptions.Parse(options.Dsn); + configuration.AbortOnConnectFail = false; + + if (options.DriverSettings.TryGetValue("clientName", out var clientName) && !string.IsNullOrWhiteSpace(clientName)) + { + configuration.ClientName = clientName; + } + + if (options.DriverSettings.TryGetValue("ssl", out var sslValue) && bool.TryParse(sslValue, out var ssl)) + { + configuration.Ssl = ssl; + } + + if (options.DriverSettings.TryGetValue("password", out var password) && !string.IsNullOrWhiteSpace(password)) + { + configuration.Password = password; + } + + _connection = await _connectionFactory.ConnectAsync(configuration, cancellationToken).ConfigureAwait(false); + _logger.LogInformation("Connected graph job publisher to Redis stream {Stream}.", options.Stream); + } + } + finally + { + _connectionGate.Release(); + } + + return _connection!.GetDatabase(); + } + + private void LogEnvelope(GraphJobCompletionNotification notification) + { + var envelope = GraphJobEventFactory.Create(notification); + var json = JsonSerializer.Serialize(envelope, SerializerOptions); + _logger.LogInformation("{EventJson}", json); + } + + public async ValueTask DisposeAsync() + { + if (_disposed) + { + return; + } + + _disposed = true; + + if (_connection is not null) + { + try + { + await _connection.CloseAsync(); + } + catch (Exception ex) + { + _logger.LogDebug(ex, "Error while closing graph job Redis connection."); + } + + _connection.Dispose(); + } + + _connectionGate.Dispose(); + } +} diff --git a/src/Scheduler/StellaOps.Scheduler.WebService/GraphJobs/Events/IRedisConnectionFactory.cs b/src/Scheduler/StellaOps.Scheduler.WebService/GraphJobs/Events/IRedisConnectionFactory.cs new file mode 100644 index 00000000..888968a9 --- /dev/null +++ b/src/Scheduler/StellaOps.Scheduler.WebService/GraphJobs/Events/IRedisConnectionFactory.cs @@ -0,0 +1,8 @@ +using StackExchange.Redis; + +namespace StellaOps.Scheduler.WebService.GraphJobs.Events; + +internal interface IRedisConnectionFactory +{ + Task ConnectAsync(ConfigurationOptions options, CancellationToken cancellationToken); +} diff --git a/src/Scheduler/StellaOps.Scheduler.WebService/GraphJobs/Events/RedisConnectionFactory.cs b/src/Scheduler/StellaOps.Scheduler.WebService/GraphJobs/Events/RedisConnectionFactory.cs new file mode 100644 index 00000000..31180c33 --- /dev/null +++ b/src/Scheduler/StellaOps.Scheduler.WebService/GraphJobs/Events/RedisConnectionFactory.cs @@ -0,0 +1,26 @@ +using StackExchange.Redis; + +namespace StellaOps.Scheduler.WebService.GraphJobs.Events; + +internal sealed class RedisConnectionFactory : IRedisConnectionFactory +{ + public async Task ConnectAsync(ConfigurationOptions options, CancellationToken cancellationToken) + { + ArgumentNullException.ThrowIfNull(options); + + var completionSource = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); + cancellationToken.Register(() => completionSource.TrySetCanceled(cancellationToken)); + + try + { + var connection = await ConnectionMultiplexer.ConnectAsync(options).ConfigureAwait(false); + completionSource.TrySetResult(connection); + } + catch (Exception ex) + { + completionSource.TrySetException(ex); + } + + return await completionSource.Task.ConfigureAwait(false); + } +} diff --git a/src/Scheduler/StellaOps.Scheduler.WebService/Options/SchedulerEventsOptions.cs b/src/Scheduler/StellaOps.Scheduler.WebService/Options/SchedulerEventsOptions.cs index d27b26f3..d04b8984 100644 --- a/src/Scheduler/StellaOps.Scheduler.WebService/Options/SchedulerEventsOptions.cs +++ b/src/Scheduler/StellaOps.Scheduler.WebService/Options/SchedulerEventsOptions.cs @@ -3,7 +3,8 @@ namespace StellaOps.Scheduler.WebService.Options; /// /// Scheduler WebService event options (outbound + inbound). /// -using System; +using System; +using System.Collections.Generic; public sealed class SchedulerEventsOptions { @@ -12,13 +13,43 @@ public sealed class SchedulerEventsOptions public SchedulerInboundWebhooksOptions Webhooks { get; set; } = new(); } -public sealed class GraphJobEventsOptions -{ - /// - /// Enables emission of legacy scheduler.graph.job.completed@1 events. - /// - public bool Enabled { get; set; } -} +public sealed class GraphJobEventsOptions +{ + /// + /// Enables emission of legacy scheduler.graph.job.completed@1 events. + /// + public bool Enabled { get; set; } + + /// + /// Event transport driver (defaults to redis). + /// + public string Driver { get; set; } = "redis"; + + /// + /// Connection string for the event transport. + /// + public string Dsn { get; set; } = string.Empty; + + /// + /// Stream/topic identifier for published events. + /// + public string Stream { get; set; } = "stella.events"; + + /// + /// Maximum time in seconds to wait for the transport to accept an event. + /// + public double PublishTimeoutSeconds { get; set; } = 5; + + /// + /// Maximum number of events to retain in the stream. + /// + public long MaxStreamLength { get; set; } = 10000; + + /// + /// Additional transport-specific settings (e.g., clientName, ssl). + /// + public IDictionary DriverSettings { get; set; } = new Dictionary(StringComparer.OrdinalIgnoreCase); +} public sealed class SchedulerInboundWebhooksOptions { diff --git a/src/Scheduler/StellaOps.Scheduler.WebService/Program.cs b/src/Scheduler/StellaOps.Scheduler.WebService/Program.cs index 60d5d268..8103adba 100644 --- a/src/Scheduler/StellaOps.Scheduler.WebService/Program.cs +++ b/src/Scheduler/StellaOps.Scheduler.WebService/Program.cs @@ -48,8 +48,8 @@ if (authorityOptions.Audiences.Count == 0) authorityOptions.Validate(); builder.Services.AddSingleton(authorityOptions); -builder.Services.AddOptions() - .Bind(builder.Configuration.GetSection("Scheduler:Events")) +builder.Services.AddOptions() + .Bind(builder.Configuration.GetSection("Scheduler:Events")) .PostConfigure(options => { options.Webhooks ??= new SchedulerInboundWebhooksOptions(); @@ -67,10 +67,11 @@ builder.Services.AddOptions() options.Webhooks.Vexer.Validate(); }); -builder.Services.AddMemoryCache(); -builder.Services.AddSingleton(); -builder.Services.AddSingleton(); -builder.Services.AddSingleton(); +builder.Services.AddMemoryCache(); +builder.Services.AddSingleton(); +builder.Services.AddSingleton(); +builder.Services.AddSingleton(); +builder.Services.AddSingleton(); var cartographerOptions = builder.Configuration.GetSection("Scheduler:Cartographer").Get() ?? new SchedulerCartographerOptions(); builder.Services.AddSingleton(cartographerOptions); diff --git a/src/Scheduler/StellaOps.Scheduler.WebService/StellaOps.Scheduler.WebService.csproj b/src/Scheduler/StellaOps.Scheduler.WebService/StellaOps.Scheduler.WebService.csproj index 420267be..fa490073 100644 --- a/src/Scheduler/StellaOps.Scheduler.WebService/StellaOps.Scheduler.WebService.csproj +++ b/src/Scheduler/StellaOps.Scheduler.WebService/StellaOps.Scheduler.WebService.csproj @@ -13,4 +13,7 @@ - \ No newline at end of file + + + + diff --git a/src/Scheduler/StellaOps.Scheduler.WebService/TASKS.completed.md b/src/Scheduler/StellaOps.Scheduler.WebService/TASKS.completed.md new file mode 100644 index 00000000..8a7ab8a8 --- /dev/null +++ b/src/Scheduler/StellaOps.Scheduler.WebService/TASKS.completed.md @@ -0,0 +1,12 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| SCHED-WEB-16-101 | DONE (2025-10-27) | Scheduler WebService Guild | SCHED-MODELS-16-101 | Bootstrap Minimal API host with Authority OpTok + DPoP, health endpoints, plug-in discovery per architecture §§1–2. | Service boots with config validation; `/healthz`/`/readyz` pass; restart-only plug-ins enforced. | +| SCHED-WEB-16-102 | DONE (2025-10-26) | Scheduler WebService Guild | SCHED-WEB-16-101 | Implement schedules CRUD (tenant-scoped) with cron validation, pause/resume, audit logging. | CRUD operations tested; invalid cron inputs rejected; audit entries persisted. | +| SCHED-WEB-16-103 | DONE (2025-10-26) | Scheduler WebService Guild | SCHED-WEB-16-102 | Runs API (list/detail/cancel), ad-hoc run POST, and impact preview endpoints. | Integration tests cover run lifecycle; preview returns counts/sample; cancellation honoured. | +| SCHED-WEB-16-104 | DONE (2025-10-27) | Scheduler WebService Guild | SCHED-QUEUE-16-401, SCHED-STORAGE-16-201 | Webhook endpoints for Feeder/Vexer exports with mTLS/HMAC validation and rate limiting. | Webhooks validated via tests; invalid signatures rejected; rate limits documented. | +| SCHED-WEB-20-001 | DONE (2025-10-29) | Scheduler WebService Guild, Policy Guild | SCHED-WEB-16-101, POLICY-ENGINE-20-000 | Expose policy run scheduling APIs (`POST /policy/runs`, `GET /policy/runs`) with tenant scoping and RBAC enforcement for `policy:run`. | Endpoints documented; integration tests cover run creation/status; unauthorized access blocked. | +| SCHED-WEB-21-001 | DONE (2025-10-26) | Scheduler WebService Guild, Cartographer Guild | SCHED-WEB-16-101, SCHED-MODELS-21-001 | Expose graph build/overlay job APIs (`POST /graphs/build`, `GET /graphs/jobs`) with `graph:write`/`graph:read` enforcement and tenant scoping. | APIs documented in `docs/SCHED-WEB-21-001-GRAPH-APIS.md`; integration tests cover submission/status; unauthorized requests blocked; scope checks now reference `StellaOpsScopes`. | +| SCHED-WEB-21-002 | DONE (2025-10-26) | Scheduler WebService Guild | SCHED-WEB-21-001, CARTO-GRAPH-21-007 | Provide overlay lag metrics endpoint and webhook to notify Cartographer of job completions; include correlation IDs. | `POST /graphs/hooks/completed` + `GET /graphs/overlays/lag` documented in `docs/SCHED-WEB-21-001-GRAPH-APIS.md`; integration tests cover completion + metrics. | +| SCHED-WEB-21-003 | DONE (2025-10-26) | Scheduler WebService Guild, Authority Core Guild | AUTH-GRAPH-21-001 | Replace temporary `X-Scopes`/`X-Tenant-Id` headers with Authority-issued OpTok verification and scope enforcement for graph endpoints. | Authentication configured via `AddStellaOpsResourceServerAuthentication`; authority scopes enforced end-to-end with `StellaOpsScopes`; header fallback limited to dev mode; tests updated. | diff --git a/src/Scheduler/StellaOps.Scheduler.WebService/TASKS.md b/src/Scheduler/StellaOps.Scheduler.WebService/TASKS.md index 9823336f..7716e03b 100644 --- a/src/Scheduler/StellaOps.Scheduler.WebService/TASKS.md +++ b/src/Scheduler/StellaOps.Scheduler.WebService/TASKS.md @@ -2,16 +2,11 @@ | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| SCHED-WEB-16-101 | DONE (2025-10-27) | Scheduler WebService Guild | SCHED-MODELS-16-101 | Bootstrap Minimal API host with Authority OpTok + DPoP, health endpoints, plug-in discovery per architecture §§1–2. | Service boots with config validation; `/healthz`/`/readyz` pass; restart-only plug-ins enforced. | -| SCHED-WEB-16-102 | DONE (2025-10-26) | Scheduler WebService Guild | SCHED-WEB-16-101 | Implement schedules CRUD (tenant-scoped) with cron validation, pause/resume, audit logging. | CRUD operations tested; invalid cron inputs rejected; audit entries persisted. | -| SCHED-WEB-16-103 | DONE (2025-10-26) | Scheduler WebService Guild | SCHED-WEB-16-102 | Runs API (list/detail/cancel), ad-hoc run POST, and impact preview endpoints. | Integration tests cover run lifecycle; preview returns counts/sample; cancellation honoured. | -| SCHED-WEB-16-104 | DONE (2025-10-27) | Scheduler WebService Guild | SCHED-QUEUE-16-401, SCHED-STORAGE-16-201 | Webhook endpoints for Feeder/Vexer exports with mTLS/HMAC validation and rate limiting. | Webhooks validated via tests; invalid signatures rejected; rate limits documented. | ## Policy Engine v2 (Sprint 20) | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| SCHED-WEB-20-001 | DONE (2025-10-29) | Scheduler WebService Guild, Policy Guild | SCHED-WEB-16-101, POLICY-ENGINE-20-000 | Expose policy run scheduling APIs (`POST /policy/runs`, `GET /policy/runs`) with tenant scoping and RBAC enforcement for `policy:run`. | Endpoints documented; integration tests cover run creation/status; unauthorized access blocked. | > 2025-10-29: Added `/api/v1/scheduler/policy/runs` create/list/get endpoints with in-memory queue, scope/tenant enforcement, and contract docs (`docs/SCHED-WEB-20-001-POLICY-RUNS.md`). Tests cover happy path + auth failures. > 2025-10-26: Use canonical request/response samples from `samples/api/scheduler/policy-*.json`; serializer contract defined in `src/Scheduler/__Libraries/StellaOps.Scheduler.Models/docs/SCHED-MODELS-20-001-POLICY-RUNS.md`. | SCHED-WEB-20-002 | BLOCKED (waiting on SCHED-WORKER-20-301) | Scheduler WebService Guild | SCHED-WEB-20-001, SCHED-WORKER-20-301 | Provide simulation trigger endpoint returning diff preview metadata and job state for UI/CLI consumption. | Simulation endpoint returns deterministic diffs metadata; rate limits enforced; tests cover concurrency. | @@ -21,10 +16,8 @@ | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| SCHED-WEB-21-001 | DONE (2025-10-26) | Scheduler WebService Guild, Cartographer Guild | SCHED-WEB-16-101, SCHED-MODELS-21-001 | Expose graph build/overlay job APIs (`POST /graphs/build`, `GET /graphs/jobs`) with `graph:write`/`graph:read` enforcement and tenant scoping. | APIs documented in `docs/SCHED-WEB-21-001-GRAPH-APIS.md`; integration tests cover submission/status; unauthorized requests blocked; scope checks now reference `StellaOpsScopes`. | -| SCHED-WEB-21-002 | DONE (2025-10-26) | Scheduler WebService Guild | SCHED-WEB-21-001, CARTO-GRAPH-21-007 | Provide overlay lag metrics endpoint and webhook to notify Cartographer of job completions; include correlation IDs. | `POST /graphs/hooks/completed` + `GET /graphs/overlays/lag` documented in `docs/SCHED-WEB-21-001-GRAPH-APIS.md`; integration tests cover completion + metrics. | -| SCHED-WEB-21-003 | DONE (2025-10-26) | Scheduler WebService Guild, Authority Core Guild | AUTH-GRAPH-21-001 | Replace temporary `X-Scopes`/`X-Tenant-Id` headers with Authority-issued OpTok verification and scope enforcement for graph endpoints. | Authentication configured via `AddStellaOpsResourceServerAuthentication`; authority scopes enforced end-to-end with `StellaOpsScopes`; header fallback limited to dev mode; tests updated. | | SCHED-WEB-21-004 | DOING (2025-10-26) | Scheduler WebService Guild, Scheduler Storage Guild | SCHED-WEB-21-001, SCHED-STORAGE-16-201 | Persist graph job lifecycle to Mongo storage and publish `scheduler.graph.job.completed@1` events + outbound webhook to Cartographer. | Storage repositories updated; events emitted; webhook payload documented; integration tests cover storage + event flow. **Note:** Events currently log JSON envelopes while the shared platform bus is provisioned. Cartographer webhook now posts JSON payloads when configured; replace inline logging with bus publisher once the shared event transport is online. | +> 2025-10-30: Implemented Redis-backed publisher (`Scheduler:Events:GraphJobs`) emitting `scheduler.graph.job.completed@1` to configured stream with optional logging fallback; docs/configs to be validated with DevOps before closing. ## StellaOps Console (Sprint 23) | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | diff --git a/src/Scheduler/__Libraries/StellaOps.Scheduler.ImpactIndex/TASKS.completed.md b/src/Scheduler/__Libraries/StellaOps.Scheduler.ImpactIndex/TASKS.completed.md new file mode 100644 index 00000000..2f2e064c --- /dev/null +++ b/src/Scheduler/__Libraries/StellaOps.Scheduler.ImpactIndex/TASKS.completed.md @@ -0,0 +1,7 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| SCHED-IMPACT-16-300 | DONE (2025-10-20) | Scheduler ImpactIndex Guild | SAMPLES-10-001 | **STUB** ingest/query using fixtures to unblock Scheduler planning (remove by SP16 end). | Stub merges fixture BOM-Index, query API returns deterministic results, removal note tracked. | +| SCHED-IMPACT-16-301 | DONE (2025-10-26) | Scheduler ImpactIndex Guild | SCANNER-EMIT-10-605 | Implement ingestion of per-image BOM-Index sidecars into roaring bitmap store (contains/usedBy). | Ingestion tests process sample SBOM index; bitmaps persisted; deterministic IDs assigned. | +| SCHED-IMPACT-16-302 | DONE (2025-10-26) | Scheduler ImpactIndex Guild | SCHED-IMPACT-16-301 | Provide query APIs (ResolveByPurls, ResolveByVulns, ResolveAll, selectors) with tenant/namespace filters. | Query functions tested; performance benchmarks documented; selectors enforce filters. | diff --git a/src/Scheduler/__Libraries/StellaOps.Scheduler.ImpactIndex/TASKS.md b/src/Scheduler/__Libraries/StellaOps.Scheduler.ImpactIndex/TASKS.md index c2ca1c09..49521c5e 100644 --- a/src/Scheduler/__Libraries/StellaOps.Scheduler.ImpactIndex/TASKS.md +++ b/src/Scheduler/__Libraries/StellaOps.Scheduler.ImpactIndex/TASKS.md @@ -2,9 +2,6 @@ | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| SCHED-IMPACT-16-300 | DONE (2025-10-20) | Scheduler ImpactIndex Guild | SAMPLES-10-001 | **STUB** ingest/query using fixtures to unblock Scheduler planning (remove by SP16 end). | Stub merges fixture BOM-Index, query API returns deterministic results, removal note tracked. | -| SCHED-IMPACT-16-301 | DONE (2025-10-26) | Scheduler ImpactIndex Guild | SCANNER-EMIT-10-605 | Implement ingestion of per-image BOM-Index sidecars into roaring bitmap store (contains/usedBy). | Ingestion tests process sample SBOM index; bitmaps persisted; deterministic IDs assigned. | -| SCHED-IMPACT-16-302 | DONE (2025-10-26) | Scheduler ImpactIndex Guild | SCHED-IMPACT-16-301 | Provide query APIs (ResolveByPurls, ResolveByVulns, ResolveAll, selectors) with tenant/namespace filters. | Query functions tested; performance benchmarks documented; selectors enforce filters. | | SCHED-IMPACT-16-303 | TODO | Scheduler ImpactIndex Guild | SCHED-IMPACT-16-301 | Snapshot/compaction + invalidation for removed images; persistence to RocksDB/Redis per architecture. | Snapshot routine implemented; invalidation tests pass; docs describe recovery. | > Removal tracking note: see `src/Scheduler/__Libraries/StellaOps.Scheduler.ImpactIndex/REMOVAL_NOTE.md` for follow-up actions once the roaring bitmap implementation lands. diff --git a/src/Scheduler/__Libraries/StellaOps.Scheduler.Models/TASKS.completed.md b/src/Scheduler/__Libraries/StellaOps.Scheduler.Models/TASKS.completed.md new file mode 100644 index 00000000..81052bc5 --- /dev/null +++ b/src/Scheduler/__Libraries/StellaOps.Scheduler.Models/TASKS.completed.md @@ -0,0 +1,11 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| SCHED-MODELS-16-101 | DONE (2025-10-19) | Scheduler Models Guild | — | Define DTOs (Schedule, Run, ImpactSet, Selector, DeltaSummary, AuditRecord) with validation + canonical JSON. | DTOs merged with tests; documentation snippet added; serialization deterministic. | +| SCHED-MODELS-16-102 | DONE (2025-10-19) | Scheduler Models Guild | SCHED-MODELS-16-101 | Publish schema docs & sample payloads for UI/Notify integration. | Samples committed; docs referenced; contract tests pass. | +| SCHED-MODELS-16-103 | DONE (2025-10-20) | Scheduler Models Guild | SCHED-MODELS-16-101 | Versioning/migration helpers (schedule evolution, run state transitions). | Migration helpers implemented; tests cover upgrade/downgrade; guidelines documented. | +| SCHED-MODELS-20-001 | DONE (2025-10-26) | Scheduler Models Guild, Policy Guild | POLICY-ENGINE-20-000 | Define DTOs/schemas for policy runs, diffs, and explain traces (`PolicyRunRequest`, `PolicyRunStatus`, `PolicyDiffSummary`). | DTOs serialize deterministically; schema samples committed; validation helpers added. | +| SCHED-MODELS-20-002 | DONE (2025-10-29) | Scheduler Models Guild | SCHED-MODELS-20-001 | Extend scheduler schema docs to include policy run lifecycle, environment metadata, and diff payloads. | Docs updated with compliance checklist; samples validated against JSON schema; consumers notified. | +| SCHED-MODELS-21-001 | DONE (2025-10-26) | Scheduler Models Guild, Cartographer Guild | CARTO-GRAPH-21-007 | Define job DTOs for graph builds/overlay refresh (`GraphBuildJob`, `GraphOverlayJob`) with deterministic serialization and status enums. | DTOs serialized deterministically; schema snippets documented in `src/Scheduler/__Libraries/StellaOps.Scheduler.Models/docs/SCHED-MODELS-21-001-GRAPH-JOBS.md`; tests cover transitions. | +| SCHED-MODELS-21-002 | DONE (2025-10-26) | Scheduler Models Guild | SCHED-MODELS-21-001 | Publish schema docs/sample payloads for graph jobs and overlay events for downstream workers/UI. | Docs updated with compliance checklist; samples validated; notifications sent to guilds. | diff --git a/src/Scheduler/__Libraries/StellaOps.Scheduler.Models/TASKS.md b/src/Scheduler/__Libraries/StellaOps.Scheduler.Models/TASKS.md index 391be273..3bef531b 100644 --- a/src/Scheduler/__Libraries/StellaOps.Scheduler.Models/TASKS.md +++ b/src/Scheduler/__Libraries/StellaOps.Scheduler.Models/TASKS.md @@ -2,21 +2,14 @@ | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| SCHED-MODELS-16-101 | DONE (2025-10-19) | Scheduler Models Guild | — | Define DTOs (Schedule, Run, ImpactSet, Selector, DeltaSummary, AuditRecord) with validation + canonical JSON. | DTOs merged with tests; documentation snippet added; serialization deterministic. | -| SCHED-MODELS-16-102 | DONE (2025-10-19) | Scheduler Models Guild | SCHED-MODELS-16-101 | Publish schema docs & sample payloads for UI/Notify integration. | Samples committed; docs referenced; contract tests pass. | -| SCHED-MODELS-16-103 | DONE (2025-10-20) | Scheduler Models Guild | SCHED-MODELS-16-101 | Versioning/migration helpers (schedule evolution, run state transitions). | Migration helpers implemented; tests cover upgrade/downgrade; guidelines documented. | ## Policy Engine v2 (Sprint 20) | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| SCHED-MODELS-20-001 | DONE (2025-10-26) | Scheduler Models Guild, Policy Guild | POLICY-ENGINE-20-000 | Define DTOs/schemas for policy runs, diffs, and explain traces (`PolicyRunRequest`, `PolicyRunStatus`, `PolicyDiffSummary`). | DTOs serialize deterministically; schema samples committed; validation helpers added. | -| SCHED-MODELS-20-002 | DONE (2025-10-29) | Scheduler Models Guild | SCHED-MODELS-20-001 | Extend scheduler schema docs to include policy run lifecycle, environment metadata, and diff payloads. | Docs updated with compliance checklist; samples validated against JSON schema; consumers notified. | > 2025-10-29: Added lifecycle table, environment metadata section, and diff payload breakdown to `SCHED-MODELS-20-001-POLICY-RUNS.md`; compliance checklist extended to cover new documentation. ## Graph Explorer v1 (Sprint 21) | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| SCHED-MODELS-21-001 | DONE (2025-10-26) | Scheduler Models Guild, Cartographer Guild | CARTO-GRAPH-21-007 | Define job DTOs for graph builds/overlay refresh (`GraphBuildJob`, `GraphOverlayJob`) with deterministic serialization and status enums. | DTOs serialized deterministically; schema snippets documented in `src/Scheduler/__Libraries/StellaOps.Scheduler.Models/docs/SCHED-MODELS-21-001-GRAPH-JOBS.md`; tests cover transitions. | -| SCHED-MODELS-21-002 | DONE (2025-10-26) | Scheduler Models Guild | SCHED-MODELS-21-001 | Publish schema docs/sample payloads for graph jobs and overlay events for downstream workers/UI. | Docs updated with compliance checklist; samples validated; notifications sent to guilds. | diff --git a/src/Scheduler/__Libraries/StellaOps.Scheduler.Queue/TASKS.completed.md b/src/Scheduler/__Libraries/StellaOps.Scheduler.Queue/TASKS.completed.md new file mode 100644 index 00000000..4f5ff9e3 --- /dev/null +++ b/src/Scheduler/__Libraries/StellaOps.Scheduler.Queue/TASKS.completed.md @@ -0,0 +1,7 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| SCHED-QUEUE-16-401 | DONE (2025-10-20) | Scheduler Queue Guild | SCHED-MODELS-16-101 | Implement queue abstraction + Redis Streams adapter (planner inputs, runner segments) with ack/lease semantics. | Integration tests cover enqueue/dequeue/ack; lease renewal implemented; ordering preserved. | +| SCHED-QUEUE-16-402 | DONE (2025-10-20) | Scheduler Queue Guild | SCHED-QUEUE-16-401 | Add NATS JetStream adapter with configuration binding, health probes, failover. | Health endpoints verified; failover documented; adapter tested. | +| SCHED-QUEUE-16-403 | DONE (2025-10-20) | Scheduler Queue Guild | SCHED-QUEUE-16-401 | Dead-letter handling + metrics (queue depth, retry counts), configuration toggles. | Dead-letter policy tested; metrics exported; docs updated. | diff --git a/src/Scheduler/__Libraries/StellaOps.Scheduler.Queue/TASKS.md b/src/Scheduler/__Libraries/StellaOps.Scheduler.Queue/TASKS.md index 2ca7feda..37698bf1 100644 --- a/src/Scheduler/__Libraries/StellaOps.Scheduler.Queue/TASKS.md +++ b/src/Scheduler/__Libraries/StellaOps.Scheduler.Queue/TASKS.md @@ -1,9 +1,6 @@ -# Scheduler Queue Task Board (Sprint 16) - -> **Status note (2025-10-19):** Scheduler DTOs and sample payloads are now available (SCHED-MODELS-16-102). Queue tasks remain pending on this board. - -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| SCHED-QUEUE-16-401 | DONE (2025-10-20) | Scheduler Queue Guild | SCHED-MODELS-16-101 | Implement queue abstraction + Redis Streams adapter (planner inputs, runner segments) with ack/lease semantics. | Integration tests cover enqueue/dequeue/ack; lease renewal implemented; ordering preserved. | -| SCHED-QUEUE-16-402 | DONE (2025-10-20) | Scheduler Queue Guild | SCHED-QUEUE-16-401 | Add NATS JetStream adapter with configuration binding, health probes, failover. | Health endpoints verified; failover documented; adapter tested. | -| SCHED-QUEUE-16-403 | DONE (2025-10-20) | Scheduler Queue Guild | SCHED-QUEUE-16-401 | Dead-letter handling + metrics (queue depth, retry counts), configuration toggles. | Dead-letter policy tested; metrics exported; docs updated. | +# Scheduler Queue Task Board (Sprint 16) + +> **Status note (2025-10-19):** Scheduler DTOs and sample payloads are now available (SCHED-MODELS-16-102). Queue tasks remain pending on this board. + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| diff --git a/src/Scheduler/__Libraries/StellaOps.Scheduler.Storage.Mongo/TASKS.completed.md b/src/Scheduler/__Libraries/StellaOps.Scheduler.Storage.Mongo/TASKS.completed.md new file mode 100644 index 00000000..7248d0a3 --- /dev/null +++ b/src/Scheduler/__Libraries/StellaOps.Scheduler.Storage.Mongo/TASKS.completed.md @@ -0,0 +1,7 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| SCHED-STORAGE-16-201 | DONE (2025-10-19) | Scheduler Storage Guild | SCHED-MODELS-16-101 | Create Mongo collections (schedules, runs, impact_cursors, locks, audit) with indexes/migrations per architecture. | Migration scripts and indexes implemented; integration tests cover CRUD paths. | +| SCHED-STORAGE-16-202 | DONE (2025-10-26) | Scheduler Storage Guild | SCHED-STORAGE-16-201 | Implement repositories/services with tenant scoping, soft delete, TTL for completed runs, and causal consistency options. | Unit tests pass; TTL/soft delete validated; documentation updated. | +| SCHED-STORAGE-16-203 | DONE (2025-10-26) | Scheduler Storage Guild | SCHED-STORAGE-16-201 | Audit/logging pipeline + run stats materialized views for UI. | Audit entries persisted; stats queries efficient; docs capture usage. | diff --git a/src/Scheduler/__Libraries/StellaOps.Scheduler.Storage.Mongo/TASKS.md b/src/Scheduler/__Libraries/StellaOps.Scheduler.Storage.Mongo/TASKS.md index 1a879b46..84462c8e 100644 --- a/src/Scheduler/__Libraries/StellaOps.Scheduler.Storage.Mongo/TASKS.md +++ b/src/Scheduler/__Libraries/StellaOps.Scheduler.Storage.Mongo/TASKS.md @@ -1,9 +1,6 @@ -# Scheduler Storage Task Board (Sprint 16) - -> **Status note (2025-10-19):** Scheduler models/samples delivered in SCHED-MODELS-16-102. Tasks below remain pending for the Storage guild. - -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| SCHED-STORAGE-16-201 | DONE (2025-10-19) | Scheduler Storage Guild | SCHED-MODELS-16-101 | Create Mongo collections (schedules, runs, impact_cursors, locks, audit) with indexes/migrations per architecture. | Migration scripts and indexes implemented; integration tests cover CRUD paths. | -| SCHED-STORAGE-16-202 | DONE (2025-10-26) | Scheduler Storage Guild | SCHED-STORAGE-16-201 | Implement repositories/services with tenant scoping, soft delete, TTL for completed runs, and causal consistency options. | Unit tests pass; TTL/soft delete validated; documentation updated. | -| SCHED-STORAGE-16-203 | DONE (2025-10-26) | Scheduler Storage Guild | SCHED-STORAGE-16-201 | Audit/logging pipeline + run stats materialized views for UI. | Audit entries persisted; stats queries efficient; docs capture usage. | +# Scheduler Storage Task Board (Sprint 16) + +> **Status note (2025-10-19):** Scheduler models/samples delivered in SCHED-MODELS-16-102. Tasks below remain pending for the Storage guild. + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| diff --git a/src/Scheduler/__Libraries/StellaOps.Scheduler.Worker/TASKS.completed.md b/src/Scheduler/__Libraries/StellaOps.Scheduler.Worker/TASKS.completed.md new file mode 100644 index 00000000..8a8a4dc2 --- /dev/null +++ b/src/Scheduler/__Libraries/StellaOps.Scheduler.Worker/TASKS.completed.md @@ -0,0 +1,14 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| SCHED-WORKER-16-201 | DONE (2025-10-28) | Scheduler Worker Guild | SCHED-QUEUE-16-401 | Planner loop (cron + event triggers) with lease management, fairness, and rate limiting (§6). | Planner integration tests cover cron/event triggers; rate limits enforced; logs include run IDs. | +| SCHED-WORKER-16-202 | DONE (2025-10-27) | Scheduler Worker Guild | SCHED-IMPACT-16-301 | Wire ImpactIndex targeting (ResolveByPurls/vulns), dedupe, shard planning. | Targeting tests confirm correct image selection; dedupe documented; shards evenly distributed. | +| SCHED-WORKER-16-203 | DONE (2025-10-27) | Scheduler Worker Guild | SCHED-WORKER-16-202 | Runner execution: call Scanner `/reports` (analysis-only) or `/scans` when configured; collect deltas; handle retries. | Runner tests stub Scanner; retries/backoff validated; deltas aggregated deterministically. | +| SCHED-WORKER-16-204 | DONE (2025-10-27) | Scheduler Worker Guild | SCHED-WORKER-16-203 | Emit events (`scheduler.rescan.delta`, `scanner.report.ready`) for Notify/UI with summaries. | Events published to queue; payload schema documented; integration tests verify consumption. | +| SCHED-WORKER-16-205 | DONE (2025-10-27) | Scheduler Worker Guild | SCHED-WORKER-16-201 | Metrics/telemetry: run stats, queue depth, planner latency, delta counts. | Metrics exported per spec; dashboards updated; alerts configured. | +| SCHED-WORKER-20-301 | DONE (2025-10-28) | Scheduler Worker Guild, Policy Guild | SCHED-WORKER-16-201, POLICY-ENGINE-20-000 | Extend scheduler worker to trigger policy runs (full/incremental/simulate) via Policy Engine API, with idempotent job tracking and tenant scoping. | Worker schedules policy jobs deterministically; job records persisted; integration tests cover modes + cancellation. | +| SCHED-WORKER-21-201 | DONE (2025-10-29) | Scheduler Worker Guild, Cartographer Guild | SCHED-MODELS-21-001 | Implement graph build worker that dequeues SBOM graph jobs, invokes Cartographer build APIs, and records status with retries/backoff. | Worker processes fixtures; retries handled; logs include `graph_id`; integration tests pass. | +| SCHED-WORKER-20-302 | DONE (2025-10-29) | Scheduler Worker Guild | SCHED-WORKER-20-301, POLICY-ENGINE-20-006 | Implement policy delta targeting to re-evaluate only affected SBOM sets based on change streams and policy metadata. | Targeting reduces workload per design; tests simulate advisory/vex updates; metrics show delta counts. | +| SCHED-WORKER-20-303 | DONE (2025-10-29) | Scheduler Worker Guild, Observability Guild | SCHED-WORKER-20-301 | Expose metrics (`policy_runs_scheduled`, `policy_runs_failed`, planner latency) and structured logs with policy/run identifiers. | Metrics registered; dashboards updated; logs validated in integration tests. | +| SCHED-WORKER-21-202 | DONE (2025-10-29) | Scheduler Worker Guild | SCHED-WORKER-21-201, CARTO-GRAPH-21-007 | Overlay refresh worker subscribing to policy/SBOM change events, batching affected graph overlays, and enforcing <2 min SLA. | Overlay jobs scheduled deterministically; lag metrics < 2 min in tests; alerts configured. | diff --git a/src/Scheduler/__Libraries/StellaOps.Scheduler.Worker/TASKS.md b/src/Scheduler/__Libraries/StellaOps.Scheduler.Worker/TASKS.md index b68b086b..85b10265 100644 --- a/src/Scheduler/__Libraries/StellaOps.Scheduler.Worker/TASKS.md +++ b/src/Scheduler/__Libraries/StellaOps.Scheduler.Worker/TASKS.md @@ -2,11 +2,6 @@ | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| SCHED-WORKER-16-201 | DONE (2025-10-28) | Scheduler Worker Guild | SCHED-QUEUE-16-401 | Planner loop (cron + event triggers) with lease management, fairness, and rate limiting (§6). | Planner integration tests cover cron/event triggers; rate limits enforced; logs include run IDs. | -| SCHED-WORKER-16-202 | DONE (2025-10-27) | Scheduler Worker Guild | SCHED-IMPACT-16-301 | Wire ImpactIndex targeting (ResolveByPurls/vulns), dedupe, shard planning. | Targeting tests confirm correct image selection; dedupe documented; shards evenly distributed. | -| SCHED-WORKER-16-203 | DONE (2025-10-27) | Scheduler Worker Guild | SCHED-WORKER-16-202 | Runner execution: call Scanner `/reports` (analysis-only) or `/scans` when configured; collect deltas; handle retries. | Runner tests stub Scanner; retries/backoff validated; deltas aggregated deterministically. | -| SCHED-WORKER-16-204 | DONE (2025-10-27) | Scheduler Worker Guild | SCHED-WORKER-16-203 | Emit events (`scheduler.rescan.delta`, `scanner.report.ready`) for Notify/UI with summaries. | Events published to queue; payload schema documented; integration tests verify consumption. | -| SCHED-WORKER-16-205 | DONE (2025-10-27) | Scheduler Worker Guild | SCHED-WORKER-16-201 | Metrics/telemetry: run stats, queue depth, planner latency, delta counts. | Metrics exported per spec; dashboards updated; alerts configured. | > 2025-10-27: Impact targeting sanitizes selector-constrained results, dedupes digests, and documents shard planning in `docs/SCHED-WORKER-16-202-IMPACT-TARGETING.md`. @@ -17,20 +12,15 @@ | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| SCHED-WORKER-20-301 | DONE (2025-10-28) | Scheduler Worker Guild, Policy Guild | SCHED-WORKER-16-201, POLICY-ENGINE-20-000 | Extend scheduler worker to trigger policy runs (full/incremental/simulate) via Policy Engine API, with idempotent job tracking and tenant scoping. | Worker schedules policy jobs deterministically; job records persisted; integration tests cover modes + cancellation. | > 2025-10-26: Align worker serializers with `PolicyRunRequest/Status/DiffSummary` contracts from `src/Scheduler/__Libraries/StellaOps.Scheduler.Models`. Reference fixtures in `samples/api/scheduler/` for expected payload ordering. -| SCHED-WORKER-20-302 | DONE (2025-10-29) | Scheduler Worker Guild | SCHED-WORKER-20-301, POLICY-ENGINE-20-006 | Implement policy delta targeting to re-evaluate only affected SBOM sets based on change streams and policy metadata. | Targeting reduces workload per design; tests simulate advisory/vex updates; metrics show delta counts. | > 2025-10-29: `PolicyRunTargetingService` translates change-stream metadata into SBOM sets, marks no-work jobs completed, and surfaces targeting options (`Policy.Targeting`). See `docs/SCHED-WORKER-20-302-POLICY-DELTA-TARGETING.md` for supported metadata keys and behaviour. -| SCHED-WORKER-20-303 | DONE (2025-10-29) | Scheduler Worker Guild, Observability Guild | SCHED-WORKER-20-301 | Expose metrics (`policy_runs_scheduled`, `policy_runs_failed`, planner latency) and structured logs with policy/run identifiers. | Metrics registered; dashboards updated; logs validated in integration tests. | > 2025-10-29: Added `scheduler_policy_run_events_total` + latency histogram, instrumented policy dispatch success/retry/failure/cancel paths, and upgraded structured logs with tenant/policy/run identifiers. Docs updated in `docs/SCHED-WORKER-20-301-POLICY-RUNS.md` Observability section. ## Graph Explorer v1 (Sprint 21) | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| SCHED-WORKER-21-201 | DONE (2025-10-29) | Scheduler Worker Guild, Cartographer Guild | SCHED-MODELS-21-001 | Implement graph build worker that dequeues SBOM graph jobs, invokes Cartographer build APIs, and records status with retries/backoff. | Worker processes fixtures; retries handled; logs include `graph_id`; integration tests pass. | > 2025-10-29: Graph build worker background service + execution pipeline added. Cartographer + Scheduler API options documented in `docs/SCHED-WORKER-21-201-GRAPH-BUILD.md`; unit tests cover success/retry/failure paths. -| SCHED-WORKER-21-202 | DONE (2025-10-29) | Scheduler Worker Guild | SCHED-WORKER-21-201, CARTO-GRAPH-21-007 | Overlay refresh worker subscribing to policy/SBOM change events, batching affected graph overlays, and enforcing <2 min SLA. | Overlay jobs scheduled deterministically; lag metrics < 2 min in tests; alerts configured. | > 2025-10-29: Overlay worker now polls pending jobs, posts to Cartographer overlay endpoint, and reports completion via Scheduler webhook. Config + behaviour documented in `docs/SCHED-WORKER-21-202-GRAPH-OVERLAY.md`. | SCHED-WORKER-21-203 | TODO | Scheduler Worker Guild, Observability Guild | SCHED-WORKER-21-201 | Export metrics (`graph_build_seconds`, `graph_jobs_inflight`, `overlay_lag_seconds`) and structured logs with tenant/graph identifiers. | Metrics/traces exposed; dashboards updated; integration tests verify metrics emission. | diff --git a/src/Scheduler/__Tests/StellaOps.Scheduler.WebService.Tests/GraphJobEventPublisherTests.cs b/src/Scheduler/__Tests/StellaOps.Scheduler.WebService.Tests/GraphJobEventPublisherTests.cs index a42751d1..21e7c425 100644 --- a/src/Scheduler/__Tests/StellaOps.Scheduler.WebService.Tests/GraphJobEventPublisherTests.cs +++ b/src/Scheduler/__Tests/StellaOps.Scheduler.WebService.Tests/GraphJobEventPublisherTests.cs @@ -1,25 +1,30 @@ -using Microsoft.Extensions.Logging; -using Microsoft.Extensions.Options; -using StellaOps.Auth.Abstractions; -using StellaOps.Scheduler.Models; -using StellaOps.Scheduler.WebService.GraphJobs; -using StellaOps.Scheduler.WebService.GraphJobs.Events; -using StellaOps.Scheduler.WebService.Options; +using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Options; +using StellaOps.Auth.Abstractions; +using StellaOps.Scheduler.Models; +using StellaOps.Scheduler.WebService.GraphJobs; +using StellaOps.Scheduler.WebService.GraphJobs.Events; +using StellaOps.Scheduler.WebService.Options; +using StackExchange.Redis; namespace StellaOps.Scheduler.WebService.Tests; public sealed class GraphJobEventPublisherTests { [Fact] - public async Task PublishAsync_WritesEventJson_WhenEnabled() - { - var options = Microsoft.Extensions.Options.Options.Create(new SchedulerEventsOptions - { - GraphJobs = { Enabled = true } - }); - var loggerProvider = new ListLoggerProvider(); - using var loggerFactory = LoggerFactory.Create(builder => builder.AddProvider(loggerProvider)); - var publisher = new GraphJobEventPublisher(new OptionsMonitorStub(options), loggerFactory.CreateLogger()); + public async Task PublishAsync_LogsEvent_WhenDriverUnsupported() + { + var options = Microsoft.Extensions.Options.Options.Create(new SchedulerEventsOptions + { + GraphJobs = + { + Enabled = true, + Driver = "memory" + } + }); + var loggerProvider = new ListLoggerProvider(); + using var loggerFactory = LoggerFactory.Create(builder => builder.AddProvider(loggerProvider)); + var publisher = new GraphJobEventPublisher(new OptionsMonitorStub(options), new ThrowingRedisConnectionFactory(), loggerFactory.CreateLogger()); var buildJob = new GraphBuildJob( id: "gbj_test", @@ -58,12 +63,12 @@ public sealed class GraphJobEventPublisherTests } [Fact] - public async Task PublishAsync_Suppressed_WhenDisabled() - { - var options = Microsoft.Extensions.Options.Options.Create(new SchedulerEventsOptions()); - var loggerProvider = new ListLoggerProvider(); - using var loggerFactory = LoggerFactory.Create(builder => builder.AddProvider(loggerProvider)); - var publisher = new GraphJobEventPublisher(new OptionsMonitorStub(options), loggerFactory.CreateLogger()); + public async Task PublishAsync_Suppressed_WhenDisabled() + { + var options = Microsoft.Extensions.Options.Options.Create(new SchedulerEventsOptions()); + var loggerProvider = new ListLoggerProvider(); + using var loggerFactory = LoggerFactory.Create(builder => builder.AddProvider(loggerProvider)); + var publisher = new GraphJobEventPublisher(new OptionsMonitorStub(options), new ThrowingRedisConnectionFactory(), loggerFactory.CreateLogger()); var overlayJob = new GraphOverlayJob( id: "goj_test", @@ -95,12 +100,12 @@ public sealed class GraphJobEventPublisherTests await publisher.PublishAsync(notification, CancellationToken.None); - Assert.DoesNotContain(loggerProvider.Messages, message => message.Contains(GraphJobEventKinds.GraphJobCompleted, StringComparison.Ordinal)); + Assert.DoesNotContain(loggerProvider.Messages, message => message.Contains(GraphJobEventKinds.GraphJobCompleted, StringComparison.Ordinal)); } -private sealed class OptionsMonitorStub : IOptionsMonitor where T : class - { - private readonly IOptions _options; + private sealed class OptionsMonitorStub : IOptionsMonitor where T : class + { + private readonly IOptions _options; public OptionsMonitorStub(IOptions options) { @@ -112,7 +117,13 @@ private sealed class OptionsMonitorStub : IOptionsMonitor where T : class public T Get(string? name) => _options.Value; public IDisposable? OnChange(Action listener) => null; - } + } + + private sealed class ThrowingRedisConnectionFactory : IRedisConnectionFactory + { + public Task ConnectAsync(ConfigurationOptions options, CancellationToken cancellationToken) + => throw new InvalidOperationException("Redis connection should not be established in this test."); + } private sealed class ListLoggerProvider : ILoggerProvider { diff --git a/src/Signals/StellaOps.Signals/TASKS.completed.md b/src/Signals/StellaOps.Signals/TASKS.completed.md new file mode 100644 index 00000000..363351a0 --- /dev/null +++ b/src/Signals/StellaOps.Signals/TASKS.completed.md @@ -0,0 +1,6 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| SIGNALS-24-001 | DONE (2025-10-29) | Signals Guild, Architecture Guild | SBOM-GRAPH-24-002 | Implement Signals API skeleton (ASP.NET Minimal API) with auth middleware, health checks, and configuration binding. | Service boots with configuration validation, `/healthz`/`/readyz` return 200, RBAC enforced in integration tests. | +| SIGNALS-24-002 | DONE (2025-10-29) | Signals Guild, Language Specialists | SIGNALS-24-001 | Build callgraph ingestion pipeline (Java/Node/Python/Go parsers) normalizing into `callgraphs` collection and storing artifact metadata in object storage. | Parsers accept sample artifacts; data persisted with schema validation; unit tests cover malformed inputs. | diff --git a/src/Signals/StellaOps.Signals/TASKS.md b/src/Signals/StellaOps.Signals/TASKS.md index ee25e32b..2031c3e6 100644 --- a/src/Signals/StellaOps.Signals/TASKS.md +++ b/src/Signals/StellaOps.Signals/TASKS.md @@ -1,13 +1,11 @@ -# Signals Service Task Board — Reachability v1 -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| SIGNALS-24-001 | DONE (2025-10-29) | Signals Guild, Architecture Guild | SBOM-GRAPH-24-002 | Implement Signals API skeleton (ASP.NET Minimal API) with auth middleware, health checks, and configuration binding. | Service boots with configuration validation, `/healthz`/`/readyz` return 200, RBAC enforced in integration tests. | -> 2025-10-29: Skeleton live with scope policies, stub endpoints, integration tests. Sample config added under `etc/signals.yaml.sample`. -| SIGNALS-24-002 | DONE (2025-10-29) | Signals Guild, Language Specialists | SIGNALS-24-001 | Build callgraph ingestion pipeline (Java/Node/Python/Go parsers) normalizing into `callgraphs` collection and storing artifact metadata in object storage. | Parsers accept sample artifacts; data persisted with schema validation; unit tests cover malformed inputs. | -> 2025-10-29: JSON parsers for java/nodejs/python/go implemented; artifacts stored on filesystem with SHA-256, callgraphs upserted into Mongo with unique index; integration tests cover success + malformed requests. -| SIGNALS-24-003 | BLOCKED (2025-10-27) | Signals Guild, Runtime Guild | SIGNALS-24-001 | Implement runtime facts ingestion endpoint and normalizer (process, sockets, container metadata) populating `context_facts` with AOC provenance. | Endpoint ingests fixture batches; duplicates deduped; schema enforced; tests cover privacy filters. | -> 2025-10-27: Depends on SIGNALS-24-001 for base API host + authentication plumbing. -| SIGNALS-24-004 | BLOCKED (2025-10-27) | Signals Guild, Data Science | SIGNALS-24-002, SIGNALS-24-003 | Deliver reachability scoring engine producing states/scores and writing to `reachability_facts`; expose configuration for weights. | Scoring engine deterministic; tests cover state transitions; metrics emitted. | -> 2025-10-27: Upstream ingestion pipelines (SIGNALS-24-002/003) blocked; scoring engine cannot proceed. -| SIGNALS-24-005 | BLOCKED (2025-10-27) | Signals Guild, Platform Events Guild | SIGNALS-24-004 | Implement Redis caches (`reachability_cache:*`), invalidation on new facts, and publish `signals.fact.updated` events. | Cache hit rate tracked; invalidations working; events delivered with idempotent ids; integration tests pass. | -> 2025-10-27: Awaiting scoring engine and ingestion layers before wiring cache/events. +# Signals Service Task Board — Reachability v1 +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +> 2025-10-29: Skeleton live with scope policies, stub endpoints, integration tests. Sample config added under `etc/signals.yaml.sample`. +> 2025-10-29: JSON parsers for java/nodejs/python/go implemented; artifacts stored on filesystem with SHA-256, callgraphs upserted into Mongo with unique index; integration tests cover success + malformed requests. +| SIGNALS-24-003 | BLOCKED (2025-10-27) | Signals Guild, Runtime Guild | SIGNALS-24-001 | Implement runtime facts ingestion endpoint and normalizer (process, sockets, container metadata) populating `context_facts` with AOC provenance. | Endpoint ingests fixture batches; duplicates deduped; schema enforced; tests cover privacy filters. | +> 2025-10-27: Depends on SIGNALS-24-001 for base API host + authentication plumbing. +| SIGNALS-24-004 | BLOCKED (2025-10-27) | Signals Guild, Data Science | SIGNALS-24-002, SIGNALS-24-003 | Deliver reachability scoring engine producing states/scores and writing to `reachability_facts`; expose configuration for weights. | Scoring engine deterministic; tests cover state transitions; metrics emitted. | +> 2025-10-27: Upstream ingestion pipelines (SIGNALS-24-002/003) blocked; scoring engine cannot proceed. +| SIGNALS-24-005 | BLOCKED (2025-10-27) | Signals Guild, Platform Events Guild | SIGNALS-24-004 | Implement Redis caches (`reachability_cache:*`), invalidation on new facts, and publish `signals.fact.updated` events. | Cache hit rate tracked; invalidations working; events delivered with idempotent ids; integration tests pass. | +> 2025-10-27: Awaiting scoring engine and ingestion layers before wiring cache/events. diff --git a/src/Signer/StellaOps.Signer/TASKS.completed.md b/src/Signer/StellaOps.Signer/TASKS.completed.md new file mode 100644 index 00000000..d0595a10 --- /dev/null +++ b/src/Signer/StellaOps.Signer/TASKS.completed.md @@ -0,0 +1,7 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| SIGNER-API-11-101 | DONE (2025-10-21) | Signer Guild | — | `/sign/dsse` pipeline with Authority auth, PoE introspection, release verification, DSSE signing. | ✅ `POST /api/v1/signer/sign/dsse` enforces OpTok audience/scope, DPoP/mTLS binding, PoE introspection, and rejects untrusted scanner digests.
✅ Signing pipeline supports keyless (Fulcio) plus optional KMS modes, returning DSSE bundles + cert metadata; deterministic audits persisted.
✅ Regression coverage in `SignerEndpointsTests` (`dotnet test src/Signer/StellaOps.Signer/StellaOps.Signer.Tests/StellaOps.Signer.Tests.csproj`). | +| SIGNER-REF-11-102 | DONE (2025-10-21) | Signer Guild | — | `/verify/referrers` endpoint with OCI lookup, caching, and policy enforcement. | ✅ `GET /api/v1/signer/verify/referrers` validates trusted scanner digests via release verifier and surfaces signer metadata; JSON responses served deterministically.
✅ Integration tests cover trusted/untrusted digests and validation failures (`SignerEndpointsTests`). | +| SIGNER-QUOTA-11-103 | DONE (2025-10-21) | Signer Guild | — | Enforce plan quotas, concurrency/QPS limits, artifact size caps with metrics/audit logs. | ✅ Quota middleware derives plan limits from PoE claims, applies per-tenant concurrency/QPS/size caps, and surfaces remaining capacity in responses.
✅ Unit coverage exercises throttled/artifact-too-large paths via in-memory quota service. | diff --git a/src/Signer/StellaOps.Signer/TASKS.md b/src/Signer/StellaOps.Signer/TASKS.md index 2b3e03de..65a4ff68 100644 --- a/src/Signer/StellaOps.Signer/TASKS.md +++ b/src/Signer/StellaOps.Signer/TASKS.md @@ -2,9 +2,6 @@ | ID | Status | Owner(s) | Depends on | Description | Exit Criteria | |----|--------|----------|------------|-------------|---------------| -| SIGNER-API-11-101 | DONE (2025-10-21) | Signer Guild | — | `/sign/dsse` pipeline with Authority auth, PoE introspection, release verification, DSSE signing. | ✅ `POST /api/v1/signer/sign/dsse` enforces OpTok audience/scope, DPoP/mTLS binding, PoE introspection, and rejects untrusted scanner digests.
✅ Signing pipeline supports keyless (Fulcio) plus optional KMS modes, returning DSSE bundles + cert metadata; deterministic audits persisted.
✅ Regression coverage in `SignerEndpointsTests` (`dotnet test src/Signer/StellaOps.Signer/StellaOps.Signer.Tests/StellaOps.Signer.Tests.csproj`). | -| SIGNER-REF-11-102 | DONE (2025-10-21) | Signer Guild | — | `/verify/referrers` endpoint with OCI lookup, caching, and policy enforcement. | ✅ `GET /api/v1/signer/verify/referrers` validates trusted scanner digests via release verifier and surfaces signer metadata; JSON responses served deterministically.
✅ Integration tests cover trusted/untrusted digests and validation failures (`SignerEndpointsTests`). | -| SIGNER-QUOTA-11-103 | DONE (2025-10-21) | Signer Guild | — | Enforce plan quotas, concurrency/QPS limits, artifact size caps with metrics/audit logs. | ✅ Quota middleware derives plan limits from PoE claims, applies per-tenant concurrency/QPS/size caps, and surfaces remaining capacity in responses.
✅ Unit coverage exercises throttled/artifact-too-large paths via in-memory quota service. | > Update status columns (TODO / DOING / DONE / BLOCKED) in tandem with code changes and associated tests. diff --git a/src/StellaOps.sln b/src/StellaOps.sln index 8e6fe7b9..aecd01c9 100644 --- a/src/StellaOps.sln +++ b/src/StellaOps.sln @@ -359,10 +359,14 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "PolicyEngine", "PolicyEngin EndProject Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "StellaOps.Bench.PolicyEngine", "StellaOps.Bench\PolicyEngine\StellaOps.Bench.PolicyEngine\StellaOps.Bench.PolicyEngine.csproj", "{D8B22C17-28E9-4059-97C5-4AC4600A2BD5}" EndProject -Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "StellaOps.Aoc", "StellaOps.Aoc\StellaOps.Aoc.csproj", "{6BE16682-4FB9-49C7-A2B3-ECB4EC5EF8BD}" -EndProject -Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "StellaOps.Aoc.Tests", "StellaOps.Aoc.Tests\StellaOps.Aoc.Tests.csproj", "{4D167781-1AC0-46CF-A32E-1B6E048940B2}" -EndProject +Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "StellaOps.Aoc", "StellaOps.Aoc\StellaOps.Aoc.csproj", "{6BE16682-4FB9-49C7-A2B3-ECB4EC5EF8BD}" +EndProject +Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "StellaOps.Aoc.AspNetCore", "StellaOps.Aoc.AspNetCore\StellaOps.Aoc.AspNetCore.csproj", "{D3D47993-27D3-4C90-9C8E-14652807DAF5}" +EndProject +Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "StellaOps.Aoc.Tests", "StellaOps.Aoc.Tests\StellaOps.Aoc.Tests.csproj", "{4D167781-1AC0-46CF-A32E-1B6E048940B2}" +EndProject +Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "StellaOps.Aoc.AspNetCore.Tests", "StellaOps.Aoc.AspNetCore.Tests\StellaOps.Aoc.AspNetCore.Tests.csproj", "{5F9B7682-71E2-4989-9BC9-014A2C26AF50}" +EndProject Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "StellaOps.Concelier.RawModels", "StellaOps.Concelier.RawModels\StellaOps.Concelier.RawModels.csproj", "{C3AEAEE7-038E-45FF-892B-DB18EE29F790}" EndProject Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "StellaOps.Concelier.RawModels.Tests", "StellaOps.Concelier.RawModels.Tests\StellaOps.Concelier.RawModels.Tests.csproj", "{7FACF6B4-7E12-4543-AAD4-0072FA1ECE0E}" @@ -2446,21 +2450,45 @@ Global {6BE16682-4FB9-49C7-A2B3-ECB4EC5EF8BD}.Release|Any CPU.ActiveCfg = Release|Any CPU {6BE16682-4FB9-49C7-A2B3-ECB4EC5EF8BD}.Release|Any CPU.Build.0 = Release|Any CPU {6BE16682-4FB9-49C7-A2B3-ECB4EC5EF8BD}.Release|x64.ActiveCfg = Release|Any CPU - {6BE16682-4FB9-49C7-A2B3-ECB4EC5EF8BD}.Release|x64.Build.0 = Release|Any CPU - {6BE16682-4FB9-49C7-A2B3-ECB4EC5EF8BD}.Release|x86.ActiveCfg = Release|Any CPU - {6BE16682-4FB9-49C7-A2B3-ECB4EC5EF8BD}.Release|x86.Build.0 = Release|Any CPU - {4D167781-1AC0-46CF-A32E-1B6E048940B2}.Debug|Any CPU.ActiveCfg = Debug|Any CPU - {4D167781-1AC0-46CF-A32E-1B6E048940B2}.Debug|Any CPU.Build.0 = Debug|Any CPU - {4D167781-1AC0-46CF-A32E-1B6E048940B2}.Debug|x64.ActiveCfg = Debug|Any CPU - {4D167781-1AC0-46CF-A32E-1B6E048940B2}.Debug|x64.Build.0 = Debug|Any CPU - {4D167781-1AC0-46CF-A32E-1B6E048940B2}.Debug|x86.ActiveCfg = Debug|Any CPU - {4D167781-1AC0-46CF-A32E-1B6E048940B2}.Debug|x86.Build.0 = Debug|Any CPU - {4D167781-1AC0-46CF-A32E-1B6E048940B2}.Release|Any CPU.ActiveCfg = Release|Any CPU - {4D167781-1AC0-46CF-A32E-1B6E048940B2}.Release|Any CPU.Build.0 = Release|Any CPU - {4D167781-1AC0-46CF-A32E-1B6E048940B2}.Release|x64.ActiveCfg = Release|Any CPU - {4D167781-1AC0-46CF-A32E-1B6E048940B2}.Release|x64.Build.0 = Release|Any CPU - {4D167781-1AC0-46CF-A32E-1B6E048940B2}.Release|x86.ActiveCfg = Release|Any CPU - {4D167781-1AC0-46CF-A32E-1B6E048940B2}.Release|x86.Build.0 = Release|Any CPU + {6BE16682-4FB9-49C7-A2B3-ECB4EC5EF8BD}.Release|x64.Build.0 = Release|Any CPU + {6BE16682-4FB9-49C7-A2B3-ECB4EC5EF8BD}.Release|x86.ActiveCfg = Release|Any CPU + {6BE16682-4FB9-49C7-A2B3-ECB4EC5EF8BD}.Release|x86.Build.0 = Release|Any CPU + {D3D47993-27D3-4C90-9C8E-14652807DAF5}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {D3D47993-27D3-4C90-9C8E-14652807DAF5}.Debug|Any CPU.Build.0 = Debug|Any CPU + {D3D47993-27D3-4C90-9C8E-14652807DAF5}.Debug|x64.ActiveCfg = Debug|Any CPU + {D3D47993-27D3-4C90-9C8E-14652807DAF5}.Debug|x64.Build.0 = Debug|Any CPU + {D3D47993-27D3-4C90-9C8E-14652807DAF5}.Debug|x86.ActiveCfg = Debug|Any CPU + {D3D47993-27D3-4C90-9C8E-14652807DAF5}.Debug|x86.Build.0 = Debug|Any CPU + {D3D47993-27D3-4C90-9C8E-14652807DAF5}.Release|Any CPU.ActiveCfg = Release|Any CPU + {D3D47993-27D3-4C90-9C8E-14652807DAF5}.Release|Any CPU.Build.0 = Release|Any CPU + {D3D47993-27D3-4C90-9C8E-14652807DAF5}.Release|x64.ActiveCfg = Release|Any CPU + {D3D47993-27D3-4C90-9C8E-14652807DAF5}.Release|x64.Build.0 = Release|Any CPU + {D3D47993-27D3-4C90-9C8E-14652807DAF5}.Release|x86.ActiveCfg = Release|Any CPU + {D3D47993-27D3-4C90-9C8E-14652807DAF5}.Release|x86.Build.0 = Release|Any CPU + {4D167781-1AC0-46CF-A32E-1B6E048940B2}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {4D167781-1AC0-46CF-A32E-1B6E048940B2}.Debug|Any CPU.Build.0 = Debug|Any CPU + {4D167781-1AC0-46CF-A32E-1B6E048940B2}.Debug|x64.ActiveCfg = Debug|Any CPU + {4D167781-1AC0-46CF-A32E-1B6E048940B2}.Debug|x64.Build.0 = Debug|Any CPU + {4D167781-1AC0-46CF-A32E-1B6E048940B2}.Debug|x86.ActiveCfg = Debug|Any CPU + {4D167781-1AC0-46CF-A32E-1B6E048940B2}.Debug|x86.Build.0 = Debug|Any CPU + {4D167781-1AC0-46CF-A32E-1B6E048940B2}.Release|Any CPU.ActiveCfg = Release|Any CPU + {4D167781-1AC0-46CF-A32E-1B6E048940B2}.Release|Any CPU.Build.0 = Release|Any CPU + {4D167781-1AC0-46CF-A32E-1B6E048940B2}.Release|x64.ActiveCfg = Release|Any CPU + {4D167781-1AC0-46CF-A32E-1B6E048940B2}.Release|x64.Build.0 = Release|Any CPU + {4D167781-1AC0-46CF-A32E-1B6E048940B2}.Release|x86.ActiveCfg = Release|Any CPU + {4D167781-1AC0-46CF-A32E-1B6E048940B2}.Release|x86.Build.0 = Release|Any CPU + {5F9B7682-71E2-4989-9BC9-014A2C26AF50}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {5F9B7682-71E2-4989-9BC9-014A2C26AF50}.Debug|Any CPU.Build.0 = Debug|Any CPU + {5F9B7682-71E2-4989-9BC9-014A2C26AF50}.Debug|x64.ActiveCfg = Debug|Any CPU + {5F9B7682-71E2-4989-9BC9-014A2C26AF50}.Debug|x64.Build.0 = Debug|Any CPU + {5F9B7682-71E2-4989-9BC9-014A2C26AF50}.Debug|x86.ActiveCfg = Debug|Any CPU + {5F9B7682-71E2-4989-9BC9-014A2C26AF50}.Debug|x86.Build.0 = Debug|Any CPU + {5F9B7682-71E2-4989-9BC9-014A2C26AF50}.Release|Any CPU.ActiveCfg = Release|Any CPU + {5F9B7682-71E2-4989-9BC9-014A2C26AF50}.Release|Any CPU.Build.0 = Release|Any CPU + {5F9B7682-71E2-4989-9BC9-014A2C26AF50}.Release|x64.ActiveCfg = Release|Any CPU + {5F9B7682-71E2-4989-9BC9-014A2C26AF50}.Release|x64.Build.0 = Release|Any CPU + {5F9B7682-71E2-4989-9BC9-014A2C26AF50}.Release|x86.ActiveCfg = Release|Any CPU + {5F9B7682-71E2-4989-9BC9-014A2C26AF50}.Release|x86.Build.0 = Release|Any CPU {C3AEAEE7-038E-45FF-892B-DB18EE29F790}.Debug|Any CPU.ActiveCfg = Debug|Any CPU {C3AEAEE7-038E-45FF-892B-DB18EE29F790}.Debug|Any CPU.Build.0 = Debug|Any CPU {C3AEAEE7-038E-45FF-892B-DB18EE29F790}.Debug|x64.ActiveCfg = Debug|Any CPU diff --git a/src/UI/StellaOps.UI/TASKS.completed.md b/src/UI/StellaOps.UI/TASKS.completed.md new file mode 100644 index 00000000..452dc484 --- /dev/null +++ b/src/UI/StellaOps.UI/TASKS.completed.md @@ -0,0 +1,6 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| UI-CONSOLE-23-001 | DONE (2025-10-31) | UI Guild & Security Guild | AUTH-CONSOLE-23-002 | Integrate Authority console endpoints (`/console/tenants`, `/console/profile`, `/console/token/introspect`) into UI session state, decode tenant/scopes claims, and expose signals for components. | Console session store fetches context on login, tenant header enforcement confirmed, unit tests cover store/service, and errors surface through state flags. | +| UI-CONSOLE-23-002 | DONE (2025-10-31) | UI Guild | UI-CONSOLE-23-001 | Build console profile view showing user identity, fresh-auth status, token metadata, and tenant catalog with refresh + tenant switch actions. | Component renders data from store, refresh action wired to API, accessibility checks pass, and component tests cover loading/error states. | diff --git a/src/UI/StellaOps.UI/TASKS.md b/src/UI/StellaOps.UI/TASKS.md index d23477ee..f09cdfb6 100644 --- a/src/UI/StellaOps.UI/TASKS.md +++ b/src/UI/StellaOps.UI/TASKS.md @@ -1,95 +1,93 @@ -# UI Task Board (Sprints 13 & 19) - -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| UI-POLICY-13-007 | TODO | UI Guild | POLICY-CORE-09-006, SCANNER-WEB-09-103 | Surface policy confidence metadata (band, age, quiet provenance) on preview and report views. | UI renders new columns/tooltips, accessibility and responsive checks pass, Cypress regression updated. | -| UI-AOC-19-001 | TODO | UI Guild | CONCELIER-WEB-AOC-19-001, EXCITITOR-WEB-AOC-19-001 | Add Sources dashboard tiles showing AOC pass/fail, recent violation codes, and ingest throughput per tenant. | Dashboard displays metrics from new endpoints, charts verified in e2e tests, accessibility checks pass. | -| UI-AOC-19-002 | TODO | UI Guild | UI-AOC-19-001 | Implement violation drill-down view highlighting offending document fields and provenance metadata. | Drill-down renders formatted JSON with highlights; copy-to-clipboard works; tests cover forbidden key cases. | -| UI-AOC-19-003 | TODO | UI Guild | UI-AOC-19-001, CLI-AOC-19-002 | Add "Verify last 24h" action triggering AOC verifier endpoint and surfacing CLI parity guidance. | Action wired to API, results rendered in toast/log panel, docs link to CLI usage, e2e test verifies flow. | - -## Policy Engine v2 (Sprint 20) - -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| UI-POLICY-20-001 | TODO | UI Guild | WEB-POLICY-20-001 | Ship Monaco-based policy editor with DSL syntax highlighting, inline diagnostics, and compliance checklist sidebar. | Editor renders DSL with token colors + lint; accessibility review passes; diagnostics surfaced from API compile endpoint in tests. | -| UI-POLICY-20-002 | TODO | UI Guild | UI-POLICY-20-001, WEB-POLICY-20-001, WEB-POLICY-20-002 | Build simulation panel showing before/after counts, severity deltas, and rule hit summaries with deterministic diff rendering. | Simulation view consumes API diff JSON, handles large datasets with virtualization, Cypress regression verifies charts/tables. | -| UI-POLICY-20-003 | TODO | UI Guild, Product Ops | UI-POLICY-20-001, AUTH-POLICY-27-001 | Implement submit/review/approve workflow with comments, approvals log, and RBAC checks aligned to new Policy Studio roles (`policy:author`/`policy:review`/`policy:approve`/`policy:operate`). | Workflow passes e2e tests, audit trail rendered, unauthorized roles blocked, docs linked from UI help. | -| UI-POLICY-20-004 | TODO | UI Guild, Observability Guild | WEB-POLICY-20-001, POLICY-ENGINE-20-006, POLICY-ENGINE-20-007 | Add run viewer dashboards (rule heatmap, VEX wins, suppressions) with filter/search and export. | Dashboards render aggregated metrics, export downloads CSV/JSON, accessibility/perf budgets met, telemetry charts validated. | - -## Policy Studio RBAC Alignment (Sprint 27) - -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| UI-POLICY-27-001 | TODO | UI Guild, Product Ops | AUTH-POLICY-27-001, UI-POLICY-20-003 | Update Console policy workspace RBAC guards, scope requests, and user messaging to reflect the new Policy Studio roles/scopes (`policy:author/review/approve/operate/audit/simulate`), including Cypress auth stubs and help text. | UI requests tokens with new scopes, unauthorized messaging references updated roles, Cypress/e2e tests cover scope failures, and help tooltips/docs links refreshed. | -> Heads-up: Authority & Gateway configs now reject the old `policy:write`/`policy:submit` scopes—Console policy flows will error until they request the new bundles. - -## Graph Explorer v1 (Sprint 21) - -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| UI-GRAPH-21-001 | TODO | UI Guild | WEB-GRAPH-21-001, AUTH-GRAPH-21-001 | Align Graph Explorer auth configuration with new `graph:*` scopes; consume scope identifiers from shared `StellaOpsScopes` exports (via generated SDK/config) instead of hard-coded strings. | UI requests graph tokens using shared scope constants; configuration docs updated; Cypress auth stub updated accordingly. | - -## Link-Not-Merge v1 (Sprint 22) - -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| UI-LNM-22-001 | TODO | UI Guild, Policy Guild | SCANNER-LNM-21-002, WEB-LNM-21-001 | Build Evidence panel showing policy decision with advisory observations/linksets side-by-side, conflict badges, AOC chain, and raw doc download links. Docs `DOCS-LNM-22-005` waiting on delivered UI for screenshots + flows. | Panel renders multiple sources; conflict badges accessible; e2e tests cover high-volume linksets. | -| UI-LNM-22-002 | TODO | UI Guild | UI-LNM-22-001 | Implement filters (source, severity bucket, conflict-only, CVSS vector presence) and pagination/lazy loading for large linksets. Docs depend on finalized filtering UX. | Filters respond within 500 ms; virtualization validated; unit/e2e tests added. | -| UI-LNM-22-003 | TODO | UI Guild, Excititor Guild | UI-LNM-22-001, WEB-LNM-21-002 | Add VEX tab with status/justification summaries, conflict indicators, and export actions. Required for `DOCS-LNM-22-005` coverage of VEX evidence tab. | VEX tab displays multiple observations; exports produce zipped OSV/CycloneDX; tests updated. | -| UI-LNM-22-004 | TODO | UI Guild | UI-LNM-22-001 | Provide permalink + copy-to-clipboard for selected component/linkset/policy combination; ensure high-contrast theme support. | Permalink reproduces state; accessibility audit passes; telemetry events logged. | - -## StellaOps Console (Sprint 23) - -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| UI-CONSOLE-23-001 | DONE (2025-10-31) | UI Guild & Security Guild | AUTH-CONSOLE-23-002 | Integrate Authority console endpoints (`/console/tenants`, `/console/profile`, `/console/token/introspect`) into UI session state, decode tenant/scopes claims, and expose signals for components. | Console session store fetches context on login, tenant header enforcement confirmed, unit tests cover store/service, and errors surface through state flags. | -> 2025-10-31: Added authority console API client, session store/service, and access token metadata parsing in `AuthorityAuthService`. Signals expose tenant/scopes, and unit tests cover happy/error paths. -| UI-CONSOLE-23-002 | DONE (2025-10-31) | UI Guild | UI-CONSOLE-23-001 | Build console profile view showing user identity, fresh-auth status, token metadata, and tenant catalog with refresh + tenant switch actions. | Component renders data from store, refresh action wired to API, accessibility checks pass, and component tests cover loading/error states. | -> 2025-10-31: Delivered `ConsoleProfileComponent`, hooked into navigation/header indicators, and styled cards for profile/token/tenant catalog with refresh + tenant switching. - -## Policy Engine + Editor v1 (Sprint 23) - -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| UI-POLICY-23-001 | TODO | UI Guild, Policy Guild | WEB-POLICY-23-001 | Deliver Policy Editor workspace with pack list, revision history, and scoped metadata cards. | Editor lists packs/revisions; navigation accessible; tests cover RBAC states. | -| UI-POLICY-23-002 | TODO | UI Guild | UI-POLICY-23-001 | Implement YAML editor with schema validation, lint diagnostics, and live canonicalization preview. | YAML editor surfaces inline errors sourced from compiler; keyboard shortcuts and accessibility verified. | -| UI-POLICY-23-003 | TODO | UI Guild | UI-POLICY-23-001, WEB-POLICY-23-003 | Build guided rule builder (source preferences, severity mapping, VEX precedence, exceptions) with preview JSON output. | Guided builder generates valid SPL, diff view matches YAML; tests cover rule permutations. | -| UI-POLICY-23-004 | TODO | UI Guild | UI-POLICY-23-001, WEB-POLICY-23-002, POLICY-GATEWAY-18-002..003 | Add review/approval workflow UI: checklists, comments, two-person approval indicator, scope scheduling. | Workflow screens complete; approval restrictions enforced; e2e tests cover approval -> activation. | -| UI-POLICY-23-005 | TODO | UI Guild | UI-POLICY-23-001, WEB-POLICY-23-003 | Integrate simulator panel (SBOM/component/advisory selection), run diff vs active policy, show explain tree and overlays. | Simulation results render diff/projection; explain tree interactive; performance <1s for sample data. | -| UI-POLICY-23-006 | TODO | UI Guild | UI-POLICY-23-005 | Implement explain view linking to evidence overlays and exceptions; provide export to JSON/PDF. | Explain view accessible; exports generated; analytics instrumented. | - -## Graph & Vuln Explorer v1 (Sprint 24) - -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| UI-GRAPH-24-001 | TODO | UI Guild, SBOM Service Guild | WEB-GRAPH-24-001 | Build Graph Explorer canvas with layered/radial layouts, virtualization, zoom/pan, and scope toggles; initial render <1.5s for sample asset. | Canvas meets perf budget; automated tests cover navigation; accessibility validation done. | -| UI-GRAPH-24-002 | TODO | UI Guild, Policy Guild | UI-GRAPH-24-001, WEB-GRAPH-24-001, WEB-VEX-30-007 | Implement overlays (Policy, Evidence, License, Exposure), simulation toggle, path view, and SBOM diff/time-travel with accessible tooltips/AOC indicators. | Overlays + simulation toggle respond <250 ms; path view/diff export validated; accessibility tests cover keyboard + contrast; e2e covers overlay combos. | -| UI-GRAPH-24-003 | TODO | UI Guild | UI-GRAPH-24-001 | Deliver filters/search panel with facets, saved views, permalinks, and share modal. | Filters update view <250ms; saved view persisted; permalinks reproduce state. | -| UI-GRAPH-24-004 | TODO | UI Guild | UI-GRAPH-24-001 | Add side panels (Details, What-if, History) with upgrade simulation integration and SBOM diff viewer. | Simulation results display diff + policy impact; history shows added/removed nodes; tests cover flows. | -| UI-GRAPH-24-006 | TODO | UI Guild, Accessibility Guild | UI-GRAPH-24-001..005 | Ensure accessibility (keyboard nav, screen reader labels, contrast), add hotkeys (`f`,`e`,`.`), and analytics instrumentation. | Accessibility audit passes; hotkeys documented; telemetry events captured. | - -## Exceptions v1 (Sprint 25) - -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| UI-EXC-25-001 | TODO | UI Guild, Governance Guild | WEB-EXC-25-001 | Build Exception Center (list + kanban) with filters, sorting, workflow transitions, and audit views. | Exception Center functional; state transitions via UI; accessibility validated. | -| UI-EXC-25-002 | TODO | UI Guild | UI-EXC-25-001 | Implement exception creation wizard with scope preview, justification templates, timebox guardrails. | Wizard enforces scope/timebox; previews impacted items; tests cover validation. | -| UI-EXC-25-003 | TODO | UI Guild | UI-EXC-25-001, WEB-EXC-25-002 | Add inline exception drafting/proposing from Vulnerability Explorer and Graph detail panels with live simulation. | Inline flows produce drafts; preview shows policy delta; telemetry instrumented. | -| UI-EXC-25-004 | TODO | UI Guild | UI-EXC-25-001 | Surface exception badges, countdown timers, and explain integration across Graph/Vuln Explorer and policy views. | Badges visible with SR labels; countdown updates; explain drawer shows exception info. | -| UI-EXC-25-005 | TODO | UI Guild, Accessibility Guild | UI-EXC-25-001..004 | Add keyboard shortcuts (`x`,`a`,`r`) and ensure screen-reader messaging for approvals/revocations. | Shortcuts functional; accessibility audit passes. | - -## Reachability v1 (Sprint 26) - -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| UI-SIG-26-001 | TODO | UI Guild, Signals Guild | WEB-SIG-26-001 | Add reachability columns/badges to Vulnerability Explorer with filters and tooltips. | Columns render with virtualization; filters update under 250 ms; badges accessible. | -| UI-SIG-26-002 | TODO | UI Guild | UI-SIG-26-001, WEB-SIG-26-002 | Enhance “Why” drawer with call path visualization, reachability timeline, and evidence list. | Drawer displays call path breadcrumb; copyable details; tests cover states. | -| UI-SIG-26-003 | TODO | UI Guild | UI-GRAPH-24-001, WEB-SIG-26-002 | Add reachability overlay halos/time slider to SBOM Graph along with state legend. | Overlay toggles; time slider compares snapshots; performance budget met. | -| UI-SIG-26-004 | TODO | UI Guild | WEB-SIG-26-003 | Build Reachability Center view showing asset coverage, missing sensors, and stale facts. | Center lists assets with metrics; missing sensors highlighted; accessibility validated. | - -## Orchestrator Dashboard (Sprint 32) - -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| UI-ORCH-32-001 | TODO | UI Guild, Console Guild | AUTH-ORCH-32-001, ORCH-SVC-32-003 | Update Console RBAC mappings to surface `Orch.Viewer`, request `orch:read` scope in token flows, and gate dashboard access/messaging accordingly. | Console role catalogue includes `Orch.Viewer`; auth helpers use shared scope constant; dashboard routes enforce scope and show actionable guidance; e2e tests cover authorized/unauthorized flows. | -> 2025-10-31: Authority minted `orch:read` scope; ensure Console UX aligns before orchestrator dashboards ship. +# UI Task Board (Sprints 13 & 19) + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| UI-POLICY-13-007 | TODO | UI Guild | POLICY-CORE-09-006, SCANNER-WEB-09-103 | Surface policy confidence metadata (band, age, quiet provenance) on preview and report views. | UI renders new columns/tooltips, accessibility and responsive checks pass, Cypress regression updated. | +| UI-AOC-19-001 | TODO | UI Guild | CONCELIER-WEB-AOC-19-001, EXCITITOR-WEB-AOC-19-001 | Add Sources dashboard tiles showing AOC pass/fail, recent violation codes, and ingest throughput per tenant. | Dashboard displays metrics from new endpoints, charts verified in e2e tests, accessibility checks pass. | +| UI-AOC-19-002 | TODO | UI Guild | UI-AOC-19-001 | Implement violation drill-down view highlighting offending document fields and provenance metadata. | Drill-down renders formatted JSON with highlights; copy-to-clipboard works; tests cover forbidden key cases. | +| UI-AOC-19-003 | TODO | UI Guild | UI-AOC-19-001, CLI-AOC-19-002 | Add "Verify last 24h" action triggering AOC verifier endpoint and surfacing CLI parity guidance. | Action wired to API, results rendered in toast/log panel, docs link to CLI usage, e2e test verifies flow. | + +## Policy Engine v2 (Sprint 20) + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| UI-POLICY-20-001 | TODO | UI Guild | WEB-POLICY-20-001 | Ship Monaco-based policy editor with DSL syntax highlighting, inline diagnostics, and compliance checklist sidebar. | Editor renders DSL with token colors + lint; accessibility review passes; diagnostics surfaced from API compile endpoint in tests. | +| UI-POLICY-20-002 | TODO | UI Guild | UI-POLICY-20-001, WEB-POLICY-20-001, WEB-POLICY-20-002 | Build simulation panel showing before/after counts, severity deltas, and rule hit summaries with deterministic diff rendering. | Simulation view consumes API diff JSON, handles large datasets with virtualization, Cypress regression verifies charts/tables. | +| UI-POLICY-20-003 | TODO | UI Guild, Product Ops | UI-POLICY-20-001, AUTH-POLICY-27-001 | Implement submit/review/approve workflow with comments, approvals log, and RBAC checks aligned to new Policy Studio roles (`policy:author`/`policy:review`/`policy:approve`/`policy:operate`). | Workflow passes e2e tests, audit trail rendered, unauthorized roles blocked, docs linked from UI help. | +| UI-POLICY-20-004 | TODO | UI Guild, Observability Guild | WEB-POLICY-20-001, POLICY-ENGINE-20-006, POLICY-ENGINE-20-007 | Add run viewer dashboards (rule heatmap, VEX wins, suppressions) with filter/search and export. | Dashboards render aggregated metrics, export downloads CSV/JSON, accessibility/perf budgets met, telemetry charts validated. | + +## Policy Studio RBAC Alignment (Sprint 27) + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| UI-POLICY-27-001 | TODO | UI Guild, Product Ops | AUTH-POLICY-27-001, UI-POLICY-20-003 | Update Console policy workspace RBAC guards, scope requests, and user messaging to reflect the new Policy Studio roles/scopes (`policy:author/review/approve/operate/audit/simulate`), including Cypress auth stubs and help text. | UI requests tokens with new scopes, unauthorized messaging references updated roles, Cypress/e2e tests cover scope failures, and help tooltips/docs links refreshed. | +> Heads-up: Authority & Gateway configs now reject the old `policy:write`/`policy:submit` scopes—Console policy flows will error until they request the new bundles. + +## Graph Explorer v1 (Sprint 21) + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| UI-GRAPH-21-001 | TODO | UI Guild | WEB-GRAPH-21-001, AUTH-GRAPH-21-001 | Align Graph Explorer auth configuration with new `graph:*` scopes; consume scope identifiers from shared `StellaOpsScopes` exports (via generated SDK/config) instead of hard-coded strings. | UI requests graph tokens using shared scope constants; configuration docs updated; Cypress auth stub updated accordingly. | + +## Link-Not-Merge v1 (Sprint 22) + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| UI-LNM-22-001 | TODO | UI Guild, Policy Guild | SCANNER-LNM-21-002, WEB-LNM-21-001 | Build Evidence panel showing policy decision with advisory observations/linksets side-by-side, conflict badges, AOC chain, and raw doc download links. Docs `DOCS-LNM-22-005` waiting on delivered UI for screenshots + flows. | Panel renders multiple sources; conflict badges accessible; e2e tests cover high-volume linksets. | +| UI-LNM-22-002 | TODO | UI Guild | UI-LNM-22-001 | Implement filters (source, severity bucket, conflict-only, CVSS vector presence) and pagination/lazy loading for large linksets. Docs depend on finalized filtering UX. | Filters respond within 500 ms; virtualization validated; unit/e2e tests added. | +| UI-LNM-22-003 | TODO | UI Guild, Excititor Guild | UI-LNM-22-001, WEB-LNM-21-002 | Add VEX tab with status/justification summaries, conflict indicators, and export actions. Required for `DOCS-LNM-22-005` coverage of VEX evidence tab. | VEX tab displays multiple observations; exports produce zipped OSV/CycloneDX; tests updated. | +| UI-LNM-22-004 | TODO | UI Guild | UI-LNM-22-001 | Provide permalink + copy-to-clipboard for selected component/linkset/policy combination; ensure high-contrast theme support. | Permalink reproduces state; accessibility audit passes; telemetry events logged. | + +## StellaOps Console (Sprint 23) + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +> 2025-10-31: Added authority console API client, session store/service, and access token metadata parsing in `AuthorityAuthService`. Signals expose tenant/scopes, and unit tests cover happy/error paths. +> 2025-10-31: Delivered `ConsoleProfileComponent`, hooked into navigation/header indicators, and styled cards for profile/token/tenant catalog with refresh + tenant switching. + +## Policy Engine + Editor v1 (Sprint 23) + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| UI-POLICY-23-001 | TODO | UI Guild, Policy Guild | WEB-POLICY-23-001 | Deliver Policy Editor workspace with pack list, revision history, and scoped metadata cards. | Editor lists packs/revisions; navigation accessible; tests cover RBAC states. | +| UI-POLICY-23-002 | TODO | UI Guild | UI-POLICY-23-001 | Implement YAML editor with schema validation, lint diagnostics, and live canonicalization preview. | YAML editor surfaces inline errors sourced from compiler; keyboard shortcuts and accessibility verified. | +| UI-POLICY-23-003 | TODO | UI Guild | UI-POLICY-23-001, WEB-POLICY-23-003 | Build guided rule builder (source preferences, severity mapping, VEX precedence, exceptions) with preview JSON output. | Guided builder generates valid SPL, diff view matches YAML; tests cover rule permutations. | +| UI-POLICY-23-004 | TODO | UI Guild | UI-POLICY-23-001, WEB-POLICY-23-002, POLICY-GATEWAY-18-002..003 | Add review/approval workflow UI: checklists, comments, two-person approval indicator, scope scheduling. | Workflow screens complete; approval restrictions enforced; e2e tests cover approval -> activation. | +| UI-POLICY-23-005 | TODO | UI Guild | UI-POLICY-23-001, WEB-POLICY-23-003 | Integrate simulator panel (SBOM/component/advisory selection), run diff vs active policy, show explain tree and overlays. | Simulation results render diff/projection; explain tree interactive; performance <1s for sample data. | +| UI-POLICY-23-006 | TODO | UI Guild | UI-POLICY-23-005 | Implement explain view linking to evidence overlays and exceptions; provide export to JSON/PDF. | Explain view accessible; exports generated; analytics instrumented. | + +## Graph & Vuln Explorer v1 (Sprint 24) + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| UI-GRAPH-24-001 | TODO | UI Guild, SBOM Service Guild | WEB-GRAPH-24-001 | Build Graph Explorer canvas with layered/radial layouts, virtualization, zoom/pan, and scope toggles; initial render <1.5s for sample asset. | Canvas meets perf budget; automated tests cover navigation; accessibility validation done. | +| UI-GRAPH-24-002 | TODO | UI Guild, Policy Guild | UI-GRAPH-24-001, WEB-GRAPH-24-001, WEB-VEX-30-007 | Implement overlays (Policy, Evidence, License, Exposure), simulation toggle, path view, and SBOM diff/time-travel with accessible tooltips/AOC indicators. | Overlays + simulation toggle respond <250 ms; path view/diff export validated; accessibility tests cover keyboard + contrast; e2e covers overlay combos. | +| UI-GRAPH-24-003 | TODO | UI Guild | UI-GRAPH-24-001 | Deliver filters/search panel with facets, saved views, permalinks, and share modal. | Filters update view <250ms; saved view persisted; permalinks reproduce state. | +| UI-GRAPH-24-004 | TODO | UI Guild | UI-GRAPH-24-001 | Add side panels (Details, What-if, History) with upgrade simulation integration and SBOM diff viewer. | Simulation results display diff + policy impact; history shows added/removed nodes; tests cover flows. | +| UI-GRAPH-24-006 | TODO | UI Guild, Accessibility Guild | UI-GRAPH-24-001..005 | Ensure accessibility (keyboard nav, screen reader labels, contrast), add hotkeys (`f`,`e`,`.`), and analytics instrumentation. | Accessibility audit passes; hotkeys documented; telemetry events captured. | + +## Exceptions v1 (Sprint 25) + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| UI-EXC-25-001 | TODO | UI Guild, Governance Guild | WEB-EXC-25-001 | Build Exception Center (list + kanban) with filters, sorting, workflow transitions, and audit views. | Exception Center functional; state transitions via UI; accessibility validated. | +| UI-EXC-25-002 | TODO | UI Guild | UI-EXC-25-001 | Implement exception creation wizard with scope preview, justification templates, timebox guardrails. | Wizard enforces scope/timebox; previews impacted items; tests cover validation. | +| UI-EXC-25-003 | TODO | UI Guild | UI-EXC-25-001, WEB-EXC-25-002 | Add inline exception drafting/proposing from Vulnerability Explorer and Graph detail panels with live simulation. | Inline flows produce drafts; preview shows policy delta; telemetry instrumented. | +| UI-EXC-25-004 | TODO | UI Guild | UI-EXC-25-001 | Surface exception badges, countdown timers, and explain integration across Graph/Vuln Explorer and policy views. | Badges visible with SR labels; countdown updates; explain drawer shows exception info. | +| UI-EXC-25-005 | TODO | UI Guild, Accessibility Guild | UI-EXC-25-001..004 | Add keyboard shortcuts (`x`,`a`,`r`) and ensure screen-reader messaging for approvals/revocations. | Shortcuts functional; accessibility audit passes. | + +## Reachability v1 (Sprint 26) + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| UI-SIG-26-001 | TODO | UI Guild, Signals Guild | WEB-SIG-26-001 | Add reachability columns/badges to Vulnerability Explorer with filters and tooltips. | Columns render with virtualization; filters update under 250 ms; badges accessible. | +| UI-SIG-26-002 | TODO | UI Guild | UI-SIG-26-001, WEB-SIG-26-002 | Enhance “Why” drawer with call path visualization, reachability timeline, and evidence list. | Drawer displays call path breadcrumb; copyable details; tests cover states. | +| UI-SIG-26-003 | TODO | UI Guild | UI-GRAPH-24-001, WEB-SIG-26-002 | Add reachability overlay halos/time slider to SBOM Graph along with state legend. | Overlay toggles; time slider compares snapshots; performance budget met. | +| UI-SIG-26-004 | TODO | UI Guild | WEB-SIG-26-003 | Build Reachability Center view showing asset coverage, missing sensors, and stale facts. | Center lists assets with metrics; missing sensors highlighted; accessibility validated. | + +## Orchestrator Dashboard (Sprint 32) + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| UI-ORCH-32-001 | TODO | UI Guild, Console Guild | AUTH-ORCH-32-001, ORCH-SVC-32-003 | Update Console RBAC mappings to surface `Orch.Viewer`, request `orch:read` scope in token flows, and gate dashboard access/messaging accordingly. | Console role catalogue includes `Orch.Viewer`; auth helpers use shared scope constant; dashboard routes enforce scope and show actionable guidance; e2e tests cover authorized/unauthorized flows. | +> 2025-10-31: Authority minted `orch:read` scope; ensure Console UX aligns before orchestrator dashboards ship. diff --git a/src/Web/StellaOps.Web/TASKS.md b/src/Web/StellaOps.Web/TASKS.md index 2a5e419e..65d07952 100644 --- a/src/Web/StellaOps.Web/TASKS.md +++ b/src/Web/StellaOps.Web/TASKS.md @@ -1,8 +1,10 @@ # TASKS — Epic 1: Aggregation-Only Contract | ID | Status | Owner(s) | Depends on | Notes | |----|--------|----------|------------|-------| -| WEB-AOC-19-001 `Shared AOC guard primitives` | DOING (2025-10-26) | BE-Base Platform Guild | — | Provide `AOCForbiddenKeys`, guard middleware/interceptor hooks, and error types (`AOCError`, `AOCViolationCode`) for ingestion services. Publish sample usage + analyzer to ensure guard registered. | -> 2025-10-26: Introduced `StellaOps.Aoc` library with forbidden key list, guard result/options, and baseline write guard + tests. Middleware/analyzer wiring still pending. +| WEB-AOC-19-001 `Shared AOC guard primitives` | DOING (2025-10-26) | BE-Base Platform Guild | — | Provide `AOCForbiddenKeys`, guard middleware/interceptor hooks, and error types (`AOCError`, `AOCViolationCode`) for ingestion services. Publish sample usage + analyzer to ensure guard registered. | +> 2025-10-26: Introduced `StellaOps.Aoc` library with forbidden key list, guard result/options, and baseline write guard + tests. Middleware/analyzer wiring still pending. +> 2025-10-30: Added `StellaOps.Aoc.AspNetCore` helpers (`AddAocGuard`, `AocHttpResults`) and switched Concelier WebService to the shared problem-details mapper; analyzer wiring remains pending. +> 2025-10-30: Published `docs/aoc/guard-library.md` covering registration patterns, endpoint filters, and error mapping for ingestion services. | WEB-AOC-19-002 `Provenance & signature helpers` | TODO | BE-Base Platform Guild | WEB-AOC-19-001 | Ship `ProvenanceBuilder`, checksum utilities, and signature verification helper integrated with guard logging. Cover DSSE/CMS formats with unit tests. | | WEB-AOC-19-003 `Analyzer + test fixtures` | TODO | QA Guild, BE-Base Platform Guild | WEB-AOC-19-001 | Author Roslyn analyzer preventing ingestion modules from writing forbidden keys without guard, and provide shared test fixtures for guard validation used by Concelier/Excititor service tests. | > Docs alignment (2025-10-26): Analyzer expectations detailed in `docs/ingestion/aggregation-only-contract.md` §3/5; CI integration tracked via DEVOPS-AOC-19-001. diff --git a/src/Zastava/StellaOps.Zastava.Observer/TASKS.completed.md b/src/Zastava/StellaOps.Zastava.Observer/TASKS.completed.md new file mode 100644 index 00000000..3ad59ac0 --- /dev/null +++ b/src/Zastava/StellaOps.Zastava.Observer/TASKS.completed.md @@ -0,0 +1,9 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| ZASTAVA-OBS-12-001 | DONE (2025-10-24) | Zastava Observer Guild | ZASTAVA-CORE-12-201 | Build container lifecycle watcher that tails CRI (containerd/cri-o/docker) events and emits deterministic runtime records with buffering + backoff. | Fixture cluster produces start/stop events with stable ordering, jitter/backoff tested, metrics/logging wired. | +| ZASTAVA-OBS-12-002 | DONE (2025-10-24) | Zastava Observer Guild | ZASTAVA-OBS-12-001 | Capture entrypoint traces and loaded libraries, hashing binaries and correlating to SBOM baseline per architecture sections 2.1 and 10. | EntryTrace parser covers shell/python/node launchers, loaded library hashes recorded, fixtures assert linkage to SBOM usage view. | +| ZASTAVA-OBS-12-003 | DONE (2025-10-24) | Zastava Observer Guild | ZASTAVA-OBS-12-002 | Implement runtime posture checks (signature/SBOM/attestation presence) with offline caching and warning surfaces. | Observer marks posture status, caches refresh across restarts, integration tests prove offline tolerance. | +| ZASTAVA-OBS-12-004 | DONE (2025-10-24) | Zastava Observer Guild | ZASTAVA-OBS-12-002 | Batch `/runtime/events` submissions with disk-backed buffer, rate limits, and deterministic envelopes. | Buffered submissions survive restart, rate-limits enforced in tests, JSON envelopes match schema in docs/events. | +| ZASTAVA-OBS-17-005 | DONE (2025-10-25) | Zastava Observer Guild | ZASTAVA-OBS-12-002 | Collect GNU build-id for ELF processes and attach it to emitted runtime events to enable symbol lookup + debug-store correlation. | Build-id extraction feeds RuntimeEvent envelopes plus Scanner policy downstream; unit tests cover capture + envelope wiring, and ops runbook documents retrieval + debug-store mapping. | diff --git a/src/Zastava/StellaOps.Zastava.Observer/TASKS.md b/src/Zastava/StellaOps.Zastava.Observer/TASKS.md index 68dc1821..b3b5efb8 100644 --- a/src/Zastava/StellaOps.Zastava.Observer/TASKS.md +++ b/src/Zastava/StellaOps.Zastava.Observer/TASKS.md @@ -1,11 +1,6 @@ -# Zastava Observer Task Board - -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| ZASTAVA-OBS-12-001 | DONE (2025-10-24) | Zastava Observer Guild | ZASTAVA-CORE-12-201 | Build container lifecycle watcher that tails CRI (containerd/cri-o/docker) events and emits deterministic runtime records with buffering + backoff. | Fixture cluster produces start/stop events with stable ordering, jitter/backoff tested, metrics/logging wired. | -| ZASTAVA-OBS-12-002 | DONE (2025-10-24) | Zastava Observer Guild | ZASTAVA-OBS-12-001 | Capture entrypoint traces and loaded libraries, hashing binaries and correlating to SBOM baseline per architecture sections 2.1 and 10. | EntryTrace parser covers shell/python/node launchers, loaded library hashes recorded, fixtures assert linkage to SBOM usage view. | -| ZASTAVA-OBS-12-003 | DONE (2025-10-24) | Zastava Observer Guild | ZASTAVA-OBS-12-002 | Implement runtime posture checks (signature/SBOM/attestation presence) with offline caching and warning surfaces. | Observer marks posture status, caches refresh across restarts, integration tests prove offline tolerance. | -| ZASTAVA-OBS-12-004 | DONE (2025-10-24) | Zastava Observer Guild | ZASTAVA-OBS-12-002 | Batch `/runtime/events` submissions with disk-backed buffer, rate limits, and deterministic envelopes. | Buffered submissions survive restart, rate-limits enforced in tests, JSON envelopes match schema in docs/events. | -| ZASTAVA-OBS-17-005 | DONE (2025-10-25) | Zastava Observer Guild | ZASTAVA-OBS-12-002 | Collect GNU build-id for ELF processes and attach it to emitted runtime events to enable symbol lookup + debug-store correlation. | Build-id extraction feeds RuntimeEvent envelopes plus Scanner policy downstream; unit tests cover capture + envelope wiring, and ops runbook documents retrieval + debug-store mapping. | +# Zastava Observer Task Board + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| > 2025-10-24: Observer unit tests pending; `dotnet restore` requires offline copies of `Google.Protobuf`, `Grpc.Net.Client`, `Grpc.Tools` in `local-nuget` before execution can be verified. diff --git a/src/Zastava/StellaOps.Zastava.Webhook/TASKS.completed.md b/src/Zastava/StellaOps.Zastava.Webhook/TASKS.completed.md new file mode 100644 index 00000000..0bc59115 --- /dev/null +++ b/src/Zastava/StellaOps.Zastava.Webhook/TASKS.completed.md @@ -0,0 +1,8 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| ZASTAVA-WEBHOOK-12-101 | DONE (2025-10-24) | Zastava Webhook Guild | — | Admission controller host with TLS bootstrap and Authority auth. | Webhook host boots with deterministic TLS bootstrap, enforces Authority-issued credentials, e2e smoke proves admission callback lifecycle, structured logs + metrics emit on each decision. | +| ZASTAVA-WEBHOOK-12-102 | DONE (2025-10-24) | Zastava Webhook Guild | — | Query Scanner `/policy/runtime`, resolve digests, enforce verdicts. | Scanner client resolves image digests + policy verdicts, unit tests cover allow/deny, integration harness rejects/admits workloads per policy with deterministic payloads. | +| ZASTAVA-WEBHOOK-12-103 | DONE (2025-10-24) | Zastava Webhook Guild | — | Caching, fail-open/closed toggles, metrics/logging for admission decisions. | Configurable cache TTL + seeds survive restart, fail-open/closed toggles verified via tests, metrics/logging exported per decision path, docs note operational knobs. | +| ZASTAVA-WEBHOOK-12-104 | DONE (2025-10-24) | Zastava Webhook Guild | ZASTAVA-WEBHOOK-12-102 | Wire `/admission` endpoint to runtime policy client and emit allow/deny envelopes. | Admission handler resolves pods to digests, invokes policy client, returns canonical `AdmissionDecisionEnvelope` with deterministic logging and metrics. | diff --git a/src/Zastava/StellaOps.Zastava.Webhook/TASKS.md b/src/Zastava/StellaOps.Zastava.Webhook/TASKS.md index 7bee28cd..116d89a6 100644 --- a/src/Zastava/StellaOps.Zastava.Webhook/TASKS.md +++ b/src/Zastava/StellaOps.Zastava.Webhook/TASKS.md @@ -1,10 +1,6 @@ -# Zastava Webhook Task Board - -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| ZASTAVA-WEBHOOK-12-101 | DONE (2025-10-24) | Zastava Webhook Guild | — | Admission controller host with TLS bootstrap and Authority auth. | Webhook host boots with deterministic TLS bootstrap, enforces Authority-issued credentials, e2e smoke proves admission callback lifecycle, structured logs + metrics emit on each decision. | -| ZASTAVA-WEBHOOK-12-102 | DONE (2025-10-24) | Zastava Webhook Guild | — | Query Scanner `/policy/runtime`, resolve digests, enforce verdicts. | Scanner client resolves image digests + policy verdicts, unit tests cover allow/deny, integration harness rejects/admits workloads per policy with deterministic payloads. | -| ZASTAVA-WEBHOOK-12-103 | DONE (2025-10-24) | Zastava Webhook Guild | — | Caching, fail-open/closed toggles, metrics/logging for admission decisions. | Configurable cache TTL + seeds survive restart, fail-open/closed toggles verified via tests, metrics/logging exported per decision path, docs note operational knobs. | -| ZASTAVA-WEBHOOK-12-104 | DONE (2025-10-24) | Zastava Webhook Guild | ZASTAVA-WEBHOOK-12-102 | Wire `/admission` endpoint to runtime policy client and emit allow/deny envelopes. | Admission handler resolves pods to digests, invokes policy client, returns canonical `AdmissionDecisionEnvelope` with deterministic logging and metrics. | - -> Status update · 2025-10-19: Confirmed no prerequisites for ZASTAVA-WEBHOOK-12-101/102/103; tasks moved to DOING for kickoff. Implementation plan covering TLS bootstrap, backend contract, caching/metrics recorded in `IMPLEMENTATION_PLAN.md`. +# Zastava Webhook Task Board + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| + +> Status update · 2025-10-19: Confirmed no prerequisites for ZASTAVA-WEBHOOK-12-101/102/103; tasks moved to DOING for kickoff. Implementation plan covering TLS bootstrap, backend contract, caching/metrics recorded in `IMPLEMENTATION_PLAN.md`. diff --git a/src/Zastava/__Libraries/StellaOps.Zastava.Core/TASKS.completed.md b/src/Zastava/__Libraries/StellaOps.Zastava.Core/TASKS.completed.md new file mode 100644 index 00000000..f900a92d --- /dev/null +++ b/src/Zastava/__Libraries/StellaOps.Zastava.Core/TASKS.completed.md @@ -0,0 +1,8 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| ZASTAVA-CORE-12-201 | DONE (2025-10-23) | Zastava Core Guild | — | Define runtime event/admission DTOs, hashing helpers, and versioning strategy. | DTOs cover runtime events and admission verdict envelopes with canonical JSON schema; hashing helpers accept payloads and yield deterministic multihash outputs; version negotiation rules documented and exercised by serialization tests. | +| ZASTAVA-CORE-12-202 | DONE (2025-10-23) | Zastava Core Guild | — | Provide configuration/logging/metrics utilities shared by Observer/Webhook. | Shared options bind from configuration with validation; logging scopes/metrics exporters registered via reusable DI extension; integration test host demonstrates Observer/Webhook consumption with deterministic instrumentation. | +| ZASTAVA-CORE-12-203 | DONE (2025-10-23) | Zastava Core Guild | — | Authority client helpers, OpTok caching, and security guardrails for runtime services. | Typed Authority client surfaces OpTok retrieval + renewal with configurable cache; guardrails enforce DPoP/mTLS expectations and emit structured audit logs; negative-path tests cover expired/invalid tokens and configuration toggles. | +| ZASTAVA-OPS-12-204 | DONE (2025-10-23) | Zastava Core Guild | — | Operational runbooks, alert rules, and dashboard exports for runtime plane. | Runbooks capture install/upgrade/rollback + incident handling; alert rules and dashboard JSON exported for Prometheus/Grafana bundle; docs reference Offline Kit packaging and verification checklist. | diff --git a/src/Zastava/__Libraries/StellaOps.Zastava.Core/TASKS.md b/src/Zastava/__Libraries/StellaOps.Zastava.Core/TASKS.md index f2146353..6d715ec9 100644 --- a/src/Zastava/__Libraries/StellaOps.Zastava.Core/TASKS.md +++ b/src/Zastava/__Libraries/StellaOps.Zastava.Core/TASKS.md @@ -1,10 +1,6 @@ -# Zastava Core Task Board - -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| ZASTAVA-CORE-12-201 | DONE (2025-10-23) | Zastava Core Guild | — | Define runtime event/admission DTOs, hashing helpers, and versioning strategy. | DTOs cover runtime events and admission verdict envelopes with canonical JSON schema; hashing helpers accept payloads and yield deterministic multihash outputs; version negotiation rules documented and exercised by serialization tests. | -| ZASTAVA-CORE-12-202 | DONE (2025-10-23) | Zastava Core Guild | — | Provide configuration/logging/metrics utilities shared by Observer/Webhook. | Shared options bind from configuration with validation; logging scopes/metrics exporters registered via reusable DI extension; integration test host demonstrates Observer/Webhook consumption with deterministic instrumentation. | -| ZASTAVA-CORE-12-203 | DONE (2025-10-23) | Zastava Core Guild | — | Authority client helpers, OpTok caching, and security guardrails for runtime services. | Typed Authority client surfaces OpTok retrieval + renewal with configurable cache; guardrails enforce DPoP/mTLS expectations and emit structured audit logs; negative-path tests cover expired/invalid tokens and configuration toggles. | -| ZASTAVA-OPS-12-204 | DONE (2025-10-23) | Zastava Core Guild | — | Operational runbooks, alert rules, and dashboard exports for runtime plane. | Runbooks capture install/upgrade/rollback + incident handling; alert rules and dashboard JSON exported for Prometheus/Grafana bundle; docs reference Offline Kit packaging and verification checklist. | - -> Remark (2025-10-19): Prerequisites reviewed—none outstanding. ZASTAVA-CORE-12-201, ZASTAVA-CORE-12-202, ZASTAVA-CORE-12-203, and ZASTAVA-OPS-12-204 moved to DOING for Wave 0 kickoff. +# Zastava Core Task Board + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| + +> Remark (2025-10-19): Prerequisites reviewed—none outstanding. ZASTAVA-CORE-12-201, ZASTAVA-CORE-12-202, ZASTAVA-CORE-12-203, and ZASTAVA-OPS-12-204 moved to DOING for Wave 0 kickoff. diff --git a/src/__Libraries/StellaOps.Configuration/AuthoritySigningOptions.cs b/src/__Libraries/StellaOps.Configuration/AuthoritySigningOptions.cs index fcb69dee..48aebe52 100644 --- a/src/__Libraries/StellaOps.Configuration/AuthoritySigningOptions.cs +++ b/src/__Libraries/StellaOps.Configuration/AuthoritySigningOptions.cs @@ -11,6 +11,11 @@ public sealed class AuthoritySigningOptions /// public bool Enabled { get; set; } = true; + /// + /// Duration that JWKS responses are cached before being rebuilt. + /// + public TimeSpan JwksCacheLifetime { get; set; } = TimeSpan.FromMinutes(15); + /// /// Signing algorithm identifier (ES256 by default). /// @@ -77,5 +82,10 @@ public sealed class AuthoritySigningOptions { key.Validate(KeySource); } + + if (JwksCacheLifetime <= TimeSpan.Zero || JwksCacheLifetime > TimeSpan.FromHours(1)) + { + throw new InvalidOperationException("Authority signing configuration requires signing.jwksCacheLifetime to be between 00:00:01 and 01:00:00."); + } } } diff --git a/src/__Libraries/StellaOps.Cryptography.Kms/TASKS.completed.md b/src/__Libraries/StellaOps.Cryptography.Kms/TASKS.completed.md new file mode 100644 index 00000000..c46f4674 --- /dev/null +++ b/src/__Libraries/StellaOps.Cryptography.Kms/TASKS.completed.md @@ -0,0 +1,6 @@ +# Completed Tasks + +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| KMS-72-001 | DONE (2025-10-30) | KMS Guild | — | Implement KMS interface (sign, verify, metadata, rotate, revoke) and file-based key driver with encrypted at-rest storage. | Interface + file driver operational; unit tests cover sign/verify/rotation; lint passes.
2025-10-29: `FileKmsClient` (ES256) file driver scaffolding committed under `StellaOps.Cryptography.Kms`; includes disk encryption + unit tests. Follow-up: address PBKDF2/AesGcm warnings and wire into Authority services.
2025-10-29 18:40Z: Hardened PBKDF2 iteration floor (≥600k), switched to tag-size explicit `AesGcm` usage, removed transient array allocations, and refreshed unit tests (`StellaOps.Cryptography.Kms.Tests`).
2025-10-30: Cleared remaining PBKDF2/AesGcm analyser warnings, validated Authority host wiring for `AddFileKms`, reran `dotnet test src/__Libraries/__Tests/StellaOps.Cryptography.Kms.Tests/StellaOps.Cryptography.Kms.Tests.csproj --no-build`, and confirmed clean `dotnet build` (no warnings). | +| KMS-72-002 | DONE (2025-10-30) | KMS Guild | KMS-72-001 | Add CLI support for importing/exporting file-based keys with password protection. | CLI commands functional; docs updated; integration tests pass.
2025-10-30: CLI requirements reviewed; new `stella kms` verb planned for file driver import/export flow with Spectre prompts + tests.
2025-10-30 20:15Z: Shipped `stella kms export|import` (passphrase/env/prompt support), wired `FileKmsClient.ImportAsync`, updated plugin manifest loader tests, and ran `dotnet build`/`dotnet test` for KMS + CLI suites. | diff --git a/src/__Libraries/StellaOps.Cryptography.Kms/TASKS.md b/src/__Libraries/StellaOps.Cryptography.Kms/TASKS.md index bafa1ce0..ddcea24e 100644 --- a/src/__Libraries/StellaOps.Cryptography.Kms/TASKS.md +++ b/src/__Libraries/StellaOps.Cryptography.Kms/TASKS.md @@ -1,13 +1,11 @@ -# KMS Task Board — Epic 19: Attestor Console - -## Sprint 72 – Abstractions & File Driver -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| KMS-72-001 | DONE (2025-10-30) | KMS Guild | — | Implement KMS interface (sign, verify, metadata, rotate, revoke) and file-based key driver with encrypted at-rest storage. | Interface + file driver operational; unit tests cover sign/verify/rotation; lint passes.
2025-10-29: `FileKmsClient` (ES256) file driver scaffolding committed under `StellaOps.Cryptography.Kms`; includes disk encryption + unit tests. Follow-up: address PBKDF2/AesGcm warnings and wire into Authority services.
2025-10-29 18:40Z: Hardened PBKDF2 iteration floor (≥600k), switched to tag-size explicit `AesGcm` usage, removed transient array allocations, and refreshed unit tests (`StellaOps.Cryptography.Kms.Tests`).
2025-10-30: Cleared remaining PBKDF2/AesGcm analyser warnings, validated Authority host wiring for `AddFileKms`, reran `dotnet test src/__Libraries/__Tests/StellaOps.Cryptography.Kms.Tests/StellaOps.Cryptography.Kms.Tests.csproj --no-build`, and confirmed clean `dotnet build` (no warnings). | -| KMS-72-002 | DONE (2025-10-30) | KMS Guild | KMS-72-001 | Add CLI support for importing/exporting file-based keys with password protection. | CLI commands functional; docs updated; integration tests pass.
2025-10-30: CLI requirements reviewed; new `stella kms` verb planned for file driver import/export flow with Spectre prompts + tests.
2025-10-30 20:15Z: Shipped `stella kms export|import` (passphrase/env/prompt support), wired `FileKmsClient.ImportAsync`, updated plugin manifest loader tests, and ran `dotnet build`/`dotnet test` for KMS + CLI suites. | - -## Sprint 73 – Cloud & HSM Integration -| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | -|----|--------|----------|------------|-------------|---------------| -| KMS-73-001 | TODO | KMS Guild | KMS-72-001 | Add cloud KMS driver (e.g., AWS KMS, GCP KMS) with signing and key metadata retrieval. | Cloud driver tested with mock; configuration documented; security review sign-off. | -| KMS-73-002 | TODO | KMS Guild | KMS-72-001 | Implement PKCS#11/HSM driver plus FIDO2 signing support for high assurance workflows. | HSM/FIDO2 drivers tested with hardware stubs; error handling documented. | +# KMS Task Board — Epic 19: Attestor Console + +## Sprint 72 – Abstractions & File Driver +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| + +## Sprint 73 – Cloud & HSM Integration +| ID | Status | Owner(s) | Depends on | Description | Exit Criteria | +|----|--------|----------|------------|-------------|---------------| +| KMS-73-001 | TODO | KMS Guild | KMS-72-001 | Add cloud KMS driver (e.g., AWS KMS, GCP KMS) with signing and key metadata retrieval. | Cloud driver tested with mock; configuration documented; security review sign-off. | +| KMS-73-002 | TODO | KMS Guild | KMS-72-001 | Implement PKCS#11/HSM driver plus FIDO2 signing support for high assurance workflows. | HSM/FIDO2 drivers tested with hardware stubs; error handling documented. | diff --git a/src/__Libraries/StellaOps.Cryptography/TASKS.completed.md b/src/__Libraries/StellaOps.Cryptography/TASKS.completed.md new file mode 100644 index 00000000..2a94113b --- /dev/null +++ b/src/__Libraries/StellaOps.Cryptography/TASKS.completed.md @@ -0,0 +1,27 @@ +# Completed Tasks + +| ID | Status | Owner | Description | Dependencies | Exit Criteria | +|----|--------|-------|-------------|--------------|---------------| +| SEC1.A | DONE (2025-10-11) | Security Guild | Introduce `Argon2idPasswordHasher` backed by Konscious defaults. Wire options into `StandardPluginOptions` (`PasswordHashOptions`) and `StellaOpsAuthorityOptions.Security.PasswordHashing`. | PLG3, CORE3 | ✅ Hashes emit PHC string `$argon2id$v=19$m=19456,t=2,p=1$...`; ✅ `NeedsRehash` promotes PBKDF2 → Argon2; ✅ Integration tests cover tamper, legacy rehash, perf p95 < 250 ms. | +| SEC1.B | DONE (2025-10-12) | Security Guild | Add compile-time switch to enable libsodium/Core variants later (`STELLAOPS_CRYPTO_SODIUM`). Document build variable. | SEC1.A | ✅ Conditional compilation path compiles; ✅ README snippet in `docs/security/password-hashing.md`. | +| SEC2.A | DONE (2025-10-13) | Security Guild + Core | Define audit event contract (`AuthEventRecord`) with subject/client/scope/IP/outcome/correlationId and PII tags. | CORE5–CORE7 | ✅ Contract shipped in `StellaOps.Cryptography` (or shared abstractions); ✅ Docs in `docs/security/audit-events.md`. | +| SEC2.B | DONE (2025-10-13) | Security Guild | Emit audit records from OpenIddict handlers (password + client creds) and bootstrap APIs. Persist via `IAuthorityLoginAttemptStore`. | SEC2.A | ✅ Tests assert three flows (success/failure/lockout); ✅ Serilog output contains correlationId + PII tagging; ✅ Mongo store holds summary rows. | +| SEC3.A | DONE (2025-10-12) | Security Guild + Core | Configure ASP.NET rate limiter (`AddRateLimiter`) with fixed-window policy keyed by IP + `client_id`. Apply to `/token` and `/internal/*`. | CORE8 completion | ✅ Middleware active; ✅ Configurable limits via options; ✅ Integration test hits 429. | +| SEC3.B | DONE (2025-10-13) | Security Guild | Document lockout + rate-limit tuning guidance and escalation thresholds. | SEC3.A | ✅ Section in `docs/security/rate-limits.md`; ✅ Includes SOC alert recommendations. | +| SEC4.A | DONE (2025-10-12) | Security Guild + DevOps | Define revocation JSON schema (`revocation_bundle.schema.json`) and detached JWS workflow. | CORE9, OPS3 | ✅ Schema + sample committed; ✅ CLI command `stellaops auth revoke export` scaffolded with acceptance tests; ✅ Verification script + docs. | +| SEC4.B | DONE (2025-10-12) | Security Guild | Integrate signing keys with crypto provider abstraction (initially ES256 via BCL). | SEC4.A, D5 | ✅ `ICryptoProvider.GetSigner` stub + default BCL signer; ✅ Unit tests verifying signature roundtrip. | +| SEC5.A | DONE (2025-10-12) | Security Guild | Author STRIDE threat model (`docs/security/authority-threat-model.md`) covering token, bootstrap, revocation, CLI, plugin surfaces. | All SEC1–SEC4 in progress | ✅ DFDs + trust boundaries drawn; ✅ Risk table with owners/actions; ✅ Follow-up backlog issues created. | +| SEC5.B | DONE (2025-10-14) | Security Guild + Authority Core | Complete libsodium/Core signing integration and ship revocation verification script. | SEC4.A, SEC4.B, SEC4.HOST | ✅ libsodium/Core signing provider wired; ✅ `stellaops auth revoke verify` script published; ✅ Revocation docs updated with verification workflow. | +| SEC5.B1 | DONE (2025-10-14) | Security Guild + Authority Core | Introduce `LibsodiumCryptoProvider` implementing ECDSA signing/verification via libsodium, register under feature flag, and validate against existing ES256 fixtures. | SEC5.B | ✅ Provider resolves via `ICryptoProviderRegistry`; ✅ Integration tests cover sign/verify parity with default provider; ✅ Fallback to managed provider documented. | +| SEC5.B2 | DONE (2025-10-14) | Security Guild + DevEx/CLI | Extend `stellaops auth revoke verify` to detect provider metadata, reuse registry for verification, and document CLI workflow. | SEC5.B | ✅ CLI uses registry signers for verification; ✅ End-to-end test invokes verify against sample bundle; ✅ docs/11_AUTHORITY.md references CLI procedure. | +| SEC5.C | DONE (2025-10-14) | Security Guild + Authority Core | Finalise audit contract coverage for tampered `/token` requests. | SEC2.A, SEC2.B | ✅ Tamper attempts logged with correlationId/PII tags; ✅ SOC runbook updated; ✅ Threat model status reviewed. | +| SEC5.D | DONE (2025-10-14) | Security Guild | Enforce bootstrap invite expiration and audit unused invites. | SEC5.A | ✅ Bootstrap tokens auto-expire; ✅ Audit entries emitted for expiration/reuse attempts; ✅ Operator docs updated. | +| SEC5.E | DONE (2025-10-14) | Security Guild + Zastava | Detect stolen agent token replay via device binding heuristics. | SEC4.A | ✅ Device binding guidance published; ✅ Alerting pipeline raises stale revocation acknowledgements; ✅ Tests cover replay detection. | +| SEC5.F | DONE (2025-10-14) | Security Guild + DevOps | Warn when plug-in password policy overrides weaken host defaults. | SEC1.A, PLG3 | ✅ Static analyser flags weaker overrides; ✅ Runtime warning surfaced; ✅ Docs call out mitigation. | +| SEC5.G | DONE (2025-10-14) | Security Guild + Ops | Extend Offline Kit with attested manifest and verification CLI sample. | OPS3 | ✅ Offline Kit build signs manifest with detached JWS; ✅ Verification CLI documented; ✅ Supply-chain attestation recorded. | +| SEC5.H | DONE (2025-10-13) | Security Guild + Authority Core | Ensure `/token` denials persist audit records with correlation IDs. | SEC2.A, SEC2.B | ✅ Audit store captures denials; ✅ Tests cover success/failure/lockout; ✅ Threat model review updated. | +| D5.A | DONE (2025-10-12) | Security Guild | Flesh out `StellaOps.Cryptography` provider registry, policy, and DI helpers enabling sovereign crypto selection. | SEC1.A, SEC4.B | ✅ `ICryptoProviderRegistry` implementation with provider selection rules; ✅ `StellaOps.Cryptography.DependencyInjection` extensions; ✅ Tests covering fallback ordering. | + +| SEC5.H | DONE (2025-10-13) | Security Guild + Authority Core | Ensure `/token` denials persist audit records with correlation IDs. | SEC2.A, SEC2.B | ✅ Audit store captures denials; ✅ Tests cover success/failure/lockout; ✅ Threat model review updated. | +| D5.A | DONE (2025-10-12) | Security Guild | Flesh out `StellaOps.Cryptography` provider registry, policy, and DI helpers enabling sovereign crypto selection. | SEC1.A, SEC4.B | ✅ `ICryptoProviderRegistry` implementation with provider selection rules; ✅ `StellaOps.Cryptography.DependencyInjection` extensions; ✅ Tests covering fallback ordering. | +| SEC6.A | DONE (2025-10-19) | Security Guild | Ship BouncyCastle-backed Ed25519 signing as a `StellaOps.Cryptography` plug-in and migrate Scanner WebService signing to consume the provider registry; codify the plug-in rule in AGENTS.
2025-10-19: Added `StellaOps.Cryptography.Plugin.BouncyCastle`, updated DI and ReportSigner, captured provider tests (`BouncyCastleEd25519CryptoProviderTests`). | D5.A | ✅ Plug-in registered via DI (`AddStellaOpsCrypto` + `AddBouncyCastleEd25519Provider`); ✅ Report signer resolves keys through registry; ✅ Unit tests cover Ed25519 sign/verify via provider. | diff --git a/src/__Libraries/StellaOps.Cryptography/TASKS.md b/src/__Libraries/StellaOps.Cryptography/TASKS.md index d2692761..d14932d4 100644 --- a/src/__Libraries/StellaOps.Cryptography/TASKS.md +++ b/src/__Libraries/StellaOps.Cryptography/TASKS.md @@ -1,51 +1,31 @@ -# Team 8 — Security Guild Task Board (UTC 2025-10-10) - -| ID | Status | Owner | Description | Dependencies | Exit Criteria | -|----|--------|-------|-------------|--------------|---------------| -| SEC1.A | DONE (2025-10-11) | Security Guild | Introduce `Argon2idPasswordHasher` backed by Konscious defaults. Wire options into `StandardPluginOptions` (`PasswordHashOptions`) and `StellaOpsAuthorityOptions.Security.PasswordHashing`. | PLG3, CORE3 | ✅ Hashes emit PHC string `$argon2id$v=19$m=19456,t=2,p=1$...`; ✅ `NeedsRehash` promotes PBKDF2 → Argon2; ✅ Integration tests cover tamper, legacy rehash, perf p95 < 250 ms. | -| SEC1.B | DONE (2025-10-12) | Security Guild | Add compile-time switch to enable libsodium/Core variants later (`STELLAOPS_CRYPTO_SODIUM`). Document build variable. | SEC1.A | ✅ Conditional compilation path compiles; ✅ README snippet in `docs/security/password-hashing.md`. | -| SEC2.A | DONE (2025-10-13) | Security Guild + Core | Define audit event contract (`AuthEventRecord`) with subject/client/scope/IP/outcome/correlationId and PII tags. | CORE5–CORE7 | ✅ Contract shipped in `StellaOps.Cryptography` (or shared abstractions); ✅ Docs in `docs/security/audit-events.md`. | -| SEC2.B | DONE (2025-10-13) | Security Guild | Emit audit records from OpenIddict handlers (password + client creds) and bootstrap APIs. Persist via `IAuthorityLoginAttemptStore`. | SEC2.A | ✅ Tests assert three flows (success/failure/lockout); ✅ Serilog output contains correlationId + PII tagging; ✅ Mongo store holds summary rows. | -| SEC3.A | DONE (2025-10-12) | Security Guild + Core | Configure ASP.NET rate limiter (`AddRateLimiter`) with fixed-window policy keyed by IP + `client_id`. Apply to `/token` and `/internal/*`. | CORE8 completion | ✅ Middleware active; ✅ Configurable limits via options; ✅ Integration test hits 429. | -| SEC3.B | DONE (2025-10-13) | Security Guild | Document lockout + rate-limit tuning guidance and escalation thresholds. | SEC3.A | ✅ Section in `docs/security/rate-limits.md`; ✅ Includes SOC alert recommendations. | -| SEC4.A | DONE (2025-10-12) | Security Guild + DevOps | Define revocation JSON schema (`revocation_bundle.schema.json`) and detached JWS workflow. | CORE9, OPS3 | ✅ Schema + sample committed; ✅ CLI command `stellaops auth revoke export` scaffolded with acceptance tests; ✅ Verification script + docs. | -| SEC4.B | DONE (2025-10-12) | Security Guild | Integrate signing keys with crypto provider abstraction (initially ES256 via BCL). | SEC4.A, D5 | ✅ `ICryptoProvider.GetSigner` stub + default BCL signer; ✅ Unit tests verifying signature roundtrip. | -| SEC5.A | DONE (2025-10-12) | Security Guild | Author STRIDE threat model (`docs/security/authority-threat-model.md`) covering token, bootstrap, revocation, CLI, plugin surfaces. | All SEC1–SEC4 in progress | ✅ DFDs + trust boundaries drawn; ✅ Risk table with owners/actions; ✅ Follow-up backlog issues created. | -| SEC5.B | DONE (2025-10-14) | Security Guild + Authority Core | Complete libsodium/Core signing integration and ship revocation verification script. | SEC4.A, SEC4.B, SEC4.HOST | ✅ libsodium/Core signing provider wired; ✅ `stellaops auth revoke verify` script published; ✅ Revocation docs updated with verification workflow. | -| SEC5.B1 | DONE (2025-10-14) | Security Guild + Authority Core | Introduce `LibsodiumCryptoProvider` implementing ECDSA signing/verification via libsodium, register under feature flag, and validate against existing ES256 fixtures. | SEC5.B | ✅ Provider resolves via `ICryptoProviderRegistry`; ✅ Integration tests cover sign/verify parity with default provider; ✅ Fallback to managed provider documented. | -| SEC5.B2 | DONE (2025-10-14) | Security Guild + DevEx/CLI | Extend `stellaops auth revoke verify` to detect provider metadata, reuse registry for verification, and document CLI workflow. | SEC5.B | ✅ CLI uses registry signers for verification; ✅ End-to-end test invokes verify against sample bundle; ✅ docs/11_AUTHORITY.md references CLI procedure. | -| SEC5.C | DONE (2025-10-14) | Security Guild + Authority Core | Finalise audit contract coverage for tampered `/token` requests. | SEC2.A, SEC2.B | ✅ Tamper attempts logged with correlationId/PII tags; ✅ SOC runbook updated; ✅ Threat model status reviewed. | -| SEC5.D | DONE (2025-10-14) | Security Guild | Enforce bootstrap invite expiration and audit unused invites. | SEC5.A | ✅ Bootstrap tokens auto-expire; ✅ Audit entries emitted for expiration/reuse attempts; ✅ Operator docs updated. | -> Remark (2025-10-14): Cleanup service wired to store; background sweep + invite audit tests added. -| SEC5.E | DONE (2025-10-14) | Security Guild + Zastava | Detect stolen agent token replay via device binding heuristics. | SEC4.A | ✅ Device binding guidance published; ✅ Alerting pipeline raises stale revocation acknowledgements; ✅ Tests cover replay detection. | -> Remark (2025-10-14): Token usage metadata persisted with replay audits + handler/unit coverage. -| SEC5.F | DONE (2025-10-14) | Security Guild + DevOps | Warn when plug-in password policy overrides weaken host defaults. | SEC1.A, PLG3 | ✅ Static analyser flags weaker overrides; ✅ Runtime warning surfaced; ✅ Docs call out mitigation. | -> Remark (2025-10-14): Analyzer surfaces warnings during CLI load; docs updated with mitigation steps. -| SEC5.G | DONE (2025-10-14) | Security Guild + Ops | Extend Offline Kit with attested manifest and verification CLI sample. | OPS3 | ✅ Offline Kit build signs manifest with detached JWS; ✅ Verification CLI documented; ✅ Supply-chain attestation recorded. | -> Remark (2025-10-14): Offline kit docs include manifest verification workflow; attestation artifacts referenced. -| SEC5.H | DONE (2025-10-13) | Security Guild + Authority Core | Ensure `/token` denials persist audit records with correlation IDs. | SEC2.A, SEC2.B | ✅ Audit store captures denials; ✅ Tests cover success/failure/lockout; ✅ Threat model review updated. | -| D5.A | DONE (2025-10-12) | Security Guild | Flesh out `StellaOps.Cryptography` provider registry, policy, and DI helpers enabling sovereign crypto selection. | SEC1.A, SEC4.B | ✅ `ICryptoProviderRegistry` implementation with provider selection rules; ✅ `StellaOps.Cryptography.DependencyInjection` extensions; ✅ Tests covering fallback ordering. | -| SEC6.A | DONE (2025-10-19) | Security Guild | Ship BouncyCastle-backed Ed25519 signing as a `StellaOps.Cryptography` plug-in and migrate Scanner WebService signing to consume the provider registry; codify the plug-in rule in AGENTS.
2025-10-19: Added `StellaOps.Cryptography.Plugin.BouncyCastle`, updated DI and ReportSigner, captured provider tests (`BouncyCastleEd25519CryptoProviderTests`). | D5.A | ✅ Plug-in registered via DI (`AddStellaOpsCrypto` + `AddBouncyCastleEd25519Provider`); ✅ Report signer resolves keys through registry; ✅ Unit tests cover Ed25519 sign/verify via provider. | - -> Remark (2025-10-13, SEC2.B): Coordinated with Authority Core — audit sinks now receive `/token` success/failure events; awaiting host test suite once signing fixture lands. -> -> Remark (2025-10-13, SEC3.B): Pinged Docs & Plugin guilds — rate limit guidance published in `docs/security/rate-limits.md` and flagged for PLG6.DOC copy lift. -> -> Remark (2025-10-13, SEC5.B): Split follow-up into SEC5.B1 (libsodium provider) and SEC5.B2 (CLI verification) after scoping registry integration; work not yet started. - -> Remark (2025-10-13, SEC2.B): Coordinated with Authority Core — audit sinks now receive `/token` success/failure events; awaiting host test suite once signing fixture lands. -> -> Remark (2025-10-13, SEC3.B): Pinged Docs & Plugin guilds — rate limit guidance published in `docs/security/rate-limits.md` and flagged for PLG6.DOC copy lift. -> -> Remark (2025-10-13, SEC5.B): Split follow-up into SEC5.B1 (libsodium provider) and SEC5.B2 (CLI verification) after scoping registry integration; work not yet started. - -## Notes -- Target Argon2 parameters follow OWASP Cheat Sheet (memory ≈ 19 MiB, iterations 2, parallelism 1). Allow overrides via configuration. -- When CORE8 lands, pair with Team 2 to expose request context information required by the rate limiter (client_id enrichment). -- Revocation bundle must be consumable offline; include issue timestamp, signing key metadata, and reasons. -- All crypto usage in Authority code should funnel through the new abstractions (`ICryptoProvider`), enabling future CryptoPro/OpenSSL providers. - -## Done Definition -- Code merges include unit/integration tests and documentation updates. -- `TASKS.md` status transitions (TODO → DOING → DONE/BLOCKED) must happen in the same PR as the work. -- Prior to marking DONE: run `dotnet test` for touched solutions and attach excerpt to PR description. +# Team 8 — Security Guild Task Board (UTC 2025-10-10) + +| ID | Status | Owner | Description | Dependencies | Exit Criteria | +|----|--------|-------|-------------|--------------|---------------| +> Remark (2025-10-14): Cleanup service wired to store; background sweep + invite audit tests added. +> Remark (2025-10-14): Token usage metadata persisted with replay audits + handler/unit coverage. +> Remark (2025-10-14): Analyzer surfaces warnings during CLI load; docs updated with mitigation steps. +> Remark (2025-10-14): Offline kit docs include manifest verification workflow; attestation artifacts referenced. + +> Remark (2025-10-13, SEC2.B): Coordinated with Authority Core — audit sinks now receive `/token` success/failure events; awaiting host test suite once signing fixture lands. +> +> Remark (2025-10-13, SEC3.B): Pinged Docs & Plugin guilds — rate limit guidance published in `docs/security/rate-limits.md` and flagged for PLG6.DOC copy lift. +> +> Remark (2025-10-13, SEC5.B): Split follow-up into SEC5.B1 (libsodium provider) and SEC5.B2 (CLI verification) after scoping registry integration; work not yet started. + +> Remark (2025-10-13, SEC2.B): Coordinated with Authority Core — audit sinks now receive `/token` success/failure events; awaiting host test suite once signing fixture lands. +> +> Remark (2025-10-13, SEC3.B): Pinged Docs & Plugin guilds — rate limit guidance published in `docs/security/rate-limits.md` and flagged for PLG6.DOC copy lift. +> +> Remark (2025-10-13, SEC5.B): Split follow-up into SEC5.B1 (libsodium provider) and SEC5.B2 (CLI verification) after scoping registry integration; work not yet started. + +## Notes +- Target Argon2 parameters follow OWASP Cheat Sheet (memory ≈ 19 MiB, iterations 2, parallelism 1). Allow overrides via configuration. +- When CORE8 lands, pair with Team 2 to expose request context information required by the rate limiter (client_id enrichment). +- Revocation bundle must be consumable offline; include issue timestamp, signing key metadata, and reasons. +- All crypto usage in Authority code should funnel through the new abstractions (`ICryptoProvider`), enabling future CryptoPro/OpenSSL providers. + +## Done Definition +- Code merges include unit/integration tests and documentation updates. +- `TASKS.md` status transitions (TODO → DOING → DONE/BLOCKED) must happen in the same PR as the work. +- Prior to marking DONE: run `dotnet test` for touched solutions and attach excerpt to PR description. diff --git a/src/__Libraries/StellaOps.Plugin/TASKS.completed.md b/src/__Libraries/StellaOps.Plugin/TASKS.completed.md new file mode 100644 index 00000000..9e77872e --- /dev/null +++ b/src/__Libraries/StellaOps.Plugin/TASKS.completed.md @@ -0,0 +1,14 @@ +# Completed Tasks + +|PLUGIN-DI-08-001 Scoped service support in plugin bootstrap|Plugin Platform Guild (DONE 2025-10-21)|StellaOps.DependencyInjection|Scoped DI metadata primitives landed; dynamic plugin integration tests now verify `RegisterPluginRoutines` honours `[ServiceBinding]` lifetimes and remains idempotent.| + +|PLUGIN-DI-08-002.COORD Authority scoped-service handshake|Plugin Platform Guild, Authority Core (DONE 2025-10-20)|PLUGIN-DI-08-001|Workshop held 2025-10-20 15:00–16:05 UTC; outcomes/notes captured in `docs/dev/authority-plugin-di-coordination.md`, follow-up action items assigned for PLUGIN-DI-08-002 implementation plan.| + +|PLUGIN-DI-08-002 Authority plugin integration updates|Plugin Platform Guild, Authority Core (DONE 2025-10-20)|PLUGIN-DI-08-001, PLUGIN-DI-08-002.COORD|Standard registrar now registers scoped credential/provisioning stores + identity-provider plugins, registry Acquire scopes instances, and regression suites (`dotnet test src/Authority/StellaOps.Authority/StellaOps.Authority.Plugin.Standard.Tests/StellaOps.Authority.Plugin.Standard.Tests.csproj`, `dotnet test src/Authority/StellaOps.Authority/StellaOps.Authority.Tests/StellaOps.Authority.Tests.csproj`) cover scoped lifetimes + handles.| + +|PLUGIN-DI-08-003 Authority registry scoped resolution|Plugin Platform Guild, Authority Core (DONE 2025-10-20)|PLUGIN-DI-08-002.COORD|Reworked `IAuthorityIdentityProviderRegistry` to expose metadata + scoped handles, updated OpenIddict flows/Program health endpoints, and added coverage via `AuthorityIdentityProviderRegistryTests`.| + +|PLUGIN-DI-08-004 Authority plugin loader DI bridge|Plugin Platform Guild, Authority Core (DONE 2025-10-20)|PLUGIN-DI-08-002.COORD|Authority plugin loader now activates registrars via scoped DI leases, registers `[ServiceBinding]` metadata, and includes regression coverage in `AuthorityPluginLoaderTests`.| + +|PLUGIN-DI-08-005 Authority plugin bootstrap scope pattern|Plugin Platform Guild, Authority Core (DONE 2025-10-20)|PLUGIN-DI-08-002.COORD|Standard bootstrapper uses `IServiceScopeFactory` per run; tests updated to validate scoped execution and documentation annotated in `authority-plugin-di-coordination.md`.| + diff --git a/src/__Libraries/StellaOps.Plugin/TASKS.md b/src/__Libraries/StellaOps.Plugin/TASKS.md index 603d4ec9..2265b2fc 100644 --- a/src/__Libraries/StellaOps.Plugin/TASKS.md +++ b/src/__Libraries/StellaOps.Plugin/TASKS.md @@ -1,9 +1,3 @@ # TASKS | Task | Owner(s) | Depends on | Notes | |---|---|---|---| -|PLUGIN-DI-08-001 Scoped service support in plugin bootstrap|Plugin Platform Guild (DONE 2025-10-21)|StellaOps.DependencyInjection|Scoped DI metadata primitives landed; dynamic plugin integration tests now verify `RegisterPluginRoutines` honours `[ServiceBinding]` lifetimes and remains idempotent.| -|PLUGIN-DI-08-002.COORD Authority scoped-service handshake|Plugin Platform Guild, Authority Core (DONE 2025-10-20)|PLUGIN-DI-08-001|Workshop held 2025-10-20 15:00–16:05 UTC; outcomes/notes captured in `docs/dev/authority-plugin-di-coordination.md`, follow-up action items assigned for PLUGIN-DI-08-002 implementation plan.| -|PLUGIN-DI-08-002 Authority plugin integration updates|Plugin Platform Guild, Authority Core (DONE 2025-10-20)|PLUGIN-DI-08-001, PLUGIN-DI-08-002.COORD|Standard registrar now registers scoped credential/provisioning stores + identity-provider plugins, registry Acquire scopes instances, and regression suites (`dotnet test src/Authority/StellaOps.Authority/StellaOps.Authority.Plugin.Standard.Tests/StellaOps.Authority.Plugin.Standard.Tests.csproj`, `dotnet test src/Authority/StellaOps.Authority/StellaOps.Authority.Tests/StellaOps.Authority.Tests.csproj`) cover scoped lifetimes + handles.| -|PLUGIN-DI-08-003 Authority registry scoped resolution|Plugin Platform Guild, Authority Core (DONE 2025-10-20)|PLUGIN-DI-08-002.COORD|Reworked `IAuthorityIdentityProviderRegistry` to expose metadata + scoped handles, updated OpenIddict flows/Program health endpoints, and added coverage via `AuthorityIdentityProviderRegistryTests`.| -|PLUGIN-DI-08-004 Authority plugin loader DI bridge|Plugin Platform Guild, Authority Core (DONE 2025-10-20)|PLUGIN-DI-08-002.COORD|Authority plugin loader now activates registrars via scoped DI leases, registers `[ServiceBinding]` metadata, and includes regression coverage in `AuthorityPluginLoaderTests`.| -|PLUGIN-DI-08-005 Authority plugin bootstrap scope pattern|Plugin Platform Guild, Authority Core (DONE 2025-10-20)|PLUGIN-DI-08-002.COORD|Standard bootstrapper uses `IServiceScopeFactory` per run; tests updated to validate scoped execution and documentation annotated in `authority-plugin-di-coordination.md`.|