wip: doctor/cli/docs/api to vector db consolidation; api hardening for descriptions, tenant, and scopes; migrations and conversions of all DALs to EF v10
This commit is contained in:
@@ -2,6 +2,22 @@
|
||||
|
||||
This directory contains architecture documentation for all StellaOps modules.
|
||||
|
||||
---
|
||||
|
||||
## Platform Statistics
|
||||
|
||||
| Metric | Count |
|
||||
|--------|-------|
|
||||
| Source modules | 63 |
|
||||
| Documented modules | 79 |
|
||||
| Runnable services (Program.cs) | 47 |
|
||||
| Modules with workers | 19 |
|
||||
| PostgreSQL databases | 30 |
|
||||
| SQL migration files | ~180 |
|
||||
| Total .csproj files | 1,105 |
|
||||
|
||||
---
|
||||
|
||||
## Module Categories
|
||||
|
||||
### Core Platform
|
||||
@@ -10,7 +26,7 @@ This directory contains architecture documentation for all StellaOps modules.
|
||||
|--------|------|-------------|
|
||||
| [Authority](./authority/) | `src/Authority/` | Authentication, authorization, OAuth/OIDC, DPoP |
|
||||
| [Gateway](./gateway/) | `src/Gateway/` | API gateway with routing and transport abstraction |
|
||||
| [Router](./router/) | `src/Router/` | Transport-agnostic messaging (TCP/TLS/UDP/RabbitMQ/Valkey) |
|
||||
| [Router](./router/) | `src/Router/` | Transport-agnostic messaging (TCP/TLS/UDP/RabbitMQ/Valkey). Note: also contains a `StellaOps.Gateway.WebService` for binary protocol bridging, separate from `src/Gateway/`. |
|
||||
| [Platform](./platform/) | `src/Platform/` | Platform architecture and Platform Service aggregation APIs |
|
||||
|
||||
### Data Ingestion
|
||||
@@ -45,7 +61,7 @@ This directory contains architecture documentation for all StellaOps modules.
|
||||
| [EvidenceLocker](./evidence-locker/) | `src/EvidenceLocker/` | Sealed evidence storage and export |
|
||||
| [ExportCenter](./export-center/) | `src/ExportCenter/` | Batch export and report generation |
|
||||
| [Provenance](./provenance/) | `src/Provenance/` | SLSA/DSSE attestation tooling |
|
||||
| [Provcache](./prov-cache/) | Library | Provenance cache utilities |
|
||||
| [Provcache](./prov-cache/) | Library | Production provenance cache shared library family |
|
||||
|
||||
### Policy & Risk
|
||||
|
||||
@@ -55,21 +71,32 @@ This directory contains architecture documentation for all StellaOps modules.
|
||||
| [RiskEngine](./risk-engine/) | `src/RiskEngine/` | Risk scoring runtime |
|
||||
| [VulnExplorer](./vuln-explorer/) | `src/VulnExplorer/` | Vulnerability exploration and triage |
|
||||
| [Unknowns](./unknowns/) | `src/Unknowns/` | Unknown component tracking registry |
|
||||
| [Findings](./findings-ledger/) | `src/Findings/` | Centralized findings aggregation and evidence graphs |
|
||||
|
||||
### Operations
|
||||
### Release & Orchestration
|
||||
|
||||
| Module | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| [Scheduler](./scheduler/) | `src/Scheduler/` | Job scheduling and queue management |
|
||||
| [ReleaseOrchestrator](./release-orchestrator/) | `src/ReleaseOrchestrator/` | Central release control plane (active development) |
|
||||
| [Orchestrator](./orchestrator/) | `src/Orchestrator/` | Workflow orchestration and task coordination |
|
||||
| [Scheduler](./scheduler/) | `src/Scheduler/` | Job scheduling and queue management |
|
||||
| [TaskRunner](./taskrunner/) | `src/TaskRunner/` | Task pack execution engine |
|
||||
| [PacksRegistry](./packs-registry/) | `src/PacksRegistry/` | Task packs registry |
|
||||
| [Remediation](./remediation/) | `src/Remediation/` | Fix template marketplace for CVE remediation |
|
||||
|
||||
### Operations & Observability
|
||||
|
||||
| Module | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| [Doctor](./doctor/) | `src/Doctor/` | Diagnostic framework for system health validation |
|
||||
| [Notify](./notify/) | `src/Notify/` | Notification toolkit (Email, Slack, Teams, Webhooks) |
|
||||
| [Notifier](./notifier/) | `src/Notifier/` | Notifications Studio host |
|
||||
| [PacksRegistry](./packs-registry/) | `src/PacksRegistry/` | Task packs registry |
|
||||
| [OpsMemory](./opsmemory/) | `src/OpsMemory/` | Decision ledger with similarity-based suggestions |
|
||||
| [Timeline](./timeline/) | `src/Timeline/` | Timeline query service for event browsing |
|
||||
| [TimelineIndexer](./timeline-indexer/) | `src/TimelineIndexer/` | Timeline event indexing |
|
||||
| [Replay](./replay/) | `src/Replay/` | Deterministic replay engine |
|
||||
|
||||
### Integration
|
||||
### Integration & Clients
|
||||
|
||||
| Module | Path | Description |
|
||||
|--------|------|-------------|
|
||||
@@ -78,17 +105,26 @@ This directory contains architecture documentation for all StellaOps modules.
|
||||
| [Web/UI](./ui/) | `src/Web/` | Angular 21 frontend SPA |
|
||||
| [API](./api/) | `src/Api/` | OpenAPI contracts and governance |
|
||||
| [Registry](./registry/) | `src/Registry/` | Container registry integration |
|
||||
| [Integrations](./integrations/) | `src/Integrations/` | Integration hub for external systems (SCM, CI, registries, secrets) |
|
||||
| [Extensions](./extensions/) | `src/Extensions/` | IDE extensions for JetBrains and VS Code |
|
||||
| [Sdk](./sdk/) | `src/Sdk/` | Client SDK generator and release SDK |
|
||||
| [DevPortal](./devportal/) | `src/DevPortal/` | Developer portal static site |
|
||||
|
||||
### Infrastructure
|
||||
### Infrastructure & Libraries
|
||||
|
||||
| Module | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| [Cryptography](./cryptography/) | `src/Cryptography/` | Crypto plugins (FIPS, eIDAS, GOST, SM, PQ) |
|
||||
| [SmRemote](./sm-remote/) | `src/SmRemote/` | Remote SM2/SM3/SM4 cryptographic operations |
|
||||
| [Telemetry](./telemetry/) | `src/Telemetry/` | OpenTelemetry traces, metrics, logging |
|
||||
| [Graph](./graph/) | `src/Graph/` | Call graph and reachability data structures |
|
||||
| [Signals](./signals/) | `src/Signals/` | Runtime signal collection and correlation |
|
||||
| [AirGap](./airgap/) | `src/AirGap/` | Air-gapped deployment support |
|
||||
| [AOC](./aoc/) | `src/Aoc/` | Append-Only Contract enforcement |
|
||||
| [Plugin](./plugin/) | `src/Plugin/` | Plugin SDK, registry, sandbox, and host framework |
|
||||
| [RuntimeInstrumentation](./runtime-instrumentation/) | `src/RuntimeInstrumentation/` | Tetragon-based eBPF runtime instrumentation |
|
||||
| [Cartographer](./cartographer/) | `src/Cartographer/` | Infrastructure topology discovery |
|
||||
| [Facet](./facet/) | Library | Production cross-module faceting library (Scanner + Policy) |
|
||||
|
||||
### Testing & Benchmarks
|
||||
|
||||
@@ -96,16 +132,830 @@ This directory contains architecture documentation for all StellaOps modules.
|
||||
|--------|------|-------------|
|
||||
| [Benchmark](./benchmark/) | Scanner library | Competitive benchmarking (accuracy comparison) |
|
||||
| [Bench](./bench/) | `src/Bench/` | Performance benchmarks |
|
||||
| [Tools](./tools/) | `src/Tools/` | Developer utility tools (fixtures, golden pairs, smoke tests) |
|
||||
| [Verifier](./verifier/) | `src/Verifier/` | Standalone evidence bundle verification CLI |
|
||||
|
||||
### Cross-Cutting Concepts
|
||||
|
||||
| Folder | Purpose |
|
||||
|--------|---------|
|
||||
| [Analytics](./analytics/) | Analytics capabilities (embedded in Platform) |
|
||||
| [Evidence](./evidence/) | Unified evidence model specification |
|
||||
| [Eventing](./eventing/) | Event envelope schemas and libraries |
|
||||
| [Snapshot](./snapshot/) | Knowledge snapshot and replay concepts |
|
||||
| [Triage](./triage/) | Vulnerability triage workflows |
|
||||
| [DevOps](./devops/) | DevOps and CI/CD infrastructure |
|
||||
| [CI](./ci/) | CI pipeline documentation |
|
||||
| [Reachability](./reachability/) | Reachability concepts (split between ReachGraph and Scanner) |
|
||||
| [SARIF Export](./sarif-export/) | SARIF export format (capability within ExportCenter) |
|
||||
|
||||
---
|
||||
|
||||
## Module Catalog
|
||||
|
||||
### AdvisoryAI
|
||||
- **Source**: `src/AdvisoryAI/`
|
||||
- **Docs**: [`docs/modules/advisory-ai/`](./advisory-ai/)
|
||||
- **Type**: Service
|
||||
- **Database**: PostgreSQL (2 SQL migrations: chat audit, knowledge search)
|
||||
- **Endpoints**: 7 (attestation, chat, evidence pack, knowledge search, LLM adapter, run, companion explain)
|
||||
|
||||
AI-powered advisory summarization and knowledge search service using LLM inference to explain vulnerability advisories, VEX evidence, and remediation guidance. Uses retrieval-augmented generation (RAG) over advisories, VEX, and platform documentation with configurable LLM backends (Claude, OpenAI, Ollama, Gemini). Includes chat audit logging, doctor-search controls, and guardrails for determinism and offline operation.
|
||||
|
||||
**Dependencies**: Concelier, VEX Lens / Excititor, Policy Engine, SBOM Service, LLM providers.
|
||||
|
||||
---
|
||||
|
||||
### AirGap
|
||||
- **Source**: `src/AirGap/`
|
||||
- **Docs**: [`docs/modules/airgap/`](./airgap/)
|
||||
- **Type**: Service
|
||||
- **Database**: PostgreSQL (1 SQL migration)
|
||||
- **Endpoints**: 1 (AirGapEndpoints)
|
||||
|
||||
Air-gapped deployment controller, importer, and cryptographic time anchor enabling StellaOps operation in fully disconnected environments. Handles bundle import/export, trust verification, and sealed-mode enforcement. Time service provides cryptographic time anchors using Roughtime or RFC3161.
|
||||
|
||||
**Dependencies**: Authority, Concelier / Excititor, Signer / Attestor.
|
||||
|
||||
---
|
||||
|
||||
### Aoc
|
||||
- **Source**: `src/Aoc/`
|
||||
- **Docs**: [`docs/modules/aoc/`](./aoc/)
|
||||
- **Type**: CLI / Library
|
||||
- **Database**: None
|
||||
- **Endpoints**: None
|
||||
|
||||
CLI tool and Roslyn analyzers enforcing the Aggregation-Only Contract (AOC) across the platform. Validates that ingestion services preserve raw upstream data without modification, maintain full provenance chains, and never merge conflicting observations.
|
||||
|
||||
**Dependencies**: None (standalone validation tool).
|
||||
|
||||
---
|
||||
|
||||
### Api
|
||||
- **Source**: `src/Api/`
|
||||
- **Docs**: [`docs/modules/api/`](./api/)
|
||||
- **Type**: Library
|
||||
- **Database**: None
|
||||
- **Endpoints**: None
|
||||
|
||||
Shared API contracts, governance rules, and OpenAPI generation libraries used across all StellaOps services. Includes `StellaOps.Api` (core contracts), `StellaOps.Api.Governance` (validators), and `StellaOps.Api.OpenApi` (specification generation).
|
||||
|
||||
**Dependencies**: None (consumed as a library).
|
||||
|
||||
---
|
||||
|
||||
### Attestor
|
||||
- **Source**: `src/Attestor/`
|
||||
- **Docs**: [`docs/modules/attestor/`](./attestor/)
|
||||
- **Type**: Service
|
||||
- **Database**: PostgreSQL (`ProofChainDbContext`, 10 SQL migrations)
|
||||
- **Endpoints**: 6 (verdict, predicate registry, core attestor, watchlist, and more)
|
||||
|
||||
Transparency-logged attestation service that submits DSSE envelopes to Rekor v2, retrieves inclusion proofs, and exposes verification APIs. Accepts DSSE only from the Signer over mTLS and enforces chain-of-trust to Stella Ops roots. Includes predicate type registry, identity watchlist, and trust verdict storage.
|
||||
|
||||
**Dependencies**: Signer, Rekor v2, Authority.
|
||||
|
||||
---
|
||||
|
||||
### Authority
|
||||
- **Source**: `src/Authority/`
|
||||
- **Docs**: [`docs/modules/authority/`](./authority/)
|
||||
- **Type**: Service
|
||||
- **Database**: PostgreSQL (7 SQL migrations)
|
||||
- **Endpoints**: 4+ (authorize, token, JWKS, discovery)
|
||||
|
||||
On-premises OIDC/OAuth2 identity service issuing short-lived, sender-constrained operational tokens (OpToks) with DPoP and mTLS binding. Supports RBAC scopes, multi-tenant claims with header-based tenant selection, deterministic validation, console admin endpoints, and LDAP/Standard client provisioning plugins.
|
||||
|
||||
**Dependencies**: Valkey (DPoP nonce store, token caching).
|
||||
|
||||
---
|
||||
|
||||
### Bench
|
||||
- **Source**: `src/Bench/`
|
||||
- **Docs**: [`docs/modules/bench/`](./bench/)
|
||||
- **Type**: Tool
|
||||
- **Database**: None
|
||||
- **Endpoints**: None
|
||||
|
||||
Performance benchmark harnesses (BenchmarkDotNet) for critical platform subsystems including Link-Not-Merge, VEX, Notify, Policy Engine, and Scanner analyzers. Results establish performance baselines and detect regressions.
|
||||
|
||||
**Dependencies**: None (standalone benchmarks).
|
||||
|
||||
---
|
||||
|
||||
### BinaryIndex
|
||||
- **Source**: `src/BinaryIndex/`
|
||||
- **Docs**: [`docs/modules/binary-index/`](./binary-index/)
|
||||
- **Type**: Service
|
||||
- **Database**: PostgreSQL (`BinaryIndexDbContext`, 5 SQL migrations)
|
||||
- **Endpoints**: Defined in WebService Program.cs
|
||||
|
||||
Vulnerable binaries database enabling detection of vulnerable code at the binary level, independent of package metadata, using fingerprint claims and delta signatures. Addresses unreliable package version strings (backports, custom builds, stripped metadata).
|
||||
|
||||
**Dependencies**: Scanner, Concelier.
|
||||
|
||||
---
|
||||
|
||||
### Cartographer
|
||||
- **Source**: `src/Cartographer/`
|
||||
- **Docs**: [`docs/modules/cartographer/`](./cartographer/)
|
||||
- **Type**: Service
|
||||
- **Database**: None
|
||||
- **Endpoints**: Defined in Program.cs
|
||||
|
||||
Infrastructure topology discovery and service mapping for container environments. Produces SBOM snapshots and topology graphs consumed by the Graph Indexer. Environment topology and promotion lanes are now owned by the Release Orchestrator.
|
||||
|
||||
**Dependencies**: Graph, Scanner.
|
||||
|
||||
---
|
||||
|
||||
### CLI
|
||||
- **Source**: `src/Cli/`
|
||||
- **Docs**: [`docs/modules/cli/`](./cli/)
|
||||
- **Type**: CLI
|
||||
- **Database**: None
|
||||
- **Endpoints**: None
|
||||
|
||||
Command-line interface for driving StellaOps workflows including scanning, policy operations, VEX/vulnerability data pulls, verification, offline kit administration, knowledge search, and migration management. Supports build-time SBOM generation via Buildx orchestration and authenticates via Authority DPoP tokens.
|
||||
|
||||
**Dependencies**: Scanner.WebService, Authority, AdvisoryAI, Policy Engine, Concelier / Excititor, Signer / Attestor, Platform.
|
||||
|
||||
---
|
||||
|
||||
### Concelier
|
||||
- **Source**: `src/Concelier/`
|
||||
- **Docs**: [`docs/modules/concelier/`](./concelier/)
|
||||
- **Type**: Service
|
||||
- **Database**: PostgreSQL (24 SQL migrations)
|
||||
- **Endpoints**: 9 (advisory source, air-gap, canonical advisory, federation, feed mirror, feed snapshot, interest score, mirror, SBOM)
|
||||
|
||||
Advisory ingestion and Link-Not-Merge (LNM) observation pipeline producing deterministic raw observations, correlation linksets, and evidence events. Acquires authoritative vulnerability advisories from vendor PSIRTs, distros, OSS ecosystems, and CERTs, persisting them as immutable observations under AOC.
|
||||
|
||||
**Dependencies**: Policy Engine, Excititor, Graph, AirGap, Feedser.
|
||||
|
||||
---
|
||||
|
||||
### Cryptography
|
||||
- **Source**: `src/Cryptography/`
|
||||
- **Docs**: [`docs/modules/cryptography/`](./cryptography/)
|
||||
- **Type**: Library
|
||||
- **Database**: None
|
||||
- **Endpoints**: None
|
||||
|
||||
Pluggable cryptographic primitives supporting regional standards (eIDAS, FIPS, GOST, SM) with algorithm-agile signing operations. Plugins include HSM integration, ECDSA, and EdDSA signing profiles. All operations are deterministic and support offline operation.
|
||||
|
||||
**Dependencies**: None (consumed as a library by Signer, Attestor, and other services).
|
||||
|
||||
---
|
||||
|
||||
### DevPortal
|
||||
- **Source**: `src/DevPortal/`
|
||||
- **Docs**: [`docs/modules/devportal/`](./devportal/)
|
||||
- **Type**: Static Site
|
||||
- **Database**: None
|
||||
- **Endpoints**: None
|
||||
|
||||
Developer portal static site providing API documentation, integration guides, SDK references, and getting-started tutorials. Aggregates OpenAPI specifications from all services for third-party developers and integrators.
|
||||
|
||||
**Dependencies**: None (static site).
|
||||
|
||||
---
|
||||
|
||||
### Doctor
|
||||
- **Source**: `src/Doctor/`
|
||||
- **Docs**: [`docs/modules/doctor/`](./doctor/)
|
||||
- **Type**: Service
|
||||
- **Database**: None (uses Valkey for caching)
|
||||
- **Endpoints**: 3 (doctor, scheduler, timestamping)
|
||||
|
||||
Diagnostic framework for validating system health, configuration, integration connectivity, and OCI registry compatibility. Plugin-based architecture with binary analysis, notification validation, and observability probing. Integrates with AdvisoryAI knowledge search for diagnostic assistance.
|
||||
|
||||
**Dependencies**: All platform services (health check targets), external registries, AdvisoryAI.
|
||||
|
||||
---
|
||||
|
||||
### EvidenceLocker
|
||||
- **Source**: `src/EvidenceLocker/`
|
||||
- **Docs**: [`docs/modules/evidence-locker/`](./evidence-locker/)
|
||||
- **Type**: Service
|
||||
- **Database**: PostgreSQL (`EvidenceDbContext`, 5 SQL migrations)
|
||||
- **Endpoints**: 4 (evidence audit, evidence thread, export, verdict)
|
||||
|
||||
Tamper-proof, immutable evidence storage for vulnerability scan evidence, audit logs, and compliance artifacts with cryptographic sealing. Evidence is content-addressable. Once sealed, evidence cannot be modified. Supports threads, verdicts, bundle packaging, and portable bundles for offline compliance audits.
|
||||
|
||||
**Dependencies**: Signer, Attestor, Authority, object storage.
|
||||
|
||||
---
|
||||
|
||||
### Excititor
|
||||
- **Source**: `src/Excititor/`
|
||||
- **Docs**: [`docs/modules/excititor/`](./excititor/)
|
||||
- **Type**: Service
|
||||
- **Database**: PostgreSQL (10 SQL migrations)
|
||||
- **Endpoints**: 11 (attestation, evidence, ingest, linkset, mirror, mirror registration, observation, policy, Rekor attestation, resolve, risk feed)
|
||||
|
||||
VEX ingestion and consensus pipeline converting heterogeneous VEX statements (OpenVEX, CSAF VEX, CycloneDX VEX) into immutable observations with provenance-preserving linksets. Does not decide PASS/FAIL; supplies evidence with statuses, justifications, and provenance weights. Conflicting observations are preserved unchanged.
|
||||
|
||||
**Dependencies**: Policy Engine, Concelier, Attestor / Signer, Graph.
|
||||
|
||||
---
|
||||
|
||||
### ExportCenter
|
||||
- **Source**: `src/ExportCenter/`
|
||||
- **Docs**: [`docs/modules/export-center/`](./export-center/)
|
||||
- **Type**: Service
|
||||
- **Database**: PostgreSQL (1 SQL migration)
|
||||
- **Endpoints**: 15 (profile management, export runs, distribution, status, download)
|
||||
|
||||
Evidence and policy overlay packaging service producing reproducible, deterministic export bundles in multiple formats (JSON, SARIF, offline kit). Enforces AOC guardrails and produces deterministic manifests with optional signing and distribution to OCI registries or object storage.
|
||||
|
||||
**Dependencies**: Findings Ledger, Policy Engine, Orchestrator, Authority, Signer, object storage.
|
||||
|
||||
---
|
||||
|
||||
### Extensions
|
||||
- **Source**: `src/Extensions/`
|
||||
- **Docs**: [`docs/modules/extensions/`](./extensions/)
|
||||
- **Type**: IDE Extensions
|
||||
- **Database**: None
|
||||
- **Endpoints**: None
|
||||
|
||||
IDE extensions for JetBrains IDEs and Visual Studio Code providing inline vulnerability information, policy status, and StellaOps workflow integration directly within the developer's editor environment.
|
||||
|
||||
**Dependencies**: Platform API.
|
||||
|
||||
---
|
||||
|
||||
### Feedser
|
||||
- **Source**: `src/Feedser/`
|
||||
- **Docs**: [`docs/modules/feedser/`](./feedser/)
|
||||
- **Type**: Library
|
||||
- **Database**: None
|
||||
- **Endpoints**: None
|
||||
|
||||
Evidence collection library for backport detection and binary fingerprinting supporting the four-tier backport proof system. Extracts patch signatures from unified diffs and binary fingerprints from compiled code. Consumed primarily by Concelier's ProofService layer. All outputs are deterministic with canonical JSON serialization.
|
||||
|
||||
**Dependencies**: None (consumed as a library by Concelier).
|
||||
|
||||
---
|
||||
|
||||
### Findings
|
||||
- **Source**: `src/Findings/`
|
||||
- **Docs**: [`docs/modules/findings-ledger/`](./findings-ledger/)
|
||||
- **Type**: Service
|
||||
- **Database**: PostgreSQL (via shared libraries)
|
||||
- **Endpoints**: 8 (backport, evidence graph, finding summary, reachability map, runtime timeline, runtime traces, scoring, webhook)
|
||||
|
||||
Centralized findings aggregation service providing backport tracking, evidence graphs, finding summaries, reachability maps, runtime timelines, scoring, and webhook notifications. Aggregates vulnerability findings from Scanner, Policy, Concelier, and Excititor into a unified query surface. Includes a LedgerReplayHarness tool for deterministic replay testing.
|
||||
|
||||
**Dependencies**: Scanner, Policy Engine, Concelier / Excititor, Attestor.
|
||||
|
||||
---
|
||||
|
||||
### Gateway
|
||||
- **Source**: `src/Gateway/`
|
||||
- **Docs**: [`docs/modules/gateway/`](./gateway/)
|
||||
- **Type**: Service
|
||||
- **Database**: None (stateless)
|
||||
- **Endpoints**: None (reverse proxy)
|
||||
|
||||
Single HTTP ingress point for all external traffic providing authentication, routing, OpenAPI aggregation, health monitoring, rate limiting, and tenant propagation. A separate `StellaOps.Gateway.WebService` also exists under `src/Router/` which serves as the transport-layer gateway for the Router's binary protocol.
|
||||
|
||||
**Dependencies**: Authority, Router, all microservices (proxied requests).
|
||||
|
||||
---
|
||||
|
||||
### Graph
|
||||
- **Source**: `src/Graph/`
|
||||
- **Docs**: [`docs/modules/graph/`](./graph/)
|
||||
- **Type**: Service
|
||||
- **Database**: PostgreSQL (1 SQL migration in Graph Indexer Persistence)
|
||||
- **Endpoints**: 3 (graph query, analytics, overlay)
|
||||
|
||||
SBOM dependency graph store with a rich node/edge model covering artifacts, SBOMs, components, advisories, VEX statements, and policy versions with reachability overlays. The Graph Indexer processes events from Cartographer/SBOM Service, Concelier/Excititor, and policy runs to maintain the graph.
|
||||
|
||||
**Dependencies**: Cartographer / SBOM Service, Concelier / Excititor, Policy Engine, Scanner.
|
||||
|
||||
---
|
||||
|
||||
### Integrations
|
||||
- **Source**: `src/Integrations/`
|
||||
- **Docs**: [`docs/modules/integrations/`](./integrations/)
|
||||
- **Type**: Service
|
||||
- **Database**: PostgreSQL (`IntegrationDbContext`, EF Core managed)
|
||||
- **Endpoints**: 2
|
||||
|
||||
Integration hub managing connections to external systems (SCM, CI, registries, secrets vaults) with health monitoring. Stores connection profiles, manages credential references, and monitors integration health. Plugins under `__Plugins` provide specific connector implementations.
|
||||
|
||||
**Dependencies**: Authority, external SCM/CI/Registry/Vault systems.
|
||||
|
||||
---
|
||||
|
||||
### IssuerDirectory
|
||||
- **Source**: `src/IssuerDirectory/`
|
||||
- **Docs**: [`docs/modules/issuer-directory/`](./issuer-directory/)
|
||||
- **Type**: Service
|
||||
- **Database**: PostgreSQL (1 SQL migration)
|
||||
- **Endpoints**: 3 (issuer, issuer key, issuer trust)
|
||||
|
||||
Centralized trusted VEX/CSAF publisher metadata registry enabling issuer identity resolution, key management, and trust weight assignment. Key lifecycle management validates Ed25519, X.509, and DSSE public keys with fingerprint deduplication. On startup, imports default CSAF publishers into the global tenant.
|
||||
|
||||
**Dependencies**: Authority.
|
||||
|
||||
---
|
||||
|
||||
### Mirror
|
||||
- **Source**: `src/Mirror/`
|
||||
- **Docs**: [`docs/modules/mirror/`](./mirror/)
|
||||
- **Type**: Library / Tool
|
||||
- **Database**: None
|
||||
- **Endpoints**: None
|
||||
|
||||
Vulnerability feed mirror and distribution utility for offline operation and reduced latency. Produces bundles for air-gapped distribution with cryptographic verification. Primary component is `StellaOps.Mirror.Creator` for assembling mirror bundles.
|
||||
|
||||
**Dependencies**: Upstream vulnerability feeds (NVD, OSV, GHSA), Concelier.
|
||||
|
||||
---
|
||||
|
||||
### Notifier
|
||||
- **Source**: `src/Notifier/`
|
||||
- **Docs**: [`docs/modules/notifier/`](./notifier/)
|
||||
- **Type**: Service
|
||||
- **Database**: None (uses Notify's persistence layer)
|
||||
- **Endpoints**: 14 (escalation, fallback, incident, localization, observability, operator override, quiet hours, rule, security, simulation, storm breaker, template, throttle, notify API)
|
||||
|
||||
Notifications Studio host application composing the Notify toolkit libraries into a full-featured notification management service. Provides 73+ routes covering escalation policies, fallback routing, incident tracking, localization, security controls, simulation testing, storm breaking (anti-spam), and more.
|
||||
|
||||
**Dependencies**: Notify libraries, Authority, Slack/Teams/Email/Webhook channels.
|
||||
|
||||
---
|
||||
|
||||
### Notify
|
||||
- **Source**: `src/Notify/`
|
||||
- **Docs**: [`docs/modules/notify/`](./notify/)
|
||||
- **Type**: Service / Library
|
||||
- **Database**: PostgreSQL (2 SQL migrations)
|
||||
- **Endpoints**: 2 (via WebService Program.cs)
|
||||
|
||||
Rules-driven, tenant-aware notification engine providing event consumption, operator-defined routing, channel-specific rendering, and reliable delivery with idempotency, throttling, and digests. Foundation library consumed by the Notifier host application. Secrets for channels are referenced, not stored raw.
|
||||
|
||||
**Dependencies**: Scanner / Scheduler / Excititor / Concelier / Attestor / Zastava (event sources), Slack/Teams/Email/Webhook, Authority.
|
||||
|
||||
---
|
||||
|
||||
### OpsMemory
|
||||
- **Source**: `src/OpsMemory/`
|
||||
- **Docs**: [`docs/modules/opsmemory/`](./opsmemory/)
|
||||
- **Type**: Service
|
||||
- **Database**: PostgreSQL (via shared infrastructure, schema managed programmatically)
|
||||
- **Endpoints**: 1 (OpsMemoryEndpoints)
|
||||
|
||||
Decision ledger capturing the lifecycle of security decisions with similarity-based suggestion retrieval for organizational learning. Uses similarity vectors to suggest relevant precedents for new situations. Deterministic with fixed similarity formulas, no randomness in ranking, and multi-tenant isolation.
|
||||
|
||||
**Dependencies**: AdvisoryAI, Authority.
|
||||
|
||||
---
|
||||
|
||||
### Orchestrator
|
||||
- **Source**: `src/Orchestrator/`
|
||||
- **Docs**: [`docs/modules/orchestrator/`](./orchestrator/)
|
||||
- **Type**: Service
|
||||
- **Database**: PostgreSQL (via shared infrastructure)
|
||||
- **Endpoints**: 25 (approvals, audit, circuit breakers, DAG, dead letter, export jobs, first signal, health, jobs, KPIs, ledger, OpenAPI, pack registry, pack runs, quotas, governance, release control v2, release dashboard, releases, runs, scale, SLOs, sources, streams, workers)
|
||||
|
||||
Source and job orchestration service managing job lifecycle, rate-limit governance, DAG execution, circuit breakers, and worker coordination. Applies quotas and rate limits per tenant/jobType, manages leasing to workers, handles completion tracking with retry policies, and supports replay. SDK bridges exist for Go and Python workers.
|
||||
|
||||
**Dependencies**: TaskRunner, Concelier / Excititor / Scheduler / ExportCenter / Policy (job producers), Valkey or NATS, Authority.
|
||||
|
||||
---
|
||||
|
||||
### PacksRegistry
|
||||
- **Source**: `src/PacksRegistry/`
|
||||
- **Docs**: [`docs/modules/packs-registry/`](./packs-registry/)
|
||||
- **Type**: Service
|
||||
- **Database**: PostgreSQL (`PacksRegistryDbContext`, EF Core managed)
|
||||
- **Endpoints**: Defined in WebService Program.cs
|
||||
|
||||
Centralized registry for distributable task packs, policy packs, and analyzer bundles with versioned management and integrity verification. All packs are content-addressed. Pack execution is handled by TaskRunner.
|
||||
|
||||
**Dependencies**: TaskRunner, object storage, Authority.
|
||||
|
||||
---
|
||||
|
||||
### Platform
|
||||
- **Source**: `src/Platform/`
|
||||
- **Docs**: [`docs/modules/platform/`](./platform/)
|
||||
- **Type**: Service
|
||||
- **Database**: PostgreSQL (57 SQL migrations, `stellaops` shared database)
|
||||
- **Endpoints**: 45+ (analytics, context, environment settings, evidence thread, federation telemetry, function map, integration read model, legacy alias, pack adapter, platform, policy interop, release control, release read model, score, security read model, topology read model, trust signing, and more)
|
||||
|
||||
Central platform service providing analytics, context management, environment settings, evidence threading, federation telemetry, function maps, integration health, release control, scoring, security, and topology read models. Largest module by migration count. Owns the migration consolidation framework (`MigrationModuleRegistry`, `MigrationModulePluginDiscovery`, `MigrationModuleConsolidation`).
|
||||
|
||||
**Dependencies**: Authority, all modules (migration coordination), PostgreSQL.
|
||||
|
||||
---
|
||||
|
||||
### Plugin
|
||||
- **Source**: `src/Plugin/`
|
||||
- **Docs**: [`docs/modules/plugin/`](./plugin/)
|
||||
- **Type**: Library / Framework
|
||||
- **Database**: PostgreSQL (Plugin Registry, 1 SQL migration)
|
||||
- **Endpoints**: None
|
||||
|
||||
Plugin SDK, registry, sandbox, and host framework enabling extensibility across the platform. Includes Abstractions (contracts), Sdk (development SDK), Host (lifecycle management), Registry (PostgreSQL persistence), Sandbox (isolated execution), Cli (management tool), Testing (test harness), and Samples.
|
||||
|
||||
**Dependencies**: None (framework consumed by other modules).
|
||||
|
||||
---
|
||||
|
||||
### Policy
|
||||
- **Source**: `src/Policy/`
|
||||
- **Docs**: [`docs/modules/policy/`](./policy/)
|
||||
- **Type**: Service (Engine + Gateway)
|
||||
- **Database**: PostgreSQL (5 SQL migrations plus demo seed)
|
||||
- **Endpoints**: 59 (49 Engine + 10 Gateway)
|
||||
|
||||
Deterministic policy evaluation engine and gateway service compiling stella-dsl policy packs into verdicts by joining SBOM inventory, advisories, and VEX evidence. The Engine provides extensive coverage across advisory AI knobs, attestation reports, batch evaluation, budgets, conflicts, CVSS receipts, determinization, effective policy, merge previews, risk profiles/simulations, sealed mode, verification policy, and violation tracking. The Gateway handles admission decisions, deltas, exception approvals, gates, governance, and registry webhooks.
|
||||
|
||||
**Dependencies**: Concelier, Excititor, SBOM Service, Scanner, Authority, Scheduler.
|
||||
|
||||
---
|
||||
|
||||
### Provenance
|
||||
- **Source**: `src/Provenance/`
|
||||
- **Docs**: [`docs/modules/provenance/`](./provenance/)
|
||||
- **Type**: Library / Tool
|
||||
- **Database**: None
|
||||
- **Endpoints**: None
|
||||
|
||||
Provenance attestation library and CLI tool for generating and verifying supply-chain provenance records. Creates in-toto attestation statements linking build artifacts to source materials, build systems, and parameters. A separate provenance cache library exists at `src/__Libraries/StellaOps.Provcache.Postgres/`.
|
||||
|
||||
**Dependencies**: Signer, Attestor.
|
||||
|
||||
---
|
||||
|
||||
### ReachGraph
|
||||
- **Source**: `src/ReachGraph/`
|
||||
- **Docs**: [`docs/modules/reach-graph/`](./reach-graph/)
|
||||
- **Type**: Service
|
||||
- **Database**: PostgreSQL (1 SQL migration in shared library)
|
||||
- **Endpoints**: Defined in Program.cs
|
||||
|
||||
Unified reachability subgraph store providing fast, deterministic, audit-ready answers about why a dependency is reachable. Consolidates reachability data from Scanner (call graphs), Signals (runtime observations), and Attestor (PoE JSON) into content-addressed artifacts with edge-level explainability.
|
||||
|
||||
**Dependencies**: Scanner, Signals, Attestor.
|
||||
|
||||
---
|
||||
|
||||
### Registry
|
||||
- **Source**: `src/Registry/`
|
||||
- **Docs**: [`docs/modules/registry/`](./registry/)
|
||||
- **Type**: Service
|
||||
- **Database**: None (stateless)
|
||||
- **Endpoints**: 2 (token issuance)
|
||||
|
||||
Docker registry bearer token service issuing short-lived tokens for private or mirrored registries with license/plan enforcement. Validates caller identity using Authority-issued tokens, authorizes requested registry scopes against a configured plan catalogue, and mints Docker-registry-compatible JWTs.
|
||||
|
||||
**Dependencies**: Authority.
|
||||
|
||||
---
|
||||
|
||||
### ReleaseOrchestrator
|
||||
- **Source**: `src/ReleaseOrchestrator/`
|
||||
- **Docs**: [`docs/modules/release-orchestrator/`](./release-orchestrator/)
|
||||
- **Type**: Service (Active Development)
|
||||
- **Database**: PostgreSQL (planned, via Platform migrations)
|
||||
- **Endpoints**: 1
|
||||
|
||||
Central release control plane for non-Kubernetes container estates governing promotion across environments (Dev, Stage, Prod), enforcing security and policy gates, and producing verifiable evidence. Contains API, agents, apps, and libraries sub-directories with 140K+ lines of production code. Owns environment topology and promotion lanes.
|
||||
|
||||
**Dependencies**: Platform, Policy Engine, Scanner, Authority.
|
||||
|
||||
---
|
||||
|
||||
### Remediation
|
||||
- **Source**: `src/Remediation/`
|
||||
- **Docs**: [`docs/modules/remediation/`](./remediation/)
|
||||
- **Type**: Service
|
||||
- **Database**: PostgreSQL (1 SQL migration)
|
||||
- **Endpoints**: 3
|
||||
|
||||
Developer-facing signed-PR remediation marketplace enabling discovery, application, and verification of community-contributed fix templates for known CVEs. Templates include unified diff content, version range applicability, and trust scores. Tracks PR submission lifecycle with attestation evidence.
|
||||
|
||||
**Dependencies**: Scanner, Signer / Attestor, Authority.
|
||||
|
||||
---
|
||||
|
||||
### Replay
|
||||
- **Source**: `src/Replay/`
|
||||
- **Docs**: [`docs/modules/replay/`](./replay/)
|
||||
- **Type**: Service
|
||||
- **Database**: PostgreSQL (via shared infrastructure)
|
||||
- **Endpoints**: 2 (point-in-time query, verdict replay)
|
||||
|
||||
Deterministic replay engine ensuring vulnerability assessments can be reproduced byte-for-byte given the same inputs. Manages replay tokens (cryptographically bound to input digests), manifests, feed snapshots, and verification workflows. Stores content-addressed references, not actual data.
|
||||
|
||||
**Dependencies**: Policy Engine, Scanner, Concelier / Excititor.
|
||||
|
||||
---
|
||||
|
||||
### RiskEngine
|
||||
- **Source**: `src/RiskEngine/`
|
||||
- **Docs**: [`docs/modules/risk-engine/`](./risk-engine/)
|
||||
- **Type**: Service
|
||||
- **Database**: PostgreSQL (via shared infrastructure)
|
||||
- **Endpoints**: 1 (exploit maturity)
|
||||
|
||||
Risk scoring runtime computing deterministic, explainable risk scores by aggregating signals from EPSS, CVSS, KEV, VEX, and reachability data. Produces audit trails and explainability payloads for every scoring decision. Does not make PASS/FAIL decisions; provides scores to the Policy Engine. Supports offline operation via factor bundles.
|
||||
|
||||
**Dependencies**: Concelier, Excititor, Signals, Policy Engine.
|
||||
|
||||
---
|
||||
|
||||
### Router
|
||||
- **Source**: `src/Router/`
|
||||
- **Docs**: [`docs/modules/router/`](./router/)
|
||||
- **Type**: Service / Framework
|
||||
- **Database**: None
|
||||
- **Endpoints**: 4
|
||||
|
||||
Internal service transport using binary protocol (TCP/TLS/UDP) for microservice-to-gateway communication with pluggable transports. Includes a unified plugin, shared libraries, and example microservices. The Router's `StellaOps.Gateway.WebService` bridges binary protocol connections to HTTP; this is separate from `src/Gateway/` which is the HTTP ingress gateway.
|
||||
|
||||
**Dependencies**: Gateway, all microservices, Valkey.
|
||||
|
||||
---
|
||||
|
||||
### RuntimeInstrumentation
|
||||
- **Source**: `src/RuntimeInstrumentation/`
|
||||
- **Docs**: [`docs/modules/runtime-instrumentation/`](./runtime-instrumentation/)
|
||||
- **Type**: Library
|
||||
- **Database**: None
|
||||
- **Endpoints**: None
|
||||
|
||||
Tetragon-based runtime instrumentation library for eBPF-based process and syscall monitoring. Provides integration with Cilium Tetragon enabling observation of actual process execution, file access, and network activity at the kernel level for runtime evidence collection.
|
||||
|
||||
**Dependencies**: Tetragon (eBPF runtime), Zastava / Signals (evidence consumers).
|
||||
|
||||
---
|
||||
|
||||
### SbomService
|
||||
- **Source**: `src/SbomService/`
|
||||
- **Docs**: [`docs/modules/sbom-service/`](./sbom-service/)
|
||||
- **Type**: Service
|
||||
- **Database**: PostgreSQL (1 SQL migration in Lineage persistence)
|
||||
- **Endpoints**: 5
|
||||
|
||||
Canonical SBOM projection, lookup, and timeline API serving deterministic, tenant-scoped SBOM data. Does not perform scanning; consumes Scanner outputs or supplied SPDX/CycloneDX blobs. SBOMs are append-only with mutations via new versions only. Owns the SBOM lineage ledger for versioned uploads, diffs, and retention pruning.
|
||||
|
||||
**Dependencies**: Scanner, Graph, AdvisoryAI, Policy Engine.
|
||||
|
||||
---
|
||||
|
||||
### Scanner
|
||||
- **Source**: `src/Scanner/`
|
||||
- **Docs**: [`docs/modules/scanner/`](./scanner/)
|
||||
- **Type**: Service
|
||||
- **Database**: PostgreSQL (36 SQL migrations, multiple DbContexts including `TriageDbContext`)
|
||||
- **Endpoints**: 77 (EPSS, observability, reachability evidence, replay, score replay, unknowns, scan, and 71 more)
|
||||
|
||||
Deterministic SBOM generation and vulnerability scanning engine for container images and filesystems with three-way diffs, per-layer caching, and attestation hand-off. Emits Inventory and Usage views. Does not produce PASS/FAIL verdicts. Second-highest migration count covering scans, findings, call graphs, EPSS, smart diffs, unknowns, reachability, binary evidence, secret detection, runtime observations, and score history.
|
||||
|
||||
**Dependencies**: Signer, Attestor, Authority, object storage / RustFS, SBOM Service, Concelier / Excititor.
|
||||
|
||||
---
|
||||
|
||||
### Scheduler
|
||||
- **Source**: `src/Scheduler/`
|
||||
- **Docs**: [`docs/modules/scheduler/`](./scheduler/)
|
||||
- **Type**: Service
|
||||
- **Database**: PostgreSQL (11 SQL migrations)
|
||||
- **Endpoints**: 8 (event webhook, failure signature, graph job, policy run, policy simulation, run, schedule, resolver job)
|
||||
|
||||
Re-evaluation scheduler keeping scan results current by pinpointing affected images when new advisories or VEX claims arrive. Default mode is analysis-only (no image pull). Includes event webhooks, failure signature tracking, graph jobs, policy runs/simulations, and vulnerability resolver jobs.
|
||||
|
||||
**Dependencies**: Scanner.WebService, Policy Engine, Concelier / Excititor, Notify, Orchestrator.
|
||||
|
||||
---
|
||||
|
||||
### Sdk
|
||||
- **Source**: `src/Sdk/`
|
||||
- **Docs**: [`docs/modules/sdk/`](./sdk/)
|
||||
- **Type**: Library / Code Generator
|
||||
- **Database**: None
|
||||
- **Endpoints**: None
|
||||
|
||||
Client SDK generator and release SDK for producing typed API clients across multiple languages from OpenAPI specifications. Includes `StellaOps.Sdk.Generator` (code generator) and `StellaOps.Sdk.Release` (publishing SDK).
|
||||
|
||||
**Dependencies**: Gateway / OpenAPI specs.
|
||||
|
||||
---
|
||||
|
||||
### Signals
|
||||
- **Source**: `src/Signals/`
|
||||
- **Docs**: [`docs/modules/signals/`](./signals/)
|
||||
- **Type**: Service
|
||||
- **Database**: PostgreSQL (6 SQL migrations)
|
||||
- **Endpoints**: 1 (SCM webhook)
|
||||
|
||||
Unified evidence-weighted scoring system aggregating reachability, runtime observations, backport detection, exploit intelligence, source trust, and mitigations into a single 0-100 score. Maintains determinism, provides score decomposition with explanations, and supports SCM webhook integration for change-triggered re-scoring.
|
||||
|
||||
**Dependencies**: Scanner, Concelier / Excititor, RuntimeInstrumentation, Policy Engine.
|
||||
|
||||
---
|
||||
|
||||
### Signer
|
||||
- **Source**: `src/Signer/`
|
||||
- **Docs**: [`docs/modules/signer/`](./signer/)
|
||||
- **Type**: Service
|
||||
- **Database**: PostgreSQL (`KeyManagementDbContext`, 2 SQL migrations)
|
||||
- **Endpoints**: 3 (ceremony, key rotation, signer)
|
||||
|
||||
The only service permitted to produce Stella Ops-verified DSSE signatures over SBOMs and reports, enforcing entitlement (PoE), sender-constrained auth, and supply-chain integrity. Does not push to Rekor (Attestor does). Stateless for the hot path with keys in KMS/HSM or ephemeral (keyless mode). Supports multi-algorithm signing (ECDSA, EdDSA, eIDAS, FIPS, GOST, SM).
|
||||
|
||||
**Dependencies**: Authority, Cryptography library, KMS/HSM.
|
||||
|
||||
---
|
||||
|
||||
### SmRemote
|
||||
- **Source**: `src/SmRemote/`
|
||||
- **Docs**: [`docs/modules/sm-remote/`](./sm-remote/)
|
||||
- **Type**: Service
|
||||
- **Database**: None
|
||||
- **Endpoints**: Defined in Program.cs
|
||||
|
||||
Remote service for Chinese SM2/SM3/SM4 cryptographic operations enabling sovereign crypto compliance. Allows deployments requiring SM compliance to offload operations to a dedicated service rather than requiring SM crypto libraries on every host.
|
||||
|
||||
**Dependencies**: Cryptography Plugin.Sm.
|
||||
|
||||
---
|
||||
|
||||
### Symbols
|
||||
- **Source**: `src/Symbols/`
|
||||
- **Docs**: [`docs/modules/symbols/`](./symbols/)
|
||||
- **Type**: Service
|
||||
- **Database**: None (content-addressed storage)
|
||||
- **Endpoints**: 1 (symbol source)
|
||||
|
||||
Symbol resolution and debug information management service for native binary analysis. Maps symbols to packages, manages debug information, and supports stripped binary analysis. Includes marketplace architecture for community-contributed symbol sources and offline symbol stores.
|
||||
|
||||
**Dependencies**: Scanner, BinaryIndex.
|
||||
|
||||
---
|
||||
|
||||
### TaskRunner
|
||||
- **Source**: `src/TaskRunner/`
|
||||
- **Docs**: [`docs/modules/taskrunner/`](./taskrunner/)
|
||||
- **Type**: Service
|
||||
- **Database**: PostgreSQL (via infrastructure layer)
|
||||
- **Endpoints**: Defined in WebService/Worker Program.cs
|
||||
|
||||
Deterministic task pack execution engine with approvals, sealed-mode enforcement, evidence capture, and DSSE attestation for every completed run. Three-phase execution: Plan (build execution graph), optional Simulation (dry-run with gates), and Execution (verify plan hash, execute steps, stream logs). Operates offline/air-gapped.
|
||||
|
||||
**Dependencies**: Orchestrator, PacksRegistry, Authority, Signer / Attestor, object storage.
|
||||
|
||||
---
|
||||
|
||||
### Telemetry
|
||||
- **Source**: `src/Telemetry/`
|
||||
- **Docs**: [`docs/modules/telemetry/`](./telemetry/)
|
||||
- **Type**: Library
|
||||
- **Database**: None
|
||||
- **Endpoints**: None
|
||||
|
||||
Observability library providing OpenTelemetry-based metrics, traces, and logs with Roslyn analyzers ensuring telemetry best practices. Supports multiple pipeline profiles (default, forensic, airgap) with allow-listed exporters for sovereign readiness. Forensic archives produce signed bundles of raw OTLP records.
|
||||
|
||||
**Dependencies**: Prometheus, Tempo/Jaeger, Loki.
|
||||
|
||||
---
|
||||
|
||||
### Timeline
|
||||
- **Source**: `src/Timeline/`
|
||||
- **Docs**: [`docs/modules/timeline/`](./timeline/)
|
||||
- **Type**: Service
|
||||
- **Database**: PostgreSQL (1 SQL migration in Timeline.Core)
|
||||
- **Endpoints**: 3 (export, replay, timeline)
|
||||
|
||||
Timeline query service providing export, replay, and timeline browsing endpoints for vulnerability history and event streams. Uses shared libraries from `StellaOps.Eventing` for event envelope schemas and `StellaOps.Timeline.Core` for core logic including critical path view.
|
||||
|
||||
**Dependencies**: All services (event sources), TimelineIndexer.
|
||||
|
||||
---
|
||||
|
||||
### TimelineIndexer
|
||||
- **Source**: `src/TimelineIndexer/`
|
||||
- **Docs**: [`docs/modules/timeline-indexer/`](./timeline-indexer/)
|
||||
- **Type**: Service
|
||||
- **Database**: PostgreSQL (1 SQL migration)
|
||||
- **Endpoints**: Defined in WebService Program.cs
|
||||
|
||||
Timeline event indexing and query service providing fast indexed access to events across all StellaOps services. Receives events from NATS/Valkey streams, indexes them, and provides efficient time-range queries with filtering. Enables vulnerability history browsing, scan timeline analysis, and policy evaluation trail inspection.
|
||||
|
||||
**Dependencies**: NATS / Valkey, Timeline.
|
||||
|
||||
---
|
||||
|
||||
### Tools
|
||||
- **Source**: `src/Tools/`
|
||||
- **Docs**: [`docs/modules/tools/`](./tools/)
|
||||
- **Type**: CLI Tools
|
||||
- **Database**: None
|
||||
- **Endpoints**: None
|
||||
|
||||
Developer utility tools including FixtureUpdater, GoldenPairs generator, LanguageAnalyzerSmoke, NotifySmokeCheck, PolicyDslValidator, PolicySchemaExporter, PolicySimulationSmoke, RustFsMigrator, WorkflowGenerator, and a Python CERT-Bund offline snapshot tool.
|
||||
|
||||
**Dependencies**: Various (tool-specific).
|
||||
|
||||
---
|
||||
|
||||
### Unknowns
|
||||
- **Source**: `src/Unknowns/`
|
||||
- **Docs**: [`docs/modules/unknowns/`](./unknowns/)
|
||||
- **Type**: Service
|
||||
- **Database**: PostgreSQL (`UnknownsDbContext`, 4 SQL migrations)
|
||||
- **Endpoints**: 2 (grey queue, unknowns)
|
||||
|
||||
Structured registry for tracking unresolved components, symbols, and incomplete mappings that Scanner and other analyzers cannot definitively identify. Includes a Grey Queue for lifecycle management (queueing, triage, resolution). Does not guess identities; records what cannot be determined.
|
||||
|
||||
**Dependencies**: Scanner, Signals.
|
||||
|
||||
---
|
||||
|
||||
### Verifier
|
||||
- **Source**: `src/Verifier/`
|
||||
- **Docs**: [`docs/modules/verifier/`](./verifier/)
|
||||
- **Type**: CLI Tool
|
||||
- **Database**: None
|
||||
- **Endpoints**: None
|
||||
|
||||
Standalone CLI tool for verifying the integrity and authenticity of signed evidence bundles produced by the platform. Validates DSSE envelope signatures, Merkle inclusion proofs, and bundle manifest checksums. Designed for operators and auditors who need independent verification without a full StellaOps installation.
|
||||
|
||||
**Dependencies**: None (standalone verification).
|
||||
|
||||
---
|
||||
|
||||
### VexHub
|
||||
- **Source**: `src/VexHub/`
|
||||
- **Docs**: [`docs/modules/vex-hub/`](./vex-hub/)
|
||||
- **Type**: Service
|
||||
- **Database**: PostgreSQL (1 SQL migration)
|
||||
- **Endpoints**: 1 (VexHubEndpointExtensions)
|
||||
|
||||
VEX document hub providing centralized management, distribution, and integration of VEX statements across the platform. Stores VEX statements from various sources and distributes data to Excititor for processing.
|
||||
|
||||
**Dependencies**: Excititor, IssuerDirectory, Authority.
|
||||
|
||||
---
|
||||
|
||||
### VexLens
|
||||
- **Source**: `src/VexLens/`
|
||||
- **Docs**: [`docs/modules/vex-lens/`](./vex-lens/)
|
||||
- **Type**: Service
|
||||
- **Database**: PostgreSQL (1 SQL migration)
|
||||
- **Endpoints**: 2 (export, VexLens)
|
||||
|
||||
VEX consensus viewer and analysis service providing issuer-aware VEX statement evaluation with export capabilities. Evaluates VEX statements from multiple issuers, computes consensus views considering trust weights and provenance, and integrates with IssuerDirectory for owner manifest resolution.
|
||||
|
||||
**Dependencies**: Excititor, IssuerDirectory, Policy Engine.
|
||||
|
||||
---
|
||||
|
||||
### VulnExplorer
|
||||
- **Source**: `src/VulnExplorer/`
|
||||
- **Docs**: [`docs/modules/vuln-explorer/`](./vuln-explorer/)
|
||||
- **Type**: Service
|
||||
- **Database**: None (reads from other modules' databases)
|
||||
- **Endpoints**: Defined in Program.cs
|
||||
|
||||
Vulnerability exploration API providing rich querying, filtering, and analysis of vulnerability findings across the platform. Aggregates data from Scanner, Concelier, Excititor, Policy, and Signals to present a unified vulnerability view with concept-based navigation and evidence links.
|
||||
|
||||
**Dependencies**: Scanner, Concelier, Excititor, Policy Engine, Signals, Findings Ledger.
|
||||
|
||||
---
|
||||
|
||||
### Web
|
||||
- **Source**: `src/Web/`
|
||||
- **Docs**: [`docs/modules/ui/`](./ui/)
|
||||
- **Type**: UI (Angular SPA)
|
||||
- **Database**: None
|
||||
- **Endpoints**: None
|
||||
|
||||
Angular single-page application providing the operator console with vulnerability dashboard, release management, policy studio, evidence browser, SBOM exploration, notification management, and administrative controls. Includes search client APIs for knowledge search integration, smart diff visualization, AI UX patterns, and competitive triage workflows.
|
||||
|
||||
**Dependencies**: Gateway (API proxy), all backend APIs.
|
||||
|
||||
---
|
||||
|
||||
### Zastava
|
||||
- **Source**: `src/Zastava/`
|
||||
- **Docs**: [`docs/modules/zastava/`](./zastava/)
|
||||
- **Type**: Service (three processes: Agent, Observer, Webhook)
|
||||
- **Database**: None (stateless agents)
|
||||
- **Endpoints**: Webhook endpoints, agent APIs defined in Program.cs
|
||||
|
||||
Runtime inspector and enforcer watching real workloads, detecting drift from scanned baselines, verifying image/SBOM/attestation posture, and optionally admitting/blocking deployments. Observer inventories containers and verifies signatures; Admission (optional Kubernetes webhook) enforces minimal posture pre-flight. On non-Kubernetes Docker hosts, runs as a host service with observer-only features.
|
||||
|
||||
**Dependencies**: Scanner.WebService, Policy Engine, Signals, Attestor, Authority.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -14,9 +14,9 @@ LLMs can still be used as optional formatters later, but AKS correctness is grou
|
||||
|
||||
## Architecture
|
||||
1. Ingestion/indexing:
|
||||
- Markdown (`docs/**`) -> section chunks.
|
||||
- OpenAPI (`openapi.json`) -> per-operation chunks + normalized operation tables.
|
||||
- Doctor seed/metadata -> doctor projection chunks.
|
||||
- Markdown allow-list/manifest -> section chunks.
|
||||
- OpenAPI aggregate (`openapi_current.json` style artifact) -> per-operation chunks + normalized operation tables.
|
||||
- Doctor seed + controls metadata (including CLI-discovered Doctor check catalog projection) -> doctor projection chunks.
|
||||
2. Storage:
|
||||
- PostgreSQL tables in schema `advisoryai` via migration `src/AdvisoryAI/StellaOps.AdvisoryAI/Storage/Migrations/002_knowledge_search.sql`.
|
||||
3. Retrieval:
|
||||
@@ -40,22 +40,32 @@ Vector support:
|
||||
|
||||
## Deterministic ingestion rules
|
||||
### Markdown
|
||||
- Source order:
|
||||
1. Allow-list file: `src/AdvisoryAI/StellaOps.AdvisoryAI/KnowledgeSearch/knowledge-docs-allowlist.json`.
|
||||
2. Generated manifest (optional, from CLI tool): `knowledge-docs-manifest.json`.
|
||||
3. Fallback scan roots (`docs/**`) only if allow-list resolves no markdown files.
|
||||
- Chunk by H2/H3 headings.
|
||||
- Stable anchors using slug + duplicate suffix.
|
||||
- Stable chunk IDs from source path + anchor + span.
|
||||
- Metadata includes path, anchor, section path, tags.
|
||||
|
||||
### OpenAPI
|
||||
- Parse `openapi.json` only for deterministic MVP.
|
||||
- Source order:
|
||||
1. Aggregated OpenAPI file path (default `devops/compose/openapi_current.json`).
|
||||
2. Fallback repository scan for `openapi.json` when aggregate is missing.
|
||||
- Parse deterministic JSON aggregate for MVP.
|
||||
- Emit one searchable chunk per HTTP operation.
|
||||
- Preserve structured operation payloads (`request_json`, `responses_json`, `security_json`).
|
||||
|
||||
### Doctor
|
||||
- Source order:
|
||||
1. Seed file `src/AdvisoryAI/StellaOps.AdvisoryAI/KnowledgeSearch/doctor-search-seed.json`.
|
||||
2. Optional Doctor endpoint metadata (`DoctorChecksEndpoint`) when configured.
|
||||
2. Controls file `src/AdvisoryAI/StellaOps.AdvisoryAI/KnowledgeSearch/doctor-search-controls.json` (contains control fields plus fallback metadata from `stella advisoryai sources prepare`).
|
||||
3. Optional Doctor endpoint metadata (`DoctorChecksEndpoint`) when configured.
|
||||
- `stella advisoryai sources prepare` merges configured seed entries with `DoctorEngine.ListChecks()` (when available in CLI runtime) and writes enriched control projection metadata (`title`, `severity`, `description`, `remediation`, `runCommand`, `symptoms`, `tags`, `references`).
|
||||
- Emit doctor chunk + projection record including:
|
||||
- `checkCode`, `title`, `severity`, `runCommand`, remediation, symptoms.
|
||||
- control metadata (`control`, `requiresConfirmation`, `isDestructive`, `inspectCommand`, `verificationCommand`).
|
||||
|
||||
## Ranking strategy
|
||||
Implemented in `src/AdvisoryAI/StellaOps.AdvisoryAI/KnowledgeSearch/KnowledgeSearchService.cs`:
|
||||
@@ -99,6 +109,7 @@ AKS commands:
|
||||
- `stella search "<query>" [--type docs|api|doctor] [--product ...] [--version ...] [--service ...] [--tag ...] [--k N] [--json]`
|
||||
- `stella doctor suggest "<symptom>" [--product ...] [--version ...] [--k N] [--json]`
|
||||
- `stella advisoryai index rebuild [--json]`
|
||||
- `stella advisoryai sources prepare [--repo-root ...] [--docs-allowlist ...] [--docs-manifest-output ...] [--openapi-output ...] [--doctor-seed ...] [--doctor-controls-output ...] [--overwrite] [--json]`
|
||||
|
||||
Output:
|
||||
- Human mode: grouped actionable references.
|
||||
@@ -127,6 +138,7 @@ Init script:
|
||||
Example workflow:
|
||||
```bash
|
||||
docker compose -f devops/compose/docker-compose.advisoryai-knowledge-test.yml up -d
|
||||
stella advisoryai sources prepare --json
|
||||
stella advisoryai index rebuild --json
|
||||
dotnet test src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/StellaOps.AdvisoryAI.Tests.csproj
|
||||
```
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# Advisory Lens Architecture
|
||||
|
||||
> **Status: Production (Shared Library).** AdvisoryLens is a standalone deterministic library at `src/__Libraries/StellaOps.AdvisoryLens/`, **not** merged into AdvisoryAI. The two modules serve different purposes: AdvisoryLens provides pattern-based case matching without AI inference; AdvisoryAI provides LLM-powered advisory analysis with guardrails. They can be composed together but are architecturally independent. The library is currently available for integration but not yet referenced from any WebService `Program.cs`.
|
||||
|
||||
## Purpose
|
||||
|
||||
StellaOps.AdvisoryLens is a deterministic, offline-first library for semantic case matching of vulnerability advisories. It produces ranked suggestions and contextual hints without AI/LLM inference.
|
||||
|
||||
@@ -19,8 +19,8 @@ AirGap manages sealed knowledge snapshot export and import for offline/air-gappe
|
||||
**Libraries:**
|
||||
- `StellaOps.AirGap.Policy` - Staleness policy evaluation
|
||||
- `StellaOps.AirGap.Time` - Time anchor validation and trust
|
||||
- `StellaOps.AirGap.Storage.Postgres` - PostgreSQL storage for snapshots
|
||||
- `StellaOps.AirGap.Storage.Postgres.Tests` - Storage integration tests
|
||||
- `StellaOps.AirGap.Persistence` - PostgreSQL persistence (EF Core v10)
|
||||
- `StellaOps.AirGap.Persistence.Tests` - Persistence integration tests
|
||||
|
||||
## Configuration
|
||||
|
||||
@@ -33,6 +33,45 @@ Key settings:
|
||||
- PostgreSQL connection (schema: `airgap`)
|
||||
- Export/import paths and validation rules
|
||||
|
||||
## EF Core Persistence Workflow
|
||||
|
||||
AirGap persistence now uses EF Core v10 models generated from the module migration schema.
|
||||
|
||||
Scaffold baseline context/models:
|
||||
|
||||
```bash
|
||||
dotnet ef dbcontext scaffold \
|
||||
"Host=...;Port=...;Database=...;Username=...;Password=..." \
|
||||
Npgsql.EntityFrameworkCore.PostgreSQL \
|
||||
--project src/AirGap/__Libraries/StellaOps.AirGap.Persistence/StellaOps.AirGap.Persistence.csproj \
|
||||
--startup-project src/AirGap/__Libraries/StellaOps.AirGap.Persistence/StellaOps.AirGap.Persistence.csproj \
|
||||
--schema airgap \
|
||||
--table state \
|
||||
--table bundle_versions \
|
||||
--table bundle_version_history \
|
||||
--context-dir EfCore/Context \
|
||||
--context AirGapDbContext \
|
||||
--output-dir EfCore/Models \
|
||||
--namespace StellaOps.AirGap.Persistence.EfCore.Models \
|
||||
--context-namespace StellaOps.AirGap.Persistence.EfCore.Context \
|
||||
--use-database-names
|
||||
```
|
||||
|
||||
Regenerate compiled model artifacts after model updates:
|
||||
|
||||
```bash
|
||||
dotnet ef dbcontext optimize \
|
||||
--project src/AirGap/__Libraries/StellaOps.AirGap.Persistence/StellaOps.AirGap.Persistence.csproj \
|
||||
--startup-project src/AirGap/__Libraries/StellaOps.AirGap.Persistence/StellaOps.AirGap.Persistence.csproj \
|
||||
--context AirGapDbContext \
|
||||
--output-dir EfCore/CompiledModels \
|
||||
--namespace StellaOps.AirGap.Persistence.EfCore.CompiledModels
|
||||
```
|
||||
|
||||
Runtime behavior:
|
||||
- The static compiled model is used explicitly for the default `airgap` schema path.
|
||||
- Non-default schemas (for integration fixtures) use runtime model construction to preserve schema isolation.
|
||||
|
||||
## Bundle manifest (v2) additions
|
||||
|
||||
- `canonicalManifestHash`: sha256 of canonical JSON for deterministic verification.
|
||||
|
||||
@@ -82,6 +82,16 @@ Determinism requirements:
|
||||
- Use ordinal comparisons for keys and stable serialization settings for JSON responses.
|
||||
- Never infer state from wall-clock behavior other than the injected `TimeProvider`.
|
||||
|
||||
## Persistence backend
|
||||
|
||||
Controller state persistence is implemented in `src/AirGap/__Libraries/StellaOps.AirGap.Persistence` using EF Core v10.
|
||||
|
||||
- `PostgresAirGapStateStore` and `PostgresBundleVersionStore` use EF queries/updates with deterministic ordering guarantees preserved from the previous SQL paths.
|
||||
- `AirGapDbContextFactory` explicitly binds `AirGapDbContextModel.Instance` for the default `airgap` schema.
|
||||
- Integration fixtures that provision per-run schemas intentionally use runtime model mapping (no automatic compiled-model discovery) so table resolution stays schema-isolated.
|
||||
|
||||
Regeneration commands are documented in `docs/modules/airgap/README.md` under `EF Core Persistence Workflow`.
|
||||
|
||||
## Telemetry
|
||||
|
||||
The controller emits:
|
||||
@@ -97,4 +107,3 @@ The controller emits:
|
||||
- `docs/modules/airgap/guides/staleness-and-time.md`
|
||||
- `docs/modules/airgap/guides/time-api.md`
|
||||
- `docs/modules/airgap/guides/importer.md`
|
||||
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# Analytics Module Architecture
|
||||
|
||||
> **Implementation Note:** Analytics is a cross-cutting feature integrated into the **Platform WebService** (`src/Platform/`). There is no standalone `src/Analytics/` module. Data ingestion pipelines span Scanner, Concelier, and Attestor modules. See [Platform Architecture](../platform/architecture-overview.md) for service-level integration details.
|
||||
|
||||
## Design Philosophy
|
||||
|
||||
The Analytics module implements a **star-schema data warehouse** pattern optimized for analytical queries rather than transactional workloads. Key design principles:
|
||||
|
||||
@@ -1,19 +1,25 @@
|
||||
# component_architecture_authority.md — **Stella Ops Authority** (2025Q4)
|
||||
# component_architecture_authority.md — **Stella Ops Authority** (2025Q4)
|
||||
|
||||
> **Current tenant-selection ADR:** `docs/architecture/decisions/ADR-002-multi-tenant-same-api-key-selection.md`
|
||||
> **Service impact ledger:** `docs/technical/architecture/multi-tenant-service-impact-ledger.md`
|
||||
> **Flow sequences:** `docs/technical/architecture/multi-tenant-flow-sequences.md`
|
||||
> **Rollout policy:** `docs/operations/multi-tenant-rollout-and-compatibility.md`
|
||||
> **QA matrix:** `docs/qa/feature-checks/multi-tenant-acceptance-matrix.md`
|
||||
|
||||
> Consolidates identity and tenancy requirements documented across the AOC, Policy, and Platform guides, along with the dedicated Authority implementation plan.
|
||||
|
||||
> **Scope.** Implementation‑ready architecture for **Stella Ops Authority**: the on‑prem **OIDC/OAuth2** service that issues **short‑lived, sender‑constrained operational tokens (OpToks)** to first‑party services and tools. Covers protocols (DPoP & mTLS binding), token shapes, endpoints, storage, rotation, HA, RBAC, audit, and testing. This component is the trust anchor for *who* is calling inside a Stella Ops installation. (Entitlement is proven separately by **PoE** from the cloud Licensing Service; Authority does not issue PoE.)
|
||||
> **Scope.** Implementation‑ready architecture for **Stella Ops Authority**: the on‑prem **OIDC/OAuth2** service that issues **short‑lived, sender‑constrained operational tokens (OpToks)** to first‑party services and tools. Covers protocols (DPoP & mTLS binding), token shapes, endpoints, storage, rotation, HA, RBAC, audit, and testing. This component is the trust anchor for *who* is calling inside a Stella Ops installation. (Entitlement is proven separately by **PoE** from the cloud Licensing Service; Authority does not issue PoE.)
|
||||
|
||||
---
|
||||
|
||||
## 0) Mission & boundaries
|
||||
|
||||
**Mission.** Provide **fast, local, verifiable** authentication for Stella Ops microservices and tools by minting **very short‑lived** OAuth2/OIDC tokens that are **sender‑constrained** (DPoP or mTLS‑bound). Support RBAC scopes, multi‑tenant claims, and deterministic validation for APIs (Scanner, Signer, Attestor, Excititor, Concelier, UI, CLI, Zastava).
|
||||
**Mission.** Provide **fast, local, verifiable** authentication for Stella Ops microservices and tools by minting **very short‑lived** OAuth2/OIDC tokens that are **sender‑constrained** (DPoP or mTLS‑bound). Support RBAC scopes, multi‑tenant claims, and deterministic validation for APIs (Scanner, Signer, Attestor, Excititor, Concelier, UI, CLI, Zastava).
|
||||
|
||||
**Boundaries.**
|
||||
|
||||
* Authority **does not** validate entitlements/licensing. That’s enforced by **Signer** using **PoE** with the cloud Licensing Service.
|
||||
* Authority tokens are **operational only** (2–5 min TTL) and must not be embedded in long‑lived artifacts or stored in SBOMs.
|
||||
* Authority **does not** validate entitlements/licensing. That’s enforced by **Signer** using **PoE** with the cloud Licensing Service.
|
||||
* Authority tokens are **operational only** (2–5 min TTL) and must not be embedded in long‑lived artifacts or stored in SBOMs.
|
||||
* Authority is **stateless for validation** (JWT) and **optional introspection** for services that prefer online checks.
|
||||
|
||||
---
|
||||
@@ -23,16 +29,16 @@
|
||||
* **OIDC Discovery**: `/.well-known/openid-configuration`
|
||||
* **OAuth2** grant types:
|
||||
|
||||
* **Client Credentials** (service↔service, with mTLS or private_key_jwt)
|
||||
* **Client Credentials** (service↔service, with mTLS or private_key_jwt)
|
||||
* **Device Code** (CLI login on headless agents; optional)
|
||||
* **Authorization Code + PKCE** (browser login for UI; optional)
|
||||
* **Sender constraint options** (choose per caller or per audience):
|
||||
|
||||
* **DPoP** (Demonstration of Proof‑of‑Possession): proof JWT on each HTTP request, bound to the access token via `cnf.jkt`.
|
||||
* **OAuth 2.0 mTLS** (certificate‑bound tokens): token bound to client certificate thumbprint via `cnf.x5t#S256`.
|
||||
* **Signing algorithms**: **EdDSA (Ed25519)** preferred; fallback **ES256 (P‑256)**. Rotation is supported via **kid** in JWKS.
|
||||
* **DPoP** (Demonstration of Proof‑of‑Possession): proof JWT on each HTTP request, bound to the access token via `cnf.jkt`.
|
||||
* **OAuth 2.0 mTLS** (certificate‑bound tokens): token bound to client certificate thumbprint via `cnf.x5t#S256`.
|
||||
* **Signing algorithms**: **EdDSA (Ed25519)** preferred; fallback **ES256 (P‑256)**. Rotation is supported via **kid** in JWKS.
|
||||
* **Token format**: **JWT** access tokens (compact), optionally opaque reference tokens for services that insist on introspection.
|
||||
* **Clock skew tolerance**: ±60 s; issue `nbf`, `iat`, `exp` accordingly.
|
||||
* **Clock skew tolerance**: ±60 s; issue `nbf`, `iat`, `exp` accordingly.
|
||||
|
||||
---
|
||||
|
||||
@@ -41,7 +47,7 @@
|
||||
* **Incident mode tokens** require the `obs:incident` scope, a human-supplied `incident_reason`, and remain valid only while `auth_time` stays within a five-minute freshness window. Resource servers enforce the same window and persist `incident.reason`, `incident.auth_time`, and the fresh-auth verdict in `authority.resource.authorize` events. Authority exposes `/authority/audit/incident` so auditors can review recent activations.
|
||||
|
||||
|
||||
### 2.1 Access token (OpTok) — short‑lived (120–300 s)
|
||||
### 2.1 Access token (OpTok) — short‑lived (120–300 s)
|
||||
|
||||
**Registered claims**
|
||||
|
||||
@@ -56,7 +62,7 @@ jti = <uuid>
|
||||
scope = "scanner.scan scanner.export signer.sign ..."
|
||||
```
|
||||
|
||||
**Sender‑constraint (`cnf`)**
|
||||
**Sender‑constraint (`cnf`)**
|
||||
|
||||
* **DPoP**:
|
||||
|
||||
@@ -78,11 +84,11 @@ roles = [ "svc.scanner", "svc.signer", "ui.admin", ... ]
|
||||
plan? = <plan name> // optional hint for UIs; not used for enforcement
|
||||
```
|
||||
|
||||
> **Note**: Do **not** copy PoE claims into OpTok; OpTok ≠ entitlement. Only **Signer** checks PoE.
|
||||
> **Note**: Do **not** copy PoE claims into OpTok; OpTok ≠entitlement. Only **Signer** checks PoE.
|
||||
|
||||
### 2.2 Refresh tokens (optional)
|
||||
|
||||
* Default **disabled**. If enabled (for UI interactive logins), pair with **DPoP‑bound** refresh tokens or **mTLS** client sessions; short TTL (≤ 8 h), rotating on use (replay‑safe).
|
||||
* Default **disabled**. If enabled (for UI interactive logins), pair with **DPoP‑bound** refresh tokens or **mTLS** client sessions; short TTL (≤ 8 h), rotating on use (replay‑safe).
|
||||
|
||||
### 2.3 ID tokens (optional)
|
||||
|
||||
@@ -94,8 +100,8 @@ plan? = <plan name> // optional hint for UIs; not used for e
|
||||
|
||||
### 3.1 OIDC discovery & keys
|
||||
|
||||
* `GET /.well-known/openid-configuration` → endpoints, algs, jwks_uri
|
||||
* `GET /jwks` → JSON Web Key Set (rotating, at least 2 active keys during transition)
|
||||
* `GET /.well-known/openid-configuration` → endpoints, algs, jwks_uri
|
||||
* `GET /jwks` → JSON Web Key Set (rotating, at least 2 active keys during transition)
|
||||
|
||||
> **KMS-backed keys.** When the signing provider is `kms`, Authority fetches only the public coordinates (`Qx`, `Qy`) and version identifiers from the backing KMS. Private scalars never leave the provider; JWKS entries are produced by re-exporting the public material via the `kms.version` metadata attached to each key. Retired keys keep the same `kms.version` metadata so audits can trace which cloud KMS version produced a token.
|
||||
|
||||
@@ -105,12 +111,12 @@ plan? = <plan name> // optional hint for UIs; not used for e
|
||||
|
||||
> Legacy aliases under `/oauth/token` are deprecated as of 1 November 2025 and now emit `Deprecation/Sunset/Warning` headers. See [`docs/api/authority-legacy-auth-endpoints.md`](../../api/authority-legacy-auth-endpoints.md) for timelines and migration guidance.
|
||||
|
||||
* **Client Credentials** (service→service):
|
||||
* **Client Credentials** (service→service):
|
||||
|
||||
* **mTLS**: mutual TLS + `client_id` → bound token (`cnf.x5t#S256`)
|
||||
* **mTLS**: mutual TLS + `client_id` → bound token (`cnf.x5t#S256`)
|
||||
* `security.senderConstraints.mtls.enforceForAudiences` forces the mTLS path when requested `aud`/`resource` values intersect high-value audiences (defaults include `signer`). Authority rejects clients attempting to use DPoP/basic secrets for these audiences.
|
||||
* Stored `certificateBindings` are authoritative: thumbprint, subject, issuer, serial number, and SAN values are matched against the presented certificate, with rotation grace applied to activation windows. Failures surface deterministic error codes (e.g. `certificate_binding_subject_mismatch`).
|
||||
* **private_key_jwt**: JWT‑based client auth + **DPoP** header (preferred for tools and CLI)
|
||||
* **private_key_jwt**: JWT‑based client auth + **DPoP** header (preferred for tools and CLI)
|
||||
* **Device Code** (CLI): `POST /oauth/device/code` + `POST /oauth/token` poll
|
||||
* **Authorization Code + PKCE** (UI): standard
|
||||
|
||||
@@ -128,7 +134,7 @@ plan? = <plan name> // optional hint for UIs; not used for e
|
||||
|
||||
signed with the DPoP private key; header carries JWK.
|
||||
3. Authority validates proof; issues access token with `cnf.jkt=<thumbprint(JWK)>`.
|
||||
4. Client uses the same DPoP key to sign **every subsequent API request** to services (Signer, Scanner, …).
|
||||
4. Client uses the same DPoP key to sign **every subsequent API request** to services (Signer, Scanner, …).
|
||||
|
||||
**mTLS flow**
|
||||
|
||||
@@ -136,11 +142,11 @@ plan? = <plan name> // optional hint for UIs; not used for e
|
||||
|
||||
### 3.3 Introspection & revocation (optional)
|
||||
|
||||
* `POST /introspect` → `{ active, sub, scope, aud, exp, cnf, ... }`
|
||||
* `POST /revoke` → revokes refresh tokens or opaque access tokens.
|
||||
* `POST /introspect` → `{ active, sub, scope, aud, exp, cnf, ... }`
|
||||
* `POST /revoke` → revokes refresh tokens or opaque access tokens.
|
||||
|
||||
> Requests targeting the legacy `/oauth/{introspect|revoke}` paths receive deprecation headers and are scheduled for removal after 1 May 2026.
|
||||
* **Replay prevention**: maintain **DPoP `jti` cache** (TTL ≤ 10 min) to reject duplicate proofs when services supply DPoP nonces (Signer requires nonce for high‑value operations).
|
||||
* **Replay prevention**: maintain **DPoP `jti` cache** (TTL ≤ 10 min) to reject duplicate proofs when services supply DPoP nonces (Signer requires nonce for high‑value operations).
|
||||
|
||||
### 3.4 UserInfo (optional for UI)
|
||||
|
||||
@@ -150,19 +156,19 @@ plan? = <plan name> // optional hint for UIs; not used for e
|
||||
|
||||
### 3.5 Vuln Explorer workflow safeguards
|
||||
|
||||
* **Anti-forgery flow** — Vuln Explorer’s mutation verbs call
|
||||
* **Anti-forgery flow** — Vuln Explorer’s mutation verbs call
|
||||
* `POST /vuln/workflow/anti-forgery/issue`
|
||||
* `POST /vuln/workflow/anti-forgery/verify`
|
||||
|
||||
Callers must hold `vuln:operate` scopes. Issued tokens embed the actor, tenant, whitelisted actions, ABAC selectors (environment/owner/business tier), and optional context key/value pairs. Tokens are EdDSA/ES256 signed via the primary Authority signing key and default to a 10‑minute TTL (cap: 30 minutes). Verification enforces nonce reuse prevention, tenant match, and action membership before forwarding the request to Vuln Explorer.
|
||||
Callers must hold `vuln:operate` scopes. Issued tokens embed the actor, tenant, whitelisted actions, ABAC selectors (environment/owner/business tier), and optional context key/value pairs. Tokens are EdDSA/ES256 signed via the primary Authority signing key and default to a 10‑minute TTL (cap: 30 minutes). Verification enforces nonce reuse prevention, tenant match, and action membership before forwarding the request to Vuln Explorer.
|
||||
|
||||
* **Attachment access** — Evidence bundles and attachments reference a ledger hash. Vuln Explorer obtains a scoped download token through:
|
||||
* **Attachment access** — Evidence bundles and attachments reference a ledger hash. Vuln Explorer obtains a scoped download token through:
|
||||
* `POST /vuln/attachments/tokens/issue`
|
||||
* `POST /vuln/attachments/tokens/verify`
|
||||
|
||||
These tokens bind the ledger event hash, attachment identifier, optional finding/content metadata, and the actor. They default to a 30‑minute TTL (cap: 4 hours) and require `vuln:investigate`.
|
||||
These tokens bind the ledger event hash, attachment identifier, optional finding/content metadata, and the actor. They default to a 30‑minute TTL (cap: 4 hours) and require `vuln:investigate`.
|
||||
|
||||
* **Audit trail** — Both flows emit `vuln.workflow.csrf.*` and `vuln.attachment.token.*` audit records with tenant, actor, ledger hash, nonce, and filtered context metadata so Offline Kit operators can reconcile actions against ledger entries.
|
||||
* **Audit trail** — Both flows emit `vuln.workflow.csrf.*` and `vuln.attachment.token.*` audit records with tenant, actor, ledger hash, nonce, and filtered context metadata so Offline Kit operators can reconcile actions against ledger entries.
|
||||
|
||||
* **Configuration**
|
||||
|
||||
@@ -194,7 +200,7 @@ plan? = <plan name> // optional hint for UIs; not used for e
|
||||
|
||||
### 4.1 Audiences
|
||||
|
||||
* `signer` — only the **Signer** service should accept tokens with `aud=signer`.
|
||||
* `signer` — only the **Signer** service should accept tokens with `aud=signer`.
|
||||
* `attestor`, `scanner`, `concelier`, `excititor`, `ui`, `zastava` similarly.
|
||||
|
||||
Services **must** verify `aud` and **sender constraint** (DPoP/mTLS) per their policy.
|
||||
@@ -221,13 +227,13 @@ Services **must** verify `aud` and **sender constraint** (DPoP/mTLS) per their p
|
||||
| `authority:branding.read` / `authority:branding.write` | Authority | Branding admin |
|
||||
| `zastava.emit` / `zastava.enforce` | Scanner/Zastava | Runtime events / admission |
|
||||
|
||||
**Roles → scopes mapping** is configured centrally (Authority policy) and pushed during token issuance.
|
||||
**Roles → scopes mapping** is configured centrally (Authority policy) and pushed during token issuance.
|
||||
|
||||
---
|
||||
|
||||
## 5) Storage & state
|
||||
|
||||
* **Configuration DB** (PostgreSQL/MySQL): clients, audiences, role→scope maps, tenant/installation registry, device code grants, persistent consents (if any).
|
||||
* **Configuration DB** (PostgreSQL/MySQL): clients, audiences, role→scope maps, tenant/installation registry, device code grants, persistent consents (if any).
|
||||
* **Cache** (Valkey):
|
||||
|
||||
* DPoP **jti** replay cache (short TTL)
|
||||
@@ -241,20 +247,20 @@ Services **must** verify `aud` and **sender constraint** (DPoP/mTLS) per their p
|
||||
|
||||
* Maintain **at least 2 signing keys** active during rotation; tokens carry `kid`.
|
||||
* Prefer **Ed25519** for compact tokens; maintain **ES256** fallback for FIPS contexts.
|
||||
* Rotation cadence: 30–90 days; emergency rotation supported.
|
||||
* Publish new JWKS **before** issuing tokens with the new `kid` to avoid cold‑start validation misses.
|
||||
* Rotation cadence: 30–90 days; emergency rotation supported.
|
||||
* Publish new JWKS **before** issuing tokens with the new `kid` to avoid cold‑start validation misses.
|
||||
* Keep **old keys** available **at least** for max token TTL + 5 minutes.
|
||||
|
||||
---
|
||||
|
||||
## 7) HA & performance
|
||||
|
||||
* **Stateless issuance** (except device codes/refresh) → scale horizontally behind a load‑balancer.
|
||||
* **DB** only for client metadata and optional flows; token checks are JWT‑local; introspection endpoints hit cache/DB minimally.
|
||||
* **Stateless issuance** (except device codes/refresh) → scale horizontally behind a load‑balancer.
|
||||
* **DB** only for client metadata and optional flows; token checks are JWT‑local; introspection endpoints hit cache/DB minimally.
|
||||
* **Targets**:
|
||||
|
||||
* Token issuance P95 ≤ **20 ms** under warm cache.
|
||||
* DPoP proof validation ≤ **1 ms** extra per request at resource servers (Signer/Scanner).
|
||||
* Token issuance P95 ≤ **20 ms** under warm cache.
|
||||
* DPoP proof validation ≤ **1 ms** extra per request at resource servers (Signer/Scanner).
|
||||
* 99.9% uptime; HPA on CPU/latency.
|
||||
|
||||
---
|
||||
@@ -263,7 +269,7 @@ Services **must** verify `aud` and **sender constraint** (DPoP/mTLS) per their p
|
||||
|
||||
* **Strict TLS** (1.3 preferred); HSTS; modern cipher suites.
|
||||
* **mTLS** enabled where required (Signer/Attestor paths).
|
||||
* **Replay protection**: DPoP `jti` cache, nonce support for **Signer** (add `DPoP-Nonce` header on 401; clients re‑sign).
|
||||
* **Replay protection**: DPoP `jti` cache, nonce support for **Signer** (add `DPoP-Nonce` header on 401; clients re‑sign).
|
||||
* **Rate limits** per client & per IP; exponential backoff on failures.
|
||||
* **Secrets**: clients use **private_key_jwt** or **mTLS**; never basic secrets over the wire.
|
||||
* **CSP/CSRF** hardening on UI flows; `SameSite=Lax` cookies; PKCE enforced.
|
||||
@@ -271,10 +277,10 @@ Services **must** verify `aud` and **sender constraint** (DPoP/mTLS) per their p
|
||||
|
||||
---
|
||||
|
||||
## 9) Multi‑tenancy & installations
|
||||
## 9) Multi‑tenancy & installations
|
||||
|
||||
* **Tenant (`tid`)** and **Installation (`inst`)** registries define which audiences/scopes a client can request.
|
||||
* Cross‑tenant isolation enforced at issuance (disallow rogue `aud`), and resource servers **must** check that `tid` matches their configured tenant.
|
||||
* Cross‑tenant isolation enforced at issuance (disallow rogue `aud`), and resource servers **must** check that `tid` matches their configured tenant.
|
||||
|
||||
---
|
||||
|
||||
@@ -287,7 +293,7 @@ Authority exposes two admin tiers:
|
||||
```
|
||||
POST /admin/clients # create/update client (confidential/public)
|
||||
POST /admin/audiences # register audience resource URIs
|
||||
POST /admin/roles # define role→scope mappings
|
||||
POST /admin/roles # define role→scope mappings
|
||||
POST /admin/tenants # create tenant/install entries
|
||||
POST /admin/keys/rotate # rotate signing key (zero-downtime)
|
||||
GET /admin/metrics # Prometheus exposition (token issue rates, errors)
|
||||
@@ -300,10 +306,10 @@ Declared client `audiences` flow through to the issued JWT `aud` claim and the t
|
||||
|
||||
## 11) Integration hard lines (what resource servers must enforce)
|
||||
|
||||
Every Stella Ops service that consumes Authority tokens **must**:
|
||||
Every Stella Ops service that consumes Authority tokens **must**:
|
||||
|
||||
1. Verify JWT signature (`kid` in JWKS), `iss`, `aud`, `exp`, `nbf`.
|
||||
2. Enforce **sender‑constraint**:
|
||||
2. Enforce **sender‑constraint**:
|
||||
|
||||
* **DPoP**: validate DPoP proof (`htu`, `htm`, `iat`, `jti`) and match `cnf.jkt`; cache `jti` for replay defense; honor nonce challenges.
|
||||
* **mTLS**: match presented client cert thumbprint to token `cnf.x5t#S256`.
|
||||
@@ -316,8 +322,8 @@ Every Stella Ops service that consumes Authority tokens **must**:
|
||||
## 12) Error surfaces & UX
|
||||
|
||||
* Token endpoint errors follow OAuth2 (`invalid_client`, `invalid_grant`, `invalid_scope`, `unauthorized_client`).
|
||||
* Resource servers use RFC 6750 style (`WWW-Authenticate: DPoP error="invalid_token", error_description="…", dpop_nonce="…" `).
|
||||
* For DPoP nonce challenges, clients retry with the server‑supplied nonce once.
|
||||
* Resource servers use RFC 6750 style (`WWW-Authenticate: DPoP error="invalid_token", error_description="…", dpop_nonce="…" `).
|
||||
* For DPoP nonce challenges, clients retry with the server‑supplied nonce once.
|
||||
|
||||
---
|
||||
|
||||
@@ -425,8 +431,8 @@ authority:
|
||||
* **JWT validation**: wrong `aud`, expired `exp`, skewed `nbf`, stale `kid`.
|
||||
* **DPoP**: invalid `htu`/`htm`, replayed `jti`, stale `iat`, wrong `jkt`, nonce dance.
|
||||
* **mTLS**: wrong client cert, wrong CA, thumbprint mismatch.
|
||||
* **RBAC**: scope enforcement per audience; over‑privileged client denied.
|
||||
* **Rotation**: JWKS rotation while load‑testing; zero‑downtime verification.
|
||||
* **RBAC**: scope enforcement per audience; over‑privileged client denied.
|
||||
* **Rotation**: JWKS rotation while load‑testing; zero‑downtime verification.
|
||||
* **HA**: kill one Authority instance; verify issuance continues; JWKS served by peers.
|
||||
* **Performance**: 1k token issuance/sec on 2 cores with Valkey enabled for jti caching.
|
||||
|
||||
@@ -436,18 +442,18 @@ authority:
|
||||
|
||||
| Threat | Vector | Mitigation |
|
||||
| ------------------- | ---------------- | ------------------------------------------------------------------------------------------ |
|
||||
| Token theft | Copy of JWT | **Short TTL**, **sender‑constraint** (DPoP/mTLS); replay blocked by `jti` cache and nonces |
|
||||
| Token theft | Copy of JWT | **Short TTL**, **sender‑constraint** (DPoP/mTLS); replay blocked by `jti` cache and nonces |
|
||||
| Replay across hosts | Reuse DPoP proof | Enforce `htu`/`htm`, `iat` freshness, `jti` uniqueness; services may require **nonce** |
|
||||
| Impersonation | Fake client | mTLS or `private_key_jwt` with pinned JWK; client registration & rotation |
|
||||
| Key compromise | Signing key leak | HSM/KMS storage, key rotation, audit; emergency key revoke path; narrow token TTL |
|
||||
| Cross‑tenant abuse | Scope elevation | Enforce `aud`, `tid`, `inst` at issuance and resource servers |
|
||||
| Cross‑tenant abuse | Scope elevation | Enforce `aud`, `tid`, `inst` at issuance and resource servers |
|
||||
| Downgrade to bearer | Strip DPoP | Resource servers require DPoP/mTLS based on `aud`; reject bearer without `cnf` |
|
||||
|
||||
---
|
||||
|
||||
## 17) Deployment & HA
|
||||
|
||||
* **Stateless** microservice, containerized; run ≥ 2 replicas behind LB.
|
||||
* **Stateless** microservice, containerized; run ≥ 2 replicas behind LB.
|
||||
* **DB**: HA Postgres (or MySQL) for clients/roles; **Valkey** for device codes, DPoP nonces/jtis.
|
||||
* **Secrets**: mount client JWKs via K8s Secrets/HashiCorp Vault; signing keys via KMS.
|
||||
* **Backups**: DB daily; Valkey not critical (ephemeral).
|
||||
@@ -464,7 +470,7 @@ authority:
|
||||
|
||||
---
|
||||
|
||||
## 19) Quick reference — wire examples
|
||||
## 19) Quick reference — wire examples
|
||||
|
||||
**Access token (payload excerpt)**
|
||||
|
||||
@@ -501,7 +507,7 @@ Signer validates that `hash(JWK)` in the proof matches `cnf.jkt` in the token.
|
||||
|
||||
## 20) Rollout plan
|
||||
|
||||
1. **MVP**: Client Credentials (private_key_jwt + DPoP), JWKS, short OpToks, per‑audience scopes.
|
||||
2. **Add**: mTLS‑bound tokens for Signer/Attestor; device code for CLI; optional introspection.
|
||||
1. **MVP**: Client Credentials (private_key_jwt + DPoP), JWKS, short OpToks, per‑audience scopes.
|
||||
2. **Add**: mTLS‑bound tokens for Signer/Attestor; device code for CLI; optional introspection.
|
||||
3. **Hardening**: DPoP nonce support; full audit pipeline; HA tuning.
|
||||
4. **UX**: Tenant/installation admin UI; role→scope editors; client bootstrap wizards.
|
||||
4. **UX**: Tenant/installation admin UI; role→scope editors; client bootstrap wizards.
|
||||
|
||||
@@ -1,5 +1,9 @@
|
||||
# Benchmark
|
||||
|
||||
> **Dual Purpose:** This documentation covers two aspects:
|
||||
> - **Performance Benchmarking** (Production) — BenchmarkDotNet harnesses in `src/Bench/` for scanner, policy, and notification performance testing
|
||||
> - **Competitive Benchmarking** (Planned) — Accuracy comparison framework in `src/Scanner/__Libraries/StellaOps.Scanner.Benchmark/`
|
||||
|
||||
**Status:** Implemented
|
||||
**Source:** `src/Bench/`
|
||||
**Owner:** Platform Team
|
||||
|
||||
@@ -64,6 +64,25 @@ stella advisoryai index rebuild [--json]
|
||||
|
||||
Rebuilds the deterministic AKS index from local markdown, OpenAPI, and Doctor metadata sources.
|
||||
|
||||
### `stella advisoryai sources prepare`
|
||||
|
||||
```bash
|
||||
stella advisoryai sources prepare \
|
||||
[--repo-root <path>] \
|
||||
[--docs-allowlist <path>] \
|
||||
[--docs-manifest-output <path>] \
|
||||
[--openapi-output <path>] \
|
||||
[--doctor-seed <path>] \
|
||||
[--doctor-controls-output <path>] \
|
||||
[--overwrite] \
|
||||
[--json]
|
||||
```
|
||||
|
||||
Generates deterministic AKS seed artifacts before index rebuild:
|
||||
- docs manifest from the allow-list
|
||||
- aggregated OpenAPI artifact output path
|
||||
- doctor controls projection JSON enriched from configured seed + discovered Doctor check catalog metadata when available
|
||||
|
||||
## 2 · `stella sources ingest --dry-run`
|
||||
|
||||
### 2.1 Synopsis
|
||||
|
||||
8
docs/modules/eventing/README.md
Normal file
8
docs/modules/eventing/README.md
Normal file
@@ -0,0 +1,8 @@
|
||||
# Eventing Module
|
||||
|
||||
> **Status: Draft/Planned.** The event envelope SDK is currently in design phase. Implementation is planned for the Timeline and TimelineIndexer modules and will be integrated across all services via `src/__Libraries/StellaOps.Eventing/`. No standalone `src/Eventing/` module exists.
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Event Envelope Schema](event-envelope-schema.md)
|
||||
- [Timeline UI](timeline-ui.md)
|
||||
40
docs/modules/extensions/README.md
Normal file
40
docs/modules/extensions/README.md
Normal file
@@ -0,0 +1,40 @@
|
||||
# Extensions (IDE Plugins)
|
||||
|
||||
> IDE integration plugins for Stella Ops, enabling release management and configuration validation from within VS Code and JetBrains IDEs.
|
||||
|
||||
## Purpose
|
||||
|
||||
Provides IDE integration for Stella Ops via VS Code and JetBrains plugins, allowing developers to manage releases, view environments, and validate configurations without leaving their editor. Extensions act as thin clients consuming existing Orchestrator and Router APIs, bringing operational visibility directly into the development workflow.
|
||||
|
||||
## Quick Links
|
||||
|
||||
- [Architecture](./architecture.md) - Technical design and implementation details
|
||||
|
||||
## Status
|
||||
|
||||
| Attribute | Value |
|
||||
|-----------|-------|
|
||||
| **Maturity** | Beta |
|
||||
| **Source** | `src/Extensions/` |
|
||||
|
||||
## Key Features
|
||||
|
||||
- **VS Code extension:** Tree views for releases and environments, CodeLens annotations for `stella.yaml` files, command palette integration, status bar widgets
|
||||
- **JetBrains plugin:** Tool windows with Releases/Environments/Deployments tabs, YAML annotator for configuration validation, status bar integration, action menus
|
||||
- **Unified configuration:** Both plugins share the same Orchestrator API surface and authentication flow
|
||||
- **Real-time updates:** Live status refresh for release pipelines and environment health
|
||||
|
||||
## Dependencies
|
||||
|
||||
### Upstream (this module depends on)
|
||||
- **Orchestrator** - Release state, pipeline status, and environment data via HTTP API
|
||||
- **Authority** - OAuth token-based authentication and scope enforcement
|
||||
|
||||
### Downstream (modules that depend on this)
|
||||
- None (end-user development tools; no other modules consume Extensions)
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Orchestrator](../orchestrator/) - Backend API consumed by extensions
|
||||
- [Authority](../authority/) - Authentication provider
|
||||
- [CLI](../cli/) - Command-line alternative for the same operations
|
||||
117
docs/modules/extensions/architecture.md
Normal file
117
docs/modules/extensions/architecture.md
Normal file
@@ -0,0 +1,117 @@
|
||||
# Extensions (IDE Plugins) Architecture
|
||||
|
||||
> Technical architecture for VS Code and JetBrains IDE plugins providing Stella Ops integration.
|
||||
|
||||
## Overview
|
||||
|
||||
The Extensions module consists of two independent IDE plugins that provide developer-facing integration with the Stella Ops platform. Both plugins are pure HTTP clients that consume the Orchestrator and Router APIs; they do not host any services, expose endpoints, or maintain local databases. Authentication is handled through OAuth tokens obtained from the Authority service.
|
||||
|
||||
## Design Principles
|
||||
|
||||
1. **Thin client** - Extensions contain no business logic; all state and decisions live in backend services
|
||||
2. **Consistent experience** - Both plugins expose equivalent functionality despite different technology stacks
|
||||
3. **Non-blocking** - All API calls are asynchronous; the IDE remains responsive during network operations
|
||||
4. **Offline-tolerant** - Graceful degradation when the Stella Ops backend is unreachable
|
||||
|
||||
## Components
|
||||
|
||||
```
|
||||
Extensions/
|
||||
├── vscode-stella-ops/ # VS Code extension (TypeScript)
|
||||
│ ├── src/
|
||||
│ │ ├── extension.ts # Entry point and activation
|
||||
│ │ ├── providers/
|
||||
│ │ │ ├── ReleaseTreeProvider.ts # TreeView: releases
|
||||
│ │ │ ├── EnvironmentTreeProvider.ts# TreeView: environments
|
||||
│ │ │ └── CodeLensProvider.ts # CodeLens for stella.yaml
|
||||
│ │ ├── commands/ # Command palette handlers
|
||||
│ │ ├── views/
|
||||
│ │ │ └── webview/ # Webview panels (detail views)
|
||||
│ │ ├── statusbar/
|
||||
│ │ │ └── StatusBarManager.ts # Status bar integration
|
||||
│ │ └── api/
|
||||
│ │ └── OrchestratorClient.ts # HTTP client for Orchestrator API
|
||||
│ ├── package.json # Extension manifest
|
||||
│ └── tsconfig.json
|
||||
│
|
||||
└── jetbrains-stella-ops/ # JetBrains plugin (Kotlin)
|
||||
├── src/main/kotlin/
|
||||
│ ├── toolwindow/
|
||||
│ │ ├── ReleasesToolWindow.kt # Tool window: Releases tab
|
||||
│ │ ├── EnvironmentsToolWindow.kt # Tool window: Environments tab
|
||||
│ │ └── DeploymentsToolWindow.kt # Tool window: Deployments tab
|
||||
│ ├── annotator/
|
||||
│ │ └── StellaYamlAnnotator.kt # YAML file annotator
|
||||
│ ├── actions/ # Action menu handlers
|
||||
│ ├── statusbar/
|
||||
│ │ └── StellaStatusBarWidget.kt # Status bar widget
|
||||
│ └── api/
|
||||
│ └── OrchestratorClient.kt # HTTP client for Orchestrator API
|
||||
├── src/main/resources/
|
||||
│ └── META-INF/plugin.xml # Plugin descriptor
|
||||
└── build.gradle.kts
|
||||
```
|
||||
|
||||
## Data Flow
|
||||
|
||||
```
|
||||
[Developer IDE] --> [Extension/Plugin]
|
||||
│
|
||||
├── GET /api/v1/releases/* ──────> [Orchestrator API]
|
||||
├── GET /api/v1/environments/* ──> [Orchestrator API]
|
||||
├── POST /api/v1/promotions/* ──-> [Orchestrator API]
|
||||
└── POST /oauth/token ──────────-> [Authority]
|
||||
```
|
||||
|
||||
1. **Authentication:** On activation, the extension initiates an OAuth device-code or browser-redirect flow against Authority. The obtained access token is stored in the IDE's secure credential store (VS Code `SecretStorage`, JetBrains `PasswordSafe`).
|
||||
2. **Data retrieval:** Tree views and tool windows issue HTTP GET requests to the Orchestrator API on initial load and on manual/timed refresh.
|
||||
3. **Actions:** Approve/reject/promote commands issue HTTP POST requests to the Orchestrator release control endpoints.
|
||||
4. **Configuration validation:** The CodeLens provider (VS Code) and YAML annotator (JetBrains) parse `stella.yaml` files locally and highlight configuration issues inline.
|
||||
|
||||
## VS Code Extension Details
|
||||
|
||||
### Tree Views
|
||||
- **Releases:** Hierarchical view of releases grouped by environment, showing status, version, and promotion eligibility
|
||||
- **Environments:** Flat list of configured environments with health indicators
|
||||
|
||||
### CodeLens
|
||||
- Inline annotations above `stella.yaml` entries showing the current deployment status of the referenced release
|
||||
- Click-to-promote actions directly from the YAML file
|
||||
|
||||
### Status Bar
|
||||
- Compact widget showing the number of pending promotions and overall platform health
|
||||
|
||||
### Webview Panels
|
||||
- Detail panels for release timelines, evidence summaries, and deployment logs
|
||||
|
||||
## JetBrains Plugin Details
|
||||
|
||||
### Tool Windows
|
||||
- **Releases tab:** Table view of all releases with sortable columns (version, environment, status, timestamp)
|
||||
- **Environments tab:** Environment cards with health status and current deployments
|
||||
- **Deployments tab:** Active and recent deployment history with log links
|
||||
|
||||
### YAML Annotator
|
||||
- Real-time validation of `stella.yaml` files with gutter icons and tooltip messages for configuration issues
|
||||
|
||||
### Action Menus
|
||||
- Context-sensitive actions (promote, approve, reject) available from tool window rows and editor context menus
|
||||
|
||||
## Security Considerations
|
||||
|
||||
- **Token storage:** OAuth tokens are stored exclusively in the IDE's built-in secure credential store; never persisted to disk in plaintext
|
||||
- **Scope enforcement:** Extensions request only the scopes necessary for read operations and promotions (`release:read`, `release:promote`, `env:read`)
|
||||
- **TLS enforcement:** All HTTP communication uses HTTPS; certificate validation is not bypassed
|
||||
- **No secrets in configuration:** The `stella.yaml` file contains no credentials; integration secrets are managed by the Authority and Integrations modules
|
||||
|
||||
## Performance Characteristics
|
||||
|
||||
- Tree view refresh is debounced to avoid excessive API calls (default: 30-second minimum interval)
|
||||
- API responses are cached locally with short TTL (60 seconds) to reduce latency on repeated navigation
|
||||
- Webview panels and tool windows load data lazily on first open
|
||||
|
||||
## References
|
||||
|
||||
- [Module README](./README.md)
|
||||
- [Orchestrator Architecture](../orchestrator/architecture.md)
|
||||
- [Authority Architecture](../authority/architecture.md)
|
||||
@@ -1,5 +1,7 @@
|
||||
# Facet Sealing Architecture
|
||||
|
||||
> **Status: Production (Cross-Module Library).** Facet Sealing is a fully implemented subsystem with its core library at `src/__Libraries/StellaOps.Facet/` (30 source files) and integration points spanning **Scanner** (extraction via `FacetSealExtractor`, storage via `PostgresFacetSealStore` in `scanner.facet_seals` table), **Policy** (drift and quota enforcement via `FacetQuotaGate`), **Zastava** (admission validation via `FacetAdmissionValidator`), and **CLI** (`seal`, `drift`, `vex-gen` commands). Comprehensive test coverage exists across 17 test files. This documentation covers the cross-cutting architecture.
|
||||
|
||||
> **Ownership:** Scanner Guild, Policy Guild
|
||||
> **Audience:** Service owners, platform engineers, security architects
|
||||
> **Related:** [Platform Architecture](../platform/architecture-overview.md), [Scanner Architecture](../scanner/architecture.md), [Replay Architecture](../replay/architecture.md), [Policy Engine](../policy/architecture.md)
|
||||
|
||||
@@ -2,6 +2,12 @@
|
||||
|
||||
> Derived from Reference Architecture Advisory and Router Architecture Specification
|
||||
|
||||
> **Dual-location clarification (updated 2026-02-22).** Both `src/Gateway/` and `src/Router/` contain a project named `StellaOps.Gateway.WebService`. They are **different implementations** serving complementary roles:
|
||||
> - **`src/Gateway/`** (this module) — the simplified HTTP ingress gateway focused on authentication, routing to microservices via binary protocol, and OpenAPI aggregation.
|
||||
> - **`src/Router/`** — the evolved "Front Door" gateway with advanced features: configurable route tables (`GatewayRouteCatalog`), reverse proxy, SPA hosting, WebSocket support, Valkey messaging transport, and extended Authority integration.
|
||||
>
|
||||
> The Router version (`src/Router/`) appears to be the current canonical deployment target. This Gateway version may represent a simplified or legacy configuration. Operators should verify which is deployed in their environment. See also [Router Architecture](../router/architecture.md).
|
||||
|
||||
> **Scope.** The Gateway WebService is the single HTTP ingress point for all external traffic. It authenticates requests via Authority (DPoP/mTLS), routes to microservices via the Router binary protocol, aggregates OpenAPI specifications, and enforces tenant isolation.
|
||||
> **Ownership:** Platform Guild
|
||||
|
||||
|
||||
@@ -37,7 +37,14 @@ Graph Indexer + Graph API build the tenant-scoped knowledge graph that powers bl
|
||||
|
||||
## Operations & runbook (Sprint 030)
|
||||
- Dashboards: import `Observability/graph-api-grafana.json` (panels for latency, budget denials, overlay cache ratio, export latency). Apply tenant filter in every panel.
|
||||
- Health checks: `/healthz` should be 200; search/query/paths/diff/export endpoints require `X-Stella-Tenant`, `Authorization`, and scopes (`graph:read/query/export`).
|
||||
- Health checks: `/healthz` should be 200; search/query/paths/diff/export endpoints require tenant context, `Authorization`, and graph scopes (`graph:read/query/export`).
|
||||
- Tenant context resolution:
|
||||
- Canonical header: `X-StellaOps-Tenant`.
|
||||
- Compatibility headers: `X-Stella-Tenant`, `X-Tenant-Id` (migration-only).
|
||||
- Conflicting tenant values across headers/claims are rejected deterministically with `400 GRAPH_VALIDATION_FAILED`.
|
||||
- Scope enforcement:
|
||||
- Graph endpoints authorize against claim-based policies (`Graph.ReadOrQuery`, `Graph.Query`, `Graph.Export`).
|
||||
- Header scope compatibility (`X-StellaOps-Scopes`, `X-Stella-Scopes`) is bridged once at authentication and then evaluated only through policies.
|
||||
- Key metrics (new):
|
||||
- `graph_tile_latency_seconds` histogram (label `route`); alert when p95 > 1.5s for 5m.
|
||||
- `graph_query_budget_denied_total` counter (label `reason`); investigate spikes (>50 in 5m).
|
||||
|
||||
@@ -35,11 +35,25 @@
|
||||
- `POST /graph/edges/metadata` — batch query for edge explanations; request contains `EdgeIds[]`, response includes `EdgeTileWithMetadata[]` with full provenance.
|
||||
- `GET /graph/edges/{edgeId}/metadata` — single edge metadata with explanation, via, provenance, and evidence references.
|
||||
- `GET /graph/edges/path/{sourceNodeId}/{targetNodeId}` — returns all edges on the shortest path between two nodes, each with metadata.
|
||||
- `GET /graph/edges/by-reason/{reason}` — query edges by `EdgeReason` enum (e.g., `SbomDependency`, `AdvisoryAffects`, `VexStatement`, `RuntimeTrace`).
|
||||
- `GET /graph/edges/by-evidence?evidenceType=&evidenceRef=` — query edges by evidence reference.
|
||||
- Legacy: `GET /graph/nodes/{id}`, `POST /graph/query/saved`, `GET /graph/impact/{advisoryKey}`, `POST /graph/overlay/policy` remain in spec but should align to the NDJSON surfaces above as they are brought forward.
|
||||
|
||||
### 3.1) Edge Metadata Contracts
|
||||
- `GET /graph/edges/by-reason/{reason}` — query edges by `EdgeReason` enum (e.g., `SbomDependency`, `AdvisoryAffects`, `VexStatement`, `RuntimeTrace`).
|
||||
- `GET /graph/edges/by-evidence?evidenceType=&evidenceRef=` — query edges by evidence reference.
|
||||
- Legacy: `GET /graph/nodes/{id}`, `POST /graph/query/saved`, `GET /graph/impact/{advisoryKey}`, `POST /graph/overlay/policy` remain in spec but should align to the NDJSON surfaces above as they are brought forward.
|
||||
|
||||
### 3.1) Tenant and auth resolution contract (Sprint 20260222.058)
|
||||
|
||||
- Graph uses a single tenant resolver path (`GraphRequestContextResolver`) across search/query/paths/diff/lineage/export and edge-metadata endpoints.
|
||||
- Tenant source precedence and compatibility:
|
||||
- claim: `stellaops:tenant` (with bounded aliases `tid`, `tenant_id`)
|
||||
- headers: `X-StellaOps-Tenant` (canonical), then migration headers `X-Stella-Tenant` and `X-Tenant-Id`
|
||||
- Deterministic failures:
|
||||
- missing tenant: `400 GRAPH_VALIDATION_FAILED`
|
||||
- conflicting tenant claim/header values: `400 GRAPH_VALIDATION_FAILED`
|
||||
- missing auth: `401 GRAPH_UNAUTHORIZED`
|
||||
- missing scope: `403 GRAPH_FORBIDDEN`
|
||||
- Scope checks are policy-driven (`Graph.ReadOrQuery`, `Graph.Query`, `Graph.Export`) and no endpoint directly trusts raw scope headers.
|
||||
- Rate limiting and audit logging use the resolved tenant context; authenticated flows no longer collapse to ambiguous `"unknown"` tenant keys.
|
||||
|
||||
### 3.2) Edge Metadata Contracts
|
||||
|
||||
The edge metadata system provides explainability for graph relationships:
|
||||
|
||||
|
||||
53
docs/modules/integrations/README.md
Normal file
53
docs/modules/integrations/README.md
Normal file
@@ -0,0 +1,53 @@
|
||||
# Integrations
|
||||
|
||||
> Central catalog and connector hub for managing external tool integrations across the Stella Ops platform.
|
||||
|
||||
## Purpose
|
||||
|
||||
Integrations provides a unified API for registering, configuring, and health-checking third-party service connectors such as GitHub, GitLab, and Harbor. It serves as the single source of truth for all external tool configurations, enabling other modules to discover and consume integration details without embedding provider-specific logic.
|
||||
|
||||
## Quick Links
|
||||
|
||||
- [Architecture](./architecture.md) - Technical design and implementation details
|
||||
|
||||
## Status
|
||||
|
||||
| Attribute | Value |
|
||||
|-----------|-------|
|
||||
| **Maturity** | Production |
|
||||
| **Source** | `src/Integrations/` |
|
||||
|
||||
## Key Features
|
||||
|
||||
- **Plugin-based connector architecture:** Extensible provider system with built-in connectors for GitHub App, GitLab, Harbor, and in-memory testing
|
||||
- **Health checks:** Per-integration health probing with status tracking and alerting
|
||||
- **Credential management:** AuthRef URI scheme for vault-referenced credentials; no plaintext secrets stored in the integration catalog
|
||||
- **Tenant isolation:** All integrations are scoped to a tenant; cross-tenant access is prohibited
|
||||
- **AiCodeGuard pipeline support:** Integrations participate in AI-assisted code review pipelines
|
||||
- **Plugin discovery:** Automatic detection and registration of available connector plugins
|
||||
|
||||
## Dependencies
|
||||
|
||||
### Upstream (this module depends on)
|
||||
- **Authority** - Authentication and authorization for API access
|
||||
- **Plugin Framework** - Plugin lifecycle and discovery infrastructure
|
||||
|
||||
### Downstream (modules that depend on this)
|
||||
- **Scanner** - Retrieves repository and registry connection details for scanning operations
|
||||
- **Orchestrator** - Reads integration configs for CI/CD pipeline orchestration
|
||||
- **Signals** - Uses integration metadata for SCM webhook routing and validation
|
||||
|
||||
## Configuration
|
||||
|
||||
Key settings:
|
||||
- PostgreSQL connection (database: `stellaops_integrations`)
|
||||
- Authority audiences and scopes
|
||||
- Plugin search paths for connector discovery
|
||||
- Health check intervals and timeout thresholds
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Plugin Framework](../plugin/) - Underlying plugin infrastructure
|
||||
- [Scanner](../scanner/) - Primary consumer of integration configs
|
||||
- [Orchestrator](../orchestrator/) - Pipeline orchestration using integrations
|
||||
- [Signals](../signals/) - SCM webhook processing
|
||||
122
docs/modules/integrations/architecture.md
Normal file
122
docs/modules/integrations/architecture.md
Normal file
@@ -0,0 +1,122 @@
|
||||
# Integrations Architecture
|
||||
|
||||
> Technical architecture for the central integration catalog and connector hub.
|
||||
|
||||
## Overview
|
||||
|
||||
The Integrations module provides a unified service for managing connections to external tools (SCM providers, container registries, CI systems). It uses a plugin-based architecture where each external provider is implemented as a connector plugin, enabling new integrations to be added without modifying core logic. All integration state is persisted in PostgreSQL with tenant-scoped isolation.
|
||||
|
||||
## Design Principles
|
||||
|
||||
1. **Plugin extensibility** - New providers are added as plugins, not core code changes
|
||||
2. **Credential indirection** - Secrets are referenced via AuthRef URIs pointing to external vaults; the integration catalog never stores raw credentials
|
||||
3. **Tenant isolation** - Every integration record is scoped to a tenant; queries are filtered at the data layer
|
||||
4. **Health-first** - Every integration has a health check contract; unhealthy integrations surface alerts
|
||||
|
||||
## Components
|
||||
|
||||
```
|
||||
Integrations/
|
||||
├── StellaOps.Integrations.WebService/ # HTTP API (Minimal API)
|
||||
├── StellaOps.Integrations.Core/ # Business logic and plugin orchestration
|
||||
├── StellaOps.Integrations.Contracts/ # Shared DTOs and interfaces
|
||||
├── StellaOps.Integrations.Persistence/ # PostgreSQL via IntegrationDbContext
|
||||
├── Plugins/
|
||||
│ ├── StellaOps.Integrations.GitHubApp/ # GitHub App connector
|
||||
│ ├── StellaOps.Integrations.GitLab/ # GitLab connector
|
||||
│ ├── StellaOps.Integrations.Harbor/ # Harbor registry connector
|
||||
│ └── StellaOps.Integrations.InMemory/ # In-memory connector for testing
|
||||
└── __Tests/
|
||||
└── StellaOps.Integrations.Tests/ # Unit and integration tests
|
||||
```
|
||||
|
||||
## Data Flow
|
||||
|
||||
```
|
||||
[External Tool] <── health check ── [Integrations WebService]
|
||||
│
|
||||
[Client Module] ── GET /integrations ──> │
|
||||
│
|
||||
[Admin/CLI] ── POST /integrations ──> │ ── [IntegrationDbContext] ── [PostgreSQL]
|
||||
│
|
||||
[Plugin Loader] ── discover plugins ──> │ ── [Plugin Framework]
|
||||
```
|
||||
|
||||
1. **Registration:** An administrator or automated onboarding flow creates an integration record via `POST /integrations`, providing the provider type, configuration JSON, and an AuthRef URI for credentials.
|
||||
2. **Discovery:** The `PluginLoader` scans configured paths for available connector plugins and registers their capabilities in the plugin metadata table.
|
||||
3. **Health checking:** A background timer invokes each integration's health check endpoint (provider-specific) and persists the result. Failures update the integration status and emit alerts.
|
||||
4. **Consumption:** Downstream modules (Scanner, Orchestrator, Signals) query `GET /integrations/{type}` to retrieve connection details for a specific provider type, filtered by tenant.
|
||||
|
||||
## Database Schema
|
||||
|
||||
Database: `stellaops_integrations` (PostgreSQL)
|
||||
|
||||
| Table | Purpose |
|
||||
|-------|---------|
|
||||
| `integrations` | Primary catalog of registered integrations (id, tenant_id, type, name, config_json, auth_ref_uri, status, created_at, updated_at) |
|
||||
| `plugin_metadata` | Discovered plugin descriptors (plugin_id, type, version, capabilities, path) |
|
||||
| `health_checks` | Health check history (integration_id, checked_at, status, latency_ms, error_message) |
|
||||
| `audit_logs` | Audit trail for integration CRUD operations (actor, action, integration_id, timestamp, details) |
|
||||
|
||||
## Endpoints
|
||||
|
||||
### Integration CRUD (`/api/v1/integrations`)
|
||||
- `GET /` - List integrations for current tenant (optional `?type=` filter)
|
||||
- `GET /{id}` - Get integration by ID
|
||||
- `POST /` - Register a new integration (body: type, name, config, authRefUri)
|
||||
- `PUT /{id}` - Update integration configuration
|
||||
- `DELETE /{id}` - Remove integration (soft delete with audit)
|
||||
|
||||
### Health (`/api/v1/integrations`)
|
||||
- `POST /{id}/health-check` - Trigger an on-demand health check for a specific integration
|
||||
- `GET /{id}/health` - Retrieve latest health status and history
|
||||
|
||||
### Type-filtered queries
|
||||
- `GET /by-type/{type}` - List integrations of a specific provider type (e.g., `github`, `harbor`)
|
||||
|
||||
### Plugin discovery
|
||||
- `POST /plugins/discover` - Trigger plugin discovery scan and register new connectors
|
||||
|
||||
## Plugin Architecture
|
||||
|
||||
Each connector plugin implements a standard interface:
|
||||
|
||||
```csharp
|
||||
public interface IIntegrationPlugin
|
||||
{
|
||||
string ProviderType { get; }
|
||||
Task<IntegrationHealthResult> CheckHealthAsync(IntegrationConfig config, CancellationToken ct);
|
||||
Task<bool> ValidateConfigAsync(JsonDocument config, CancellationToken ct);
|
||||
}
|
||||
```
|
||||
|
||||
**Built-in plugins:**
|
||||
- **GitHubApp** - GitHub App installation authentication, repository listing, webhook setup
|
||||
- **GitLab** - Personal/project access token authentication, project discovery
|
||||
- **Harbor** - Robot account authentication, project and repository enumeration
|
||||
- **InMemory** - Deterministic test double for integration tests and offline development
|
||||
|
||||
## Security Considerations
|
||||
|
||||
- **AuthRef URI credential model:** Credentials are stored in an external vault (e.g., HashiCorp Vault, Azure Key Vault). The integration catalog stores only the URI reference (`authref://vault/path/to/secret`), never the raw secret.
|
||||
- **Tenant scoping:** All database queries include a mandatory tenant filter enforced at the `DbContext` level via query filters.
|
||||
- **Audit logging:** Every create, update, and delete operation is recorded in the audit log with actor identity, timestamp, and change details.
|
||||
- **Plugin sandboxing:** Connector plugins run within the Plugin Framework's trust boundary; untrusted plugins are process-isolated.
|
||||
|
||||
## Observability
|
||||
|
||||
- **Metrics:** `integration_health_status{type,tenant}`, `integration_health_latency_ms`, `integration_crud_total{action}`
|
||||
- **Logs:** Structured logs with `integrationId`, `tenantId`, `providerType`, `action`
|
||||
- **Traces:** Spans for health checks, plugin discovery, and CRUD operations
|
||||
|
||||
## Performance Characteristics
|
||||
|
||||
- Health checks run on a configurable interval (default: 5 minutes) per integration
|
||||
- Plugin discovery is triggered on startup and on-demand; results are cached
|
||||
- Integration queries use indexed tenant_id + type columns for fast filtering
|
||||
|
||||
## References
|
||||
|
||||
- [Module README](./README.md)
|
||||
- [Plugin Framework Architecture](../plugin/architecture.md)
|
||||
- [Scanner Architecture](../scanner/architecture.md)
|
||||
@@ -12,7 +12,10 @@
|
||||
* Attachments are **links** (UI/attestation pages); Notify **does not** attach SBOMs or large blobs to messages.
|
||||
* Secrets for channels (Slack tokens, SMTP creds) are **referenced**, not stored raw in the database.
|
||||
* **2025-11-02 module boundary.** Maintain `src/Notify/` as the reusable notification toolkit (engine, storage, queue, connectors) and `src/Notifier/` as the Notifications Studio host that composes those libraries. Do not merge directories without an approved packaging RFC that covers build impacts, offline kit parity, and cross-module governance.
|
||||
* **API versioning.** `/api/v1/notify` is the canonical UI and CLI surface. `/api/v2/notify` remains compatibility-only until v2-only features are merged into v1 or explicitly deprecated; Gateway should provide v2->v1 routing where needed.
|
||||
* **API versioning (updated 2026-02-22).** The API is split across two services:
|
||||
* **Notify** (`src/Notify/`) exposes `/api/v1/notify` — the core notification toolkit (rules, channels, deliveries, templates). This is the lean, canonical API surface.
|
||||
* **Notifier** (`src/Notifier/`) exposes `/api/v2/notify` — the full Notifications Studio with enterprise features (escalation policies, on-call schedules, storm breaker, inbox, retention, simulation, quiet hours, 73+ routes). Notifier also maintains select `/api/v1/notify` endpoints for backward compatibility.
|
||||
* Both versions are **actively maintained and production**. v2 is NOT deprecated — it is the enterprise-tier API hosted by the Notifier Studio service. The previous claim that v2 was "compatibility-only" is stale and has been corrected.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -8,6 +8,10 @@ This dossier summarises the end-to-end runtime topology after the Aggregation-On
|
||||
|
||||
---
|
||||
|
||||
Tenant behavior inventory for Platform endpoints: `docs/modules/platform/tenant-endpoint-classification.md`
|
||||
|
||||
---
|
||||
|
||||
> Need a quick orientation? The [Developer Quickstart](../../dev/onboarding/dev-quickstart.md) (29-Nov-2025 advisory) captures the core repositories, determinism checks, DSSE conventions, and starter tasks that explain how the platform pieces fit together.
|
||||
|
||||
> Testing strategy models and CI lanes live in `docs/technical/testing/testing-strategy-models.md`, with the source catalog in `docs/technical/testing/TEST_CATALOG.yml`.
|
||||
|
||||
32
docs/modules/platform/tenant-endpoint-classification.md
Normal file
32
docs/modules/platform/tenant-endpoint-classification.md
Normal file
@@ -0,0 +1,32 @@
|
||||
# Platform Endpoint Tenant Classification
|
||||
|
||||
## Scope
|
||||
- Service: `src/Platform/StellaOps.Platform.WebService/Endpoints`
|
||||
- Date: 2026-02-22
|
||||
- Purpose: classify endpoint files by tenant behavior and document intentional non-resolver paths.
|
||||
|
||||
## Classification Ledger
|
||||
| Endpoint file | Category | Tenant source | Auth baseline | Notes |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| `AdministrationTrustSigningMutationEndpoints.cs` | tenant-required business | `PlatformRequestContextResolver` | platform policy groups | Tenant-scoped key/issuer/certificate operations. |
|
||||
| `AnalyticsEndpoints.cs` | tenant-required business | `PlatformRequestContextResolver` | `PlatformPolicies.AnalyticsRead` | Aggregation paths require tenant context for cache keys and result shaping. |
|
||||
| `ContextEndpoints.cs` | tenant-required business | `PlatformRequestContextResolver` | `PlatformPolicies.ContextRead/Write` | Context preferences keyed by `(tenant, actor)`. |
|
||||
| `EnvironmentSettingsEndpoints.cs` | global/system | none | `AllowAnonymous` | Setup/bootstrap configuration payload for frontend shell. |
|
||||
| `EnvironmentSettingsAdminEndpoints.cs` | global/system | none | `PlatformPolicies.SetupRead/SetupAdmin` | DB setting overrides are setup-admin operations, not tenant business data. |
|
||||
| `EvidenceThreadEndpoints.cs` | tenant-required business | `PlatformRequestContextResolver` | evidence policy groups | Evidence queries are tenant-scoped. |
|
||||
| `FederationTelemetryEndpoints.cs` | tenant-required business | `PlatformRequestContextResolver` | federation policy groups | Consent/status/bundles remain tenant scoped. |
|
||||
| `FunctionMapEndpoints.cs` | tenant-required business | `PlatformRequestContextResolver` | function-map policy groups | Tenant-scoped function map catalog and operations. |
|
||||
| `IntegrationReadModelEndpoints.cs` | tenant-required business | `PlatformRequestContextResolver` | `PlatformPolicies.IntegrationsRead` | Feed/vex source projections require tenant context. |
|
||||
| `LegacyAliasEndpoints.cs` | tenant-required business | `PlatformRequestContextResolver` | same as canonical mapped policies | Compatibility aliases enforce same tenant requirements as canonical endpoints. |
|
||||
| `MigrationAdminEndpoints.cs` | global/system | none | `PlatformPolicies.SetupAdmin` | Migration operations are control-plane/system admin functions. |
|
||||
| `PackAdapterEndpoints.cs` | tenant-required business | `PlatformRequestContextResolver` | pack adapter policies | Release-pack adaptation paths are tenant-scoped. |
|
||||
| `PlatformEndpoints.cs` | tenant-required business (plus guarded tenant-param admin reads) | `PlatformRequestContextResolver` + route tenant parity check | health/quota/onboarding/preferences/search/metadata policy groups | Route tenant IDs are now validated against resolved tenant (`tenant_forbidden` on mismatch). |
|
||||
| `PolicyInteropEndpoints.cs` | tenant-required business | `PlatformRequestContextResolver` | policy interop policy groups | Import/export and interop views are tenant-scoped. |
|
||||
| `ReleaseControlEndpoints.cs` | tenant-required business | `PlatformRequestContextResolver` | release-control policy groups | Bundle/version/materialization operations use tenant-bound store calls. |
|
||||
| `ReleaseReadModelEndpoints.cs` | tenant-required business | `PlatformRequestContextResolver` | release-read policies | Run/activity/release projections are tenant scoped. |
|
||||
| `ScoreEndpoints.cs` | tenant-required business | `PlatformRequestContextResolver` | score policies | Score history/replay/verify operations are tenant scoped. |
|
||||
| `SecurityReadModelEndpoints.cs` | tenant-required business | `PlatformRequestContextResolver` | security-read policies | Finding/disposition projections are tenant scoped. |
|
||||
| `SeedEndpoints.cs` | global/system | none | `PlatformPolicies.SetupAdmin` + `STELLAOPS_ENABLE_DEMO_SEED` gate | Explicitly system/admin for controlled demo seeding. |
|
||||
| `SetupEndpoints.cs` | tenant-aware admin | resolver when available; controlled bootstrap setup context when platform not initialized | setup policy groups | Intentional bootstrap bypass is bounded to setup lifecycle checks. |
|
||||
| `TopologyReadModelEndpoints.cs` | tenant-required business | `PlatformRequestContextResolver` | `PlatformPolicies.TopologyRead` | Topology data assembled from tenant-keyed release control stores. |
|
||||
|
||||
43
docs/modules/plugin/README.md
Normal file
43
docs/modules/plugin/README.md
Normal file
@@ -0,0 +1,43 @@
|
||||
# Plugin Framework
|
||||
|
||||
> Universal extensibility framework providing plugin lifecycle management, sandboxing, registry, and SDK for building Stella Ops plugins.
|
||||
|
||||
## Purpose
|
||||
|
||||
The Plugin Framework is a foundational library that provides a consistent plugin lifecycle, trust-based sandboxing, and a registry for managing plugins across all Stella Ops modules. It enables any module to be extended with third-party or custom logic while maintaining security boundaries and operational visibility.
|
||||
|
||||
## Quick Links
|
||||
|
||||
- [Architecture](./architecture.md) - Technical design and implementation details
|
||||
|
||||
## Status
|
||||
|
||||
| Attribute | Value |
|
||||
|-----------|-------|
|
||||
| **Maturity** | Production |
|
||||
| **Source** | `src/Plugin/` |
|
||||
|
||||
## Key Features
|
||||
|
||||
- **IPlugin interface and lifecycle:** Standard contract for all plugins with well-defined states (Discovery, Loading, Initialization, Active, Shutdown)
|
||||
- **Trust levels:** Three-tier trust model -- BuiltIn (in-process), Trusted (isolated with monitoring), Untrusted (sandboxed in separate process)
|
||||
- **Process sandboxing:** Untrusted plugins run in isolated processes with gRPC IPC for communication
|
||||
- **Plugin registry:** Persistent catalog of installed plugins with version tracking (InMemory for tests, PostgreSQL for production)
|
||||
- **SDK and test utilities:** `Plugin.Sdk` for plugin authors, `Plugin.Testing` for deterministic test harnesses
|
||||
- **Capability declarations:** Plugins declare their capabilities; the host enforces capability restrictions at runtime
|
||||
|
||||
## Dependencies
|
||||
|
||||
### Upstream (this module depends on)
|
||||
- None (foundational library with no upstream module dependencies)
|
||||
|
||||
### Downstream (modules that depend on this)
|
||||
- **Integrations** - Uses plugin framework for connector plugins (GitHub, GitLab, Harbor)
|
||||
- **Scanner** - Scanner analysis plugins
|
||||
- **Policy** - Policy evaluation plugins
|
||||
- **Orchestrator** - Worker plugins and task runner extensions
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Integrations](../integrations/) - Primary consumer of plugin framework
|
||||
- [Scanner](../scanner/) - Uses plugins for analysis extensibility
|
||||
156
docs/modules/plugin/architecture.md
Normal file
156
docs/modules/plugin/architecture.md
Normal file
@@ -0,0 +1,156 @@
|
||||
# Plugin Framework Architecture
|
||||
|
||||
> Technical architecture for the universal plugin lifecycle, sandboxing, and registry framework.
|
||||
|
||||
## Overview
|
||||
|
||||
The Plugin Framework provides the core extensibility infrastructure for the Stella Ops platform. It defines how plugins are discovered, loaded, initialized, monitored, and shut down. A three-tier trust model ensures that untrusted plugins cannot compromise the host process, while built-in plugins benefit from zero-overhead in-process execution. The framework is consumed as a library by other modules; it does not expose HTTP endpoints.
|
||||
|
||||
## Design Principles
|
||||
|
||||
1. **Security by default** - Untrusted plugins are process-isolated; capabilities are explicitly declared and enforced
|
||||
2. **Lifecycle consistency** - All plugins follow the same state machine regardless of trust level
|
||||
3. **Zero-overhead for built-ins** - BuiltIn plugins run in-process with direct method calls; no serialization or IPC cost
|
||||
4. **Testability** - Every component has an in-memory or mock alternative for deterministic testing
|
||||
|
||||
## Components
|
||||
|
||||
```
|
||||
Plugin/
|
||||
├── StellaOps.Plugin.Abstractions/ # Core interfaces (IPlugin, PluginInfo, PluginCapabilities)
|
||||
├── StellaOps.Plugin.Host/ # Plugin host, lifecycle manager, trust enforcement
|
||||
├── StellaOps.Plugin.Registry/ # Plugin catalog (InMemory + PostgreSQL backends)
|
||||
├── StellaOps.Plugin.Sandbox/ # Process isolation and gRPC IPC for untrusted plugins
|
||||
├── StellaOps.Plugin.Sdk/ # SDK for plugin authors (base classes, helpers)
|
||||
├── StellaOps.Plugin.Testing/ # Test utilities (mock host, fake registry)
|
||||
├── Samples/
|
||||
│ └── HelloWorld/ # Sample plugin demonstrating the SDK
|
||||
└── __Tests/
|
||||
└── StellaOps.Plugin.Tests/ # Unit and integration tests
|
||||
```
|
||||
|
||||
## Core Interfaces
|
||||
|
||||
### IPlugin
|
||||
|
||||
```csharp
|
||||
public interface IPlugin
|
||||
{
|
||||
PluginInfo Info { get; }
|
||||
PluginCapabilities Capabilities { get; }
|
||||
Task InitializeAsync(IPluginContext context, CancellationToken ct);
|
||||
Task StartAsync(CancellationToken ct);
|
||||
Task StopAsync(CancellationToken ct);
|
||||
}
|
||||
```
|
||||
|
||||
### PluginInfo
|
||||
|
||||
```csharp
|
||||
public sealed record PluginInfo
|
||||
{
|
||||
public required string Id { get; init; }
|
||||
public required string Name { get; init; }
|
||||
public required Version Version { get; init; }
|
||||
public required PluginTrustLevel TrustLevel { get; init; }
|
||||
public string? Description { get; init; }
|
||||
public string? Author { get; init; }
|
||||
}
|
||||
```
|
||||
|
||||
### PluginCapabilities
|
||||
|
||||
Declares what the plugin can do (e.g., `CanScan`, `CanEvaluatePolicy`, `CanConnect`). The host checks capabilities before routing work to a plugin.
|
||||
|
||||
## Plugin Lifecycle
|
||||
|
||||
```
|
||||
[Discovery] --> [Loading] --> [Initialization] --> [Active] --> [Shutdown]
|
||||
│ │ │ │
|
||||
│ │ └── failure ──> [Failed] │
|
||||
│ └── failure ──> [Failed] │
|
||||
└── not found ──> (skip)
|
||||
```
|
||||
|
||||
| State | Description |
|
||||
|-------|-------------|
|
||||
| **Discovery** | Host scans configured paths for assemblies or packages containing `IPlugin` implementations |
|
||||
| **Loading** | Assembly or process is loaded; plugin metadata is read and validated |
|
||||
| **Initialization** | `InitializeAsync` is called with an `IPluginContext` providing configuration and service access |
|
||||
| **Active** | Plugin is ready to receive work; `StartAsync` has completed |
|
||||
| **Shutdown** | `StopAsync` is called during graceful host shutdown or plugin unload |
|
||||
| **Failed** | Plugin encountered an unrecoverable error during loading or initialization; logged and excluded |
|
||||
|
||||
## Trust Levels
|
||||
|
||||
| Level | Execution Model | IPC | Use Case |
|
||||
|-------|----------------|-----|----------|
|
||||
| **BuiltIn** | In-process, direct method calls | None | First-party plugins shipped with the platform |
|
||||
| **Trusted** | In-process with monitoring | None | Vetted third-party plugins with signed manifests |
|
||||
| **Untrusted** | Separate process via `ProcessSandbox` | gRPC | Community or unverified plugins |
|
||||
|
||||
### ProcessSandbox (Untrusted Plugins)
|
||||
|
||||
Untrusted plugins run in a child process managed by `ProcessSandbox`:
|
||||
|
||||
1. **Process creation:** The sandbox spawns a new process with restricted permissions
|
||||
2. **gRPC channel:** A bidirectional gRPC channel is established for host-plugin communication
|
||||
3. **Capability enforcement:** The host proxy only forwards calls matching declared capabilities
|
||||
4. **Resource limits:** CPU and memory limits are enforced at the process level
|
||||
5. **Crash isolation:** If the plugin process crashes, the host logs the failure and marks the plugin as Failed; the host process is unaffected
|
||||
|
||||
## Database Schema
|
||||
|
||||
Database: PostgreSQL (via `PostgresPluginRegistry`)
|
||||
|
||||
| Table | Purpose |
|
||||
|-------|---------|
|
||||
| `plugins` | Registered plugins (id, name, trust_level, status, config_json, registered_at) |
|
||||
| `plugin_versions` | Version history per plugin (plugin_id, version, assembly_hash, published_at) |
|
||||
| `plugin_capabilities` | Declared capabilities per plugin version (plugin_version_id, capability, parameters) |
|
||||
|
||||
The `InMemoryPluginRegistry` provides an equivalent in-memory implementation for testing and offline scenarios.
|
||||
|
||||
## Data Flow
|
||||
|
||||
```
|
||||
[Module Host] ── discover ──> [Plugin.Host]
|
||||
│
|
||||
load plugins
|
||||
│
|
||||
┌───────────────┼───────────────┐
|
||||
│ │ │
|
||||
[BuiltIn] [Trusted] [Untrusted]
|
||||
(in-process) (in-process) (ProcessSandbox)
|
||||
│ │ │
|
||||
└───────────────┼───────────────┘
|
||||
│
|
||||
[Plugin.Registry] ── persist ──> [PostgreSQL]
|
||||
```
|
||||
|
||||
## Security Considerations
|
||||
|
||||
- **Trust level enforcement:** The host never executes untrusted plugin code in-process; all untrusted execution is delegated to the sandbox
|
||||
- **Capability restrictions:** Plugins can only perform actions matching their declared capabilities; the host rejects unauthorized calls
|
||||
- **Assembly hash verification:** Plugin assemblies are hashed at registration; the host verifies the hash at load time to detect tampering
|
||||
- **No network access for untrusted plugins:** The sandbox process has restricted network permissions; plugins that need network access must be at least Trusted
|
||||
- **Audit trail:** Plugin lifecycle events (registration, activation, failure, shutdown) are logged with timestamps and actor identity
|
||||
|
||||
## Observability
|
||||
|
||||
- **Metrics:** `plugin_active_count{trust_level}`, `plugin_load_duration_ms`, `plugin_failures_total{plugin_id}`, `sandbox_process_restarts_total`
|
||||
- **Logs:** Structured logs with `pluginId`, `trustLevel`, `lifecycleState`, `capability`
|
||||
- **Health:** The registry exposes plugin health status; modules can query whether a required plugin is active
|
||||
|
||||
## Performance Characteristics
|
||||
|
||||
- BuiltIn plugins: zero overhead (direct method dispatch)
|
||||
- Trusted plugins: negligible overhead (monitoring wrapper)
|
||||
- Untrusted plugins: gRPC serialization cost per call (~1-5ms depending on payload size)
|
||||
- Plugin discovery: runs at host startup; cached until restart or explicit re-scan
|
||||
|
||||
## References
|
||||
|
||||
- [Module README](./README.md)
|
||||
- [Integrations Architecture](../integrations/architecture.md) - Primary consumer
|
||||
- [Scanner Architecture](../scanner/architecture.md) - Plugin-based analysis
|
||||
@@ -820,12 +820,12 @@ stella exception status <request-id>
|
||||
|
||||
| Component | Source File |
|
||||
|-----------|-------------|
|
||||
| Entities | `src/Policy/__Libraries/StellaOps.Policy.Storage.Postgres/Models/ExceptionApprovalEntity.cs` |
|
||||
| Repository | `src/Policy/__Libraries/StellaOps.Policy.Storage.Postgres/Repositories/ExceptionApprovalRepository.cs` |
|
||||
| Entities | `src/Policy/__Libraries/StellaOps.Policy.Persistence/Postgres/Models/ExceptionApprovalEntity.cs` |
|
||||
| Repository | `src/Policy/__Libraries/StellaOps.Policy.Persistence/Postgres/Repositories/ExceptionApprovalRepository.cs` |
|
||||
| Rules Service | `src/Policy/StellaOps.Policy.Engine/Services/ExceptionApprovalRulesService.cs` |
|
||||
| API Endpoints | `src/Policy/StellaOps.Policy.Gateway/Endpoints/ExceptionApprovalEndpoints.cs` |
|
||||
| CLI Commands | `src/Cli/StellaOps.Cli/Commands/ExceptionCommandGroup.cs` |
|
||||
| Migration | `src/Policy/__Libraries/StellaOps.Policy.Storage.Postgres/Migrations/013_exception_approval.sql` |
|
||||
| Migration | `src/Policy/__Libraries/StellaOps.Policy.Persistence/Migrations/013_exception_approval.sql` |
|
||||
|
||||
**Related Documentation:**
|
||||
- [CI/CD Gate Integration](#62--cicd-release-gate-api)
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# Provcache Architecture Guide
|
||||
|
||||
> **Status: Production (Shared Library Family).** Provcache is a mature, heavily-used shared library family — not a planned component. The implementation spans four libraries: `src/__Libraries/StellaOps.Provcache/` (77 core files: VeriKey, DecisionDigest, chunking, write-behind queue, invalidation, telemetry), `StellaOps.Provcache.Postgres/` (16 files: EF Core persistence with `provcache` schema), `StellaOps.Provcache.Valkey/` (hot-cache layer), and `StellaOps.Provcache.Api/` (HTTP endpoints). Consumed by 89+ files across Policy Engine, Concelier, ExportCenter, CLI, and other modules. Comprehensive test coverage (89 test files). Actively maintained with recent determinism refactoring (DET-005).
|
||||
|
||||
> Detailed architecture documentation for the Provenance Cache module
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -2,7 +2,9 @@
|
||||
|
||||
> Technical architecture specification for the Release Orchestrator — Stella Ops Suite's central release control plane for non-Kubernetes container estates.
|
||||
|
||||
**Status:** Planned (not yet implemented)
|
||||
**Status:** Active Development (backend substantially implemented; API surface layer in progress)
|
||||
|
||||
> **Implementation reality (updated 2026-02-22):** The backend is substantially complete with 140,000+ lines of production code across 49 projects. Core libraries (Release, Promotion, Deployment, Workflow, Evidence, PolicyGate, Progressive, Federation, Compliance) are implemented with comprehensive tests (283 test files, 37K lines). Six agent types are operational (Compose, Docker, SSH, WinRM, ECS, Nomad). The DAG workflow engine, promotion/approval framework, and evidence generation are functional. **Remaining gaps:** HTTP API layer is minimal (1 controller), no database migrations yet (in-memory stores only), and no Program.cs bootstrapping for the WebApi project.
|
||||
|
||||
## Overview
|
||||
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# Remediation Module Architecture
|
||||
|
||||
> **Status: Planned.** The Remediation marketplace is a planned feature for developer-facing fix templates, PR generation, and contributor trust scoring. Source code at `src/Remediation/` contains initial scaffolding. This architecture document is a design specification pending full implementation.
|
||||
|
||||
## Overview
|
||||
|
||||
The Remediation module provides a developer-facing signed-PR remediation marketplace for the Stella Ops platform. It enables developers to discover, apply, and verify community-contributed or vendor-supplied fix templates for known vulnerabilities (CVEs).
|
||||
|
||||
@@ -2,6 +2,13 @@
|
||||
|
||||
This document is the canonical specification for the StellaOps Router system.
|
||||
|
||||
Tenant selection and header propagation contract: `docs/architecture/decisions/ADR-002-multi-tenant-same-api-key-selection.md`
|
||||
Service impact ledger: `docs/technical/architecture/multi-tenant-service-impact-ledger.md`
|
||||
Flow sequences: `docs/technical/architecture/multi-tenant-flow-sequences.md`
|
||||
Rollout policy: `docs/operations/multi-tenant-rollout-and-compatibility.md`
|
||||
|
||||
> **Dual-location clarification (updated 2026-02-22).** The Router (`src/Router/`) hosts the evolved `StellaOps.Gateway.WebService` with advanced features not present in `src/Gateway/`: configurable route tables via `GatewayRouteCatalog`, reverse proxy support, SPA fallback hosting, WebSocket routing, Valkey messaging transport integration, and `StellaOpsRouteResolver` for front-door dispatching. This is the current canonical deployment for HTTP ingress. A simpler version exists at `src/Gateway/` for basic ingress scenarios. See also [Gateway Architecture](../gateway/architecture.md).
|
||||
|
||||
## System Architecture
|
||||
|
||||
### Scope
|
||||
@@ -282,6 +289,16 @@ Request ─►│ ForwardedHeaders │
|
||||
▼
|
||||
```
|
||||
|
||||
### Identity Header Policy and Tenant Selection
|
||||
|
||||
- Gateway strips client-supplied reserved identity headers (`X-StellaOps-*`, legacy aliases, raw claim headers, and auth headers) before proxying.
|
||||
- Effective tenant is claim-derived from validated principal claims (`stellaops:tenant`, then bounded legacy `tid` fallback).
|
||||
- Per-request tenant override is disabled by default and only works when explicitly enabled with `Gateway:Auth:EnableTenantOverride=true` and the requested tenant exists in `stellaops:allowed_tenants`.
|
||||
- Authorization/DPoP passthrough is fail-closed:
|
||||
- route must be configured with `PreserveAuthHeaders=true`, and
|
||||
- route prefix must also be in the approved passthrough allow-list (`/connect`, `/console`, `/api/admin`).
|
||||
- Tenant override attempts are logged with deterministic fields including route, actor, requested tenant, and resolved tenant.
|
||||
|
||||
### Connection State
|
||||
|
||||
Per-connection state maintained by Gateway:
|
||||
|
||||
43
docs/modules/runtime-instrumentation/README.md
Normal file
43
docs/modules/runtime-instrumentation/README.md
Normal file
@@ -0,0 +1,43 @@
|
||||
# Runtime Instrumentation
|
||||
|
||||
> Bridges eBPF-based runtime monitoring into the Stella Ops platform, converting kernel-level events into canonical format for reachability validation and signal scoring.
|
||||
|
||||
## Purpose
|
||||
|
||||
Runtime Instrumentation adapts raw eBPF events from Tetragon into the Stella Ops canonical `RuntimeCallEvent` format. This enables the platform to incorporate live runtime observations (system calls, function probes, process lifecycle) into reachability validation and evidence-weighted vulnerability scoring without coupling downstream modules to any specific eBPF agent.
|
||||
|
||||
## Quick Links
|
||||
|
||||
- [Architecture](./architecture.md) - Technical design and implementation details
|
||||
|
||||
## Status
|
||||
|
||||
| Attribute | Value |
|
||||
|-----------|-------|
|
||||
| **Maturity** | Beta |
|
||||
| **Source** | `src/RuntimeInstrumentation/` |
|
||||
|
||||
## Key Features
|
||||
|
||||
- **Tetragon gRPC client:** Connects to the Tetragon agent's gRPC stream and ingests raw eBPF events in real time
|
||||
- **eBPF probe type mapping:** Supports all major probe types -- Kprobe, Kretprobe, Uprobe, Uretprobe, Tracepoint, USDT, Fentry, Fexit, ProcessExec, ProcessExit
|
||||
- **Stack frame canonicalization:** Converts raw kernel/user-space stack frames into `CanonicalStackFrame` with symbol resolution and address normalization
|
||||
- **Hot symbol index updates:** Publishes observed symbols to the hot symbol index for runtime reachability correlation
|
||||
- **Privacy filtering:** Strips sensitive data (environment variables, command arguments, file paths) before events leave the instrumentation boundary
|
||||
|
||||
## Dependencies
|
||||
|
||||
### Upstream (this module depends on)
|
||||
- **Tetragon** - External eBPF agent providing kernel-level event streams via gRPC
|
||||
|
||||
### Downstream (modules that depend on this)
|
||||
- **Signals** - Consumes `RuntimeCallEvent` data for runtime signal scoring (RTS dimension)
|
||||
- **Scanner** - Uses runtime observations for reachability validation
|
||||
- **Policy** - Incorporates runtime evidence into policy evaluation and verdicts
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Signals](../signals/) - Runtime signal scoring using RTS dimension
|
||||
- [Signals eBPF Contract](../signals/contracts/ebpf-micro-witness-determinism-profile.md) - Determinism profile for eBPF witnesses
|
||||
- [Scanner](../scanner/) - Reachability validation
|
||||
- [Policy](../policy/) - Runtime evidence in policy decisions
|
||||
152
docs/modules/runtime-instrumentation/architecture.md
Normal file
152
docs/modules/runtime-instrumentation/architecture.md
Normal file
@@ -0,0 +1,152 @@
|
||||
# Runtime Instrumentation Architecture
|
||||
|
||||
> Technical architecture for the eBPF event adapter bridging Tetragon into Stella Ops.
|
||||
|
||||
## Overview
|
||||
|
||||
The Runtime Instrumentation module is a stream-processing library that connects to the Tetragon eBPF agent via gRPC, receives raw kernel and user-space events, and converts them into the platform's canonical `RuntimeCallEvent` format. It does not expose HTTP endpoints or maintain a database -- it is consumed as a library by services that need runtime observation data (Signals, Scanner, Policy). The adapter decouples the rest of the platform from Tetragon's wire format and probe semantics.
|
||||
|
||||
## Design Principles
|
||||
|
||||
1. **Provider abstraction** - Downstream modules consume `RuntimeCallEvent`, not Tetragon-specific types; replacing the eBPF agent requires only a new adapter
|
||||
2. **Privacy by default** - Sensitive data is filtered at the adapter boundary before events propagate into the platform
|
||||
3. **Minimal allocation** - Event conversion is designed for high-throughput streaming with minimal object allocation
|
||||
4. **Deterministic canonicalization** - Stack frame normalization produces stable, comparable output regardless of ASLR or load order
|
||||
|
||||
## Components
|
||||
|
||||
```
|
||||
RuntimeInstrumentation/
|
||||
├── StellaOps.RuntimeInstrumentation.Tetragon/ # Core adapter library
|
||||
│ ├── TetragonEventAdapter.cs # Raw event -> RuntimeCallEvent conversion
|
||||
│ ├── Models/
|
||||
│ │ ├── TetragonEvent.cs # Raw Tetragon event representation
|
||||
│ │ ├── RuntimeCallEvent.cs # Canonical platform event
|
||||
│ │ ├── CanonicalStackFrame.cs # Normalized stack frame
|
||||
│ │ └── ProbeType.cs # eBPF probe type enumeration
|
||||
│ ├── StackCanonicalization/
|
||||
│ │ ├── StackFrameCanonicalizer.cs # Symbol resolution and normalization
|
||||
│ │ └── SymbolResolver.cs # Address-to-symbol mapping
|
||||
│ ├── Privacy/
|
||||
│ │ └── PrivacyFilter.cs # Sensitive data stripping
|
||||
│ └── HotSymbol/
|
||||
│ └── HotSymbolPublisher.cs # Publishes observed symbols to index
|
||||
│
|
||||
├── StellaOps.Agent.Tetragon/ # gRPC client for Tetragon agent
|
||||
│ ├── TetragonGrpcClient.cs # gRPC stream consumer
|
||||
│ ├── TetragonStreamReader.cs # Backpressure-aware stream reader
|
||||
│ └── Proto/ # Tetragon protobuf definitions
|
||||
│
|
||||
└── __Tests/
|
||||
└── StellaOps.RuntimeInstrumentation.Tests/ # Unit tests with fixture events
|
||||
```
|
||||
|
||||
## Core Models
|
||||
|
||||
### RuntimeCallEvent (canonical output)
|
||||
|
||||
```csharp
|
||||
public sealed record RuntimeCallEvent
|
||||
{
|
||||
public required string EventId { get; init; }
|
||||
public required DateTimeOffset Timestamp { get; init; }
|
||||
public required ProbeType ProbeType { get; init; }
|
||||
public required ProcessInfo Process { get; init; }
|
||||
public ThreadInfo? Thread { get; init; }
|
||||
public required string Syscall { get; init; }
|
||||
public IReadOnlyList<CanonicalStackFrame> StackFrames { get; init; }
|
||||
public string? ContainerId { get; init; }
|
||||
public string? PodName { get; init; }
|
||||
public string? Namespace { get; init; }
|
||||
}
|
||||
```
|
||||
|
||||
### CanonicalStackFrame
|
||||
|
||||
```csharp
|
||||
public sealed record CanonicalStackFrame
|
||||
{
|
||||
public required string Module { get; init; }
|
||||
public required string Symbol { get; init; }
|
||||
public ulong Offset { get; init; }
|
||||
public bool IsKernelSpace { get; init; }
|
||||
public string? SourceFile { get; init; }
|
||||
public int? LineNumber { get; init; }
|
||||
}
|
||||
```
|
||||
|
||||
### ProbeType Enumeration
|
||||
|
||||
| Probe Type | Description | Origin |
|
||||
|------------|-------------|--------|
|
||||
| `ProcessExec` | New process execution | Tetragon process tracker |
|
||||
| `ProcessExit` | Process termination | Tetragon process tracker |
|
||||
| `Kprobe` | Kernel function entry | Kernel dynamic tracing |
|
||||
| `Kretprobe` | Kernel function return | Kernel dynamic tracing |
|
||||
| `Uprobe` | User-space function entry | User-space dynamic tracing |
|
||||
| `Uretprobe` | User-space function return | User-space dynamic tracing |
|
||||
| `Tracepoint` | Static kernel tracepoint | Kernel static tracing |
|
||||
| `USDT` | User-space static tracepoint | Application-defined probes |
|
||||
| `Fentry` | Kernel function entry (BPF trampoline) | Modern kernel tracing (5.5+) |
|
||||
| `Fexit` | Kernel function exit (BPF trampoline) | Modern kernel tracing (5.5+) |
|
||||
|
||||
## Data Flow
|
||||
|
||||
```
|
||||
[Tetragon Agent]
|
||||
│
|
||||
│ gRPC stream (protobuf)
|
||||
▼
|
||||
[TetragonGrpcClient]
|
||||
│
|
||||
│ TetragonEvent (raw)
|
||||
▼
|
||||
[TetragonEventAdapter]
|
||||
│
|
||||
├── [StackFrameCanonicalizer] ── symbol resolution ──> CanonicalStackFrame[]
|
||||
│
|
||||
├── [PrivacyFilter] ── strip sensitive data
|
||||
│
|
||||
├── [HotSymbolPublisher] ── publish to hot symbol index
|
||||
│
|
||||
▼
|
||||
[RuntimeCallEvent] (canonical)
|
||||
│
|
||||
├──> [Signals] (RTS scoring)
|
||||
├──> [Scanner] (reachability validation)
|
||||
└──> [Policy] (runtime evidence)
|
||||
```
|
||||
|
||||
1. **Stream connection:** `TetragonGrpcClient` establishes a persistent gRPC stream to the Tetragon agent running on the same node.
|
||||
2. **Raw event ingestion:** `TetragonStreamReader` reads events with backpressure handling; if the consumer falls behind, oldest events are dropped with a metric increment.
|
||||
3. **Adaptation:** `TetragonEventAdapter` maps the raw `TetragonEvent` to a `RuntimeCallEvent`, invoking the stack canonicalizer and privacy filter.
|
||||
4. **Stack canonicalization:** `StackFrameCanonicalizer` resolves addresses to symbols using the `SymbolResolver`, normalizes module paths, and separates kernel-space from user-space frames.
|
||||
5. **Privacy filtering:** `PrivacyFilter` removes or redacts environment variables, sensitive command-line arguments, and file paths matching configurable patterns.
|
||||
6. **Symbol publishing:** `HotSymbolPublisher` emits observed symbols to the hot symbol index, enabling runtime reachability correlation without requiring full re-analysis.
|
||||
7. **Downstream consumption:** The resulting `RuntimeCallEvent` stream is consumed by Signals (for RTS scoring), Scanner (for reachability validation), and Policy (for runtime evidence in verdicts).
|
||||
|
||||
## Security Considerations
|
||||
|
||||
- **Privacy filtering:** All events pass through `PrivacyFilter` before leaving the instrumentation boundary. Configurable patterns control what gets redacted (default: environment variables, home directory paths, credential file paths).
|
||||
- **Kernel vs user-space separation:** `CanonicalStackFrame.IsKernelSpace` flag ensures downstream consumers can distinguish privilege levels and avoid conflating kernel internals with application code.
|
||||
- **No credential exposure:** The gRPC connection to Tetragon uses mTLS when available; connection parameters are configured via environment variables or mounted secrets, not hardcoded.
|
||||
- **Minimal privilege:** The adapter library itself requires no elevated privileges; only the Tetragon agent (running as a DaemonSet) requires kernel access.
|
||||
|
||||
## Performance Characteristics
|
||||
|
||||
- **Throughput target:** Sustain 50,000 events/second per node without dropping events under normal load
|
||||
- **Latency:** Event-to-canonical conversion target under 1ms per event
|
||||
- **Backpressure:** When the consumer cannot keep up, `TetragonStreamReader` applies backpressure via gRPC flow control; persistent overload triggers event dropping with `events_dropped_total` metric
|
||||
- **Memory:** Pooled buffers for protobuf deserialization to minimize GC pressure
|
||||
|
||||
## Observability
|
||||
|
||||
- **Metrics:** `runtime_events_received_total{probe_type}`, `runtime_events_converted_total`, `runtime_events_dropped_total`, `runtime_event_conversion_duration_ms`, `hot_symbols_published_total`
|
||||
- **Logs:** Structured logs with `eventId`, `probeType`, `containerId`, `processName`
|
||||
- **Health:** gRPC connection status and stream lag exposed for monitoring
|
||||
|
||||
## References
|
||||
|
||||
- [Module README](./README.md)
|
||||
- [Signals Architecture](../signals/architecture.md) - RTS scoring consumer
|
||||
- [Scanner Architecture](../scanner/architecture.md) - Reachability validation
|
||||
@@ -1,5 +1,14 @@
|
||||
# SARIF Export Module Architecture
|
||||
|
||||
> **Implementation Status:**
|
||||
> - SARIF 2.1.0 Models: **Implemented** (`src/Scanner/__Libraries/StellaOps.Scanner.Sarif/`)
|
||||
> - Export Service: **Implemented**
|
||||
> - SmartDiff Integration: **Implemented**
|
||||
> - Fingerprint Generator: **Implemented**
|
||||
> - GitHub Upload Client: **Planned**
|
||||
>
|
||||
> There is no standalone `src/SarifExport/` module; SARIF export is a capability within the Scanner module.
|
||||
|
||||
## Overview
|
||||
|
||||
The **SARIF Export** module provides SARIF 2.1.0 compliant output for StellaOps Scanner findings, enabling integration with GitHub Code Scanning, GitLab SAST, Azure DevOps, and other platforms that consume SARIF.
|
||||
|
||||
@@ -2,7 +2,10 @@
|
||||
|
||||
Scanner analyses container images layer-by-layer, producing deterministic SBOM fragments, diffs, and signed reports.
|
||||
|
||||
## Latest updates (2025-12-12)
|
||||
## Latest updates (2026-02-22)
|
||||
- Unknowns API surface is now registered in Scanner (`/api/v1/unknowns`) with tenant-scoped query predicates and tenant conflict handling via shared request-context resolution.
|
||||
- Tenant isolation hardening for triage/finding evidence APIs (Sprint `20260222.057`): triage query/status/rationale/replay services now require explicit tenant context, triage persistence includes `tenant_id`, and cross-tenant finding lookups resolve as deterministic misses. See `./endpoint-registration-matrix.md`.
|
||||
- Tenant-argument parity hardening for API-backed tenant tables (Sprint `20260222.057`, `SCAN-TEN-13`): source-run and secret-exception APIs now enforce tenant-scoped repository lookups for `scanner.sbom_source_runs` and `secret_exception_pattern`.
|
||||
- Deterministic SBOM composition fixture published at `docs/modules/scanner/fixtures/deterministic-compose/` with DSSE, `_composition.json`, BOM, and hashes; doc `deterministic-sbom-compose.md` promoted to Ready v1.0 with offline verification steps.
|
||||
- Node analyzer now ingests npm/yarn/pnpm lockfiles, emitting `DeclaredOnly` components with lock provenance. The CLI companion command `stella node lock-validate` runs the collector offline, surfaces declared-only or missing-lock packages, and emits telemetry via `stellaops.cli.node.lock_validate.count`. See `docs/modules/scanner/analyzers-node.md` and bench scenario `node_detection_gaps_fixture`.
|
||||
- Python analyzer picks up `requirements*.txt`, `Pipfile.lock`, and `poetry.lock`, tagging installed distributions with lock provenance and generating declared-only components for policy. Use `stella python lock-validate` to run the same checks locally before images are built.
|
||||
@@ -37,6 +40,7 @@ Scanner analyses container images layer-by-layer, producing deterministic SBOM f
|
||||
- ./operations/analyzers-grafana-dashboard.json
|
||||
- ./operations/rustfs-migration.md
|
||||
- ./operations/entrypoint.md
|
||||
- ./endpoint-registration-matrix.md
|
||||
- ./analyzers-node.md
|
||||
- ./analyzers-go.md
|
||||
- ./operations/secret-leak-detection.md
|
||||
|
||||
@@ -416,17 +416,42 @@ Scanner now exposes a deterministic VEX+reachability matrix filter for triage pr
|
||||
- API surface: `POST /api/v1/scans/vex-reachability/filter` accepts finding batches and returns annotated decisions plus action summary counts.
|
||||
- Determinism: batch order is preserved, rule IDs are explicit, and no network lookups are required for matrix evaluation.
|
||||
|
||||
### 5.5.7 Vulnerability-first triage clustering APIs (Sprint 20260208_063)
|
||||
### 5.5.7 Vulnerability-first triage clustering APIs (Sprint 20260208_063)
|
||||
|
||||
Scanner triage now includes deterministic exploit-path clustering primitives for vulnerability-first triage workflows:
|
||||
|
||||
- Core clustering service: `StellaOps.Scanner.Triage/Services/ExploitPathGroupingService` groups findings using common call-chain prefix similarity with configurable thresholds.
|
||||
- Inbox enhancements: `GET /api/v1/triage/inbox` supports `similarityThreshold`, `sortBy`, and `descending` for deterministic cluster filtering/sorting.
|
||||
- Cluster statistics: `GET /api/v1/triage/inbox/clusters/stats` returns per-cluster severity counts, reachability distribution, and priority scores.
|
||||
- Batch triage actions: `POST /api/v1/triage/inbox/clusters/{pathId}/actions` applies one action to all findings in the cluster and emits deterministic action records.
|
||||
- Offline/determinism posture: no network calls, stable ordering by IDs/path IDs, deterministic path-ID hashing, and replayable batch payload digests.
|
||||
|
||||
### 5.6 DSSE attestation (via Signer/Attestor)
|
||||
- Batch triage actions: `POST /api/v1/triage/inbox/clusters/{pathId}/actions` applies one action to all findings in the cluster and emits deterministic action records.
|
||||
- Offline/determinism posture: no network calls, stable ordering by IDs/path IDs, deterministic path-ID hashing, and replayable batch payload digests.
|
||||
|
||||
### 5.5.8 Triage tenant isolation contract (Sprint 20260222_057)
|
||||
|
||||
Scanner triage and finding evidence APIs enforce tenant-aware access at endpoint, service, and persistence layers:
|
||||
|
||||
- Tenant context is resolved by `ScannerRequestContextResolver` (canonical claim `tenant`, compatibility claim aliases, compatibility header aliases, and conflict detection).
|
||||
- Triage/finding service contracts require explicit `tenantId` and all retrieval/update paths filter by tenant before resolving finding/scan identity.
|
||||
- Triage schema includes tenant discriminators (`triage_scan.tenant_id`, `triage_finding.tenant_id`), and active-case uniqueness includes `tenant_id` to prevent cross-tenant collisions.
|
||||
- Cross-tenant finding lookups resolve as deterministic not-found responses rather than revealing record existence.
|
||||
|
||||
### 5.5.9 Unknowns API tenant activation (Sprint 20260222_057 follow-up)
|
||||
|
||||
Scanner now registers the `/api/v1/unknowns` endpoint group in `Program.cs` with explicit `scanner.scans.read` authorization and tenant-aware query semantics:
|
||||
|
||||
- Request tenant resolution uses `ScannerRequestContextResolver` with canonical/compatibility claim-header handling and deterministic conflict failures (`tenant_conflict`).
|
||||
- Unknown list/detail/evidence/history/stats/bands handlers call a tenant-scoped query service that filters by `tenant_id`.
|
||||
- Cross-tenant detail lookups resolve as deterministic not-found responses (`404`).
|
||||
|
||||
### 5.5.10 API-backed tenant table parity (Sprint 20260222_057 SCAN-TEN-13)
|
||||
|
||||
Scanner API flows that operate on tenant-partitioned tables now require tenant arguments at repository boundaries:
|
||||
|
||||
- Source run APIs (`/api/v1/sources/{sourceId}/runs`, `/api/v1/sources/{sourceId}/runs/{runId}`) pass tenant into `ISbomSourceRunRepository` for `GetByIdAsync`, `ListForSourceAsync`, and `GetStatsAsync`; SQL predicates include `tenant_id = @tenantId`.
|
||||
- Secret exception APIs (`/api/v1/secrets/config/exceptions/{tenantId}/{exceptionId}`) use tenant-scoped repository methods for get/update/delete on `secret_exception_pattern`, removing ID-only tenant-agnostic operations.
|
||||
- Generic webhook ingress by `sourceId` remains compatibility-tolerant when tenant context is absent, but enforces tenant ownership when context is present.
|
||||
|
||||
### 5.6 DSSE attestation (via Signer/Attestor)
|
||||
|
||||
* WebService constructs **predicate** with `image_digest`, `stellaops_version`, `license_id`, `policy_digest?` (when emitting **final reports**), timestamps.
|
||||
* Calls **Signer** (requires **OpTok + PoE**); Signer verifies **entitlement + scanner image integrity** and returns **DSSE bundle**.
|
||||
|
||||
88
docs/modules/scanner/endpoint-registration-matrix.md
Normal file
88
docs/modules/scanner/endpoint-registration-matrix.md
Normal file
@@ -0,0 +1,88 @@
|
||||
# Scanner Endpoint Registration Matrix
|
||||
|
||||
Last updated: 2026-02-23 (Sprint `20260222.057`, tasks `SCAN-TEN-08`, `SCAN-TEN-11`, `SCAN-TEN-12`, and `SCAN-TEN-13`).
|
||||
|
||||
This file is the Scanner WebService source-of-truth for endpoint registration intent and authorization posture.
|
||||
|
||||
## Active endpoint maps (registered in `Program.cs`)
|
||||
|
||||
| Map method | Base path | Authorization posture | Notes |
|
||||
| --- | --- | --- | --- |
|
||||
| `MapHealthEndpoints` | `/healthz`, `/readyz` | `scanner.scans.read` | Health and readiness are authenticated. |
|
||||
| `MapObservabilityEndpoints` | `/metrics` | `AllowAnonymous` | Prometheus scrape path is explicitly anonymous. |
|
||||
| `MapOfflineKitEndpoints` | `/offline-kit` | Mixed explicit policies | Offline import/status/manifest/validate policies are explicit per route. |
|
||||
| `MapScanEndpoints` | `/api/v1/scans` | Explicit policies per route | Includes nested scan maps (`callgraphs`, `sbom`, `reachability`, `export`, `evidence`, `approval`, `manifest`, `github-code-scanning`). |
|
||||
| `MapSourcesEndpoints` | `/api/v1/sources` | `scanner.sources.read/write/admin` | Tenant-aware source CRUD and source-run operations (`sbom_source_runs` reads/writes are tenant-filtered). |
|
||||
| `MapWebhookEndpoints` | `/api/v1/webhooks` | `AllowAnonymous` | External provider ingress; tenant-scoped source resolution for name routes. |
|
||||
| `MapSbomUploadEndpoints` | `/api/v1/sbom` | Explicit policies per route | Includes SBOM upload and hot lookup maps. |
|
||||
| `MapReachabilityDriftRootEndpoints` | `/api/v1/reachability` | `scanner.scans.read/write` | Root drift query/compute routes. |
|
||||
| `MapDeltaCompareEndpoints` | `/api/v1/delta` | Explicit policies per route | Delta compare operations. |
|
||||
| `MapSmartDiffEndpoints` | `/api/v1/smart-diff` | Explicit policies per route | Smart diff + VEX candidate workflows. |
|
||||
| `MapBaselineEndpoints` | `/api/v1/baselines` | Explicit policies per route | Baseline operations. |
|
||||
| `MapActionablesEndpoints` | `/api/v1/actionables` | Explicit policies per route | Actionable findings operations. |
|
||||
| `MapCounterfactualEndpoints` | `/api/v1/counterfactuals` | Explicit policies per route | Counterfactual analysis operations. |
|
||||
| `MapProofSpineEndpoints` | `/api/v1/spines` | Explicit policies per route | Proof spine APIs. |
|
||||
| `MapReplayEndpoints` | `/api/v1/replay` | Explicit policies per route | Replay command and verification APIs. |
|
||||
| `MapScoreReplayEndpoints` (feature-gated) | `/api/v1/replay/score` | Explicit policies per route | Enabled only when `scanner:scoreReplay:enabled=true`. |
|
||||
| `MapWitnessEndpoints` | `/api/v1/witnesses` | Explicit policies per route | Witness management APIs. |
|
||||
| `MapEpssEndpoints` | `/api/v1/epss` | Explicit policies per route | EPSS ingestion/query APIs. |
|
||||
| `MapTriageStatusEndpoints` | `/api/v1/triage` | `scanner.triage.read/write` | Triage status workflows. |
|
||||
| `MapTriageInboxEndpoints` | `/api/v1/triage` | `scanner.triage.read/write` | Triage inbox workflows. |
|
||||
| `MapBatchTriageEndpoints` | `/api/v1/triage` | `scanner.triage.read/write` | Bulk triage operations. |
|
||||
| `MapProofBundleEndpoints` | `/api/v1/triage` | `scanner.triage.read` | Triage proof bundle retrieval. |
|
||||
| `MapUnknownsEndpoints` | `/api/v1/unknowns` | `scanner.scans.read` | Tenant-scoped unknown listing/detail/evidence/history/stats/bands routes. |
|
||||
| `MapSecretDetectionSettingsEndpoints` | `/api/v1/secrets/config` | `scanner.secret-settings.*` / `scanner.secret-exceptions.*` | Secret detection config APIs with tenant-scoped exception get/update/delete lookups. |
|
||||
| `MapSecurityAdapterEndpoints` | `/api/v1/security` | Explicit policies per route | Security adapter read-model routes. |
|
||||
| `MapPolicyEndpoints` (feature-gated) | `/api/v1/policy` | Explicit policies per route | Enabled only when policy preview feature flag is true. |
|
||||
| `MapReportEndpoints` | `/api/v1/reports` | `scanner.reports` | Report generation and retrieval routes. |
|
||||
| `MapRuntimeEndpoints` | `/api/v1/runtime` | `scanner.runtime.ingest` and scan policies | Runtime ingest/reconcile APIs. |
|
||||
| `MapSliceEndpoints` | `/api/slices` | Explicit policies per route | Reachability slice query/replay APIs. |
|
||||
|
||||
## Webhook tenant contract (name routes)
|
||||
|
||||
- Name-based webhook routes are:
|
||||
- `POST /api/v1/webhooks/docker/{sourceName}`
|
||||
- `POST /api/v1/webhooks/github/{sourceName}`
|
||||
- `POST /api/v1/webhooks/gitlab/{sourceName}`
|
||||
- `POST /api/v1/webhooks/harbor/{sourceName}`
|
||||
- Tenant context is required for name-based routes (claim or `X-StellaOps-Tenant`/compatibility alias). Missing tenant returns `400` with `tenant_missing`; claim/header mismatch returns `400` with `tenant_conflict`.
|
||||
- Source lookup is tenant-scoped via `GetByNameAsync(tenantId, sourceName)`, so same-name sources in different tenants cannot cross-dispatch.
|
||||
- `POST /api/v1/webhooks/{sourceId}` accepts optional tenant context; if present and it does not match source ownership, the endpoint returns `404`.
|
||||
|
||||
## Triage tenant contract
|
||||
|
||||
- Triage and finding-evidence routes resolve tenant context via `ScannerRequestContextResolver`.
|
||||
- Resolution precedence: claim (`tenant`, compatibility aliases `tid`/`tenant_id`), then headers (`X-StellaOps-Tenant`, `X-Stella-Tenant`, `X-Tenant-Id`), then fallback tenant `default` for triage flows configured with `allowDefaultTenant`.
|
||||
- If multiple claim/header values conflict, Scanner returns `400` with `tenant_conflict`.
|
||||
- Triage-facing contracts now require tenant input (`ITriageStatusService`, `ITriageQueryService`, `IFindingQueryService`, `IGatingReasonService`, `IUnifiedEvidenceService`, `IReplayCommandService`, `IFindingRationaleService`) and enforce tenant-filtered data access.
|
||||
- Triage persistence is tenant partitioned (`triage_scan.tenant_id`, `triage_finding.tenant_id`), and the active-case uniqueness key includes `tenant_id` to avoid cross-tenant collisions.
|
||||
- Cross-tenant finding lookups are treated as deterministic not-found responses (`404`) on triage/finding endpoints.
|
||||
|
||||
## Unknowns tenant contract
|
||||
|
||||
- Unknowns routes resolve tenant context via `ScannerRequestContextResolver` using canonical and compatibility claim/header paths with conflict detection.
|
||||
- Unknown queries are tenant-scoped by data access predicates (`tenant_id::text = @tenantId`) in the Scanner WebService query service.
|
||||
- Missing records across tenant boundaries resolve as deterministic `404` for detail/evidence/history endpoints.
|
||||
|
||||
## Reachability + SmartDiff tenant contract (SCAN-TEN-11)
|
||||
|
||||
- Reachability drift routes (`/api/v1/scans/{scanId}/drift`, `/api/v1/drift/{driftId}/sinks`) now resolve tenant context via `ScannerRequestContextResolver` before repository access.
|
||||
- Drift persistence/retrieval paths are tenant-parameterized end-to-end (`call_graph_snapshots`, `code_changes`, `reachability_drift_results`, `drifted_sinks`) instead of using a fixed tenant scope.
|
||||
- SmartDiff routes (`/api/v1/smart-diff/**`) now resolve tenant context and pass it explicitly to material-change and VEX-candidate stores.
|
||||
- SmartDiff persistence/retrieval paths are tenant-parameterized (`material_risk_changes`, `vex_candidates`), so candidate ID collisions across tenants resolve as deterministic not-found.
|
||||
- Remaining reachability/SmartDiff storage adapters used by these flows are tenant-parameterized as well (`risk_state_snapshots`, `reachability_results`), removing fixed-tenant constants from Postgres adapter implementations.
|
||||
|
||||
## Source-runs + secret-exception tenant contract (SCAN-TEN-13)
|
||||
|
||||
- Source run routes (`/api/v1/sources/{sourceId}/runs`, `/api/v1/sources/{sourceId}/runs/{runId}`) now pass resolved tenant context into `ISbomSourceRunRepository` and SQL predicates (`tenant_id = @tenantId`) for `scanner.sbom_source_runs`.
|
||||
- Secret exception routes (`/api/v1/secrets/config/exceptions/{tenantId}/{exceptionId}`) now use tenant-scoped repository methods for get/update/delete against `secret_exception_pattern` rather than ID-only lookups.
|
||||
- The generic webhook route (`POST /api/v1/webhooks/{sourceId}`) remains compatibility-tolerant for missing tenant context, but enforces ownership match when tenant context is present.
|
||||
|
||||
## Deferred endpoint maps (not registered by design)
|
||||
|
||||
| Map method | Current state | Reason for deferral | Activation prerequisite |
|
||||
| --- | --- | --- | --- |
|
||||
| `MapValidationEndpoints` | Deferred | Validation endpoint stack is not wired in `Program.cs` (validator service graph and option bindings are not part of default webservice startup contract). | Add explicit validation DI and contract docs, then register in `Program.cs`. |
|
||||
| `MapReachabilityEvidenceEndpoints` | Deferred | Legacy standalone `/api/reachability` surface is not part of current `/api/v1/scans`/triage scanner API contract. | Contract decision to promote or remove this surface. |
|
||||
| `MapReachabilityStackEndpoints` | Deferred | Stack endpoints depend on optional repository path and are not part of the current public API registration set. | Contract decision and stable backing repository implementation. |
|
||||
| `MapFidelityEndpoints` | Deferred | Fidelity analyzer endpoints are experimental and require explicit analyzer service contract decisions before exposure. | Product sign-off and DI contract hardening. |
|
||||
38
docs/modules/sm-remote/README.md
Normal file
38
docs/modules/sm-remote/README.md
Normal file
@@ -0,0 +1,38 @@
|
||||
# SM Remote (SM Cipher Suite Service)
|
||||
|
||||
> Stateless cryptographic operations microservice for Chinese national standard algorithms (SM2/SM3/SM4).
|
||||
|
||||
## Purpose
|
||||
|
||||
SM Remote provides Chinese national standard cryptographic algorithms (SM2 signing/verification, SM3 hashing, SM4 encryption/decryption) as a stateless microservice for regional compliance requirements. It enables Stella Ops deployments to satisfy GB/T standards by offering both soft-provider (BouncyCastle) and optional HSM/remote provider modes for production key management.
|
||||
|
||||
## Quick Links
|
||||
|
||||
- [Architecture](./architecture.md)
|
||||
|
||||
## Status
|
||||
|
||||
| Attribute | Value |
|
||||
|-------------|----------------------|
|
||||
| **Maturity** | Production |
|
||||
| **Source** | `src/SmRemote/` |
|
||||
|
||||
## Key Features
|
||||
|
||||
- SM2 digital signatures (P-256v1 curve)
|
||||
- SM3 cryptographic hashing
|
||||
- SM4-ECB encryption with PKCS7 padding
|
||||
- Ephemeral key management
|
||||
- Soft provider and optional HSM/remote provider modes
|
||||
|
||||
## Dependencies
|
||||
|
||||
### Upstream
|
||||
|
||||
- Authority - authentication for service-to-service calls
|
||||
- Cryptography - shared cryptographic primitives and abstractions
|
||||
|
||||
### Downstream
|
||||
|
||||
- Signer - SM cipher operations for signing workflows
|
||||
- AirGap - regional crypto support in offline environments
|
||||
60
docs/modules/sm-remote/architecture.md
Normal file
60
docs/modules/sm-remote/architecture.md
Normal file
@@ -0,0 +1,60 @@
|
||||
# SM Remote Architecture
|
||||
|
||||
> Stateless microservice providing Chinese national standard cryptographic operations (SM2/SM3/SM4) via HTTP endpoints.
|
||||
|
||||
## Overview
|
||||
|
||||
SM Remote is a single-project ASP.NET Core microservice that exposes SM-series cryptographic operations over HTTP. It is designed to be stateless: no database is required, and ephemeral key pairs are seeded on demand per `keyId`. The service supports two provider modes -- a soft provider backed by BouncyCastle for development and testing, and a remote provider that delegates to an HSM for production deployments.
|
||||
|
||||
## Components
|
||||
|
||||
```
|
||||
src/SmRemote/
|
||||
StellaOps.SmRemote.Service/ # ASP.NET Core web service
|
||||
Program.cs # Host configuration and endpoint registration
|
||||
Providers/
|
||||
SoftSmProvider.cs # BouncyCastle-based SM2/SM3/SM4 (cn.sm.soft)
|
||||
RemoteSmProvider.cs # HSM-delegating provider (cn.sm.remote.http)
|
||||
Models/ # Request/response DTOs
|
||||
```
|
||||
|
||||
## Data Flow
|
||||
|
||||
1. An upstream service (Signer, AirGap) sends a cryptographic operation request (sign, verify, hash, encrypt, decrypt) to SM Remote.
|
||||
2. SM Remote selects the configured provider (soft or remote).
|
||||
3. For signing/verification, the service seeds an ephemeral SM2 key pair on first use for the given `keyId` using EC domain parameters (SM2P256V1 curve).
|
||||
4. The provider executes the operation and returns the result.
|
||||
5. No state is persisted between requests; key material is ephemeral within the process lifecycle.
|
||||
|
||||
## Database Schema
|
||||
|
||||
Not applicable. SM Remote is a stateless service with no persistent storage.
|
||||
|
||||
## Dependencies
|
||||
|
||||
| Service/Library | Purpose |
|
||||
|---------------------|----------------------------------------------|
|
||||
| BouncyCastle | SM2/SM3/SM4 algorithm implementations (soft provider) |
|
||||
| Pkcs11Interop | PKCS#11 interface for HSM integration (remote provider) |
|
||||
| Router | Service mesh routing and discovery |
|
||||
| Authority | JWT/OAuth token validation for inbound requests |
|
||||
|
||||
## Endpoints
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|-------------|---------------------------------------------|
|
||||
| POST | `/hash` | Compute SM3 hash of input data |
|
||||
| POST | `/encrypt` | SM4-ECB encrypt with PKCS7 padding |
|
||||
| POST | `/decrypt` | SM4-ECB decrypt with PKCS7 padding |
|
||||
| POST | `/sign` | SM2 digital signature (P-256v1) |
|
||||
| POST | `/verify` | SM2 signature verification |
|
||||
| GET | `/status` | Provider status and configuration |
|
||||
| GET | `/health` | Health check |
|
||||
|
||||
## Security Considerations
|
||||
|
||||
- **mTLS**: Inter-service calls use mutual TLS for transport security.
|
||||
- **Ephemeral key lifecycle**: SM2 key pairs are generated per `keyId` and exist only in process memory. Key material is not persisted to disk.
|
||||
- **HSM offloading**: Production deployments should use the remote provider (`cn.sm.remote.http`) to delegate key operations to a hardware security module, ensuring key material never leaves the HSM boundary.
|
||||
- **No key export**: The service does not expose endpoints for exporting private key material.
|
||||
- **Provider selection**: The active provider is configured at startup; switching providers requires a service restart to prevent mixed-mode operation.
|
||||
@@ -16,7 +16,7 @@ TimelineIndexer provides fast, indexed access to timeline events across all Stel
|
||||
| Attribute | Value |
|
||||
|-----------|-------|
|
||||
| **Maturity** | Production |
|
||||
| **Last Reviewed** | 2025-12-29 |
|
||||
| **Last Reviewed** | 2026-02-22 |
|
||||
| **Maintainer** | Platform Guild |
|
||||
|
||||
## Key Features
|
||||
@@ -25,11 +25,14 @@ TimelineIndexer provides fast, indexed access to timeline events across all Stel
|
||||
- **Time-Range Queries**: Efficient time-series queries with filtering
|
||||
- **Event Stream Integration**: Consume from NATS/Valkey event streams
|
||||
- **PostgreSQL Storage**: Time-series indexes for fast retrieval
|
||||
- **EF Core DAL**: Database-first scaffolded model baseline with regeneration-safe partial overlays
|
||||
- **Compiled EF Model**: Static compiled model module is used at runtime for context initialization
|
||||
|
||||
## Dependencies
|
||||
|
||||
### Upstream (this module depends on)
|
||||
- **PostgreSQL** - Event storage with time-series indexes
|
||||
- **EF Core 10 + Npgsql provider** - DAL and schema mapping
|
||||
- **NATS/Valkey** - Event stream consumption
|
||||
- **Authority** - Authentication
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# component_architecture_timelineindexer.md - **Stella Ops TimelineIndexer** (2025Q4)
|
||||
# component_architecture_timelineindexer.md - **Stella Ops TimelineIndexer** (2026Q1)
|
||||
|
||||
> Timeline event indexing and query service.
|
||||
|
||||
@@ -22,18 +22,31 @@
|
||||
|
||||
```
|
||||
src/TimelineIndexer/StellaOps.TimelineIndexer/
|
||||
├─ StellaOps.TimelineIndexer.Core/ # Event models, indexing logic
|
||||
├─ StellaOps.TimelineIndexer.Infrastructure/ # Storage adapters
|
||||
├─ StellaOps.TimelineIndexer.WebService/ # Query API
|
||||
├─ StellaOps.TimelineIndexer.Worker/ # Event consumer
|
||||
└─ StellaOps.TimelineIndexer.Tests/
|
||||
|- StellaOps.TimelineIndexer.Core/ # Event models, indexing logic
|
||||
|- StellaOps.TimelineIndexer.Infrastructure/ # Storage adapters and DAL
|
||||
|- StellaOps.TimelineIndexer.WebService/ # Query API
|
||||
|- StellaOps.TimelineIndexer.Worker/ # Event consumer
|
||||
`- StellaOps.TimelineIndexer.Tests/
|
||||
```
|
||||
|
||||
### 1.1 Persistence implementation (2026-02-22)
|
||||
|
||||
* TimelineIndexer persistence uses **EF Core 10** with database-first scaffolded models.
|
||||
* Generated artifacts are stored in:
|
||||
* `src/TimelineIndexer/StellaOps.TimelineIndexer/StellaOps.TimelineIndexer.Infrastructure/EfCore/Context`
|
||||
* `src/TimelineIndexer/StellaOps.TimelineIndexer/StellaOps.TimelineIndexer.Infrastructure/EfCore/Models`
|
||||
* `src/TimelineIndexer/StellaOps.TimelineIndexer/StellaOps.TimelineIndexer.Infrastructure/EfCore/CompiledModels`
|
||||
* Store adapters (`TimelineEventStore`, `TimelineQueryStore`) run through `TimelineIndexerDataSource` tenant-scoped sessions, preserving `app.current_tenant` and RLS behavior.
|
||||
* Manual model corrections (enum mapping and FK relationship configuration) are implemented in partial files, so scaffolded files remain regeneratable.
|
||||
* Runtime context initialization uses the static compiled model module:
|
||||
* `options.UseModel(TimelineIndexerDbContextModel.Instance)`
|
||||
|
||||
---
|
||||
|
||||
## 2) External dependencies
|
||||
|
||||
* **PostgreSQL** - Event storage with time-series indexes
|
||||
* **EF Core 10 + Npgsql provider** - DAL and model mapping for timeline schema
|
||||
* **NATS/Valkey** - Event stream consumption
|
||||
* **Authority** - Authentication
|
||||
|
||||
@@ -79,5 +92,3 @@ GET /healthz | /readyz | /metrics
|
||||
|
||||
* Signals: `../signals/architecture.md`
|
||||
* Scanner: `../scanner/architecture.md`
|
||||
|
||||
|
||||
|
||||
@@ -2,12 +2,12 @@
|
||||
|
||||
> **Imposed rule:** Timeline is append-only; events may not be deleted or rewritten. Redactions require creating a compensating `redaction_notice` event that references the original ULID.
|
||||
|
||||
The Timeline Indexer service aggregates structured events (scanner runs, policy verdicts, runtime posture, evidence locker activity) so operators can audit changes over time. This guide summarises the event schema, query surfaces, and integration points.
|
||||
The Timeline Indexer service aggregates structured events (scanner runs, policy verdicts, runtime posture, evidence locker activity) so operators can audit changes over time. This guide summarizes the event schema, query surfaces, persistence implementation, and integration points.
|
||||
|
||||
## 1. Event Model
|
||||
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| --- | --- |
|
||||
| `event_id` | ULID identifying the event. |
|
||||
| `tenant` | Tenant scope. |
|
||||
| `timestamp` | UTC ISO-8601 time the event occurred. |
|
||||
@@ -19,22 +19,23 @@ Events are stored append-only with tenant-specific partitions. Producers include
|
||||
|
||||
### Event kinds (normative)
|
||||
|
||||
- `scan.completed` – scanner job finished; includes SBOM digest, findings counts, determinism score.
|
||||
- `policy.verdict` – policy engine decision with overlay cache version and allow/deny rationale.
|
||||
- `attestation.verified` – attestation verification result with Rekor UUID and bundle digest.
|
||||
- `evidence.ingested` – Evidence Locker write with WORM vault identifier.
|
||||
- `notify.sent` – outbound notification with target channel and template id.
|
||||
- `runtime.alert` – runtime enforcement or observation event from Zastava Observer.
|
||||
- `redaction_notice` – compensating entry when data is logically withdrawn; must include `supersedes` ULID.
|
||||
- `scan.completed` - scanner job finished; includes SBOM digest, findings counts, determinism score.
|
||||
- `policy.verdict` - policy engine decision with overlay cache version and allow/deny rationale.
|
||||
- `attestation.verified` - attestation verification result with Rekor UUID and bundle digest.
|
||||
- `evidence.ingested` - Evidence Locker write with WORM vault identifier.
|
||||
- `notify.sent` - outbound notification with target channel and template id.
|
||||
- `runtime.alert` - runtime enforcement or observation event from Zastava Observer.
|
||||
- `redaction_notice` - compensating entry when data is logically withdrawn; must include `supersedes` ULID.
|
||||
|
||||
## 2. APIs
|
||||
|
||||
- Native endpoints:
|
||||
Native endpoints:
|
||||
- `GET /timeline` - query timeline entries with filter parameters.
|
||||
- `GET /timeline/{eventId}` - fetch a single timeline entry.
|
||||
- `GET /timeline/{eventId}/evidence` - fetch evidence linked to a timeline entry.
|
||||
- `POST /timeline/events` - ingestion ack endpoint.
|
||||
- Router/Gateway aliases for microservice transport routing:
|
||||
|
||||
Router/Gateway aliases for microservice transport routing:
|
||||
- `GET /api/v1/timeline`
|
||||
- `GET /api/v1/timeline/{eventId}`
|
||||
- `GET /api/v1/timeline/{eventId}/evidence`
|
||||
@@ -50,10 +51,10 @@ API headers: `X-Stella-Tenant`, optional `X-Stella-TraceId`, and `If-None-Match`
|
||||
- For WORM verification, filter `category=evidence` and join on Evidence Locker bundle digest.
|
||||
- Use `category=attestation.verified` and `details.rekor_uuid` to reconcile transparency proofs.
|
||||
|
||||
Example queries
|
||||
Example queries:
|
||||
|
||||
```sh
|
||||
# Recent scan → policy → notify chain for a digest
|
||||
# Recent scan -> policy -> notify chain for a digest
|
||||
stella timeline list --tenant acme --category scan.completed --subject sha256:abc... --limit 5
|
||||
stella timeline list --tenant acme --category policy.verdict --trace-id <trace>
|
||||
stella timeline list --tenant acme --category notify.sent --trace-id <trace>
|
||||
@@ -64,7 +65,51 @@ curl -H "X-Stella-Tenant: acme" \
|
||||
-o timeline-2025-11-01.ndjson
|
||||
```
|
||||
|
||||
## 4. Integration
|
||||
## 4. Persistence (EF Core)
|
||||
|
||||
- TimelineIndexer DAL is EF Core 10 based (`StellaOps.TimelineIndexer.Infrastructure/Db`).
|
||||
- Database-first scaffolded context/models are under:
|
||||
- `src/TimelineIndexer/StellaOps.TimelineIndexer/StellaOps.TimelineIndexer.Infrastructure/EfCore/Context`
|
||||
- `src/TimelineIndexer/StellaOps.TimelineIndexer/StellaOps.TimelineIndexer.Infrastructure/EfCore/Models`
|
||||
- Keep generated files regeneration-safe. Place manual fixes in partial classes/files.
|
||||
|
||||
Scaffold command used for baseline generation:
|
||||
|
||||
```sh
|
||||
dotnet ef dbcontext scaffold "Host=localhost;Port=55433;Database=postgres;Username=postgres;Password=postgres" \
|
||||
Npgsql.EntityFrameworkCore.PostgreSQL \
|
||||
--project src/TimelineIndexer/StellaOps.TimelineIndexer/StellaOps.TimelineIndexer.Infrastructure/StellaOps.TimelineIndexer.Infrastructure.csproj \
|
||||
--startup-project src/TimelineIndexer/StellaOps.TimelineIndexer/StellaOps.TimelineIndexer.WebService/StellaOps.TimelineIndexer.WebService.csproj \
|
||||
--context TimelineIndexerDbContext \
|
||||
--context-dir EfCore/Context \
|
||||
--output-dir EfCore/Models \
|
||||
--schema timeline \
|
||||
--table timeline_events \
|
||||
--table timeline_event_details \
|
||||
--table timeline_event_digests \
|
||||
--no-onconfiguring \
|
||||
--use-database-names \
|
||||
--force
|
||||
```
|
||||
|
||||
Expected scaffold caveat: enum `timeline.event_severity` and composite FKs on `(event_id, tenant_id)` are not fully inferred by the scaffold command; these are restored in partial mapping files under `EfCore/Context` and `EfCore/Models`.
|
||||
|
||||
Generate/refresh compiled EF model artifacts:
|
||||
|
||||
```sh
|
||||
dotnet ef dbcontext optimize \
|
||||
--project src/TimelineIndexer/StellaOps.TimelineIndexer/StellaOps.TimelineIndexer.Infrastructure/StellaOps.TimelineIndexer.Infrastructure.csproj \
|
||||
--startup-project src/TimelineIndexer/StellaOps.TimelineIndexer/StellaOps.TimelineIndexer.WebService/StellaOps.TimelineIndexer.WebService.csproj \
|
||||
--context TimelineIndexerDbContext \
|
||||
--output-dir EfCore/CompiledModels \
|
||||
--namespace StellaOps.TimelineIndexer.Infrastructure.EfCore.CompiledModels
|
||||
```
|
||||
|
||||
Design-time connection can be overridden with `STELLAOPS_TIMELINEINDEXER_EF_CONNECTION`.
|
||||
|
||||
Runtime usage is explicit in `TimelineIndexerDbContextFactory` via `UseModel(TimelineIndexerDbContextModel.Instance)` so the static compiled model module is always used by DAL stores.
|
||||
|
||||
## 5. Integration
|
||||
|
||||
- Evidence Locker attaches evidence bundle digests; the console links from timeline to evidence viewer.
|
||||
- Notifier creates acknowledgement events for incident workflows.
|
||||
@@ -74,10 +119,9 @@ Retention: events are retained per-tenant for at least `timeline.retention_days`
|
||||
|
||||
Privacy/PII: producers must redact PII before emission; once emitted, redactions occur via `redaction_notice` only.
|
||||
|
||||
## 5. References
|
||||
## 6. References
|
||||
|
||||
- `docs/modules/telemetry/architecture.md`
|
||||
- `docs/modules/zastava/architecture.md`
|
||||
- `docs/modules/export-center/architecture.md`
|
||||
- `src/TimelineIndexer/StellaOps.TimelineIndexer`
|
||||
|
||||
|
||||
42
docs/modules/timeline/README.md
Normal file
42
docs/modules/timeline/README.md
Normal file
@@ -0,0 +1,42 @@
|
||||
# Timeline
|
||||
|
||||
> Unified event timeline service with HLC-ordered queries, critical path analysis, and deterministic replay.
|
||||
|
||||
## Purpose
|
||||
|
||||
Timeline is the query and presentation layer for cross-service event correlation and auditing. It provides Hybrid Logical Clock (HLC)-ordered event queries, critical path analysis for latency diagnosis, and deterministic replay capabilities. Timeline reads events that have been indexed by the separate TimelineIndexer module, serving as the read-side of the event timeline infrastructure.
|
||||
|
||||
## Quick Links
|
||||
|
||||
- [Architecture](./architecture.md)
|
||||
|
||||
## Status
|
||||
|
||||
| Attribute | Value |
|
||||
|-------------|---------------------|
|
||||
| **Maturity** | Production |
|
||||
| **Source** | `src/Timeline/` |
|
||||
|
||||
## Key Features
|
||||
|
||||
- HLC-based causal event ordering
|
||||
- Correlation ID queries
|
||||
- Critical path analysis (latency stages)
|
||||
- Service-filtered timeline views
|
||||
- Event export and deterministic replay
|
||||
|
||||
## Dependencies
|
||||
|
||||
### Upstream
|
||||
|
||||
- Eventing infrastructure (StellaOps.Eventing) - event storage and indexing
|
||||
- HybridLogicalClock library - causal timestamp generation and comparison
|
||||
|
||||
### Downstream
|
||||
|
||||
- Platform - timeline views for operator dashboards
|
||||
- CLI - event query commands
|
||||
- Web - timeline UI components
|
||||
- Replay - deterministic replay of event sequences
|
||||
|
||||
> **Note:** Timeline is the query/presentation service. TimelineIndexer (separate module) handles event ingestion and indexing. They are independently deployable services.
|
||||
84
docs/modules/timeline/architecture.md
Normal file
84
docs/modules/timeline/architecture.md
Normal file
@@ -0,0 +1,84 @@
|
||||
# Timeline Architecture
|
||||
|
||||
> Query and presentation service for HLC-ordered cross-service event timelines.
|
||||
|
||||
## Overview
|
||||
|
||||
Timeline provides a REST API for querying, analyzing, and replaying events that have been indexed by the TimelineIndexer module. All events carry Hybrid Logical Clock (HLC) timestamps that establish causal ordering across distributed services. The service supports correlation-based queries, critical path analysis for latency diagnosis, service-scoped views, and deterministic event replay.
|
||||
|
||||
## Components
|
||||
|
||||
```
|
||||
src/Timeline/
|
||||
StellaOps.Timeline.WebService/ # REST API (ASP.NET Core)
|
||||
Endpoints/
|
||||
TimelineEndpoints.cs # Core timeline query endpoints
|
||||
ExportEndpoints.cs # Event export endpoints
|
||||
ReplayEndpoints.cs # Deterministic replay endpoints
|
||||
Program.cs # Host configuration
|
||||
StellaOps.Timeline.Core/ # Query service and models
|
||||
ITimelineQueryService.cs # Core query interface
|
||||
TimelineQueryService.cs # Query implementation
|
||||
Models/
|
||||
TimelineEvent.cs # Event with HLC timestamp + correlation ID
|
||||
CriticalPathResult.cs # Stages with durations
|
||||
TimelineQueryOptions.cs # Filters + pagination
|
||||
```
|
||||
|
||||
## Data Flow
|
||||
|
||||
1. Events are produced by various Stella Ops services and carry HLC timestamps.
|
||||
2. TimelineIndexer (separate module) ingests and indexes these events into the event store.
|
||||
3. Timeline WebService receives query requests from Platform, CLI, Web, or Replay.
|
||||
4. Timeline Core executes queries against the indexed event store, applying correlation, service, and time-range filters.
|
||||
5. Results are returned in HLC-sorted order, with optional critical path analysis computing latency stages between correlated events.
|
||||
|
||||
## Database Schema
|
||||
|
||||
Timeline reads from the event store managed by the Eventing infrastructure (PostgreSQL). Key columns queried:
|
||||
|
||||
| Column | Type | Description |
|
||||
|-----------------|-------------|------------------------------------------|
|
||||
| `event_id` | UUID | Unique event identifier |
|
||||
| `hlc_timestamp` | BIGINT | Hybrid Logical Clock timestamp |
|
||||
| `correlation_id` | VARCHAR | Cross-service correlation identifier |
|
||||
| `service` | VARCHAR | Originating service name |
|
||||
| `event_type` | VARCHAR | Event type discriminator |
|
||||
| `payload` | JSONB | Event payload |
|
||||
| `created_at` | TIMESTAMPTZ | Wall-clock ingestion time |
|
||||
|
||||
## Dependencies
|
||||
|
||||
| Service/Library | Purpose |
|
||||
|--------------------------|-------------------------------------------|
|
||||
| StellaOps.Eventing | Event store access and query primitives |
|
||||
| StellaOps.HybridLogicalClock | HLC timestamp parsing and comparison |
|
||||
| Router | Service mesh routing and discovery |
|
||||
| Authority | JWT/OAuth token validation |
|
||||
|
||||
## Endpoints
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|-----------------------------------------------|--------------------------------------------------|
|
||||
| GET | `/timeline/by-correlation/{correlationId}` | Query events by correlation ID (HLC-ordered) |
|
||||
| GET | `/timeline/critical-path/{correlationId}` | Critical path analysis with latency stages |
|
||||
| GET | `/timeline/by-service/{service}` | Service-filtered timeline view |
|
||||
| POST | `/timeline/export` | Export events matching query criteria |
|
||||
| POST | `/timeline/replay` | Deterministic replay of an event sequence |
|
||||
|
||||
## Security Considerations
|
||||
|
||||
- **Authentication**: All endpoints require a valid JWT issued by Authority.
|
||||
- **Tenant isolation**: Queries are scoped to the authenticated tenant; cross-tenant event access is prohibited.
|
||||
- **Read-only surface**: Timeline exposes only read and replay operations. Event mutation is handled exclusively by TimelineIndexer.
|
||||
- **Export controls**: Exported event payloads may contain sensitive operational data; exports are audit-logged.
|
||||
- **Replay determinism**: Replay operations produce identical output given identical input sequences, supporting audit and compliance verification.
|
||||
|
||||
## Relationship to TimelineIndexer
|
||||
|
||||
Timeline and TimelineIndexer are separate deployable services with distinct responsibilities:
|
||||
|
||||
- **TimelineIndexer**: Consumes events from the message bus, assigns HLC timestamps, and writes indexed events to the event store.
|
||||
- **Timeline**: Reads from the event store and serves query, analysis, export, and replay requests.
|
||||
|
||||
This separation allows independent scaling of ingestion and query workloads.
|
||||
41
docs/modules/tools/README.md
Normal file
41
docs/modules/tools/README.md
Normal file
@@ -0,0 +1,41 @@
|
||||
# Developer Tools
|
||||
|
||||
> Collection of CLI utilities for fixture management, policy validation, smoke testing, and workflow generation.
|
||||
|
||||
## Purpose
|
||||
|
||||
Developer Tools is a collection of standalone CLI utilities used by Stella Ops developers and operators during development and CI workflows. Each tool addresses a specific concern -- refreshing golden test fixtures from live APIs, validating policy DSL files, running smoke tests, or generating CI workflow definitions. The tools are not deployed as services; they run locally or in CI pipelines.
|
||||
|
||||
## Quick Links
|
||||
|
||||
- [Architecture](./architecture.md)
|
||||
|
||||
## Status
|
||||
|
||||
| Attribute | Value |
|
||||
|-------------|-------------------|
|
||||
| **Maturity** | Production |
|
||||
| **Source** | `src/Tools/` |
|
||||
|
||||
## Key Features
|
||||
|
||||
- FixtureUpdater: golden fixture refresh from live APIs
|
||||
- GoldenPairs: SBOM/advisory corpus management
|
||||
- PolicyDslValidator: policy language validation
|
||||
- PolicySchemaExporter: JSON schema export for IDE autocomplete
|
||||
- PolicySimulationSmoke: policy simulation smoke tests
|
||||
- LanguageAnalyzerSmoke: language detection tests
|
||||
- RustFsMigrator: filesystem migration for RustFS (S3-compatible) storage
|
||||
- WorkflowGenerator: CI workflow generation with F# DSL
|
||||
|
||||
## Dependencies
|
||||
|
||||
### Upstream
|
||||
|
||||
- Policy Engine libraries - policy DSL parsing and schema definitions
|
||||
- Scanner libraries - language analyzer and SBOM processing
|
||||
|
||||
### Downstream
|
||||
|
||||
- CI pipelines - consume generated workflow definitions
|
||||
- Test suites - consume golden fixtures and SBOM/advisory pairs
|
||||
98
docs/modules/tools/architecture.md
Normal file
98
docs/modules/tools/architecture.md
Normal file
@@ -0,0 +1,98 @@
|
||||
# Developer Tools Architecture
|
||||
|
||||
> Standalone CLI utilities for development, testing, and CI support workflows.
|
||||
|
||||
## Overview
|
||||
|
||||
The Tools directory contains a set of independent CLI applications, each with its own `Program.cs` entry point. These tools are not deployed as services -- they are invoked locally by developers or executed in CI pipelines. Each tool is narrowly scoped to a single responsibility, from fixture management to workflow generation.
|
||||
|
||||
## Components
|
||||
|
||||
```
|
||||
src/Tools/
|
||||
FixtureUpdater/ # Golden fixture refresh from live APIs
|
||||
Program.cs
|
||||
GoldenPairs/ # SBOM/advisory corpus management
|
||||
Program.cs
|
||||
PolicyDslValidator/ # Policy DSL file validation
|
||||
Program.cs
|
||||
PolicySchemaExporter/ # JSON schema export for IDE support
|
||||
Program.cs
|
||||
PolicySimulationSmoke/ # Policy simulation smoke tests
|
||||
Program.cs
|
||||
LanguageAnalyzerSmoke/ # Language detection accuracy tests
|
||||
Program.cs
|
||||
RustFsMigrator/ # RustFS data migration between schema versions
|
||||
Program.cs
|
||||
WorkflowGenerator/ # CI workflow generation (F# DSL)
|
||||
Program.fs
|
||||
```
|
||||
|
||||
## Tool Descriptions
|
||||
|
||||
### FixtureUpdater
|
||||
|
||||
Pulls latest test data from running Stella Ops services and updates frozen golden fixtures deterministically. Ensures test suites use realistic, version-controlled data without manual fixture authoring.
|
||||
|
||||
### GoldenPairs
|
||||
|
||||
Manages SBOM/advisory pairs used for testing. Provides version tracking and diff tools for the test corpus, ensuring changes to upstream advisory formats are detected and accommodated.
|
||||
|
||||
### PolicyDslValidator
|
||||
|
||||
Validates policy DSL files against the current schema. Used in CI gates to catch policy syntax errors before merge.
|
||||
|
||||
### PolicySchemaExporter
|
||||
|
||||
Exports the Policy DSL schema to JSON format for documentation and IDE autocomplete support. Enables policy authors to get inline validation and completion in their editors.
|
||||
|
||||
### PolicySimulationSmoke
|
||||
|
||||
Runs end-to-end policy simulation smoke tests against a configured Policy Engine instance. Validates that policy evaluation produces expected verdicts for a known set of inputs.
|
||||
|
||||
### LanguageAnalyzerSmoke
|
||||
|
||||
Tests the language analyzer's detection accuracy against a curated set of source files. Reports precision and recall metrics for supported languages.
|
||||
|
||||
### RustFsMigrator
|
||||
|
||||
Migrates data stored in RustFS (S3-compatible object storage) between schema versions. Handles object key transformations and metadata updates required during platform upgrades.
|
||||
|
||||
### WorkflowGenerator
|
||||
|
||||
Generates GitHub Actions and .NET test workflow definitions from an F# DSL. Ensures CI workflow files are consistent, auditable, and derived from a single source of truth rather than hand-edited YAML.
|
||||
|
||||
## Data Flow
|
||||
|
||||
Tools are consumers and producers of artifacts:
|
||||
|
||||
1. **FixtureUpdater** and **GoldenPairs** pull data from live services or local corpora and write deterministic fixture files to the repository.
|
||||
2. **PolicyDslValidator** and **PolicySchemaExporter** read policy definitions and produce validation results or schema files.
|
||||
3. **PolicySimulationSmoke** and **LanguageAnalyzerSmoke** execute tests against upstream services/libraries and produce pass/fail reports.
|
||||
4. **RustFsMigrator** reads from and writes to S3-compatible storage.
|
||||
5. **WorkflowGenerator** reads F# DSL definitions and writes CI workflow YAML files.
|
||||
|
||||
## Database Schema
|
||||
|
||||
Not applicable. Tools are CLI utilities with no persistent database.
|
||||
|
||||
## Endpoints
|
||||
|
||||
Not applicable. Tools are client-side CLI applications with no HTTP endpoints.
|
||||
|
||||
## Dependencies
|
||||
|
||||
| Library/Tool | Purpose |
|
||||
|---------------------|------------------------------------------------|
|
||||
| Policy Engine libs | Policy DSL parsing, schema definitions |
|
||||
| Scanner libs | Language analyzer, SBOM processing |
|
||||
| F# compiler | WorkflowGenerator DSL compilation |
|
||||
| DotNet.Glob | File pattern matching in fixture tools |
|
||||
| AWS SDK (S3) | RustFsMigrator object storage access |
|
||||
|
||||
## Security Considerations
|
||||
|
||||
- **No network listeners**: Tools do not expose HTTP endpoints or accept inbound connections.
|
||||
- **Credential handling**: Tools that connect to live services (FixtureUpdater, PolicySimulationSmoke) use the same Authority-issued tokens as other Stella Ops services. Credentials are never embedded in tool binaries or fixture files.
|
||||
- **Deterministic output**: FixtureUpdater and GoldenPairs produce deterministic output to ensure reproducible test runs and prevent fixture drift.
|
||||
- **CI isolation**: Tools run in isolated CI containers with scoped permissions; they do not have access to production secrets.
|
||||
@@ -84,7 +84,7 @@ graph TD
|
||||
Key interactions:
|
||||
|
||||
- **Auth bootstrap:** UI retrieves Authority metadata and exchanges an authorization code + PKCE verifier for a DPoP-bound token (`aud=console`, `tenant=<id>`). Tokens expire in 120 s; refresh tokens rotate, triggering new DPoP proofs.
|
||||
- **Tenant switch:** Picker issues `Authority /fresh-auth` when required, then refreshes UI caches (`ui.tenant.switch` log). Gateway injects `X-Stella-Tenant` headers downstream.
|
||||
- **Tenant switch:** Picker issues `Authority /fresh-auth` when required, then refreshes UI caches (`ui.tenant.switch` log). Gateway injects canonical `X-StellaOps-Tenant` headers downstream (legacy `X-Stella-Tenant`/`X-Tenant-Id` aliases are compatibility-only during migration).
|
||||
- **Aggregation-only reads:** Gateway proxies `/console/advisories`, `/console/vex`, `/console/findings`, etc., without mutating Concelier or Policy data. Provenance badges and merge hashes come directly from upstream responses.
|
||||
- **Downloads parity:** `/console/downloads` merges DevOps signed manifest and Offline Kit metadata; UI renders digest, signature, and CLI parity command.
|
||||
- **Offline resilience:** Gateway exposes `/console/status` heartbeat. If unavailable, UI enters offline mode, disables SSE, and surfaces CLI fallbacks.
|
||||
@@ -161,7 +161,7 @@ Optimisation levers:
|
||||
|
||||
- **DPoP + PKCE:** Every request carries `Authorization` + `DPoP` header and gateway enforces nonce replay protection. Private keys live in IndexedDB and never leave the browser.
|
||||
- **Scope enforcement:** Gateway checks scope claims before proxying (`ui.read`, `runs.manage`, `downloads.read`, etc.) and propagates denials as `Problem+JSON` with `ERR_*` codes.
|
||||
- **Tenant propagation:** `X-Stella-Tenant` header derived from token; downstream services reject mismatches. Tenant switches log `ui.tenant.switch` and require fresh-auth for privileged actions.
|
||||
- **Tenant propagation:** canonical `X-StellaOps-Tenant` header is derived from validated token context; downstream services reject mismatches. Legacy aliases are compatibility-only during migration. Tenant switches log `ui.tenant.switch` and require fresh-auth for privileged actions.
|
||||
- **CSP & headers:** Default CSP forbids third-party scripts, only allows same-origin `connect-src`. HSTS, Referrer-Policy `no-referrer`, and `Permissions-Policy` configured via gateway (`deploy/console.md`).
|
||||
- **Evidence handling:** Downloads never cache secrets; UI renders SHA-256 + signature references and steers users to CLI for sensitive exports.
|
||||
- See [Console security posture](../../security/console-security.md) for full scope table and threat model alignment.
|
||||
|
||||
40
docs/modules/verifier/README.md
Normal file
40
docs/modules/verifier/README.md
Normal file
@@ -0,0 +1,40 @@
|
||||
# Verifier
|
||||
|
||||
> Standalone CLI tool for offline verification of evidence bundles in air-gapped environments.
|
||||
|
||||
## Purpose
|
||||
|
||||
Verifier is a self-contained, cross-platform CLI binary that validates evidence bundles without requiring network access or external dependencies. It checks DSSE signatures, RFC 3161 timestamps, SHA-256 digests, and SBOM integrity, enabling compliance verification in air-gapped environments where no Stella Ops services are reachable.
|
||||
|
||||
## Quick Links
|
||||
|
||||
- [Architecture](./architecture.md)
|
||||
|
||||
## Status
|
||||
|
||||
| Attribute | Value |
|
||||
|-------------|---------------------|
|
||||
| **Maturity** | Production |
|
||||
| **Source** | `src/Verifier/` |
|
||||
|
||||
## Key Features
|
||||
|
||||
- Self-contained single-file binary (cross-platform: win-x64, linux-x64, linux-musl-x64, osx-x64, osx-arm64)
|
||||
- Bundle extraction (gzip+tar)
|
||||
- Manifest validation
|
||||
- DSSE signature verification
|
||||
- RFC 3161 timestamp verification
|
||||
- SHA-256 digest checking
|
||||
- Trust profile support (key whitelisting)
|
||||
- Output formats (text/JSON/markdown)
|
||||
|
||||
## Dependencies
|
||||
|
||||
### Upstream
|
||||
|
||||
- None (standalone, offline-first design with zero runtime dependencies on Stella Ops services)
|
||||
|
||||
### Downstream
|
||||
|
||||
- AirGap - offline bundle verification workflows
|
||||
- CLI - integrated verification commands for operator use
|
||||
106
docs/modules/verifier/architecture.md
Normal file
106
docs/modules/verifier/architecture.md
Normal file
@@ -0,0 +1,106 @@
|
||||
# Verifier Architecture
|
||||
|
||||
> Standalone, offline-first CLI tool for cryptographic verification of evidence bundles.
|
||||
|
||||
## Overview
|
||||
|
||||
Verifier is a single-project, self-contained .NET CLI application published as a trimmed, single-file binary for multiple platforms. It takes an evidence bundle (a gzipped tar archive) as input, extracts it, and runs a six-stage verification pipeline that validates the manifest, signatures, timestamps, digests, and SBOM/DSSE pair integrity. The tool requires no network access, no database, and no running Stella Ops services.
|
||||
|
||||
## Components
|
||||
|
||||
```
|
||||
src/Verifier/
|
||||
Verifier/ # Single project (self-contained CLI)
|
||||
Program.cs # Entry point and CLI argument parsing
|
||||
BundleExtractor.cs # gzip+tar extraction
|
||||
ManifestLoader.cs # manifest.json parsing and validation
|
||||
SignatureVerifier.cs # DSSE signature verification
|
||||
TimestampVerifier.cs # RFC 3161 timestamp verification
|
||||
DigestVerifier.cs # SHA-256 digest checking
|
||||
PairVerifier.cs # SBOM + DSSE pair matching
|
||||
TrustProfile.cs # Trusted key whitelisting
|
||||
OutputFormatter.cs # Text / JSON / Markdown output
|
||||
```
|
||||
|
||||
## Bundle Format
|
||||
|
||||
The input evidence bundle is a gzipped tar archive with the following structure:
|
||||
|
||||
```
|
||||
bundle.tar.gz
|
||||
manifest.json # Bundle manifest (pairs, metadata, digests)
|
||||
manifest.json.sig # DSSE signature over the manifest
|
||||
pairs/
|
||||
{pairId}/
|
||||
sbom.spdx.json # SPDX SBOM document
|
||||
delta-sig.dsse.json # DSSE envelope for the delta signature
|
||||
{pairId}/
|
||||
...
|
||||
timestamps/ # Optional RFC 3161 timestamps
|
||||
*.tsr # Timestamp request files
|
||||
*.tst # Timestamp token files
|
||||
```
|
||||
|
||||
## Verification Pipeline
|
||||
|
||||
The verification pipeline executes six stages sequentially. Each stage must pass before the next begins:
|
||||
|
||||
| Stage | Name | Description |
|
||||
|-------|--------------------------|--------------------------------------------------------------|
|
||||
| 1 | Extract bundle | Decompress gzip, unpack tar to temporary directory |
|
||||
| 2 | Load manifest | Parse `manifest.json`, validate required fields and structure |
|
||||
| 3 | Signature verification | Verify `manifest.json.sig` DSSE signature against trusted key list |
|
||||
| 4 | Timestamp verification | Validate RFC 3161 timestamp tokens (`.tsr`/`.tst`) if present |
|
||||
| 5 | Digest verification | Recompute SHA-256 digests for all referenced files, compare to manifest |
|
||||
| 6 | Pair verification | Verify each SBOM + DSSE pair matches and is internally consistent |
|
||||
|
||||
## Data Flow
|
||||
|
||||
1. Operator provides a bundle file path and optional trust profile (key whitelist) via CLI arguments.
|
||||
2. Verifier extracts the bundle to a temporary directory.
|
||||
3. The manifest is loaded and parsed.
|
||||
4. The DSSE signature on the manifest is verified against the trust profile's allowed public keys.
|
||||
5. Any RFC 3161 timestamps are validated for structural and cryptographic correctness.
|
||||
6. SHA-256 digests are recomputed for every file referenced in the manifest and compared to the declared values.
|
||||
7. Each SBOM/DSSE pair is validated for internal consistency.
|
||||
8. A verification report is written to stdout in the requested format (text, JSON, or markdown).
|
||||
|
||||
## Database Schema
|
||||
|
||||
Not applicable. Verifier is a standalone CLI tool with no persistent storage.
|
||||
|
||||
## Endpoints
|
||||
|
||||
Not applicable. Verifier is a CLI tool with no HTTP endpoints.
|
||||
|
||||
## Cross-Platform Targets
|
||||
|
||||
| Runtime Identifier | Platform |
|
||||
|-------------------|-----------------------------------|
|
||||
| `win-x64` | Windows x64 |
|
||||
| `linux-x64` | Linux x64 (glibc) |
|
||||
| `linux-musl-x64` | Linux x64 (musl/Alpine) |
|
||||
| `osx-x64` | macOS x64 (Intel) |
|
||||
| `osx-arm64` | macOS ARM64 (Apple Silicon) |
|
||||
|
||||
All targets produce a single-file, self-contained, trimmed binary with no external runtime dependencies.
|
||||
|
||||
## Dependencies
|
||||
|
||||
| Library | Purpose |
|
||||
|------------------------------------|----------------------------------------|
|
||||
| System.CommandLine | CLI argument parsing and help generation |
|
||||
| System.Security.Cryptography | SHA-256, RSA/ECDSA signature verification |
|
||||
| System.Formats.Tar | Tar archive extraction |
|
||||
| System.IO.Compression | Gzip decompression |
|
||||
| System.Text.Json | JSON parsing for manifests and DSSE envelopes |
|
||||
| BouncyCastle (optional) | Extended algorithm support (SM2, EdDSA) |
|
||||
|
||||
## Security Considerations
|
||||
|
||||
- **Air-gap first**: Verifier requires no network access. All verification is performed locally using only the bundle contents and the trust profile.
|
||||
- **No key export or generation**: Verifier only reads public keys from the trust profile; it never generates or exports key material.
|
||||
- **Trust profiles**: Operators define which public keys are trusted for signature verification via a key whitelist file. Bundles signed by unknown keys are rejected.
|
||||
- **Deterministic output**: Given the same bundle and trust profile, Verifier produces identical verification results, supporting audit reproducibility.
|
||||
- **Temporary file cleanup**: Extracted bundle contents are written to a temporary directory and cleaned up after verification completes, minimizing residual data on disk.
|
||||
- **No code execution**: Verifier does not execute any code or scripts from within the bundle. It only reads and verifies data.
|
||||
Reference in New Issue
Block a user