Frontend gaps fill work. Testing fixes work. Auditing in progress.
This commit is contained in:
@@ -11,7 +11,7 @@ This directory contains architecture documentation for all StellaOps modules.
|
||||
| [Authority](./authority/) | `src/Authority/` | Authentication, authorization, OAuth/OIDC, DPoP |
|
||||
| [Gateway](./gateway/) | `src/Gateway/` | API gateway with routing and transport abstraction |
|
||||
| [Router](./router/) | `src/Router/` | Transport-agnostic messaging (TCP/TLS/UDP/RabbitMQ/Valkey) |
|
||||
| [Platform](./platform/) | Cross-cutting | Platform architecture overview |
|
||||
| [Platform](./platform/) | `src/Platform/` | Platform architecture and Platform Service aggregation APIs |
|
||||
|
||||
### Data Ingestion
|
||||
|
||||
|
||||
@@ -75,7 +75,7 @@ src/
|
||||
* Streams progress; exits early unless `--wait`.
|
||||
* `diff image --old <digest> --new <digest> [--view ...]` — show layer‑attributed changes.
|
||||
* `export sbom <digest> [--view ... --format ... --out file]` — download artifact.
|
||||
* `sbom upload --file <path> --artifact <ref> [--format cyclonedx|spdx]` - BYOS upload into the scanner analysis pipeline (ledger join uses the SBOM digest).
|
||||
* `sbom upload --file <path> --artifact <ref> [--format cyclonedx|spdx]` - BYOS upload into the scanner analysis pipeline (ledger join uses the SBOM digest).
|
||||
* `report final <digest> [--policy-revision ... --attest]` — request PASS/FAIL report from backend (policy+vex) and optional attestation.
|
||||
|
||||
### 2.4 Policy & data
|
||||
@@ -128,6 +128,38 @@ src/
|
||||
|
||||
* Imports a previously exported bundle into the local KMS root (`kms/` by default), promotes the imported version to `Active`, and preserves existing versions by marking them `PendingRotation`. Prompts for the passphrase when not provided to keep automation password-safe.
|
||||
|
||||
### 2.11 CI Template Generation (Sprint 015)
|
||||
|
||||
* `ci init --platform <github|gitlab|gitea|all> [--template <gate|scan|verify|full>] [--mode <scan-only|scan-attest|scan-vex>] [--output <dir>] [--force] [--offline] [--scanner-image <ref>]`
|
||||
|
||||
* Generates ready-to-run CI workflow templates for the specified platform(s).
|
||||
* Template types:
|
||||
* `gate` - PR gating workflow that blocks merges on policy violations.
|
||||
* `scan` - Scheduled/push scan workflow for container images.
|
||||
* `verify` - Verification workflow for attestations and signatures.
|
||||
* `full` - All templates combined.
|
||||
* Modes control attestation behavior:
|
||||
* `scan-only` - Scan without attestation.
|
||||
* `scan-attest` - Scan and create attestations (default).
|
||||
* `scan-vex` - Scan with VEX document generation.
|
||||
* `--offline` generates templates with pinned digests for air-gapped environments.
|
||||
|
||||
* `ci list`
|
||||
|
||||
* Lists available template types and supported platforms.
|
||||
|
||||
* `ci validate <workflow-file>`
|
||||
|
||||
* Validates a generated workflow file for correctness.
|
||||
* Checks integration IDs, registry endpoints, and AuthRef references.
|
||||
|
||||
**Generated files:**
|
||||
- GitHub: `.github/workflows/stellaops-{gate,scan,verify}.yml`
|
||||
- GitLab: `.gitlab-ci.yml` or `.gitlab/stellaops-{scan,verify}.yml`
|
||||
- Gitea: `.gitea/workflows/stellaops-{gate,scan,verify}.yml`
|
||||
|
||||
**Implementation:** `CiCommandGroup.cs`, `CiTemplates.cs` in `src/Cli/StellaOps.Cli/Commands/`.
|
||||
|
||||
Both subcommands honour offline-first expectations (no network access) and normalise relative roots via `--root` when operators mirror the credential store.
|
||||
|
||||
### 2.11 Advisory AI (RAG summaries)
|
||||
|
||||
@@ -12,6 +12,7 @@
|
||||
* Attachments are **links** (UI/attestation pages); Notify **does not** attach SBOMs or large blobs to messages.
|
||||
* Secrets for channels (Slack tokens, SMTP creds) are **referenced**, not stored raw in the database.
|
||||
* **2025-11-02 module boundary.** Maintain `src/Notify/` as the reusable notification toolkit (engine, storage, queue, connectors) and `src/Notifier/` as the Notifications Studio host that composes those libraries. Do not merge directories without an approved packaging RFC that covers build impacts, offline kit parity, and cross-module governance.
|
||||
* **API versioning.** `/api/v1/notify` is the canonical UI and CLI surface. `/api/v2/notify` remains compatibility-only until v2-only features are merged into v1 or explicitly deprecated; Gateway should provide v2->v1 routing where needed.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -1,11 +1,12 @@
|
||||
# Platform agent guide
|
||||
|
||||
## Mission
|
||||
Platform module describes cross-cutting architecture, contracts, and guardrails that bind the services together.
|
||||
Platform module describes cross-cutting architecture and now owns the Platform Service that aggregates health, quotas, onboarding, preferences, and global search for the Console UI.
|
||||
|
||||
## Key docs
|
||||
- [Module README](./README.md)
|
||||
- [Architecture](./architecture.md)
|
||||
- [Platform Service](./platform-service.md)
|
||||
- [Implementation plan](./implementation_plan.md)
|
||||
- [Task board](./TASKS.md)
|
||||
- [Architecture overview](./architecture-overview.md)
|
||||
@@ -24,6 +25,7 @@ Platform module describes cross-cutting architecture, contracts, and guardrails
|
||||
## Required Reading
|
||||
- `docs/modules/platform/README.md`
|
||||
- `docs/modules/platform/architecture.md`
|
||||
- `docs/modules/platform/platform-service.md`
|
||||
- `docs/modules/platform/implementation_plan.md`
|
||||
- `docs/modules/platform/architecture-overview.md`
|
||||
|
||||
|
||||
@@ -2,15 +2,16 @@
|
||||
|
||||
Platform module describes cross-cutting architecture, contracts, and guardrails that bind the services together.
|
||||
|
||||
## Latest updates (2025-11-30)
|
||||
- Sprint tracker `docs/implplan/SPRINT_0324_0001_0001_docs_modules_platform.md` and module `TASKS.md` added to mirror status.
|
||||
- README now points to architecture overview, AOC references, and offline guidance entry points.
|
||||
- Platform module remains docs-only; no runtime services.
|
||||
## Latest updates (2025-12-29)
|
||||
- Added Platform Service definition (`platform-service.md`) and sprint scope for service foundation.
|
||||
- README updated to reflect runtime service ownership alongside cross-cutting docs.
|
||||
- Platform-facing UI sprints now depend on Platform Service aggregation endpoints.
|
||||
|
||||
## Responsibilities
|
||||
- Maintain the system-wide architecture overview and integration diagrams.
|
||||
- Capture Aggregation-Only Contract guidance and migration playbooks.
|
||||
- Document shared services such as API gateway, tenancy, quotas, and offline posture.
|
||||
- Define and maintain Platform Service API contracts and aggregation behavior.
|
||||
- Coordinate platform-wide epics and compliance checklists.
|
||||
|
||||
## Key components
|
||||
@@ -24,7 +25,7 @@ Platform module describes cross-cutting architecture, contracts, and guardrails
|
||||
- Docs guild for cross-module onboarding.
|
||||
|
||||
## Operational notes
|
||||
- Docs-only module; focus is architectural governance and cross-module guardrails.
|
||||
- Module spans architecture governance and the Platform Service aggregation APIs.
|
||||
- Glossaries and guardrails cross-linked across docs; keep AOC references current.
|
||||
- Status mirrors: sprint file and `docs/modules/platform/TASKS.md`.
|
||||
|
||||
@@ -58,9 +59,9 @@ Platform module describes cross-cutting architecture, contracts, and guardrails
|
||||
- Glossaries and guardrails: cross-linked across all module documentation
|
||||
|
||||
### Technical Notes
|
||||
- Platform is docs-only; no runtime services
|
||||
- Focus on architectural governance and cross-module guardrails
|
||||
- Ensures discoverability of Offline Kit and AOC references from README/architecture
|
||||
- Platform includes the Platform Service runtime plus docs ownership
|
||||
- Focus on architectural governance and cross-module aggregation endpoints
|
||||
- Ensure discoverability of Offline Kit and AOC references from README/architecture
|
||||
|
||||
## Epic alignment
|
||||
- Aligns with the Aggregation-Only Contract reference, Policy and Policy Studio guides, Graph/Vulnerability Explorer documentation, and the Orchestrator, Advisory AI, and Notifications implementation plans to keep platform guardrails consistent across services.
|
||||
|
||||
19
docs/modules/platform/TASKS.md
Normal file
19
docs/modules/platform/TASKS.md
Normal file
@@ -0,0 +1,19 @@
|
||||
# Platform Module Task Board
|
||||
|
||||
This board mirrors the active platform sprint(s). Update alongside the sprint tracker.
|
||||
|
||||
## Active sprint tasks
|
||||
Source of truth: `docs/implplan/SPRINT_20251229_043_PLATFORM_platform_service_foundation.md`.
|
||||
|
||||
| Task ID | Status | Notes |
|
||||
| --- | --- | --- |
|
||||
| PLAT-SVC-001 | DONE | Platform Service project scaffold. |
|
||||
| PLAT-SVC-002 | DONE | Health aggregation endpoints. |
|
||||
| PLAT-SVC-003 | DONE | Quota aggregation endpoints. |
|
||||
| PLAT-SVC-004 | DONE | Onboarding state storage + APIs. |
|
||||
| PLAT-SVC-005 | DONE | Preferences storage + APIs. |
|
||||
| PLAT-SVC-006 | DONE | Global search aggregation. |
|
||||
| PLAT-SVC-007 | DONE | Gateway route registration + scopes. |
|
||||
| PLAT-SVC-008 | DONE | Observability metrics/logging. |
|
||||
| PLAT-SVC-009 | DONE | Determinism/offline tests. |
|
||||
| PLAT-SVC-010 | DONE | Docs/runbooks update. |
|
||||
@@ -53,7 +53,7 @@ graph TD
|
||||
Notify[Notifier]
|
||||
end
|
||||
subgraph Experience["UX & Export"]
|
||||
UIService[Console Backend]
|
||||
PlatformSvc[Platform Service<br/>(Console Backend)]
|
||||
Exporters[Export / Offline Kit]
|
||||
end
|
||||
Observability[Telemetry Stack]
|
||||
@@ -70,9 +70,9 @@ graph TD
|
||||
RawStore --> Policy
|
||||
Policy --> Scheduler
|
||||
Policy --> Notify
|
||||
Policy --> UIService
|
||||
Scheduler --> UIService
|
||||
UIService --> Exporters
|
||||
Policy --> PlatformSvc
|
||||
Scheduler --> PlatformSvc
|
||||
PlatformSvc --> Exporters
|
||||
Exporters --> CLI
|
||||
Exporters --> Offline[Offline Kit]
|
||||
Observability -.-> ScannerWeb
|
||||
@@ -83,6 +83,8 @@ graph TD
|
||||
Observability -.-> Notify
|
||||
```
|
||||
|
||||
Platform Service (StellaOps.Platform.WebService) aggregates cross-service status for the Console UI (health, quotas, onboarding, preferences, global search) and does not mutate raw evidence.
|
||||
|
||||
Key boundaries:
|
||||
|
||||
- **AOC border.** Everything inside the Ingestion subgraph writes only immutable raw facts plus link hints. Derived severity, consensus, and risk remain outside the border.
|
||||
|
||||
@@ -5,12 +5,14 @@ This module aggregates cross-cutting contracts and guardrails that every StellaO
|
||||
## Anchors
|
||||
- High-level system view: `../../07_HIGH_LEVEL_ARCHITECTURE.md`
|
||||
- Platform overview: `architecture-overview.md`
|
||||
- Platform service definition: `platform-service.md`
|
||||
- Aggregation-Only Contract: `../../aoc/aggregation-only-contract.md` (referenced across ingestion/observability docs)
|
||||
|
||||
## Scope
|
||||
- **Identity & tenancy**: Authority-issued OpToks, tenant scoping, RBAC, short TTLs; see Authority module docs.
|
||||
- **AOC & provenance**: services ingest evidence without mutating/merging; provenance preserved; determinism required.
|
||||
- **Offline posture**: Offline Kit parity, sealed-mode defaults, deterministic bundles.
|
||||
- **Platform Service**: aggregation endpoints for health, quotas, onboarding, preferences, and global search.
|
||||
- **Observability baseline**: metrics/logging/tracing patterns reused across modules; collectors documented under Telemetry module.
|
||||
- **Determinism**: stable ordering, UTC timestamps, content-addressed artifacts, reproducible exports.
|
||||
|
||||
|
||||
32
docs/modules/platform/implementation_plan.md
Normal file
32
docs/modules/platform/implementation_plan.md
Normal file
@@ -0,0 +1,32 @@
|
||||
# Platform Module Implementation Plan
|
||||
|
||||
## Purpose
|
||||
Provide a lightweight, living plan for platform-level capabilities, with the Platform Service as the primary delivery vehicle for cross-service aggregation needed by the Console UI and CLI.
|
||||
|
||||
## Active work
|
||||
- `docs/implplan/SPRINT_20251229_043_PLATFORM_platform_service_foundation.md` (Platform Service foundation).
|
||||
|
||||
## Near-term deliverables
|
||||
- `StellaOps.Platform.WebService` skeleton with health endpoint and auth wiring.
|
||||
- Aggregation endpoints for health, quotas, onboarding, preferences, and global search.
|
||||
- Deterministic caching behavior with "data as of" metadata for offline display.
|
||||
- Platform metadata endpoint and capability listing for UI bootstrapping.
|
||||
- Gateway registration and scoped access policies.
|
||||
- Unit + integration tests for ordering, caching, and error handling.
|
||||
- Postgres schema stub for platform state (`docs/db/schemas/platform.sql`).
|
||||
|
||||
## Dependencies
|
||||
- Authority (identity, scopes, tenant claims).
|
||||
- Gateway/Router (route registration, auth headers, rate limits).
|
||||
- Orchestrator, Notifier, and storage services for aggregation inputs.
|
||||
|
||||
## Evidence of completion
|
||||
- Service project under `src/Platform/StellaOps.Platform.WebService`.
|
||||
- Updated platform documentation and runbooks.
|
||||
- Postgres schema spec at `docs/db/schemas/platform.sql`.
|
||||
- Passing deterministic aggregation tests.
|
||||
|
||||
## Reference docs
|
||||
- `docs/modules/platform/platform-service.md`
|
||||
- `docs/modules/platform/architecture.md`
|
||||
- `docs/modules/platform/architecture-overview.md`
|
||||
86
docs/modules/platform/platform-service.md
Normal file
86
docs/modules/platform/platform-service.md
Normal file
@@ -0,0 +1,86 @@
|
||||
# Platform Service (StellaOps.Platform.WebService)
|
||||
|
||||
## Purpose
|
||||
Provide a single, deterministic aggregation layer for cross-service UX workflows (health, quotas, onboarding, preferences, global search) so the Console UI and CLI do not fan out to multiple modules directly.
|
||||
|
||||
## Non-goals
|
||||
- Replace module-owned APIs (Authority, Policy, Scanner, Orchestrator, etc.).
|
||||
- Ingest or mutate raw evidence or policy overlays.
|
||||
- Store high-volume evidence payloads (SBOMs, VEX, audit bundles).
|
||||
|
||||
## Responsibilities
|
||||
- Aggregate platform health and dependency status.
|
||||
- Aggregate quota usage across Authority, Gateway, Orchestrator, and storage backends.
|
||||
- Persist onboarding progress and tenant setup milestones.
|
||||
- Persist dashboard personalization and layout preferences.
|
||||
- Provide global search aggregation across entities.
|
||||
- Surface platform metadata for UI bootstrapping (version, build, offline status).
|
||||
|
||||
## API surface (v1)
|
||||
|
||||
### Health aggregation
|
||||
- GET `/api/v1/platform/health/summary`
|
||||
- GET `/api/v1/platform/health/dependencies`
|
||||
- GET `/api/v1/platform/health/incidents`
|
||||
- GET `/api/v1/platform/health/metrics`
|
||||
|
||||
### Quota aggregation
|
||||
- GET `/api/v1/platform/quotas/summary`
|
||||
- GET `/api/v1/platform/quotas/tenants/{tenantId}`
|
||||
- GET `/api/v1/platform/quotas/alerts`
|
||||
- POST `/api/v1/platform/quotas/alerts`
|
||||
|
||||
### Onboarding
|
||||
- GET `/api/v1/platform/onboarding/status`
|
||||
- POST `/api/v1/platform/onboarding/complete/{step}`
|
||||
- POST `/api/v1/platform/onboarding/skip`
|
||||
- GET `/api/v1/platform/tenants/{tenantId}/setup-status`
|
||||
|
||||
### Preferences
|
||||
- GET `/api/v1/platform/preferences/dashboard`
|
||||
- PUT `/api/v1/platform/preferences/dashboard`
|
||||
- GET `/api/v1/platform/dashboard/profiles`
|
||||
- GET `/api/v1/platform/dashboard/profiles/{profileId}`
|
||||
- POST `/api/v1/platform/dashboard/profiles`
|
||||
|
||||
### Global search
|
||||
- GET `/api/v1/search` (alias to `/api/v1/platform/search`)
|
||||
- GET `/api/v1/platform/search`
|
||||
|
||||
### Metadata
|
||||
- GET `/api/v1/platform/metadata`
|
||||
|
||||
## Data model
|
||||
- `platform.dashboard_preferences` (dashboard layout, widgets, filters)
|
||||
- `platform.dashboard_profiles` (saved profiles per tenant)
|
||||
- `platform.onboarding_state` (step state, timestamps, actor)
|
||||
- `platform.quota_alerts` (per-tenant quota alert thresholds)
|
||||
- `platform.search_history` (optional, user-scoped, append-only)
|
||||
- Schema reference: `docs/db/schemas/platform.sql` (PostgreSQL; in-memory stores used until storage driver switches).
|
||||
|
||||
## Dependencies
|
||||
- Authority (tenant/user identity, quotas, RBAC)
|
||||
- Gateway (rate-limit status and request telemetry)
|
||||
- Orchestrator (job quotas, SLO state)
|
||||
- Notifier (alert policies and delivery status)
|
||||
- Policy/Scanner/Registry/VexHub (search aggregation sources)
|
||||
|
||||
## Security and scopes
|
||||
- Health: `ops.health` (summary), `ops.admin` (metrics)
|
||||
- Quotas: `quota.read` (summary), `quota.admin` (alerts/config)
|
||||
- Onboarding: `onboarding.read`, `onboarding.write`
|
||||
- Preferences: `ui.preferences.read`, `ui.preferences.write`
|
||||
- Search: `search.read` plus downstream service scopes (`findings:read`, `policy:read`, etc.)
|
||||
- Metadata: `platform.metadata.read`
|
||||
|
||||
## Determinism and offline posture
|
||||
- Stable ordering with explicit sort keys and deterministic tiebreakers.
|
||||
- All timestamps in UTC ISO-8601.
|
||||
- Cache last-known snapshots for offline rendering with "data as of" markers.
|
||||
|
||||
## Observability
|
||||
- Metrics: `platform.aggregate.latency_ms`, `platform.aggregate.errors_total`, `platform.aggregate.cache_hits_total`
|
||||
- Logs include `traceId`, `tenantId`, `operation`, and cache-hit indicators.
|
||||
|
||||
## Gateway exposure
|
||||
The Platform Service is exposed via Gateway and registered through Router discovery. It does not expose direct ingress outside Gateway in production.
|
||||
@@ -105,6 +105,53 @@ Operational rules:
|
||||
- Logs: structured, include tenant + artifact digest + sbomVersion; classify ingest failures (schema, storage, orchestrator, validation).
|
||||
- Alerts: backlog thresholds for outbox/event delivery; high latency on path/timeline endpoints.
|
||||
|
||||
## 8.1) Registry Source Management (Sprint 012)
|
||||
|
||||
The service manages container registry sources for automated image discovery and scanning:
|
||||
|
||||
### Models
|
||||
- `RegistrySource` — registry connection with URL, filters, schedule, credentials (via AuthRef).
|
||||
- `RegistrySourceRun` — run history with status, discovered images, triggered scans, error details.
|
||||
- `RegistrySourceStatus` — `Draft`, `Active`, `Paused`, `Error`, `Deleted`.
|
||||
- `RegistrySourceProvider` — `Generic`, `Harbor`, `DockerHub`, `ACR`, `ECR`, `GCR`, `GHCR`.
|
||||
|
||||
### APIs
|
||||
- `GET/POST/PUT/DELETE /api/v1/registry-sources` — CRUD operations.
|
||||
- `POST /api/v1/registry-sources/{id}/test` — test registry connection and credentials.
|
||||
- `POST /api/v1/registry-sources/{id}/trigger` — manually trigger discovery and scanning.
|
||||
- `POST /api/v1/registry-sources/{id}/pause` / `/resume` — pause/resume scheduled scans.
|
||||
- `GET /api/v1/registry-sources/{id}/runs` — run history with health metrics.
|
||||
- `GET /api/v1/registry-sources/{id}/discover/repositories` — discover repositories matching filters.
|
||||
- `GET /api/v1/registry-sources/{id}/discover/tags/{repository}` — discover tags for a repository.
|
||||
- `GET /api/v1/registry-sources/{id}/discover/images` — full image discovery.
|
||||
- `POST /api/v1/registry-sources/{id}/discover-and-scan` — discover and submit scan jobs.
|
||||
|
||||
### Webhook Ingestion
|
||||
- `POST /api/v1/webhooks/registry/{sourceId}` — receive push notifications from registries.
|
||||
- Supported providers: Harbor, DockerHub, ACR, ECR, GCR, GHCR.
|
||||
- HMAC-SHA256 signature validation using webhook secret from AuthRef.
|
||||
- Auto-detection of provider from request headers.
|
||||
|
||||
### Discovery Service
|
||||
- OCI Distribution Spec compliant repository/tag enumeration.
|
||||
- Pagination via RFC 5988 Link headers.
|
||||
- Allowlist/denylist filtering for repositories and tags (glob patterns).
|
||||
- Manifest digest retrieval via HEAD requests.
|
||||
|
||||
### Scan Job Emission
|
||||
- Batch submission to Scanner service with rate limiting.
|
||||
- Deduplication (skips if job already exists).
|
||||
- Metadata includes source ID, trigger type, client request ID.
|
||||
|
||||
### Configuration
|
||||
- `SbomService:ScannerUrl` — Scanner service endpoint (default: `http://localhost:5100`).
|
||||
- `SbomService:BatchScanSize` — max images per batch (default: 10).
|
||||
- `SbomService:BatchScanDelayMs` — delay between batch submissions (default: 100ms).
|
||||
|
||||
### Credentials
|
||||
- All credentials via AuthRef URIs: `authref://{vault}/{path}#{key}`.
|
||||
- Supports basic auth (`basic:user:pass`) and bearer tokens (`bearer:token`) for development.
|
||||
|
||||
## 9) Configuration (PostgreSQL-backed catalog & lookup)
|
||||
- Enable PostgreSQL storage for `/console/sboms` and `/components/lookup` by setting `SbomService:PostgreSQL:ConnectionString` (env: `SBOM_SbomService__PostgreSQL__ConnectionString`).
|
||||
- Optional overrides: `SbomService:PostgreSQL:Schema`, `SbomService:PostgreSQL:CatalogTable`, `SbomService:PostgreSQL:ComponentLookupTable`; defaults are `sbom_service`, `sbom_catalog`, `sbom_component_neighbors`.
|
||||
|
||||
307
docs/modules/sbomservice/lineage/ui-architecture.md
Normal file
307
docs/modules/sbomservice/lineage/ui-architecture.md
Normal file
@@ -0,0 +1,307 @@
|
||||
# Lineage Smart-Diff UI Architecture
|
||||
|
||||
> Sprint: SPRINT_20251229_001_FE_lineage_smartdiff_overview
|
||||
> Last Updated: 2025-12-29
|
||||
|
||||
## 1. Overview
|
||||
|
||||
The Lineage Smart-Diff feature provides a comprehensive UI for visualizing SBOM lineage graphs, comparing artifact versions, and explaining security state changes between builds.
|
||||
|
||||
### 1.1 Completion Status
|
||||
|
||||
| Area | Completion | Notes |
|
||||
|------|------------|-------|
|
||||
| **Lineage Graph SVG** | 95% | Full DAG visualization with lanes, pan/zoom, nodes |
|
||||
| **Hover Cards** | 85% | Basic info displayed; needs CGS integration |
|
||||
| **SBOM Diff View** | 90% | 3-column diff exists; needs row expanders |
|
||||
| **VEX Diff View** | 90% | Status change display; needs reachability gates |
|
||||
| **Compare Mode** | 85% | Three-pane layout exists; needs explainer timeline |
|
||||
| **Export Dialog** | 80% | Basic export; needs audit pack format |
|
||||
| **Proof Tree** | 75% | Merkle tree viz; needs confidence breakdown |
|
||||
| **Reachability Diff** | 60% | Basic view; needs gate visualization |
|
||||
|
||||
## 2. UI Data Contracts
|
||||
|
||||
### 2.1 CGS/Lineage API Integration
|
||||
|
||||
```typescript
|
||||
// Lineage Graph Response
|
||||
interface LineageGraph {
|
||||
artifact: string;
|
||||
nodes: LineageNode[];
|
||||
edges: LineageEdge[];
|
||||
metadata: {
|
||||
totalNodes: number;
|
||||
maxDepth: number;
|
||||
generatedAt: string;
|
||||
};
|
||||
}
|
||||
|
||||
interface LineageNode {
|
||||
id: string;
|
||||
artifactDigest: string;
|
||||
artifactRef: string;
|
||||
sequenceNumber: number;
|
||||
createdAt: string;
|
||||
source: string;
|
||||
parentDigests: string[];
|
||||
badges: {
|
||||
newVulns: number;
|
||||
resolvedVulns: number;
|
||||
signatureStatus: 'valid' | 'invalid' | 'unknown';
|
||||
reachabilityStatus: 'analyzed' | 'pending' | 'unavailable';
|
||||
};
|
||||
replayHash: string;
|
||||
cgsHash?: string;
|
||||
}
|
||||
|
||||
interface LineageEdge {
|
||||
fromDigest: string;
|
||||
toDigest: string;
|
||||
relationship: 'parent' | 'build' | 'base' | 'derived';
|
||||
}
|
||||
```
|
||||
|
||||
### 2.2 Diff Response Schema
|
||||
|
||||
```typescript
|
||||
interface LineageDiffResponse {
|
||||
fromDigest: string;
|
||||
toDigest: string;
|
||||
sbomDiff: {
|
||||
added: ComponentDiff[];
|
||||
removed: ComponentDiff[];
|
||||
versionChanged: VersionChange[];
|
||||
licenseChanged: LicenseChange[];
|
||||
};
|
||||
vexDiff: VexDelta[];
|
||||
reachabilityDiff: ReachabilityDelta[];
|
||||
replayHash: string;
|
||||
generatedAt: string;
|
||||
}
|
||||
|
||||
interface ComponentDiff {
|
||||
purl: string;
|
||||
name: string;
|
||||
version: string;
|
||||
ecosystem: string;
|
||||
license?: string;
|
||||
scope: 'runtime' | 'development' | 'optional';
|
||||
}
|
||||
|
||||
interface VersionChange {
|
||||
purl: string;
|
||||
name: string;
|
||||
fromVersion: string;
|
||||
toVersion: string;
|
||||
changeType: 'upgrade' | 'downgrade' | 'patch';
|
||||
}
|
||||
|
||||
interface VexDelta {
|
||||
cveId: string;
|
||||
purl: string;
|
||||
fromStatus: VexStatus;
|
||||
toStatus: VexStatus;
|
||||
justification?: string;
|
||||
effectiveAt: string;
|
||||
}
|
||||
|
||||
interface ReachabilityDelta {
|
||||
cveId: string;
|
||||
purl: string;
|
||||
fromReachable: boolean;
|
||||
toReachable: boolean;
|
||||
paths?: string[][];
|
||||
}
|
||||
```
|
||||
|
||||
## 3. Explainer Timeline Requirements
|
||||
|
||||
### 3.1 Engine Steps
|
||||
|
||||
The explainer timeline shows the sequence of analysis steps:
|
||||
|
||||
| Step | Description | Visual Cue |
|
||||
|------|-------------|------------|
|
||||
| SBOM Parse | Initial SBOM ingestion | Document icon |
|
||||
| Component Match | CVE-to-component matching | Link icon |
|
||||
| VEX Lookup | VEX document resolution | Shield icon |
|
||||
| Reachability | Call graph analysis | Graph icon |
|
||||
| Policy Gate | Policy rule evaluation | Gate icon |
|
||||
| Verdict | Final determination | Checkmark/X |
|
||||
|
||||
### 3.2 Data Binding
|
||||
|
||||
```typescript
|
||||
interface ExplainerStep {
|
||||
stepId: string;
|
||||
stepType: 'sbom_parse' | 'component_match' | 'vex_lookup' |
|
||||
'reachability' | 'policy_gate' | 'verdict';
|
||||
timestamp: string;
|
||||
duration: number;
|
||||
status: 'success' | 'warning' | 'error';
|
||||
inputs: Record<string, unknown>;
|
||||
outputs: Record<string, unknown>;
|
||||
evidence?: {
|
||||
type: string;
|
||||
hash: string;
|
||||
downloadUrl?: string;
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
## 4. Node Diff Table UX
|
||||
|
||||
### 4.1 Expander Pattern
|
||||
|
||||
Each diff row can expand to show:
|
||||
- Full component details (license, scope, dependencies)
|
||||
- CVE associations and status
|
||||
- Reachability paths (if analyzed)
|
||||
- VEX statements affecting the component
|
||||
|
||||
### 4.2 Visual Design
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Component │ Before │ After │ Status│
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ ▶ lodash │ 4.17.20 │ 4.17.21 │ ↑ │
|
||||
│ └─ CVE-2021-23337 (fixed in 4.17.21) │
|
||||
│ └─ License: MIT │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ ▶ axios │ 0.21.1 │ 0.21.4 │ ↑ │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ ▶ express │ - │ 4.18.2 │ + NEW │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## 5. Reachability Gate Diff UI
|
||||
|
||||
### 5.1 Visual Cues
|
||||
|
||||
| Gate Status | Icon | Color |
|
||||
|-------------|------|-------|
|
||||
| Reachable | ● | Red |
|
||||
| Not Reachable | ○ | Green |
|
||||
| Unknown | ? | Gray |
|
||||
| Changed | ↔ | Orange |
|
||||
|
||||
### 5.2 Path Display
|
||||
|
||||
When reachability changes, show the call path:
|
||||
```
|
||||
entrypoint.ts → handler.ts → vulnerable_fn.ts → lodash.get()
|
||||
```
|
||||
|
||||
## 6. Audit Pack Export UI
|
||||
|
||||
### 6.1 Export Options
|
||||
|
||||
| Option | Description | Default |
|
||||
|--------|-------------|---------|
|
||||
| Include SBOMs | Both before/after SBOMs | ✓ |
|
||||
| Include Diff | Component/VEX/reachability diff | ✓ |
|
||||
| Include Attestations | DSSE envelopes | ✓ |
|
||||
| Include Evidence | Supporting evidence files | ✗ |
|
||||
| Sign Bundle | Sign with tenant key | ✓ |
|
||||
|
||||
### 6.2 Manifest Schema
|
||||
|
||||
```json
|
||||
{
|
||||
"version": "1.0",
|
||||
"generatedAt": "2025-12-29T10:00:00Z",
|
||||
"artifactA": { "digest": "sha256:...", "name": "...", "createdAt": "..." },
|
||||
"artifactB": { "digest": "sha256:...", "name": "...", "createdAt": "..." },
|
||||
"contents": [
|
||||
{ "type": "sbom", "filename": "before.cdx.json", "sha256": "..." },
|
||||
{ "type": "sbom", "filename": "after.cdx.json", "sha256": "..." },
|
||||
{ "type": "diff", "filename": "diff.json", "sha256": "..." },
|
||||
{ "type": "attestation", "filename": "attestations.dsse.json", "sha256": "..." }
|
||||
],
|
||||
"merkleRoot": "sha256:...",
|
||||
"summary": {
|
||||
"componentsAdded": 5,
|
||||
"componentsRemoved": 2,
|
||||
"vexUpdates": 3,
|
||||
"attestationCount": 2
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 7. Copy-Safe Workflow
|
||||
|
||||
### 7.1 Pinned Explanation UX
|
||||
|
||||
Operators can "pin" explanations for ticket export:
|
||||
|
||||
1. Click pin icon on any verdict/finding
|
||||
2. Explanation captures current state with hash
|
||||
3. Export as Markdown/JSON for JIRA/ServiceNow
|
||||
|
||||
### 7.2 Ticket Export Format
|
||||
|
||||
```markdown
|
||||
## Security Finding Report
|
||||
|
||||
**Artifact**: myorg/myapp:v1.2.3 (sha256:abc123...)
|
||||
**Generated**: 2025-12-29T10:00:00Z
|
||||
**Replay Hash**: sha256:def456...
|
||||
|
||||
### Finding: CVE-2021-23337 in lodash
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| Status | Not Affected |
|
||||
| Reason | Not Reachable |
|
||||
| Evidence | Call graph analysis |
|
||||
| VEX ID | VEX-2025-001 |
|
||||
|
||||
### Verification
|
||||
|
||||
To reproduce this verdict:
|
||||
```
|
||||
stella verdict replay --hash sha256:def456...
|
||||
```
|
||||
```
|
||||
|
||||
## 8. Confidence Breakdown Charts
|
||||
|
||||
### 8.1 Metrics Sources
|
||||
|
||||
| Metric | Source | Weight |
|
||||
|--------|--------|--------|
|
||||
| SBOM Completeness | Scanner analysis | 30% |
|
||||
| VEX Coverage | VEX Hub aggregation | 25% |
|
||||
| Reachability Depth | Call graph analysis | 25% |
|
||||
| Attestation Count | Sigstore/local | 20% |
|
||||
|
||||
### 8.2 Visualization
|
||||
|
||||
- Donut chart showing confidence breakdown
|
||||
- Hover for detailed explanations
|
||||
- Color coding: Green (>80%), Yellow (50-80%), Red (<50%)
|
||||
|
||||
## 9. Component Inventory
|
||||
|
||||
### 9.1 Lineage Feature Components
|
||||
|
||||
| Component | Location | Status |
|
||||
|-----------|----------|--------|
|
||||
| `LineageGraphComponent` | `lineage-graph.component.ts` | Complete |
|
||||
| `LineageNodeComponent` | `lineage-node.component.ts` | Complete |
|
||||
| `LineageEdgeComponent` | `lineage-edge.component.ts` | Complete |
|
||||
| `LineageHoverCardComponent` | `lineage-hover-card.component.ts` | Needs CGS |
|
||||
| `LineageMiniMapComponent` | `lineage-minimap.component.ts` | Complete |
|
||||
| `LineageControlsComponent` | `lineage-controls.component.ts` | Complete |
|
||||
| `LineageSbomDiffComponent` | `lineage-sbom-diff.component.ts` | Needs expanders |
|
||||
| `LineageVexDiffComponent` | `lineage-vex-diff.component.ts` | Needs gates |
|
||||
| `LineageCompareComponent` | `lineage-compare.component.ts` | Needs timeline |
|
||||
| `LineageExportDialogComponent` | `lineage-export-dialog.component.ts` | Needs audit pack |
|
||||
|
||||
## 10. Related Documentation
|
||||
|
||||
- [SbomService Lineage API](../sbomservice/lineage/architecture.md)
|
||||
- [UI Architecture](../ui/architecture.md)
|
||||
- [Graph Module Architecture](../graph/architecture.md)
|
||||
333
docs/modules/sbomservice/sources/architecture.md
Normal file
333
docs/modules/sbomservice/sources/architecture.md
Normal file
@@ -0,0 +1,333 @@
|
||||
# SBOM Sources Architecture
|
||||
|
||||
> Sprint: SPRINT_20251229_000_PLATFORM_sbom_sources_overview
|
||||
> Last Updated: 2025-12-29
|
||||
|
||||
## 1. Overview
|
||||
|
||||
The SBOM Sources subsystem provides a unified configuration and management layer for all SBOM ingestion pathways: Zastava (registry webhooks), Docker (direct image scans), CLI (external submissions), and Git (repository scans).
|
||||
|
||||
### 1.1 Problem Statement
|
||||
|
||||
**Current State:**
|
||||
- Fragmented source management across Orchestrator, Concelier, and Scanner
|
||||
- No unified UI for configuring ingestion sources
|
||||
- Credentials scattered without centralized management
|
||||
- No visibility into source health and scan history
|
||||
|
||||
**Target State:**
|
||||
- Single "Sources" module for all SBOM ingestion configuration
|
||||
- Unified UI with type-specific wizards
|
||||
- Centralized credential management via AuthRef pattern
|
||||
- Real-time status monitoring and run history
|
||||
|
||||
## 2. Source Types Taxonomy
|
||||
|
||||
| Type | Trigger Mode | Target | Auth Pattern | Discovery |
|
||||
|------|--------------|--------|--------------|-----------|
|
||||
| **Zastava** | Webhook (push) | Registry images | Registry creds + webhook secret | From webhook payload |
|
||||
| **Docker** | Scheduled/Manual | Specific images | Registry credentials | Tag patterns |
|
||||
| **CLI** | External submission | Any SBOM | API token | N/A (receives SBOMs) |
|
||||
| **Git** | Webhook/Scheduled | Repository code | PAT/SSH + webhook secret | Branch patterns |
|
||||
|
||||
## 3. Health Signals
|
||||
|
||||
Each source maintains the following health signals:
|
||||
|
||||
| Signal | Description | Metric |
|
||||
|--------|-------------|--------|
|
||||
| `status` | Current operational state | `active`, `paused`, `error`, `disabled`, `pending` |
|
||||
| `lastRunAt` | Timestamp of most recent run | ISO-8601 UTC |
|
||||
| `lastRunStatus` | Status of most recent run | `succeeded`, `failed`, `partial-success` |
|
||||
| `consecutiveFailures` | Count of consecutive failed runs | Integer, resets on success |
|
||||
| `currentHourScans` | Scans executed in rate-limit window | Integer, resets hourly |
|
||||
|
||||
## 4. API Contract
|
||||
|
||||
### 4.1 Source CRUD Endpoints
|
||||
|
||||
```
|
||||
GET /api/v1/sources List sources with filtering/pagination
|
||||
POST /api/v1/sources Create new source
|
||||
GET /api/v1/sources/{sourceId} Get source details
|
||||
PUT /api/v1/sources/{sourceId} Update source configuration
|
||||
DELETE /api/v1/sources/{sourceId} Delete source
|
||||
```
|
||||
|
||||
### 4.2 Operational Endpoints
|
||||
|
||||
```
|
||||
POST /api/v1/sources/{sourceId}/test Test source connection
|
||||
POST /api/v1/sources/{sourceId}/trigger Manual scan trigger
|
||||
POST /api/v1/sources/{sourceId}/pause Pause source (with reason)
|
||||
POST /api/v1/sources/{sourceId}/resume Resume paused source
|
||||
```
|
||||
|
||||
### 4.3 Run History Endpoints
|
||||
|
||||
```
|
||||
GET /api/v1/sources/{sourceId}/runs List run history (paginated)
|
||||
GET /api/v1/sources/{sourceId}/runs/{runId} Get run details
|
||||
```
|
||||
|
||||
### 4.4 Webhook Endpoints
|
||||
|
||||
```
|
||||
POST /api/v1/webhooks/zastava/{sourceId} Zastava registry webhook
|
||||
POST /api/v1/webhooks/git/{sourceId} Git repository webhook
|
||||
```
|
||||
|
||||
## 5. Domain Events
|
||||
|
||||
| Event | Payload | When Emitted |
|
||||
|-------|---------|--------------|
|
||||
| `source.created` | `{ sourceId, sourceType, tenantId }` | New source registered |
|
||||
| `source.updated` | `{ sourceId, changedFields[] }` | Source configuration updated |
|
||||
| `source.deleted` | `{ sourceId }` | Source removed |
|
||||
| `source.paused` | `{ sourceId, reason, pausedBy }` | Source paused |
|
||||
| `source.resumed` | `{ sourceId, resumedBy }` | Source resumed |
|
||||
| `source.run.started` | `{ runId, sourceId, trigger }` | Run initiated |
|
||||
| `source.run.completed` | `{ runId, sourceId, status, metrics }` | Run finished |
|
||||
|
||||
## 6. Credential Lifecycle (AuthRef Pattern)
|
||||
|
||||
All source credentials use the AuthRef pattern for secure storage:
|
||||
|
||||
1. **Storage**: Credentials stored in Authority vault, never inline in source configs
|
||||
2. **Reference**: Sources hold `authRef` identifiers pointing to vault entries
|
||||
3. **Rotation**: Rotate-on-demand API; old refs invalidated, new ref issued
|
||||
4. **Audit**: All credential access logged with source context
|
||||
|
||||
### 6.1 Supported Credential Types
|
||||
|
||||
| Source Type | Credential Types |
|
||||
|-------------|------------------|
|
||||
| Zastava | Registry auth (basic/token), webhook secret |
|
||||
| Docker | Registry auth (basic/token/ECR/GCR/ACR) |
|
||||
| CLI | API token, OIDC identity |
|
||||
| Git | PAT, SSH key, webhook secret |
|
||||
|
||||
## 7. Telemetry Schema
|
||||
|
||||
### 7.1 Metrics
|
||||
|
||||
| Metric | Type | Labels | Description |
|
||||
|--------|------|--------|-------------|
|
||||
| `sbom_source_runs_total` | Counter | `source_type`, `status`, `trigger` | Total runs by type and outcome |
|
||||
| `sbom_source_run_duration_seconds` | Histogram | `source_type` | Run execution time |
|
||||
| `sbom_source_items_scanned_total` | Counter | `source_type` | Items processed per run |
|
||||
| `sbom_source_connection_test_duration_seconds` | Histogram | `source_type` | Connection test latency |
|
||||
| `sbom_source_active_count` | Gauge | `source_type`, `status` | Active sources by type |
|
||||
|
||||
### 7.2 Structured Logs
|
||||
|
||||
All source operations emit structured logs including:
|
||||
- `tenantId`, `sourceId`, `sourceType`
|
||||
- `runId` (when applicable)
|
||||
- `correlationId` for cross-service tracing
|
||||
|
||||
## 8. Module Ownership Matrix
|
||||
|
||||
| Component | Owner Module | Interface |
|
||||
|-----------|--------------|-----------|
|
||||
| Source entity persistence | SbomService | PostgreSQL + Repository |
|
||||
| Credential storage | Authority | AuthRef vault API |
|
||||
| Webhook signature verification | SbomService | HMAC validation |
|
||||
| Scheduled trigger dispatch | Scheduler | Cron job registration |
|
||||
| Scan job execution | Scanner | Job queue interface |
|
||||
| UI configuration | Web | Sources Manager feature |
|
||||
|
||||
## 9. UI Information Architecture
|
||||
|
||||
### 9.1 Navigation Placement
|
||||
|
||||
| Section | Route | Purpose |
|
||||
|---------|-------|---------|
|
||||
| Sources List | `/sources` | Primary view with filtering, status overview, bulk actions |
|
||||
| Source Detail | `/sources/{id}` | Configuration view, run history, health metrics |
|
||||
| Add Source Wizard | `/sources/new` | Multi-step creation flow |
|
||||
| Edit Source | `/sources/{id}/edit` | Modify existing source configuration |
|
||||
|
||||
### 9.2 Wizard Flow Architecture
|
||||
|
||||
The Add/Edit Source Wizard follows a 6-step progressive disclosure pattern:
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ Step 1: Source Type Selection │
|
||||
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
|
||||
│ │ Zastava │ │ Docker │ │ CLI │ │ Git │ │
|
||||
│ │ (Webhook) │ │ (Scanner) │ │ (Receiver) │ │ (Repository)│ │
|
||||
│ └─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ Step 2: Basic Information │
|
||||
│ • Name (required, unique per tenant) │
|
||||
│ • Description (optional) │
|
||||
│ • Tags (multi-select, for filtering) │
|
||||
│ • Metadata key-value pairs (optional) │
|
||||
└─────────────────────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ Step 3: Type-Specific Configuration (varies by source type) │
|
||||
│ │
|
||||
│ Zastava: Docker: │
|
||||
│ • Registry URL • Registry URL │
|
||||
│ • Registry type dropdown • Image references (multi-add) │
|
||||
│ • Repository filters • Tag patterns (include/exclude) │
|
||||
│ • Tag patterns • Platform selection │
|
||||
│ • Webhook path (generated) • Scan options (analyzers, reachability) │
|
||||
│ │
|
||||
│ CLI: Git: │
|
||||
│ • Allowed tools list • Provider (GitHub/GitLab/Gitea/etc.) │
|
||||
│ • Allowed CI systems • Repository URL │
|
||||
│ • Validation rules • Branch patterns (include/exclude) │
|
||||
│ • Attribution requirements • Trigger modes (push/PR/tag/scheduled) │
|
||||
│ • Max SBOM size limit • Scan paths and exclusions │
|
||||
└─────────────────────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ Step 4: Credentials (AuthRef Pattern) │
|
||||
│ • Credential type selector (per source type) │
|
||||
│ • Create new credential vs. select existing │
|
||||
│ • Inline credential entry (stored via Authority vault) │
|
||||
│ • Webhook secret generation (for Zastava/Git) │
|
||||
│ • Test credential validity button │
|
||||
└─────────────────────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ Step 5: Schedule (Optional - for Docker/Git scheduled triggers) │
|
||||
│ • Enable scheduled scans toggle │
|
||||
│ • Cron expression builder or preset selector │
|
||||
│ • Timezone picker │
|
||||
│ • Rate limit (max scans per hour) │
|
||||
│ • Next run preview │
|
||||
└─────────────────────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ Step 6: Review & Test Connection │
|
||||
│ • Configuration summary (read-only) │
|
||||
│ • Test Connection button with status indicator │
|
||||
│ • Error details expansion if test fails │
|
||||
│ • Create Source / Save Changes button │
|
||||
│ • Webhook endpoint display (for Zastava/Git - copy to clipboard) │
|
||||
└─────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### 9.3 Wizard Step Validation
|
||||
|
||||
| Step | Required Fields | Validation |
|
||||
|------|-----------------|------------|
|
||||
| 1 - Type | `sourceType` | Must select one type |
|
||||
| 2 - Basic Info | `name` | Unique name, 3-100 chars |
|
||||
| 3 - Config | Varies by type | Type-specific required fields |
|
||||
| 4 - Credentials | `authRef` (if required) | Valid credential for source type |
|
||||
| 5 - Schedule | None (optional step) | Valid cron if enabled |
|
||||
| 6 - Review | None | Connection test recommended |
|
||||
|
||||
### 9.4 Source List Page Components
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ SBOM Sources [+ Add Source] │
|
||||
├─────────────────────────────────────────────────────────────────────────────┤
|
||||
│ Status Cards: │
|
||||
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
|
||||
│ │ Active │ │ Paused │ │ Error │ │ Pending │ │
|
||||
│ │ 12 │ │ 3 │ │ 2 │ │ 1 │ │
|
||||
│ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │
|
||||
├─────────────────────────────────────────────────────────────────────────────┤
|
||||
│ Filters: [Type ▼] [Status ▼] [Tags ▼] [Search: ________] │
|
||||
├─────────────────────────────────────────────────────────────────────────────┤
|
||||
│ □ │ Name │ Type │ Status │ Last Run │ Next Run │ Actions │
|
||||
│ ──┼───────────────┼─────────┼──────────┼────────────┼──────────┼─────────│
|
||||
│ □ │ DockerHub Prd │ Zastava │ ● Active │ 5 min ago │ - │ ⋮ │
|
||||
│ □ │ Harbor Dev │ Zastava │ ● Active │ 12 min ago │ - │ ⋮ │
|
||||
│ □ │ Nightly Scans │ Docker │ ● Active │ 2h ago │ 02:00 AM │ ⋮ │
|
||||
│ □ │ CI Pipeline │ CLI │ ⏸ Paused │ 1 day ago │ - │ ⋮ │
|
||||
│ □ │ Monorepo │ Git │ ⚠ Error │ Failed │ - │ ⋮ │
|
||||
├─────────────────────────────────────────────────────────────────────────────┤
|
||||
│ Showing 1-5 of 18 [< Prev] [1] [2] [3] [4] [Next >] │
|
||||
└─────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### 9.5 Row Actions Menu
|
||||
|
||||
| Action | Icon | Availability |
|
||||
|--------|------|--------------|
|
||||
| View Details | 👁 | Always |
|
||||
| Edit | ✏️ | Always |
|
||||
| Trigger Scan | ▶ | Active sources |
|
||||
| Test Connection | 🔌 | Always |
|
||||
| Pause | ⏸ | Active sources |
|
||||
| Resume | ▶ | Paused sources |
|
||||
| Delete | 🗑 | Always (with confirmation) |
|
||||
|
||||
### 9.6 Source Detail Page Layout
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ ← Back to Sources │
|
||||
├─────────────────────────────────────────────────────────────────────────────┤
|
||||
│ Docker Hub Production [Trigger] [Test] [Edit] [⋮] │
|
||||
│ Type: Zastava │ Status: ● Active │ Created: Dec 15, 2025 │
|
||||
├─────────────────────────────────────────────────────────────────────────────┤
|
||||
│ [Overview] [Configuration] [Run History] [Metrics] │
|
||||
├─────────────────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ Configuration: │
|
||||
│ ┌───────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Registry URL: https://registry-1.docker.io │ │
|
||||
│ │ Registry Type: Docker Hub │ │
|
||||
│ │ Webhook Path: /api/v1/webhooks/zastava/src-abc123 │ │
|
||||
│ │ Repository Filter: myorg/*, prod-* │ │
|
||||
│ │ Tag Filter: v*, latest (excluding: *-dev, *-test) │ │
|
||||
│ │ Analyzers: OS, Node.js, Python, Go │ │
|
||||
│ │ Reachability: Enabled │ │
|
||||
│ │ VEX Lookup: Enabled │ │
|
||||
│ └───────────────────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ Run History: │
|
||||
│ ┌───────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Run ID │ Trigger │ Status │ Started │ Duration │ Items │ │
|
||||
│ │ run-456 │ Webhook │ ✓ Success │ 5 min ago │ 45s │ 3/3 │ │
|
||||
│ │ run-455 │ Webhook │ ✓ Success │ 12 min ago │ 38s │ 1/1 │ │
|
||||
│ │ run-454 │ Manual │ ✓ Success │ 1h ago │ 2m 15s │ 12/12 │ │
|
||||
│ │ run-453 │ Webhook │ ⚠ Partial │ 2h ago │ 1m 30s │ 4/5 │ │
|
||||
│ │ run-452 │ Schedule │ ✗ Failed │ 3h ago │ 12s │ 0/0 │ │
|
||||
│ └───────────────────────────────────────────────────────────────────────┘ │
|
||||
│ [Load More] │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## 10. Security Considerations
|
||||
|
||||
| Concern | Mitigation |
|
||||
|---------|------------|
|
||||
| Credential exposure | AuthRef pattern - never inline, always reference vault |
|
||||
| Webhook forgery | HMAC signature verification for all webhooks |
|
||||
| Unauthorized access | Scope-based authorization (`sources:read`, `sources:write`, `sources:admin`) |
|
||||
| Secret logging | Audit logging excludes credential values |
|
||||
| Webhook secret rotation | Rotate-on-demand API endpoint |
|
||||
|
||||
## 10. Configuration
|
||||
|
||||
Environment variables (prefix `SBOM_SbomService__Sources__`):
|
||||
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `MaxSourcesPerTenant` | 100 | Maximum sources per tenant |
|
||||
| `DefaultMaxScansPerHour` | 60 | Default rate limit per source |
|
||||
| `RunHistoryRetentionDays` | 90 | Run history retention period |
|
||||
| `WebhookSignatureAlgorithm` | `HMAC-SHA256` | Webhook signature algorithm |
|
||||
| `ConnectionTestTimeoutSeconds` | 30 | Connection test timeout |
|
||||
|
||||
## 11. Related Documentation
|
||||
|
||||
- [SbomService Architecture](../architecture.md)
|
||||
- [Lineage Ledger](../lineage-ledger.md)
|
||||
- [Authority Architecture](../../authority/architecture.md)
|
||||
- [Scanner Architecture](../../scanner/architecture.md)
|
||||
- [UI Architecture](../../ui/architecture.md)
|
||||
@@ -213,3 +213,94 @@ The Signals module maintains strict determinism:
|
||||
- Backport Detection: `docs/modules/concelier/backport-detection.md`
|
||||
- EPSS Enrichment: `docs/modules/scanner/epss-enrichment.md`
|
||||
- Trust Vector: `docs/modules/excititor/trust-vector.md`
|
||||
|
||||
---
|
||||
|
||||
## SCM/CI Integration (Webhooks)
|
||||
|
||||
The Signals module also handles webhook ingestion from SCM (Source Code Management) and CI (Continuous Integration) providers. This enables:
|
||||
- Triggering scans on push/PR/release events
|
||||
- SBOM uploads from CI pipelines
|
||||
- Image push detection and automated scanning
|
||||
|
||||
### Location
|
||||
|
||||
```
|
||||
src/Signals/StellaOps.Signals/Scm/
|
||||
├── Models/
|
||||
│ ├── NormalizedScmEvent.cs # Provider-agnostic event payload
|
||||
│ ├── ScmEventType.cs # Event type enumeration
|
||||
│ └── ScmProvider.cs # Provider enumeration
|
||||
├── Webhooks/
|
||||
│ ├── IWebhookSignatureValidator.cs
|
||||
│ ├── GitHubWebhookValidator.cs # HMAC-SHA256 validation
|
||||
│ ├── GitLabWebhookValidator.cs # Token-based validation
|
||||
│ ├── GiteaWebhookValidator.cs # HMAC-SHA256 validation
|
||||
│ ├── IScmEventMapper.cs
|
||||
│ ├── GitHubEventMapper.cs # GitHub -> NormalizedScmEvent
|
||||
│ ├── GitLabEventMapper.cs # GitLab -> NormalizedScmEvent
|
||||
│ └── GiteaEventMapper.cs # Gitea -> NormalizedScmEvent
|
||||
├── Services/
|
||||
│ ├── IScmWebhookService.cs
|
||||
│ ├── ScmWebhookService.cs # Orchestrates validation + mapping
|
||||
│ ├── IScmTriggerService.cs
|
||||
│ └── ScmTriggerService.cs # Routes events to Scanner/Orchestrator
|
||||
└── ScmWebhookEndpoints.cs # Minimal API webhook endpoints
|
||||
```
|
||||
|
||||
### Supported Providers
|
||||
|
||||
| Provider | Webhook Endpoint | Signature Header | Validation |
|
||||
|----------|------------------|------------------|------------|
|
||||
| GitHub | `/webhooks/github` | `X-Hub-Signature-256` | HMAC-SHA256 |
|
||||
| GitLab | `/webhooks/gitlab` | `X-Gitlab-Token` | Token match |
|
||||
| Gitea | `/webhooks/gitea` | `X-Gitea-Signature` | HMAC-SHA256 |
|
||||
|
||||
### Event Types
|
||||
|
||||
| Event Type | Description | Triggers |
|
||||
|------------|-------------|----------|
|
||||
| `Push` | Code push to branch | Scan (main/release branches) |
|
||||
| `PullRequestOpened` | PR opened | — |
|
||||
| `PullRequestMerged` | PR merged | Scan |
|
||||
| `ReleasePublished` | Release created | Scan |
|
||||
| `ImagePushed` | Container image pushed | Scan |
|
||||
| `PipelineSucceeded` | CI pipeline completed | Scan |
|
||||
| `SbomUploaded` | SBOM artifact uploaded | SBOM ingestion |
|
||||
|
||||
### Webhook Payload Normalization
|
||||
|
||||
All provider-specific payloads are normalized to `NormalizedScmEvent`:
|
||||
|
||||
```csharp
|
||||
public sealed record NormalizedScmEvent
|
||||
{
|
||||
public required string EventId { get; init; }
|
||||
public ScmProvider Provider { get; init; }
|
||||
public ScmEventType EventType { get; init; }
|
||||
public DateTimeOffset Timestamp { get; init; }
|
||||
public required ScmRepository Repository { get; init; }
|
||||
public ScmActor? Actor { get; init; }
|
||||
public string? Ref { get; init; }
|
||||
public string? CommitSha { get; init; }
|
||||
public ScmPullRequest? PullRequest { get; init; }
|
||||
public ScmRelease? Release { get; init; }
|
||||
public ScmPipeline? Pipeline { get; init; }
|
||||
public ScmArtifact? Artifact { get; init; }
|
||||
public string? TenantId { get; init; }
|
||||
public string? IntegrationId { get; init; }
|
||||
}
|
||||
```
|
||||
|
||||
### Trigger Routing
|
||||
|
||||
The `ScmTriggerService` determines which events should trigger:
|
||||
1. **Scans:** Push to main/release, PR merges, releases, image pushes, successful pipelines
|
||||
2. **SBOM uploads:** Explicit `SbomUploaded` events or artifact releases with SBOM content
|
||||
|
||||
### Security
|
||||
|
||||
- **Signature verification:** All webhooks validate signatures before processing
|
||||
- **AuthRef integration:** Webhook secrets are managed via AuthRef (not stored in code)
|
||||
- **Rate limiting:** Built-in rate limiting to prevent webhook floods
|
||||
- **Audit trail:** All webhook deliveries are logged with delivery ID and result
|
||||
|
||||
@@ -28,9 +28,9 @@
|
||||
* **State**: Angular **Signals** + `@ngrx/signals` store for cross‑page slices.
|
||||
* **Transport**: `fetch` + RxJS interop; **SSE** (EventSource) for progress streams.
|
||||
* **Build**: Angular CLI + Vite builder.
|
||||
* **Testing**: Jest + Testing Library, Playwright for e2e.
|
||||
* **Packaging**: Containerized NGINX (immutable assets, ETag + content hashing).
|
||||
* **Observability docs**: runbook + Grafana JSON stub in `operations/observability.md` and `operations/dashboards/console-ui-observability.json` (offline import).
|
||||
* **Testing**: Jest + Testing Library, Playwright for e2e.
|
||||
* **Packaging**: Containerized NGINX (immutable assets, ETag + content hashing).
|
||||
* **Observability docs**: runbook + Grafana JSON stub in `operations/observability.md` and `operations/dashboards/console-ui-observability.json` (offline import).
|
||||
|
||||
---
|
||||
|
||||
@@ -44,9 +44,9 @@
|
||||
├─ scans/ # scan list, detail, SBOM viewer, diff-by-layer, EntryTrace
|
||||
├─ runtime/ # Zastava posture, drift events, admission decisions
|
||||
├─ policy/ # rules editor (YAML/Rego), exemptions, previews
|
||||
├─ vex/ # VEX explorer (claims, consensus, conflicts)
|
||||
├─ triage/ # vulnerability triage (artifact-first), VEX decisions, audit bundles
|
||||
├─ concelier/ # source health, export cursors, rebuild/export triggers
|
||||
├─ vex/ # VEX explorer (claims, consensus, conflicts)
|
||||
├─ triage/ # vulnerability triage (artifact-first), VEX decisions, audit bundles
|
||||
├─ concelier/ # source health, export cursors, rebuild/export triggers
|
||||
├─ attest/ # attestation proofs, verification bundles, Rekor links
|
||||
├─ admin/ # tenants, roles, clients, quotas, licensing posture
|
||||
└─ plugins/ # route plug-ins (lazy remote modules, governed)
|
||||
@@ -107,24 +107,80 @@ Each feature folder builds as a **standalone route** (lazy loaded). All HTTP sha
|
||||
* **Proofs list**: last 7 days Rekor entries; filter by kind (sbom/report/vex).
|
||||
* **Verification**: paste UUID or upload bundle → verify; result with explanations (chain, Merkle path).
|
||||
|
||||
### 3.8 Admin
|
||||
|
||||
* **Tenants/Installations**: view/edit, isolation hints.
|
||||
* **Clients & roles**: Authority clients, role→scope mapping, rotation hints.
|
||||
* **Quotas**: per license plan, counters, throttle events.
|
||||
* **Licensing posture**: last PoE introspection snapshot (redacted), release window.
|
||||
* **Branding**: tenant logo, title, and theme tokens with preview/apply (fresh-auth).
|
||||
|
||||
### 3.9 Vulnerability triage (VEX-first)
|
||||
|
||||
* **Routes**: `/triage/artifacts`, `/triage/artifacts/:artifactId`, `/triage/audit-bundles`, `/triage/audit-bundles/new`.
|
||||
* **Workspace**: artifact-first split layout (finding cards on the left; explainability tabs on the right: Overview, Reachability, Policy, Attestations).
|
||||
* **VEX decisions**: evidence-first VEX modal with scope + validity + evidence links; bulk apply supported; uses `/v1/vex-decisions`.
|
||||
* **Audit bundles**: "Create immutable audit bundle" UX to build and download an evidence pack; uses `/v1/audit-bundles`.
|
||||
* **Schemas**: `docs/schemas/vex-decision.schema.json`, `docs/schemas/attestation-vuln-scan.schema.json`, `docs/schemas/audit-bundle-index.schema.json`.
|
||||
* **Reference**: `docs/product-advisories/archived/27-Nov-2025-superseded/28-Nov-2025 - Vulnerability Triage UX & VEX-First Decisioning.md`.
|
||||
|
||||
---
|
||||
### 3.8 Admin
|
||||
|
||||
* **Tenants/Installations**: view/edit, isolation hints.
|
||||
* **Clients & roles**: Authority clients, role→scope mapping, rotation hints.
|
||||
* **Quotas**: per license plan, counters, throttle events.
|
||||
* **Licensing posture**: last PoE introspection snapshot (redacted), release window.
|
||||
* **Branding**: tenant logo, title, and theme tokens with preview/apply (fresh-auth).
|
||||
|
||||
### 3.9 Vulnerability triage (VEX-first)
|
||||
|
||||
* **Routes**: `/triage/artifacts`, `/triage/artifacts/:artifactId`, `/triage/audit-bundles`, `/triage/audit-bundles/new`.
|
||||
* **Workspace**: artifact-first split layout (finding cards on the left; explainability tabs on the right: Overview, Reachability, Policy, Attestations).
|
||||
* **VEX decisions**: evidence-first VEX modal with scope + validity + evidence links; bulk apply supported; uses `/v1/vex-decisions`.
|
||||
* **Audit bundles**: "Create immutable audit bundle" UX to build and download an evidence pack; uses `/v1/audit-bundles`.
|
||||
* **Schemas**: `docs/schemas/vex-decision.schema.json`, `docs/schemas/attestation-vuln-scan.schema.json`, `docs/schemas/audit-bundle-index.schema.json`.
|
||||
* **Reference**: `docs/product-advisories/archived/27-Nov-2025-superseded/28-Nov-2025 - Vulnerability Triage UX & VEX-First Decisioning.md`.
|
||||
|
||||
### 3.10 Integration Hub (Sprint 011)
|
||||
|
||||
* **Routes**: `/integrations`, `/integrations/:id`, `/integrations/activity`.
|
||||
* **Navigation placement**: Under Ops for operators; advanced settings under Admin > Integrations.
|
||||
* **Integration types**: SCM (GitHub/GitLab/Gitea), CI (GitHub Actions/GitLab CI/Jenkins), Registry (Docker Hub/Harbor/ECR/ACR/GCR/GHCR), Hosts (Zastava observer), Feeds (Concelier/Excititor mirrors), Artifacts (SBOM/VEX uploads).
|
||||
* **List view**:
|
||||
- KPI strip: total integrations, active, degraded, failed.
|
||||
- Filters: type chips, status, provider, owner, search.
|
||||
- Table columns: name, provider, type, status badge, last sync, owner, actions.
|
||||
- CTA: "Add Integration" button.
|
||||
* **Detail view**:
|
||||
- Summary header: status badge, type, provider, last test timestamp.
|
||||
- Tabs: Overview, Health, Activity, Permissions, Secrets (AuthRef), Webhooks, Inventory.
|
||||
- Actions: Test Connection, Edit, Pause/Resume, Delete.
|
||||
* **Activity view**:
|
||||
- Chronological timeline of all integration events.
|
||||
- Filters: event type, integration, date range.
|
||||
- Event types: created, updated, deleted, test_success, test_failure, health_ok, health_degraded, health_failed, paused, resumed, credential_rotated, sync_started, sync_completed, sync_failed.
|
||||
- Stats: total events, success count, warning count, failure count.
|
||||
- Auto-refresh every 30 seconds.
|
||||
* **Role gating**: `integrations.read` for list/detail; `integrations.admin` for CRUD and test actions.
|
||||
* **API backend**: `src/Integrations/StellaOps.Integrations.WebService` providing CRUD, test, trigger, pause/resume endpoints.
|
||||
* **Credentials**: All secrets via AuthRef URIs only; no raw credentials stored in UI state.
|
||||
|
||||
### 3.11 Integration Wizard (Sprint 014)
|
||||
|
||||
* **Routes**: Wizard is modal-based, launched from Integration Hub via "Add Integration" CTA.
|
||||
* **Location**: `src/Web/StellaOps.Web/src/app/features/integrations/integration-wizard.component.ts`.
|
||||
* **Wizard steps**:
|
||||
1. **Provider selection**: Choose provider from type-specific lists (registry, SCM, CI, host).
|
||||
2. **Authentication**: Configure auth method with AuthRef-managed credentials.
|
||||
3. **Scope**: Define repository/branch/namespace filters.
|
||||
4. **Schedule**: Set sync schedule (manual, interval, cron).
|
||||
5. **Preflight checks**: Run connection tests with detailed failure states.
|
||||
6. **Review**: Summary and create confirmation.
|
||||
* **Provider profiles**:
|
||||
- **Registry**: Docker Hub, Harbor, ECR, ACR, GCR, GHCR with type-specific auth (basic, token, IAM).
|
||||
- **SCM**: GitHub, GitLab, Gitea with OAuth apps or PAT auth.
|
||||
- **CI**: GitHub Actions, GitLab CI, Gitea Actions with webhook configuration.
|
||||
- **Host**: Kubernetes, VM, Baremetal with agent install templates.
|
||||
* **Auth methods**: Token, OAuth, Service Account, API Key depending on provider.
|
||||
* **Copy-safe UX**:
|
||||
- Webhook URLs and secrets are copy-button enabled.
|
||||
- Secret fields use `type="password"` with reveal toggle.
|
||||
- Setup instructions are Markdown-formatted and copy-safe.
|
||||
* **Preflight checks**:
|
||||
- Network connectivity validation.
|
||||
- Credential verification.
|
||||
- Permission/scope sufficiency checks.
|
||||
- Provider-specific health probes.
|
||||
* **Host wizard additions** (Sprint 014 extension):
|
||||
- Kernel/privilege preflight checks for eBPF/ETW observers.
|
||||
- Helm and systemd install templates.
|
||||
- Agent download and registration flow.
|
||||
* **Models**: `integration.models.ts` defines `IntegrationDraft`, `IntegrationProvider`, `WizardStep`, `PreflightCheck`, `AuthMethod`, and provider constants.
|
||||
|
||||
---
|
||||
|
||||
## 4) Auth, sessions & RBAC
|
||||
|
||||
@@ -156,11 +212,13 @@ Each feature folder builds as a **standalone route** (lazy loaded). All HTTP sha
|
||||
* **SSE** helper (EventSource) with auto‑reconnect & backpressure.
|
||||
* **DPoP** injector & nonce handling.
|
||||
|
||||
* Typed API clients (DTOs in `core/api/models.ts`):
|
||||
|
||||
* `ScannerApi`, `PolicyApi`, `ExcititorApi`, `ConcelierApi`, `AttestorApi`, `AuthorityApi`.
|
||||
|
||||
**DTO examples (abbrev):**
|
||||
* Typed API clients (DTOs in `core/api/models.ts`):
|
||||
|
||||
* `ScannerApi`, `PolicyApi`, `ExcititorApi`, `ConcelierApi`, `AttestorApi`, `AuthorityApi`.
|
||||
|
||||
* **Offline-first UX**: Ops dashboards must display a "data as of" banner with staleness thresholds when serving cached snapshots.
|
||||
|
||||
**DTO examples (abbrev):**
|
||||
|
||||
```ts
|
||||
export type ImageDigest = `sha256:${string}`;
|
||||
@@ -238,16 +296,16 @@ export interface NotifyDelivery {
|
||||
* **A11y**: WCAG 2.2 AA; keyboard navigation, focus management, ARIA roles; color‑contrast tokens verified by unit tests.
|
||||
* **I18n**: Angular i18n + runtime translation loader (`/locales/{lang}.json`); dates/numbers localized via `Intl`.
|
||||
* **Languages**: English default; Bulgarian, German, Japanese as initial additions.
|
||||
* **Theming**: dark/light via CSS variables; persisted in `prefers-color-scheme` aware store.
|
||||
* **Branding**: tenant-scoped theme tokens and logo pulled from Authority `/console/branding` after login.
|
||||
* **Theming**: dark/light via CSS variables; persisted in `prefers-color-scheme` aware store.
|
||||
* **Branding**: tenant-scoped theme tokens and logo pulled from Authority `/console/branding` after login.
|
||||
|
||||
---
|
||||
|
||||
## 10) Performance budgets
|
||||
|
||||
* **SBOM Graph overlays**: maintain >= 45 FPS pan/zoom/hover up to ~2,500 nodes / 10,000 edges (baseline laptop); degrade via LOD + sampling above this.
|
||||
* **Reachability halo limits**: cap visible halos to <= 2,000 at once; beyond this, aggregate (counts/heat) and require zoom-in or filtering to expand.
|
||||
|
||||
## 10) Performance budgets
|
||||
|
||||
* **SBOM Graph overlays**: maintain >= 45 FPS pan/zoom/hover up to ~2,500 nodes / 10,000 edges (baseline laptop); degrade via LOD + sampling above this.
|
||||
* **Reachability halo limits**: cap visible halos to <= 2,000 at once; beyond this, aggregate (counts/heat) and require zoom-in or filtering to expand.
|
||||
|
||||
* **TTI** ≤ 1.5 s on 4G/slow CPU (first visit), ≤ 0.6 s repeat (HTTP/2, cached).
|
||||
* **JS** initial < 300 KB gz (lazy routes).
|
||||
* **SBOM list**: render 10k rows in < 70 ms with virtualization; filter in < 150 ms.
|
||||
|
||||
Reference in New Issue
Block a user