Fix build and code structure improvements. New but essential UI functionality. CI improvements. Documentation improvements. AI module improvements.

2025-12-26 21:54:17 +02:00
parent 335ff7da16
commit c2b9cd8d1f
3717 changed files with 264714 additions and 48202 deletions
--- a/docs/replay/replay-manifest-guide.md
+++ b/docs/replay/replay-manifest-guide.md
@@ -0,0 +1,455 @@
+# Replay Manifest Guide
+
+> **Sprint:** SPRINT_20251228_001_BE_replay_manifest_ci (T6)
+> **Purpose:** Complete reference for Replay Manifest export, verification, and CI integration.
+
+## Overview
+
+The Replay Manifest is a self-contained JSON document that captures everything needed to reproduce a scan: inputs, toolchain versions, policies, and expected outputs. When verified, it provides cryptographic proof that a scan is deterministic and reproducible.
+
+## Quick Start
+
+```bash
+# Export replay manifest after scanning
+stella replay export --scan-id <scan-uuid> --output replay.json
+
+# Or export for a specific image
+stella replay export --image myregistry/app:v1.0.0 --output replay.json
+
+# Verify determinism (strict mode)
+stella replay verify --manifest replay.json --strict-mode
+
+# Verify with drift failure (for CI)
+stella replay verify --manifest replay.json --fail-on-drift
+```
+
+---
+
+## Schema Reference
+
+### Schema Version
+
+Current version: `1.0.0`
+
+Schema location: `src/__Libraries/StellaOps.Replay.Core/Schemas/replay-export.schema.json`
+
+### Top-Level Structure
+
+```json
+{
+  "version": "1.0.0",
+  "snapshot": { ... },
+  "toolchain": { ... },
+  "inputs": { ... },
+  "outputs": { ... },
+  "verification": { ... }
+}
+```
+
+### `snapshot` Object
+
+Identifies the scan snapshot this manifest represents.
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `id` | string | Unique snapshot ID (`snapshot:<sha256>`) |
+| `createdAt` | ISO 8601 | UTC timestamp when scan completed |
+| `artifact` | object | Reference to scanned artifact (digest, repository, tag) |
+
+Example:
+```json
+{
+  "id": "snapshot:a1b2c3d4e5f6...",
+  "createdAt": "2025-12-28T14:30:00Z",
+  "artifact": {
+    "digest": "sha256:abc123...",
+    "repository": "myregistry/app",
+    "tag": "v1.0.0"
+  }
+}
+```
+
+### `toolchain` Object
+
+Captures exact versions of all tools used during the scan.
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `scannerVersion` | string | StellaOps Scanner version |
+| `policyEngineVersion` | string | Policy Engine version |
+| `platform` | string | Platform identifier (e.g., `linux-x64`) |
+| `sbomerVersion` | string | SBOM generator version |
+| `vexerVersion` | string | VEX processor version |
+
+Example:
+```json
+{
+  "scannerVersion": "0.42.0",
+  "policyEngineVersion": "0.42.0",
+  "platform": "linux-x64",
+  "sbomerVersion": "0.42.0",
+  "vexerVersion": "0.42.0"
+}
+```
+
+### `inputs` Object
+
+All inputs consumed during the scan, with content hashes.
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `sboms` | array | SBOM inputs (if layered) |
+| `vex` | array | VEX documents used |
+| `feeds` | array | Vulnerability feed snapshots |
+| `policies` | object | Policy bundle reference |
+
+Feed snapshot example:
+```json
+{
+  "feeds": [
+    {
+      "name": "nvd",
+      "snapshotId": "nvd:2025-12-28T00:00:00Z",
+      "digest": "sha256:def456...",
+      "recordCount": 245678
+    }
+  ]
+}
+```
+
+### `outputs` Object
+
+Expected outputs from the scan, used for verification.
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `verdictDigest` | string | SHA256 of verdict JSON |
+| `decision` | enum | `allow`, `deny`, or `review` |
+| `sbomDigest` | string | SHA256 of generated SBOM |
+| `findingsDigest` | string | SHA256 of findings JSON |
+
+### `verification` Object
+
+Helper commands and expected hashes for verification.
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `command` | string | CLI command to reproduce scan |
+| `expectedSbomHash` | string | Expected SBOM content hash |
+| `expectedVerdictHash` | string | Expected verdict content hash |
+
+---
+
+## CLI Commands
+
+### `stella replay export`
+
+Export a replay manifest from a completed scan.
+
+```bash
+stella replay export [OPTIONS]
+```
+
+| Option | Required | Description |
+|--------|----------|-------------|
+| `--scan-id <uuid>` | One of | Scan ID to export |
+| `--image <ref>` | One of | Image reference (uses latest scan) |
+| `--output <path>` | No | Output path (default: `replay.json`) |
+| `--include-feed-snapshots` | No | Include full feed snapshot refs |
+| `--no-verification-script` | No | Skip verification command generation |
+
+### `stella replay verify`
+
+Verify a replay manifest by re-executing the scan and comparing outputs.
+
+```bash
+stella replay verify [OPTIONS]
+```
+
+| Option | Required | Description |
+|--------|----------|-------------|
+| `--manifest <path>` | Yes | Path to replay manifest |
+| `--strict-mode` | No | Require bit-for-bit identical outputs |
+| `--fail-on-drift` | No | Exit code 1 on any drift |
+| `--output-diff <path>` | No | Write diff report to file |
+
+### Exit Codes
+
+| Code | Meaning |
+|------|---------|
+| `0` | Verification passed, outputs match |
+| `1` | Drift detected, outputs differ |
+| `2` | Verification error (missing inputs, invalid manifest, etc.) |
+
+---
+
+## CI Integration
+
+### Gitea Actions
+
+```yaml
+name: SBOM Replay Verification
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+jobs:
+  verify-determinism:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Build image
+        run: docker build -t ${{ github.repository }}:${{ github.sha }} .
+
+      - name: Scan with replay export
+        run: |
+          stellaops scan \
+            --image ${{ github.repository }}:${{ github.sha }} \
+            --output-sbom sbom.json \
+            --output-replay replay.json
+
+      - name: Verify determinism
+        run: |
+          stellaops replay verify \
+            --manifest replay.json \
+            --fail-on-drift \
+            --strict-mode
+
+      - name: Upload replay manifest
+        uses: actions/upload-artifact@v4
+        with:
+          name: replay-manifest
+          path: replay.json
+          retention-days: 90
+```
+
+### GitHub Actions
+
+```yaml
+name: SBOM Replay Verification
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+jobs:
+  verify-determinism:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up StellaOps
+        uses: stellaops/setup-stella@v1
+        with:
+          version: '0.42.0'
+
+      - name: Build and scan
+        run: |
+          docker build -t myapp:${{ github.sha }} .
+          stella scan --image myapp:${{ github.sha }} \
+            --output-sbom sbom.json \
+            --output-replay replay.json
+
+      - name: Verify replay
+        run: stella replay verify --manifest replay.json --fail-on-drift
+
+      - name: Upload attestations
+        uses: actions/upload-artifact@v4
+        with:
+          name: sbom-attestations
+          path: |
+            sbom.json
+            replay.json
+```
+
+### GitLab CI
+
+```yaml
+sbom-replay:
+  stage: security
+  image: stellaops/cli:latest
+  script:
+    - docker build -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA .
+    - stella scan --image $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA --output-replay replay.json
+    - stella replay verify --manifest replay.json --fail-on-drift
+  artifacts:
+    paths:
+      - replay.json
+    expire_in: 90 days
+  rules:
+    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
+    - if: $CI_COMMIT_BRANCH == "main"
+```
+
+---
+
+## Troubleshooting Drift Detection
+
+### Common Drift Causes
+
+| Cause | Symptom | Fix |
+|-------|---------|-----|
+| Feed update | `findingsDigest` differs | Pin feed snapshot version |
+| Policy change | `verdictDigest` differs | Version policy bundles |
+| Tool upgrade | All digests differ | Lock toolchain versions |
+| Non-deterministic SBOM | `sbomDigest` differs | Enable deterministic mode |
+| Timezone issues | Timestamps drift | Ensure UTC everywhere |
+
+### Debugging Steps
+
+1. **Export diff report:**
+   ```bash
+   stella replay verify --manifest replay.json --output-diff drift-report.json
+   ```
+
+2. **Compare inputs:**
+   ```bash
+   stella replay diff --manifest-a old.json --manifest-b new.json --show-inputs
+   ```
+
+3. **Check feed versions:**
+   ```bash
+   stella feeds list --show-snapshots
+   ```
+
+4. **Verify toolchain:**
+   ```bash
+   stella version --all
+   ```
+
+### Feed Snapshot Pinning
+
+For reproducible CI, pin feed snapshots:
+
+```bash
+# List available snapshots
+stella feeds snapshots --feed nvd
+
+# Pin specific snapshot
+stella scan --image myapp:v1.0.0 \
+  --feed-snapshot nvd:2025-12-28T00:00:00Z \
+  --output-replay replay.json
+```
+
+---
+
+## Best Practices for Deterministic Builds
+
+### 1. Lock All Dependencies
+
+```yaml
+# In CI, always specify exact versions
+stellaops/cli:0.42.0  # Not :latest
+```
+
+### 2. Pin Feed Snapshots
+
+```bash
+# Export current snapshot ID
+stella feeds export-snapshot --output feeds-snapshot.json
+
+# Use in subsequent scans
+stella scan --feed-snapshot-file feeds-snapshot.json
+```
+
+### 3. Version Policy Bundles
+
+```bash
+# Store policies in version control
+git add policies/
+git commit -m "Policy bundle v2.3.0"
+
+# Reference by commit in manifest
+stella scan --policy-ref policies@abc123
+```
+
+### 4. Use Strict Mode in CI
+
+```bash
+# Always use strict mode in CI pipelines
+stella replay verify --manifest replay.json --strict-mode --fail-on-drift
+```
+
+### 5. Archive Replay Manifests
+
+Store replay manifests alongside release artifacts for audit trail:
+
+```bash
+# Archive with release
+cp replay.json releases/v1.0.0/replay.json
+```
+
+---
+
+## API Reference
+
+### `IReplayManifestExporter`
+
+```csharp
+public interface IReplayManifestExporter
+{
+    /// <summary>
+    /// Exports a replay manifest for a completed scan.
+    /// </summary>
+    Task<ReplayExportResult> ExportAsync(
+        string scanId,
+        ReplayExportOptions options,
+        CancellationToken ct = default);
+}
+```
+
+### `ReplayExportOptions`
+
+```csharp
+public sealed record ReplayExportOptions
+{
+    /// <summary>Include exact toolchain versions.</summary>
+    public bool IncludeToolchainVersions { get; init; } = true;
+
+    /// <summary>Include feed snapshot references.</summary>
+    public bool IncludeFeedSnapshots { get; init; } = true;
+
+    /// <summary>Generate verification shell command.</summary>
+    public bool GenerateVerificationScript { get; init; } = true;
+
+    /// <summary>Output file path.</summary>
+    public string OutputPath { get; init; } = "replay.json";
+}
+```
+
+### `ReplayExportResult`
+
+```csharp
+public sealed record ReplayExportResult
+{
+    /// <summary>Path to exported manifest.</summary>
+    public required string ManifestPath { get; init; }
+
+    /// <summary>SHA256 digest of manifest content.</summary>
+    public required string ManifestDigest { get; init; }
+
+    /// <summary>Path to verification script (if generated).</summary>
+    public string? VerificationScriptPath { get; init; }
+}
+```
+
+---
+
+## Related Documentation
+
+- [Deterministic Replay](DETERMINISTIC_REPLAY.md) - Core concepts and architecture
+- [Developer Guide: Replay](DEVS_GUIDE_REPLAY.md) - Implementation details
+- [Replay Manifest v2 Acceptance](replay-manifest-v2-acceptance.md) - Schema evolution
+- [Test Strategy](TEST_STRATEGY.md) - Replay testing approach
+
+---
+
+## Changelog
+
+| Version | Date | Changes |
+|---------|------|---------|
+| 1.0.0 | 2025-12-28 | Initial schema and CLI commands |