Files
git.stella-ops.org/docs/qa/DEEP_ADVISORY_MIRROR_AUDIT_20260316.md
master 534aabfa2a First-time user experience fixes and platform contract repairs
FTUX fixes (Sprint 316-001):
- Remove all hardcoded fake data from dashboard — fresh installs show
  honest setup guide instead of fake crisis data (5 fake criticals gone)
- Curate advisory source defaults: 32 sources disabled by default
  (ecosystem, geo-restricted, exploit, hardware, mirror). ~43 core
  sources remain enabled. StellaOps Mirror no longer enabled at priority 1.
- Filter Mirror-category sources from Create Domain wizard to prevent
  circular mirror-from-mirror chains
- Add 404 catch-all route — unknown URLs show "Page Not Found" instead
  of silently rendering the dashboard
- Fix arrow characters in release target path dropdown (? → →)
- Add login credentials to quickstart documentation
- Update Feature Matrix: 14 release orchestration features marked as
  shipped (was marked planned)

Platform contract repairs (from prior session):
- Add /api/v1/jobengine/quotas/summary endpoint on Platform
- Fix gateway route prefix matching for /policy/shadow/* and
  /policy/simulations/* (regex routes instead of exact match)
- Fix VexHub PostgresVexSourceRepository missing interface method
- Fix advisory-vex-sources sweep text expectation
- Fix mirror operator journey auth (session storage token extraction)

Verified: 110/111 canonical routes passing (1 unrelated stale approval ref)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-16 02:05:38 +02:00

9.8 KiB

Deep Advisory & Mirror Audit — Stella Ops

Date: 2026-03-16 Method: Interactive Playwright walkthrough + source code analysis Focus: Advisory source catalog, mirror architecture, mode confusion, first-time operator experience


The Core Problem: Mirror Source vs Mirror Mode Are Disconnected

Stella Ops has three advisory operating modes:

  • Direct: Fetch advisories from upstream sources (NVD, OSV, GHSA, etc.) directly
  • Mirror: Consume pre-aggregated advisory bundles from an upstream Stella Ops mirror
  • Hybrid: Both direct and mirror sources active

The mirror feature has two roles:

  • Producer: Create mirror domains that bundle advisories for downstream consumers
  • Consumer: Connect to an upstream mirror and pull bundles

The problem: The advisory source catalog and the mirror mode are independent systems that don't coordinate:

Issue 1: StellaOps Mirror Source Enabled by Default in Direct Mode

File: src/Concelier/__Libraries/StellaOps.Concelier.Core/Sources/SourceDefinitions.cs:1351-1366

The StellaMirror source definition has:

BaseEndpoint = "https://mirror.stella-ops.org/api/v1",
DefaultPriority = 1,  // HIGHEST priority
EnabledByDefault = true  // inherited from base class

On a fresh self-hosted install:

  • Mode badge shows "Direct"
  • But "StellaOps Mirror" source is enabled at priority 1 (highest)
  • Its endpoint mirror.stella-ops.org is an external SaaS URL that doesn't exist for self-hosted
  • Health check fails: "SSL/TLS error connecting to StellaOps Mirror"

Impact: The highest-priority advisory source is a broken external URL. If it were reachable, it would override all local direct sources. The system silently operates with a failed priority-1 source.

Fix: Either:

  • Set EnabledByDefault = false for the StellaMirror source
  • Or: gate it behind mirror mode — only enable when mode is "Mirror" or "Hybrid"
  • Or: replace the hardcoded URL with a placeholder that requires explicit configuration

Issue 2: Mirror Source Appears in Create Mirror Domain Wizard

When creating a new mirror domain (producer role), the source selection wizard lists ALL sources including "StellaOps Mirror" under the Mirror (1) category. An operator can select it and create a mirror domain that sources from... the broken external mirror.

Impact: Creates a circular or broken dependency chain. A mirror domain that sources from a nonexistent upstream.

Fix: Exclude sources of type StellaMirror from the Create Mirror Domain source picker. A mirror domain should only aggregate direct-fetch sources, not other mirrors.

Issue 3: No Relationship Between Catalog Toggle and Mirror Consumer Setup

Two separate ways to "use a mirror" exist:

  1. Catalog toggle: Enable "StellaOps Mirror" source in the advisory catalog → hardcoded to mirror.stella-ops.org
  2. Mirror Consumer Wizard: "Connect to Mirror" → 4-step wizard where you enter a custom mirror URL

These are independent. Enabling the catalog mirror source does NOT invoke the consumer wizard. Completing the consumer wizard does NOT update the catalog mirror source's endpoint.

Impact: A user who completes the consumer wizard and enters https://my-corp-mirror.internal still has the catalog's mirror source pointing at mirror.stella-ops.org. Conversely, a user who toggles the catalog source thinks they've configured mirror mode, but the system mode stays "Direct".

Fix: Unify these:

  • When the mirror consumer wizard is completed, it should: (a) update the StellaMirror source endpoint to the configured URL, (b) enable the source, (c) switch mode to "Mirror" or "Hybrid"
  • The catalog's StellaMirror source toggle should redirect to the consumer wizard if no mirror is configured
  • Or: remove the StellaMirror source from the catalog entirely and make it purely managed through the mirror dashboard

Issue 4: "Direct" Mode Label Is Misleading When Mirror Source Is Enabled

The mirror context header shows "Direct" (green badge) but a mirror source is enabled at highest priority. The mode badge doesn't reflect actual source configuration.

Fix: Derive the mode from actual source state:

  • If only direct sources are enabled → "Direct"
  • If only mirror source is enabled → "Mirror"
  • If both → "Hybrid" Or at minimum, show a warning: "Mirror source is enabled but unreachable"

Advisory Source Health: Fresh Install Reality

After "Check All" on a fresh install:

  • 55 healthy (sources reachable from Docker network)
  • 18 failed (unreachable from Docker network or nonexistent)

Failed sources by category:

Source Likely Reason
Oracle Security DNS/network from Docker
npm/PyPI/RubyGems/Maven/Packagist/Hex.pm Advisories (6) OSV-based, likely endpoint differences
CERT-In (India) Geo-restricted or slow
FSTEC BDU (Russia) Geo-restricted
CSAF Aggregator Endpoint format
VEX Hub Not a real endpoint yet
MITRE D3FEND Endpoint format
Exploit-DB Likely rate-limited
Docker Official CVEs Endpoint format
AMD Security DNS/network
Siemens ProductCERT Endpoint format
Ruby Advisory DB Endpoint format
StellaOps Mirror Nonexistent external URL

Finding: 74 out of 75 sources are enabled by default (EnabledByDefault = true). On a fresh install, 18 immediately fail health checks. The stats bar shows "55 enabled, 55 healthy, 19 failed" — but this is confusing because "55 enabled" should be 74 (or 75). The enabled count in the stats bar and the actual enabled count don't match.

Fix:

  • Don't enable all 75 sources by default. Enable a curated set (Primary 4 + major vendors + major distributions = ~20-25 sources)
  • Disable ecosystem-specific sources (npm, PyPI, etc.) by default — let users enable for their stack
  • Disable geo-restricted sources (FSTEC, NKCKI) by default
  • Show a "Recommended for your platform" selection during first-time setup

Mirror Domain Builder: Good UX, Missing Guardrails

The 3-step domain builder wizard is well-designed:

  1. Select Sources — categorized picker with quick-select buttons, search, selection summary
  2. Configure Domain — auto-generated ID/name, 5 export formats (JSON/JSONL/OpenVEX/CSAF/CycloneDX), rate limits, auth, signing
  3. Review & Create — summary + JSON filter preview + "generate immediately" option

Good:

  • Auto-generated domain ID from selection (mirror-primary-4src)
  • Multiple export formats including OpenVEX and CSAF
  • Rate limiting per domain
  • Optional signing and authentication
  • "Generate immediately" checkbox

Issues:

  • Can select "StellaOps Mirror" as a source (circular, see Issue 2)
  • No indication of which sources are actually healthy — I might build a domain from 4 sources where 2 are failing
  • No estimate of bundle size or generation time

Mirror Consumer Wizard: Clean but Disconnected

The 4-step consumer wizard:

  1. Connect — Enter mirror base URL + test connection
  2. Signature — Verify signing key
  3. Sync & Mode — Schedule + mode selection
  4. Review & Activate — Confirm

Good:

  • Proper signature verification step
  • "Test Connection" button
  • Mode selection in the workflow

Issues:

  • Completing this wizard doesn't update the catalog's StellaMirror source (see Issue 3)
  • Placeholder URL https://mirror.stella-ops.org is the same broken SaaS URL as the catalog source
  • No way to test with the local instance's own mirror domains (self-mirroring for verification)

Short-term (S/M effort):

  1. Set EnabledByDefault = false for StellaMirror source — prevents broken priority-1 source on fresh install
  2. Exclude StellaMirror from Create Domain wizard — prevents circular mirror chains
  3. Curate default-enabled sources — only enable Primary + top 15 vendor/distribution sources by default
  4. Fix stats bar count — "enabled" count should match actual enabled sources
  5. Show health status in Create Domain source picker — badge healthy/failed next to each source

Medium-term (L effort):

  1. Unify catalog mirror source and consumer wizard — catalog toggle should invoke wizard, wizard should update catalog
  2. Derive mode from source state — "Direct"/"Mirror"/"Hybrid" computed from actual enabled sources
  3. Add "Recommended sources" first-run flow — during setup, suggest sources based on detected platform (Docker → Container sources, Python → PyPI, etc.)

Long-term (XL effort):

  1. Mirror domain ↔ consumer federation — Instance A's domain auto-discoverable by Instance B's consumer wizard via DNS SRV or well-known URL
  2. Self-test mirror consumer — "Connect to localhost" option that validates the producer flow by consuming from own mirror domains

Files to Modify

Fix File Change
#1 src/Concelier/__Libraries/StellaOps.Concelier.Core/Sources/SourceDefinitions.cs:1351 Add EnabledByDefault = false to StellaMirror
#2 src/Web/StellaOps.Web/src/app/features/integrations/advisory-vex-sources/mirror-domain-builder.component.ts Filter out sources where category === 'Mirror' from the picker
#3 src/Concelier/__Libraries/StellaOps.Concelier.Core/Sources/SourceDefinitions.cs Set EnabledByDefault = false on ecosystem/geo-restricted sources
#4 src/Web/StellaOps.Web/src/app/features/integrations/advisory-vex-sources/advisory-source-catalog.component.ts Fix enabled count computation
#5 src/Web/StellaOps.Web/src/app/features/integrations/advisory-vex-sources/mirror-domain-builder.component.ts Show health badge next to source names
#6 Multiple files Wire consumer wizard completion to update catalog source + mode
#7 src/Concelier/StellaOps.Concelier.WebService/Extensions/MirrorDomainManagementEndpointExtensions.cs Compute mode from source state