The 15-minute cron (0,15,30,45 * * * *) caused the fetch/parse/map
pipeline to fire 4x per hour, creating constant DB write pressure.
This overlapped with e2e test runs and caused advisory-source API
timeouts due to shared Postgres contention.
Changed to every 4 hours (0 */4 * * *) which is appropriate for
advisory data freshness — Red Hat advisories don't update every 15min.
Parse/map stages staggered at +10min and +20min offsets.
Manual sync via POST /advisory-sources/redhat/sync remains available
for on-demand refreshes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three-layer defense against Concelier overload during bulk advisory sync:
Layer 1 — Freshness query cache (30s TTL):
GET /advisory-sources, /advisory-sources/summary, and
/{id}/freshness now cache their results in IMemoryCache for 30s.
Eliminates the expensive 4-table LEFT JOIN with computed freshness
on every call during sync storms.
Layer 2 — Backpressure on sync endpoint (429 + Retry-After):
POST /{sourceId}/sync checks active job count via GetActiveRunsAsync().
When active runs >= MaxConcurrentJobs, returns 429 Too Many Requests
with Retry-After: 30 header. Clients get a clear signal to back off.
Layer 3 — Staged sync-all with inter-batch delay:
POST /sync now triggers sources in batches of MaxConcurrentJobs
(default: 6) with SyncBatchDelaySeconds (default: 5s) between batches.
21 sources → 4 batches over ~15s instead of 21 instant triggers.
Each batch triggers in parallel (Task.WhenAll), then delays.
New config: JobScheduler:SyncBatchDelaySeconds (default: 5)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The topology wizard creates environments and targets via POST /api/v1/environments
and POST /api/v1/targets. These were routed to JobEngine which doesn't have
the identity envelope middleware, causing 404 on ReverseProxy routes.
Fix: Add environment CRUD, target CRUD, and agent list endpoints directly
to Concelier's TopologySetupEndpointExtensions. These use the same
Topology.Read/Manage authorization policies that work with the identity
envelope middleware.
Routes updated:
- /api/v1/environments → Concelier (was JobEngine)
- /api/v1/agents → Concelier (new)
Topology wizard now completes steps 1-4:
1. Region: CREATE OK
2. Environment: CREATE OK
3. Stage Order: OK (skip)
4. Target: CREATE OK
5. Agent: BLOCKED (expected — no agents deployed on fresh install)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The identity envelope PostConfigure on JwtBearerOptions didn't work because
AddStellaOpsResourceServerAuthentication configures its own events that
override PostConfigure. The OnMessageReceived handler was only in the
TestSigningSecret branch, never in the OIDC discovery branch used in prod.
Fix: Add a middleware BEFORE UseAuthentication() that reads
X-StellaOps-Identity-Envelope headers, verifies HMAC-SHA256 signature
using Router:IdentityEnvelopeSigningKey (from router-microservice-defaults),
and sets HttpContext.User with claims from the envelope.
Also fixed: read signing key from Router:IdentityEnvelopeSigningKey config
path (matches the compose env var Router__IdentityEnvelopeSigningKey from
x-router-microservice-defaults).
Verified: Topology wizard "Create Region" now succeeds — Next button enables.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fix#20 — Audit log empty:
Wire app.MapAuditEndpoints() in JobEngine Program.cs. The endpoint file
existed but was never registered, so /api/v1/jobengine/audit returned 404
and the Timeline unified aggregation service got 0 events.
Fix#22 — Registry search returns mock data:
Replace the catchError() synthetic mock fallback in searchImages() with
an empty array return. The release wizard will now show "no results"
instead of fabricating fake "payment-service" with "sha256:payment..."
digests. getImageDigests() returns an empty-tags placeholder on failure.
Fix#13 — Topology wizard 401 (identity envelope passthrough):
Add TryAuthenticateFromIdentityEnvelope() to Concelier's JwtBearer
OnMessageReceived handler. When no JWT bearer token is present (stripped
by gateway's IdentityHeaderPolicyMiddleware on ReverseProxy routes),
the handler reads X-StellaOps-Identity-Envelope + signature headers,
verifies the HMAC-SHA256 signature using the shared signing key, and
populates ClaimsPrincipal with subject/tenant/scopes/roles from the
envelope. This enables ReverseProxy routes to Concelier topology
endpoints to authenticate the same way Microservice/Valkey routes do.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Concelier:
- Register Topology.Read, Topology.Manage, Topology.Admin authorization
policies mapped to OrchRead/OrchOperate/PlatformContextRead/IntegrationWrite
scopes. Previously these policies were referenced by endpoints but never
registered, causing System.InvalidOperationException on every topology
API call.
Gateway routes:
- Simplified targets/environments routes (removed specific sub-path routes,
use catch-all patterns instead)
- Changed environments base route to JobEngine (where CRUD lives)
- Changed to ReverseProxy type for all topology routes
KNOWN ISSUE (not yet fixed):
- ReverseProxy routes don't forward the gateway's identity envelope to
Concelier. The regions/targets/bindings endpoints return 401 because
hasPrincipal=False — the gateway authenticates the user but doesn't
pass the identity to the backend via ReverseProxy. Microservice routes
use Valkey transport which includes envelope headers. Topology endpoints
need either: (a) Valkey transport registration in Concelier, or
(b) Concelier configured to accept raw bearer tokens on ReverseProxy paths.
This is an architecture-level fix.
Journey findings collected so far:
- Integration wizard (Harbor + GitHub App): works end-to-end
- Advisory Check All: fixed (parallel individual checks)
- Mirror domain creation: works, generate-immediately fails silently
- Topology wizard Step 1 (Region): blocked by auth passthrough issue
- Topology wizard Step 2 (Environment): POST to JobEngine needs verify
- User ID resolution: raw hashes shown everywhere
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sprint: SPRINT_20251229_049_BE_csproj_audit_maint_tests
Tasks: AUDIT-0001 through AUDIT-0147 APPLY tasks (approved decisions 1-9)
Changes:
- Set TreatWarningsAsErrors=true for all production .NET projects
- Fixed nullable warnings in Scanner.EntryTrace, Scanner.Evidence,
Scheduler.Worker, Concelier connectors, and other modules
- Injected TimeProvider/IGuidProvider for deterministic time/ID generation
- Added path traversal validation in AirGap.Bundle
- Fixed NULL handling in various cursor classes
- Third-party GostCryptography retains TreatWarningsAsErrors=false (preserves original)
- Test projects excluded per user decision (rejected decision 10)
Note: All 17 ACSC connector tests pass after snapshot fixture sync
- Implement `SbomVexOrderingDeterminismProperties` for testing component list and vulnerability metadata hash consistency.
- Create `UnicodeNormalizationDeterminismProperties` to validate NFC normalization and Unicode string handling.
- Add project file for `StellaOps.Testing.Determinism.Properties` with necessary dependencies.
- Introduce CI/CD template validation tests including YAML syntax checks and documentation content verification.
- Create validation script for CI/CD templates ensuring all required files and structures are present.
- Implemented comprehensive tests for verdict artifact generation to ensure deterministic outputs across various scenarios, including identical inputs, parallel execution, and change ordering.
- Created helper methods for generating sample verdict inputs and computing canonical hashes.
- Added tests to validate the stability of canonical hashes, proof spine ordering, and summary statistics.
- Introduced a new PowerShell script to update SHA256 sums for files, ensuring accurate hash generation and file integrity checks.
- Implemented ReachabilityCenterComponent for displaying asset reachability status with summary and filtering options.
- Added ReachabilityWhyDrawerComponent to show detailed reachability evidence and call paths.
- Created unit tests for both components to ensure functionality and correctness.
- Updated accessibility test results for the new components.
- Added BerkeleyDbReader class to read and extract RPM header blobs from BerkeleyDB hash databases.
- Implemented methods to detect BerkeleyDB format and extract values, including handling of page sizes and magic numbers.
- Added tests for BerkeleyDbReader to ensure correct functionality and header extraction.
feat: Add Yarn PnP data tests
- Created YarnPnpDataTests to validate package resolution and data loading from Yarn PnP cache.
- Implemented tests for resolved keys, package presence, and loading from cache structure.
test: Add egg-info package fixtures for Python tests
- Created egg-info package fixtures for testing Python analyzers.
- Included PKG-INFO, entry_points.txt, and installed-files.txt for comprehensive coverage.
test: Enhance RPM database reader tests
- Added tests for RpmDatabaseReader to validate fallback to legacy packages when SQLite is missing.
- Implemented helper methods to create legacy package files and RPM headers for testing.
test: Implement dual signing tests
- Added DualSignTests to validate secondary signature addition when configured.
- Created stub implementations for crypto providers and key resolvers to facilitate testing.
chore: Update CI script for Playwright Chromium installation
- Modified ci-console-exports.sh to ensure deterministic Chromium binary installation for console exports tests.
- Added checks for Windows compatibility and environment variable setups for Playwright browsers.
- Introduced `NativeTestBase` class for ELF, PE, and Mach-O binary parsing helpers and assertions.
- Created `TestCryptoFactory` for SM2 cryptographic provider setup and key generation.
- Implemented `Sm2SigningTests` to validate signing functionality with environment gate checks.
- Developed console export service and store with comprehensive unit tests for export status management.
- Added ConsoleExportClient for managing export requests and responses.
- Introduced ConsoleExportRequest and ConsoleExportResponse models.
- Implemented methods for creating and retrieving exports with appropriate headers.
feat(crypto): Add Software SM2/SM3 Cryptography Provider
- Implemented SmSoftCryptoProvider for software-only SM2/SM3 cryptography.
- Added support for signing and verification using SM2 algorithm.
- Included hashing functionality with SM3 algorithm.
- Configured options for loading keys from files and environment gate checks.
test(crypto): Add unit tests for SmSoftCryptoProvider
- Created comprehensive tests for signing, verifying, and hashing functionalities.
- Ensured correct behavior for key management and error handling.
feat(api): Enhance Console Export Models
- Expanded ConsoleExport models to include detailed status and event types.
- Added support for various export formats and notification options.
test(time): Implement TimeAnchorPolicyService tests
- Developed tests for TimeAnchorPolicyService to validate time anchors.
- Covered scenarios for anchor validation, drift calculation, and policy enforcement.
- Created expected JSON files for Go modules and workspaces.
- Added go.mod and go.sum files for example projects.
- Implemented private module structure with expected JSON output.
- Introduced vendored dependencies with corresponding expected JSON.
- Developed PostgresGraphJobStore for managing graph jobs.
- Established SQL migration scripts for graph jobs schema.
- Implemented GraphJobRepository for CRUD operations on graph jobs.
- Created IGraphJobRepository interface for repository abstraction.
- Added unit tests for GraphJobRepository to ensure functionality.
- Implemented BunPackageNormalizer to deduplicate packages by name and version.
- Created BunProjectDiscoverer to identify Bun project roots in the filesystem.
- Added project files for the Bun analyzer including manifest and project configuration.
- Developed comprehensive tests for Bun language analyzer covering various scenarios.
- Included fixture files for testing standard installs, isolated linker installs, lockfile-only scenarios, and workspaces.
- Established stubs for authentication sessions to facilitate testing in the web application.
