consolidation of some of the modules, localization fixes, product advisories work, qa work

This commit is contained in:
master
2026-03-05 03:54:22 +02:00
parent 7bafcc3eef
commit 8e1cb9448d
3878 changed files with 72600 additions and 46861 deletions

View File

@@ -1,6 +1,6 @@
# Remediation Module Architecture
# Remediation Module Architecture
> **Status: Planned.** The Remediation marketplace is a planned feature for developer-facing fix templates, PR generation, and contributor trust scoring. Source code at `src/Remediation/` contains initial scaffolding. This architecture document is a design specification pending full implementation.
> **Status: Partially implemented.** Core remediation APIs exist, and marketplace source endpoints now run with persistence-backed list/get/upsert behavior. Remaining areas are still incremental (template lifecycle depth, contributor workflow hardening, and broader policy integration).
## Overview
@@ -18,47 +18,48 @@ Tracks the lifecycle of a remediation pull request from submission through scann
Community members or vendors who submit fix templates. Each contributor has a trust score computed from their verification history (verified fixes, rejections).
### Marketplace Sources
Curated collections of fix templates from community, partner, or vendor origins. Sources are rated independently and can be enabled/disabled per tenant.
Curated collections of fix templates from community, partner, or vendor origins. Sources are rated independently and can be enabled or disabled per tenant.
## Domain Model
```
```text
FixTemplate (remediation.fix_templates)
├── CveId (text, indexed)
├── Purl (text, indexed pkg:type/name)
├── VersionRange (semver range)
├── PatchContent (unified diff)
├── Status (pending/verified/rejected)
├── TrustScore (0.01.0)
├── DsseDigest (nullable signed envelope hash)
└── ContributorId / SourceId (foreign keys)
|- CveId (text, indexed)
|- Purl (text, indexed - pkg:type/name)
|- VersionRange (semver range)
|- PatchContent (unified diff)
|- Status (pending/verified/rejected)
|- TrustScore (0.0-1.0)
|- DsseDigest (nullable - signed envelope hash)
`- ContributorId / SourceId (foreign keys)
PrSubmission (remediation.pr_submissions)
├── FixTemplateId (nullable FK)
├── PrUrl, RepositoryUrl, SourceBranch, TargetBranch
├── CveId (text, indexed)
├── Status (opened/scanning/merged/verified/failed/inconclusive)
├── PreScanDigest, PostScanDigest
├── ReachabilityDeltaDigest, FixChainDsseDigest
├── Verdict (fixed/partial/not_fixed/inconclusive)
└── ContributorId
|- FixTemplateId (nullable FK)
|- PrUrl, RepositoryUrl, SourceBranch, TargetBranch
|- CveId (text, indexed)
|- Status (opened/scanning/merged/verified/failed/inconclusive)
|- PreScanDigest, PostScanDigest
|- ReachabilityDeltaDigest, FixChainDsseDigest
|- Verdict (fixed/partial/not_fixed/inconclusive)
`- ContributorId
Contributor (remediation.contributors)
├── Username (unique)
├── VerifiedFixes, TotalSubmissions, RejectedSubmissions
└── TrustScore (computed)
|- Username (unique)
|- VerifiedFixes, TotalSubmissions, RejectedSubmissions
`- TrustScore (computed)
MarketplaceSource (remediation.marketplace_sources)
├── Key (unique)
├── SourceType (community/partner/vendor)
├── Enabled, TrustScore
└── LastSyncAt
|- Key (tenant-scoped unique key)
|- SourceType (community/partner/vendor)
|- Enabled, TrustScore
`- LastSyncAt
```
## Trust Scoring
Contributor trust score formula:
```
```text
score = clamp((verified * 1.0 - rejected * 0.5) / max(total, 1), 0, 1)
```
@@ -70,30 +71,45 @@ Trust tiers:
## API Surface
All endpoints under `/api/v1/remediation/`.
All endpoints are under `/api/v1/remediation/`.
### Templates
- `GET /templates` List fix templates (filter by CVE, PURL)
- `GET /templates/{id}` Get template detail
- `POST /templates` Create template (requires `remediation.submit`)
- `GET /templates` - List fix templates (filter by CVE, PURL)
- `GET /templates/{id}` - Get template detail
- `POST /templates` - Create template (requires `remediation.submit`)
### Submissions
- `GET /submissions` List PR submissions
- `GET /submissions/{id}` Get submission with attestation chain
- `POST /submissions` Submit PR for verification
- `GET /submissions/{id}/status` Pipeline status
- `GET /submissions` - List PR submissions
- `GET /submissions/{id}` - Get submission with attestation chain
- `POST /submissions` - Submit PR for verification
- `GET /submissions/{id}/status` - Pipeline status
### Matching
- `GET /match?cve=...&purl=...&version=...` Find applicable fix templates
- `GET /match?cve=...&purl=...&version=...` - Find applicable fix templates
### Contributors
- `GET /contributors` List contributors
- `GET /contributors/{username}` Profile with trust score
- `GET /contributors` - List contributors
- `GET /contributors/{username}` - Profile with trust score
### Sources
- `GET /sources` List marketplace sources
- `GET /sources/{key}` Source detail
- `POST /sources` Create/update source (requires `remediation.manage`)
- `GET /sources` - List marketplace sources
- `GET /sources/{key}` - Source detail
- `POST /sources` - Create/update source (requires `remediation.manage`)
Implemented source API contract (2026-03-04):
- Request model for upsert (`POST /sources`): `key`, `name`, `url`, `sourceType`, `enabled`, `trustScore`, `lastSyncAt`.
- Deterministic behavior:
- key normalization uses lowercase invariant
- list ordering is key-based ordinal ordering
- upsert is idempotent by tenant + source key
- Validation:
- key pattern: `^[a-z0-9][a-z0-9._-]{0,63}$`
- sourceType allowed values: `community`, `partner`, `vendor`
- trustScore range: `0..1`
- url must be an absolute `http`/`https` URL when provided
- Tenant isolation:
- all source endpoints require tenant context (`RequireTenant`)
- repository operations are tenant-scoped for list/get/upsert behavior
## Authorization Policies
@@ -103,13 +119,41 @@ All endpoints under `/api/v1/remediation/`.
| `remediation.submit` | Create templates and submit PRs |
| `remediation.manage` | Manage marketplace sources, verify/reject templates |
## Runtime Storage Contract (2026-03-05)
Remediation runtime storage is now selected through `Remediation:Storage:Driver` (or `Storage:Driver`) with explicit startup validation:
- `postgres` (default):
- Required settings: `ConnectionStrings:Default` or `Remediation:Storage:Postgres:ConnectionString`.
- Optional schema override: `Remediation:Storage:Postgres:SchemaName` (defaults to `remediation`).
- Behavior: repositories are wired with `RemediationDataSource` and Postgres-backed constructors.
- Startup: fails fast when required connection configuration is missing.
- `inmemory`:
- Allowed only in `Test`/`Testing` environment profiles.
- Intended for deterministic automated tests only.
- Startup: fails fast outside test profiles.
This removes implicit production-like in-memory behavior and makes storage mode explicit and auditable.
Migration notes:
- Legacy webservice wiring that instantiated parameterless repository constructors has been removed.
- Existing deployments must provide a Postgres connection string (or explicitly run with `inmemory` in `Test`/`Testing` profiles).
- Integration tests should pin `REMEDIATION__STORAGE__DRIVER=inmemory` under a testing environment profile for deterministic non-network execution.
## Service Integration Baseline (2026-03-05)
- Router integration enabled (`serviceName: remediation`) with endpoint refresh on startup.
- Local alias binding/logging enabled via `remediation.stella-ops.local`.
- CORS and tenant middleware are part of the default request pipeline before endpoint execution.
## Verification Pipeline
1. PR submitted (status: `opened`)
2. Pre-merge scan captures baseline SBOM digest
3. PR merged (status: `merged`)
4. Post-merge scan captures updated SBOM digest
5. Reachability delta computed between pre/post digests
5. Reachability delta computed between pre and post digests
6. Fix-chain DSSE envelope signed
7. Verdict determined: `fixed`, `partial`, `not_fixed`, or `inconclusive`
@@ -121,12 +165,13 @@ The `RemediationPrWebhookHandler` in the Signals module detects remediation PRs
## Module Location
```
```text
src/Remediation/
├── StellaOps.Remediation.Core/ Domain models, interfaces, services
├── StellaOps.Remediation.WebService/ API endpoints, Program.cs
├── StellaOps.Remediation.Persistence/ SQL migrations, repositories
└── __Tests/StellaOps.Remediation.Tests/ — Unit tests
|- StellaOps.Remediation.Core/ - Domain models, interfaces, services
|- StellaOps.Remediation.WebService/ - API endpoints, Program.cs
|- StellaOps.Remediation.Persistence/ - SQL migrations, repositories
|- __Tests/StellaOps.Remediation.Tests/ - Repository/domain unit tests
`- __Tests/StellaOps.Remediation.WebService.Tests/ - Source endpoint integration tests
```
## Related Sprints
@@ -141,3 +186,23 @@ src/Remediation/
## Related Contracts
- `docs/contracts/remediation-pr-v1.md`
## Advisory Gap Status (2026-03-04 Batch)
Status:
- Advisory gap `REM-001` is closed for marketplace sources.
Closed behaviors:
- `GET /api/v1/remediation/sources` returns persisted tenant-scoped sources with deterministic ordering.
- `GET /api/v1/remediation/sources/{key}` resolves persisted records (no unconditional stub `source_not_found` path).
- `POST /api/v1/remediation/sources` performs validated upsert and no longer returns `501`.
- Marketplace source repository abstraction and implementation are wired through DI:
- `IMarketplaceSourceRepository`
- `PostgresMarketplaceSourceRepository`
Verification evidence:
- `dotnet test src/Remediation/__Tests/StellaOps.Remediation.Tests/StellaOps.Remediation.Tests.csproj -m:1 -v minimal` - `28` passed.
- `dotnet test src/Remediation/__Tests/StellaOps.Remediation.WebService.Tests/StellaOps.Remediation.WebService.Tests.csproj -m:1 -v minimal` - `4` passed.
Tracking sprint:
- `docs/implplan/SPRINT_20260304_308_Remediation_marketplace_sources_api_completion.md`