stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 16 Releases Wiki Activity

Refactor and update test projects, remove obsolete tests, and upgrade dependencies

Browse Source 
- Deleted obsolete test files for SchedulerAuditService and SchedulerMongoSessionFactory.
- Removed unused TestDataFactory class.
- Updated project files for Mongo.Tests to remove references to deleted files.
- Upgraded BouncyCastle.Cryptography package to version 2.6.2 across multiple projects.
- Replaced Microsoft.Extensions.Http.Polly with Microsoft.Extensions.Http.Resilience in Zastava.Webhook project.
- Updated NetEscapades.Configuration.Yaml package to version 3.1.0 in Configuration library.
- Upgraded Pkcs11Interop package to version 5.1.2 in Cryptography libraries.
- Refactored Argon2idPasswordHasher to use BouncyCastle for hashing instead of Konscious.
- Updated JsonSchema.Net package to version 7.3.2 in Microservice project.
- Updated global.json to use .NET SDK version 10.0.101.
This commit is contained in:
master
2025-12-10 19:13:29 +02:00
parent a3c7fe5e88
commit b7059d523e
369 changed files with 11125 additions and 14245 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
docs/modules/excititor/architecture.md
View File

@@ -157,7 +157,8 @@ Schema: `vex`
- `payload BYTEA NOT NULL`, `payload_hash TEXT NOT NULL`
- PRIMARY KEY (`digest`, `name`)
- **Observations/linksets** — use the append-only Postgres linkset schema already defined for `IAppendOnlyLinksetStore` (tables `vex_linksets`, `vex_linkset_observations`, `vex_linkset_disagreements`, `vex_linkset_mutations`) with indexes on `(tenant, vulnerability_id, product_key)` and `updated_at`.
- **Observations/linksets** - use the append-only Postgres linkset schema already defined for `IAppendOnlyLinksetStore` (tables `vex_linksets`, `vex_linkset_observations`, `vex_linkset_disagreements`, `vex_linkset_mutations`) with indexes on `(tenant, vulnerability_id, product_key)` and `updated_at`.
- **Graph overlays** - materialized cache table `vex_overlays` (tenant, purl, advisory_id, source) storing JSONB payloads that follow `docs/modules/excititor/schemas/vex_overlay.schema.json` (schemaVersion 1.0.0). Cache eviction via `cached_at + ttl_seconds`; overlays regenerate when linkset or observation hashes change.
**Canonicalisation & hashing**

86
docs/modules/excititor/graph-overlays.md Normal file
View File

@@ -0,0 +1,86 @@
# Excititor Graph Overlay Contract (v1.0.0)
_Updated: 2025-12-10 | Owners: Excititor Core + UI Guilds | Scope: EXCITITOR-GRAPH-21-001..005, EXCITITOR-POLICY-20-001/002, EXCITITOR-RISK-66-001_
## Purpose
Defines the graph-ready overlay built from Link-Not-Merge observations/linksets so Console, Vuln Explorer, Policy, and Risk surfaces consume a single deterministic shape. This freezes the contract for Postgres materialization and cache APIs, unblocking Sprint 0120 tasks.
## Schema
- JSON Schema: `docs/modules/excititor/schemas/vex_overlay.schema.json` (draft 2020-12, schemaVersion `1.0.0`).
- Required fields: `schemaVersion`, `generatedAt`, `tenant`, `purl`, `advisoryId`, `source`, `status`, `observations[]`, `provenance`.
- Status enum: `affected|not_affected|under_investigation|fixed|unknown`.
- Ordering: observations are sorted by `source, advisoryId, fetchedAt` (Link-Not-Merge invariant) and emitted in that order. Overlays are returned in request PURL order, then by `advisoryId`, then `source`.
- Provenance: carries `linksetId`, `linksetHash`, `observationHashes[]`, optional `policyHash`, `sbomContextHash`, and `planCacheKey` for replay.
## Postgres materialization (IAppendOnlyLinksetStore)
- Table `vex_overlays` (materialized cache):
- Primary key: `(tenant, purl, advisory_id, source)`.
- Columns: `status`, `justifications` (jsonb), `conflicts` (jsonb), `observations` (jsonb), `provenance` (jsonb), `cached_at`, `ttl_seconds`, `schema_version`.
- Indexes: unique `(tenant, purl, advisory_id, source)`, plus `(tenant, cached_at)` for TTL sweeps.
- Overlay rows are regenerated when linkset hash or observation hash set changes; cache evictions use `cached_at + ttl_seconds`.
- Linksets and observation hashes come from the append-only linkset store (`IAppendOnlyLinksetStore`) to preserve Aggregation-Only Contract guarantees.
## API shape (Graph/Vuln Explorer)
- Endpoint: `GET /v1/graph/overlays?purl=<purl>&purl=<purl>&includeJustifications=true|false`.
- Response items follow `vex_overlay.schema.json`; `cache` stanza signals `cached`, `cachedAt`, and `ttlSeconds`.
- Cursoring: stable order (input PURL list) with `nextPageToken` based on `(tenant, purl, advisoryId, source, generatedAt)`.
- Telemetry: `excititor.graph.overlays.cache{tenant,hit}` counter; `excititor.graph.overlays.latency_ms` histogram tagged with `cached`.
## Sample (abridged)
```json
{
"schemaVersion": "1.0.0",
"generatedAt": "2025-12-10T00:00:00Z",
"tenant": "tenant-default",
"purl": "pkg:maven/org.example/foo@1.2.3",
"advisoryId": "GHSA-xxxx-yyyy-zzzz",
"source": "ghsa",
"status": "affected",
"justifications": [
{
"kind": "known_affected",
"reason": "Upstream GHSA reports affected range <1.3.0.",
"evidence": ["concelier:ghsa:obs:6561e41b3e3f4a6e9d3b91c1"],
"weight": 0.8
}
],
"conflicts": [
{
"field": "affected.versions",
"reason": "vendor_range_differs",
"values": ["<1.2.0", "<=1.3.0"],
"sourceIds": ["concelier:redhat:obs:...","concelier:ghsa:obs:..."]
}
],
"observations": [
{
"id": "concelier:ghsa:obs:6561e41b3e3f4a6e9d3b91c1",
"contentHash": "sha256:1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd",
"fetchedAt": "2025-11-19T00:00:00Z"
}
],
"provenance": {
"linksetId": "concelier:ghsa:linkset:6561e41b3e3f4a6e9d3b91d0",
"linksetHash": "sha256:deaddeaddeaddeaddeaddeaddeaddeaddeaddeaddeaddeaddeaddeaddeaddead",
"observationHashes": ["sha256:1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd"],
"policyHash": "sha256:0f7c...9ad3",
"sbomContextHash": "sha256:421af53f9eeba6903098d292fbd56f98be62ea6130b5161859889bf11d699d18",
"planCacheKey": "tenant-default|pkg:maven/org.example/foo@1.2.3|GHSA-xxxx-yyyy-zzzz"
},
"cache": {
"cached": true,
"cachedAt": "2025-12-10T00:00:00Z",
"ttlSeconds": 300
}
}
```
## Validation & determinism
- Validate overlays against `vex_overlay.schema.json` in CI and during materialization; reject or warn when fields drift.
- Deterministic ordering: input PURL order, then `advisoryId`, then `source`; observation list sorted by `source, advisoryId, fetchedAt`.
- No mutation: overlays are append-only; regeneration inserts a new row/version, leaving prior cache entries for audit until TTL expires.
## Handoff
- Consumers (Console, Vuln Explorer, Policy Engine, Risk) should treat `vex_overlay.schema.json` as the authoritative contract.
- Offline kits must bundle the schema file and sample payloads under `docs/samples/excititor/` with SHA256 manifests.
- Future schema versions must bump `schemaVersion` and add migration notes to this document and `docs/modules/excititor/architecture.md`.

2
docs/modules/excititor/operations/graph-linkouts-implementation.md
View File

@@ -26,7 +26,7 @@
- `vex_observations` indexes:
- `{ tenant: 1, component.purl: 1, advisoryId: 1, source: 1, modifiedAt: -1 }`
- Sparse `{ tenant: 1, component.purl: 1, status: 1 }`
- Optional materialized `vex_overlays` cache: unique `{ tenant: 1, purl: 1 }`, TTL on `cachedAt` driven by `excititor:graph:overlayTtlSeconds` (default 300s).
- Optional materialized `vex_overlays` cache: unique `{ tenant: 1, purl: 1 }`, TTL on `cachedAt` driven by `excititor:graph:overlayTtlSeconds` (default 300s); payload must validate against `docs/modules/excititor/schemas/vex_overlay.schema.json` (schemaVersion 1.0.0). Bundle sample payload `docs/samples/excititor/vex-overlay-sample.json` in Offline Kits.
## Determinism
- Ordering: input PURL order → `advisoryId` → `source` for linkouts; overlays follow input order.

149
docs/modules/excititor/schemas/vex_overlay.schema.json Normal file
View File

@@ -0,0 +1,149 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://stellaops.dev/schemas/excititor/vex_overlay.schema.json",
"title": "Excititor VEX Overlay",
"description": "Graph-ready overlay built from Link-Not-Merge observations and linksets. Immutable and append-only; ordered for deterministic pagination and caching.",
"type": "object",
"additionalProperties": false,
"required": [
"schemaVersion",
"generatedAt",
"tenant",
"purl",
"advisoryId",
"source",
"status",
"observations",
"provenance"
],
"properties": {
"schemaVersion": {
"type": "string",
"enum": ["1.0.0"]
},
"generatedAt": {
"type": "string",
"format": "date-time"
},
"tenant": {
"type": "string",
"description": "Tenant identifier used for storage partitioning."
},
"purl": {
"type": "string",
"description": "Normalized package URL for the component."
},
"advisoryId": {
"type": "string",
"description": "Upstream advisory identifier (e.g., GHSA, RHSA, CVE)."
},
"source": {
"type": "string",
"description": "Linkset source identifier (matches Concelier linkset source)."
},
"status": {
"type": "string",
"enum": [
"affected",
"not_affected",
"under_investigation",
"fixed",
"unknown"
]
},
"justifications": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["kind", "reason"],
"properties": {
"kind": {
"type": "string",
"description": "Reason code aligned to VEX statement taxonomy."
},
"reason": {
"type": "string",
"description": "Human-readable justification text."
},
"evidence": {
"type": "array",
"items": {
"type": "string",
"description": "Observation or linkset id contributing to this justification."
}
},
"weight": {
"type": "number",
"minimum": 0,
"maximum": 1,
"description": "Optional confidence weight."
}
}
}
},
"conflicts": {
"type": "array",
"description": "Conflicts detected in linkset normalization.",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["field", "reason"],
"properties": {
"field": { "type": "string" },
"reason": { "type": "string" },
"values": {
"type": "array",
"items": { "type": "string" }
},
"sourceIds": {
"type": "array",
"items": { "type": "string" }
}
}
}
},
"observations": {
"type": "array",
"description": "Ordered list of Link-Not-Merge observation references feeding this overlay.",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["id", "contentHash", "fetchedAt"],
"properties": {
"id": { "type": "string" },
"contentHash": { "type": "string", "pattern": "^sha256:[A-Fa-f0-9]{64}$" },
"fetchedAt": { "type": "string", "format": "date-time" }
}
},
"minItems": 1
},
"provenance": {
"type": "object",
"additionalProperties": false,
"required": ["linksetId", "linksetHash", "observationHashes"],
"properties": {
"linksetId": { "type": "string" },
"linksetHash": { "type": "string", "pattern": "^sha256:[A-Fa-f0-9]{64}$" },
"observationHashes": {
"type": "array",
"items": { "type": "string", "pattern": "^sha256:[A-Fa-f0-9]{64}$" },
"minItems": 1
},
"policyHash": { "type": "string" },
"sbomContextHash": { "type": "string" },
"planCacheKey": { "type": "string" },
"generatedBy": { "type": "string" }
}
},
"cache": {
"type": "object",
"additionalProperties": false,
"properties": {
"cached": { "type": "boolean" },
"cachedAt": { "type": "string", "format": "date-time" },
"ttlSeconds": { "type": "integer", "minimum": 0 }
}
}
}
}