feat: Add MongoIdempotencyStoreOptions for MongoDB configuration
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
feat: Implement BsonJsonConverter for converting BsonDocument and BsonArray to JSON fix: Update project file to include MongoDB.Bson package test: Add GraphOverlayExporterTests to validate NDJSON export functionality refactor: Refactor Program.cs in Attestation Tool for improved argument parsing and error handling docs: Update README for stella-forensic-verify with usage instructions and exit codes feat: Enhance HmacVerifier with clock skew and not-after checks feat: Add MerkleRootVerifier and ChainOfCustodyVerifier for additional verification methods fix: Update DenoRuntimeShim to correctly handle file paths feat: Introduce ComposerAutoloadData and related parsing in ComposerLockReader test: Add tests for Deno runtime execution and verification test: Enhance PHP package tests to include autoload data verification test: Add unit tests for HmacVerifier and verification logic
This commit is contained in:
@@ -42,6 +42,7 @@
|
||||
- Each snapshot packages `nodes.jsonl`, `edges.jsonl`, `overlays/` plus manifest with hash, counts, and provenance. Export Center consumes these artefacts for graph-specific bundles.
|
||||
- Saved queries and overlays include deterministic IDs so Offline Kit consumers can import and replay results.
|
||||
- Runtime hosts register the SBOM ingest pipeline via `services.AddSbomIngestPipeline(...)`. Snapshot exports default to `./artifacts/graph-snapshots` but can be redirected with `STELLAOPS_GRAPH_SNAPSHOT_DIR` or the `SbomIngestOptions.SnapshotRootDirectory` callback.
|
||||
- Analytics overlays are exported as NDJSON (`overlays/clusters.ndjson`, `overlays/centrality.ndjson`) ordered by node id; `overlays/manifest.json` mirrors snapshot id and counts for offline parity.
|
||||
|
||||
## 6) Observability
|
||||
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
## Deployment overlays
|
||||
- Helm/Compose should expose two timers for analytics: `GRAPH_ANALYTICS_CLUSTER_INTERVAL` and `GRAPH_ANALYTICS_CENTRALITY_INTERVAL` (ISO-8601 duration, default 5m). Map to `GraphAnalyticsOptions`.
|
||||
- Change-stream/backfill worker toggles via `GRAPH_CHANGE_POLL_INTERVAL`, `GRAPH_BACKFILL_INTERVAL`, `GRAPH_CHANGE_MAX_RETRIES`, `GRAPH_CHANGE_RETRY_BACKOFF`.
|
||||
- Mongo bindings (optional): `GRAPH_CHANGE_COLLECTION`, `GRAPH_CHANGE_SEQUENCE_FIELD`, `GRAPH_CHANGE_NODE_FIELD`, `GRAPH_CHANGE_EDGE_FIELD`, `GRAPH_CHANGE_IDEMPOTENCY_COLLECTION`, `GRAPH_ANALYTICS_SNAPSHOT_COLLECTION`, `GRAPH_ANALYTICS_PROGRESS_COLLECTION`.
|
||||
- New Mongo collections:
|
||||
- `graph_cluster_overlays` — cluster assignments (`tenant`, `snapshot_id`, `node_id`, `cluster_id`, `generated_at`).
|
||||
- `graph_centrality_overlays` — degree + betweenness approximations per node.
|
||||
@@ -25,6 +26,7 @@
|
||||
## Configuration defaults
|
||||
- Cluster/centrality intervals: 5 minutes; label-propagation iterations: 6; betweenness sample size: 12.
|
||||
- Change stream: poll every 5s, backfill every 15m, max retries 3 with 3s backoff, batch size 256.
|
||||
- Overlay exports: clusters/centrality written to `overlays/clusters.ndjson` and `overlays/centrality.ndjson` with manifest; ordered by `node_id` for deterministic bundle hashes.
|
||||
|
||||
## Notes
|
||||
- Analytics writes are idempotent (upserts keyed on tenant+snapshot+node_id). Change-stream processing is idempotent via sequence tokens persisted in `IIdempotencyStore` (Mongo or in-memory for tests).
|
||||
|
||||
@@ -0,0 +1,66 @@
|
||||
# Graph API schema outline (prep for GRAPH-API-28-001)
|
||||
|
||||
Status: draft outline · Date: 2025-11-22 · Owner: Graph API Guild
|
||||
Scope: Establish OpenAPI/JSON schema primitives for search/query/paths/diff/export endpoints, tile streaming, budgets, and RBAC headers. This is a staging note for sprint 0207 Wave 1.
|
||||
|
||||
## 1) Shared headers and security
|
||||
- `X-Stella-Tenant` (required, string)
|
||||
- `Authorization: Bearer <token>`; scopes: `graph:read`, `graph:query`, `graph:export` as applicable per route.
|
||||
- `X-Request-Id` optional; echoed in responses.
|
||||
- Rate limit headers: `X-RateLimit-Remaining`, `Retry-After` for 429 cases.
|
||||
|
||||
## 2) Tile envelope (streaming NDJSON)
|
||||
```jsonc
|
||||
{
|
||||
"type": "node" | "edge" | "stats" | "cursor" | "diagnostic",
|
||||
"seq": 1,
|
||||
"cost": { "consumed": 12, "remaining": 988, "limit": 1000 },
|
||||
"data": { /* node/edge/payload depending on type */ }
|
||||
}
|
||||
```
|
||||
- Deterministic ordering: breadth-first by hop for paths/query; lexicographic by `id` inside each hop when stable ordering is needed.
|
||||
- Cursor tile: `{"type":"cursor","token":"opaque"}` for resume.
|
||||
- Stats tile: counts, depth reached, cache hit ratios.
|
||||
|
||||
## 3) `/graph/search` (POST)
|
||||
Request body (summary):
|
||||
- `query` (string; prefix/exact semantics), `kinds` (array of `node` kinds), `limit`, `tenant`, `filters` (labels, ecosystem, scope), `ordering`.
|
||||
Response: stream of tiles (`node` + minimal neighbor context), plus final cursor/diagnostic tile.
|
||||
Validation rules: enforce `limit <= 500`, reject empty query unless `filters` present.
|
||||
|
||||
## 4) `/graph/query` (POST)
|
||||
- Body: DSL expression or structured filter object; includes `budget` (nodes/edges cap, time cap), `overlays` requested (`policy`, `vex`, `advisory`), `explain`: `none|minimal|full`.
|
||||
- Response: streaming tiles with mixed `node`, `edge`, `stats`, `cursor`; budgets decremented per tile.
|
||||
- Errors: `429` budget exhausted with diagnostic tile including partial cursor.
|
||||
|
||||
## 5) `/graph/paths` (POST)
|
||||
- Body: `source_ids[]`, `target_ids[]`, `max_depth <= 6`, `constraints` (edge kinds, tenant, overlays allowed), `fanout_cap`.
|
||||
- Response: tiles grouped by path id; includes `hop` field for ordering; optional `overlay_trace` when policy layer requested.
|
||||
|
||||
## 6) `/graph/diff` (POST)
|
||||
- Body: `snapshot_a`, `snapshot_b`, optional `filters` (tenant, kinds, advisory keys).
|
||||
- Response: tiles of `node_added`, `node_removed`, `edge_added`, `edge_removed`, `overlay_delta` (policy/vex/advisory), plus `manifest` tile with counts and checksums.
|
||||
|
||||
## 7) `/graph/export` (POST)
|
||||
- Body: `formats[]` (`graphml`, `csv`, `ndjson`, `png`, `svg`), `snapshot_id` or `query_ref`, `include_overlays`.
|
||||
- Response: job ticket `{ job_id, status_url, checksum_manifest_url }`; downloads streamed with deterministic manifests.
|
||||
|
||||
## 8) Error schema (shared)
|
||||
```jsonc
|
||||
{
|
||||
"error": "GRAPH_BUDGET_EXCEEDED" | "GRAPH_VALIDATION_FAILED" | "GRAPH_RATE_LIMITED",
|
||||
"message": "human-readable",
|
||||
"details": {},
|
||||
"request_id": "..."
|
||||
}
|
||||
```
|
||||
|
||||
## 9) Open questions to settle before sign-off
|
||||
- Final DSL surface vs structured filters for `/graph/query`.
|
||||
- Exact cache key and eviction policy for overlays requested via query.
|
||||
- PNG/SVG render payload (pre-signed URL vs inline streaming) and size caps.
|
||||
|
||||
## 10) Next steps
|
||||
- Convert this outline into OpenAPI components + schemas in repo (`docs/api/graph-gateway-spec-draft.yaml`) by 2025-11-24.
|
||||
- Add JSON Schema fragments for tiles and error payloads to reuse in SDKs.
|
||||
- Review with Policy Engine Guild to lock overlay contract version for GRAPH-API-28-006.
|
||||
Reference in New Issue
Block a user