stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 16 Releases Wiki Activity
Files
e2e404e705578c36a9d19a806405bd3051bc52ce
git.stella-ops.org/docs/advisory-ai/guardrails-and-evidence.md
master b7059d523e Refactor and update test projects, remove obsolete tests, and upgrade dependencies 
- Deleted obsolete test files for SchedulerAuditService and SchedulerMongoSessionFactory.
- Removed unused TestDataFactory class.
- Updated project files for Mongo.Tests to remove references to deleted files.
- Upgraded BouncyCastle.Cryptography package to version 2.6.2 across multiple projects.
- Replaced Microsoft.Extensions.Http.Polly with Microsoft.Extensions.Http.Resilience in Zastava.Webhook project.
- Updated NetEscapades.Configuration.Yaml package to version 3.1.0 in Configuration library.
- Upgraded Pkcs11Interop package to version 5.1.2 in Cryptography libraries.
- Refactored Argon2idPasswordHasher to use BouncyCastle for hashing instead of Konscious.
- Updated JsonSchema.Net package to version 7.3.2 in Microservice project.
- Updated global.json to use .NET SDK version 10.0.101.
2025-12-10 19:13:29 +02:00

6.7 KiB
Raw Blame History

Advisory AI Guardrails & Evidence Intake

Updated: 2025-12-09 | Owner: Advisory AI Docs Guild | Status: Ready to publish (Sprint 0111 / AIAI-DOCS-31-001)

This note captures the guardrail behaviors and evidence intake boundaries required by Sprint 0111 tasks (AIAI-DOCS-31-001, AIAI-RAG-31-003). It binds Advisory AI guardrails to upstream evidence sources and clarifies how Link-Not-Merge (LNM) documents flow into Retrieval-Augmented Generation (RAG) payloads.

1) Evidence sources and contracts

Upstream readiness gates (now satisfied)

  • CLI guardrail artefacts (2025-11-19) are sealed at out/console/guardrails/cli-vuln-29-001/ and out/console/guardrails/cli-vex-30-001/; hashes live in docs/modules/cli/artefacts/guardrails-artefacts-2025-11-19.md.
  • Policy pin: set policyVersion=2025.11.19 per docs/policy/assistant-parameters.md before enabling non-default profiles.
  • SBOM context service is live: the 2025-12-08 smoke against /sbom/context produced sha256:0c705259fdf984bf300baba0abf484fc3bbae977cf8a0a2d1877481f552d600d with evidence in evidence-locker/sbom-context/2025-12-08-response.json and offline mirror offline-kit/advisory-ai/fixtures/sbom-context/2025-12-08/.
  • DEVOPS-AIAI-31-001 landed: deterministic CI harness at ops/devops/advisoryai-ci-runner/run-advisoryai-ci.sh emits binlog/TRX/hashes for Advisory AI.

Evidence feeds

  • Advisory observations (LNM) - consume immutable advisory_observations and advisory_linksets produced per docs/modules/concelier/link-not-merge-schema.md (frozen v1, 2025-11-17).
  • VEX statements - Excititor + VEX Lens linksets with trust weights; treated as structured chunks with source_id and confidence.
  • SBOM context - SBOM-AIAI-31-001 contract: timelines and dependency paths retrieved via ISbomContextRetriever (AddSbomContextHttpClient), default clamps 500 timeline entries / 200 paths.
  • Policy explain traces - Policy Engine digests referenced by policyVersion; cache keys include policy hash to keep outputs replayable.
  • Runtime posture (optional) - Zastava signals (exposure, admissionStatus) when provided by Link-Not-Merge-enabled tenants; optional chunks tagged runtime.

All evidence items must carry content_hash + source_id; Advisory AI never mutates or merges upstream facts (Aggregation-Only Contract).

2) Guardrail stages

  1. Pre-flight sanitization
    • Redact secrets (AWS-style keys, PEM blobs, generic tokens).
    • Strip prompt-injection phrases; enforce max input payload 16kB (configurable, default).
    • Reject requests missing advisoryKey or linkset-backed evidence (LNM guard).
  2. Prompt assembly
    • Deterministic section order: advisory excerpts -> VEX statements -> SBOM deltas -> policy traces -> runtime hints.
    • Vector previews capped at 600 chars + ellipsis; section budgets fixed per profile (default, fips-local, gost-local, cloud-openai) in profiles.catalog.json and hashed into DSSE provenance.
  3. LLM invocation (local/remote)
    • Profiles selected via profile field; remote profiles require Authority tenant consent plus advisory-ai:operate and aoc:verify.
  4. Validation & citation enforcement
    • Every emitted fact must map to an input chunk (source_id + content_hash); citations serialized as [n] in Markdown.
    • Block outputs lacking citations, exceeding section budgets, or including unredacted PII.
  5. Output sealing
    • Store outputHash, inputDigest, provenanceHash; wrap in DSSE when configured.
    • Cache TTL defaults to 24h; regenerate only when inputs change or forceRefresh=true.

Metrics: advisory_ai_guardrail_blocks_total, advisory_ai_outputs_stored_total, advisory_ai_citation_coverage_ratio. Logs carry output_hash, profile, and block reason; no secrets or raw prompt bodies are logged.

3) RAG payload mapping to LNM (summary)

LNM field RAG chunk field Notes
observation._id source_id Used for citations and conflict surfacing.
observation.advisoryId advisory_key Keyed alongside task type in cache.
observation.affected[].purl purl Included for remediation + SBOM joins.
observation.severities[] severity Passed through unmerged; multiple severities allowed.
linkset.conflicts[] conflicts Rendered verbatim for conflict tasks; no inference merges.
provenance.sourceArtifactSha content_hash Drives determinism and replay.

See docs/advisory-ai/evidence-payloads.md for full JSON examples and alignment rules.

4) Compliance with upstream artefacts and verification

  • References: CONSOLE-VULN-29-001, CONSOLE-VEX-30-001, CLI-VULN-29-001, CLI-VEX-30-001, EXCITITOR-CONSOLE-23-001, DEVOPS-AIAI-31-001, SBOM-AIAI-31-001.
  • CLI fixtures: expected hashes 421af53f9eeba6903098d292fbd56f98be62ea6130b5161859889bf11d699d18 (sample SBOM context) and e5aecfba5cee8d412408fb449f12fa4d5bf0a7cb7e5b316b99da3b9019897186 / 2b11b1e2043c2ec1b0cb832c29577ad1c5cbc3fbd0b379b0ca0dee46c1bc32f6 (sample vuln/vex outputs). Verify with sha256sum --check docs/modules/cli/artefacts/guardrails-artefacts-2025-11-19.md.
  • SBOM context: fixture hash sha256:421af53f9eeba6903098d292fbd56f98be62ea6130b5161859889bf11d699d18; live SbomService smoke (2025-12-08) hash sha256:0c705259fdf984bf300baba0abf484fc3bbae977cf8a0a2d1877481f552d600d stored in evidence-locker/sbom-context/2025-12-08-response.json and mirrored under offline-kit/advisory-ai/fixtures/sbom-context/2025-12-08/.
  • CI harness: ops/devops/advisoryai-ci-runner/run-advisoryai-ci.sh emits ops/devops/artifacts/advisoryai-ci/<UTC>/build.binlog, tests/advisoryai.trx, and summary.json with SHA256s; include the latest run when shipping Offline Kits.
  • Policy compatibility: guardrails must remain compatible with docs/policy/assistant-parameters.md; configuration knobs documented there are authoritative for env vars and defaults.
  • Packaging tasks (AIAI-PACKAGING-31-002) must include this guardrail summary in DSSE metadata to keep Offline Kit parity.

5) Operator checklist

  • LNM feed enabled and Concelier schemas at v1 (2025-11-17).
  • SBOM retriever configured or NullSbomContextClient left as safe default; verify latest context hash (sha256:0c705259f...d600d) or fixture hash (sha256:421af53f9...9d18) before enabling remediation tasks.
  • Policy hash pinned via policyVersion when reproducibility is required.
  • CLI guardrail artefact hashes verified against docs/modules/cli/artefacts/guardrails-artefacts-2025-11-19.md and mirrored into Offline Kits.
  • CI harness run captured from ops/devops/advisoryai-ci-runner/run-advisoryai-ci.sh; store summary.json alongside doc promotion.
  • Remote profiles only after Authority consent and profile allowlist are set.
  • Cache directories shared between web + worker hosts for DSSE sealing.