c8f3120174
- Implement `SbomVexOrderingDeterminismProperties` for testing component list and vulnerability metadata hash consistency. - Create `UnicodeNormalizationDeterminismProperties` to validate NFC normalization and Unicode string handling. - Add project file for `StellaOps.Testing.Determinism.Properties` with necessary dependencies. - Introduce CI/CD template validation tests including YAML syntax checks and documentation content verification. - Create validation script for CI/CD templates ensuring all required files and structures are present.
6.8 KiB
6.8 KiB
Sprint 20251226 · AI Artifact Attestations
Topic & Scope
- Define and implement standardized attestation types for all AI-generated artifacts
- Ensure all AI outputs are replayable, inspectable, and clearly marked as Suggestion-only vs Evidence-backed
- Integrate with existing ProofChain infrastructure for OCI attachment
- Working directory:
src/Attestor/__Libraries/StellaOps.Attestor.ProofChain/,src/ExportCenter/
Dependencies & Concurrency
- Depends on: ProofChain library (COMPLETE).
- Depends on: OCI Referrer infrastructure (COMPLETE).
- Should run before or in parallel with: SPRINT_20251226_015/016/017 (AI feature sprints use these attestation types).
Documentation Prerequisites
docs/modules/attestor/proof-chain-specification.mdsrc/Attestor/__Libraries/StellaOps.Attestor.ProofChain/Statements/- AI Assistant Advisory (this sprint's source)
Context: What Already Exists
The following predicate types are already implemented:
| Predicate | Type URI | Status |
|---|---|---|
| Build Provenance | StellaOps.BuildProvenance@1 |
COMPLETE |
| SBOM Attestation | StellaOps.SBOMAttestation@1 |
COMPLETE |
| Scan Results | StellaOps.ScanResults@1 |
COMPLETE |
| Policy Evaluation | StellaOps.PolicyEvaluation@1 |
COMPLETE |
| VEX Attestation | StellaOps.VEXAttestation@1 |
COMPLETE |
| Risk Profile Evidence | StellaOps.RiskProfileEvidence@1 |
COMPLETE |
| Reachability Witness | StellaOps.ReachabilityWitness@1 |
COMPLETE |
| Reachability Subgraph | StellaOps.ReachabilitySubgraph@1 |
COMPLETE |
| Proof Spine | StellaOps.ProofSpine@1 |
COMPLETE |
This sprint adds AI-specific predicate types with replay metadata.
Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
|---|---|---|---|---|---|
| 1 | AIATTEST-01 | DONE | None | Attestor Guild | Define AIArtifactBase predicate structure: model_id, weights_digest, prompt_template_version, decoding_params, inputs_hashes[] |
| 2 | AIATTEST-02 | DONE | AIATTEST-01 | Attestor Guild | Define AIExplanation predicate: extends AIArtifactBase + explanation_type, content, citations[], confidence_score |
| 3 | AIATTEST-03 | DONE | AIATTEST-01 | Attestor Guild | Define AIRemediationPlan predicate: extends AIArtifactBase + steps[], expected_delta, risk_assessment, verification_status |
| 4 | AIATTEST-04 | DONE | AIATTEST-01 | Attestor Guild | Define AIVexDraft predicate: extends AIArtifactBase + vex_statements[], justifications[], evidence_refs[] |
| 5 | AIATTEST-05 | DONE | AIATTEST-01 | Attestor Guild | Define AIPolicyDraft predicate: extends AIArtifactBase + rules[], test_cases[], validation_result |
| 6 | AIATTEST-06 | DONE | AIATTEST-01 | Attestor Guild | Define AIArtifactAuthority enum: Suggestion, EvidenceBacked, AuthorityThreshold (configurable threshold for each) |
| 7 | AIATTEST-07 | DONE | AIATTEST-06 | Attestor Guild | Authority classifier: rules for when artifact qualifies as EvidenceBacked (citation rate ≥ X, evidence refs valid, etc.) |
| 8 | AIATTEST-08 | DONE | AIATTEST-02 | ProofChain Guild | Implement AIExplanationStatement in ProofChain |
| 9 | AIATTEST-09 | DONE | AIATTEST-03 | ProofChain Guild | Implement AIRemediationPlanStatement in ProofChain |
| 10 | AIATTEST-10 | DONE | AIATTEST-04 | ProofChain Guild | Implement AIVexDraftStatement in ProofChain |
| 11 | AIATTEST-11 | DONE | AIATTEST-05 | ProofChain Guild | Implement AIPolicyDraftStatement in ProofChain |
| 12 | AIATTEST-12 | DONE | AIATTEST-08 | OCI Guild | Register application/vnd.stellaops.ai.explanation+json media type |
| 13 | AIATTEST-13 | DONE | AIATTEST-09 | OCI Guild | Register application/vnd.stellaops.ai.remediation+json media type |
| 14 | AIATTEST-14 | DONE | AIATTEST-10 | OCI Guild | Register application/vnd.stellaops.ai.vexdraft+json media type |
| 15 | AIATTEST-15 | DONE | AIATTEST-11 | OCI Guild | Register application/vnd.stellaops.ai.policydraft+json media type |
| 16 | AIATTEST-16 | TODO | AIATTEST-12 | ExportCenter Guild | Implement AI attestation push via OciReferrerPushClient |
| 17 | AIATTEST-17 | TODO | AIATTEST-16 | ExportCenter Guild | Implement AI attestation discovery via OciReferrerDiscovery |
| 18 | AIATTEST-18 | DONE | AIATTEST-01 | Replay Guild | Create AIArtifactReplayManifest capturing all inputs for deterministic replay |
| 19 | AIATTEST-19 | DONE | AIATTEST-18 | Replay Guild | Implement IAIArtifactReplayer for re-executing AI generation with pinned inputs |
| 20 | AIATTEST-20 | DONE | AIATTEST-19 | Replay Guild | Replay verification: compare output hash with original, flag divergence |
| 21 | AIATTEST-21 | TODO | AIATTEST-20 | Verification Guild | Add AI artifact verification to VerificationPipeline |
| 22 | AIATTEST-22 | DONE | All above | Testing Guild | Integration tests: attestation creation, OCI push/pull, replay verification |
| 23 | AIATTEST-23 | TODO | All above | Docs Guild | Document AI attestation schemas, replay semantics, authority classification |
Execution Log
| Date (UTC) | Update | Owner |
|---|---|---|
| 2025-12-26 | Sprint created from AI Assistant Advisory analysis; extends ProofChain with AI-specific attestation types. | Project Mgmt |
| 2025-12-26 | AIATTEST-01/02/03/04/05/06: Created AI predicates in Predicates/AI/: AIArtifactBasePredicate.cs, AIExplanationPredicate.cs, AIRemediationPlanPredicate.cs, AIVexDraftPredicate.cs, AIPolicyDraftPredicate.cs |
Claude |
| 2025-12-26 | AIATTEST-07: Created AIAuthorityClassifier.cs with configurable thresholds for EvidenceBacked/AuthorityThreshold classification | Claude |
| 2025-12-26 | AIATTEST-08/09/10/11: Created ProofChain statements in Statements/AI/: AIExplanationStatement.cs, AIRemediationPlanStatement.cs, AIVexDraftStatement.cs, AIPolicyDraftStatement.cs |
Claude |
| 2025-12-26 | AIATTEST-12/13/14/15: Created AIArtifactMediaTypes.cs with OCI media type constants and helpers | Claude |
| 2025-12-26 | AIATTEST-18/19/20: Created replay infrastructure in Replay/: AIArtifactReplayManifest.cs, IAIArtifactReplayer.cs |
Claude |
| 2025-12-26 | AIATTEST-22: Created AIAuthorityClassifierTests.cs with comprehensive test coverage | Claude |
Decisions & Risks
- Decision needed: Model digest format (SHA-256 of weights, version string, provider+model). Recommend: provider:model:version for cloud, SHA-256 for local.
- Decision needed: Evidence-backed threshold. Recommend: ≥80% citations valid AND all evidence_refs resolvable.
- Risk: Model version drift between attestation and replay. Mitigation: fail replay if model unavailable; document fallback.
- Risk: Large attestation sizes. Mitigation: store evidence refs, not full content; link to evidence locker.
Next Checkpoints
- 2025-12-30 | AIATTEST-07 complete | All predicate types defined |
- 2026-01-03 | AIATTEST-17 complete | OCI integration working |
- 2026-01-06 | AIATTEST-23 complete | Full documentation and replay verification |