Add Ruby language analyzer and related functionality
- Introduced global usings for Ruby analyzer. - Implemented RubyLockData, RubyLockEntry, and RubyLockParser for handling Gemfile.lock files. - Created RubyPackage and RubyPackageCollector to manage Ruby packages and vendor cache. - Developed RubyAnalyzerPlugin and RubyLanguageAnalyzer for analyzing Ruby projects. - Added tests for Ruby language analyzer with sample Gemfile.lock and expected output. - Included necessary project files and references for the Ruby analyzer. - Added third-party licenses for tree-sitter dependencies.
This commit is contained in:
master
@@ -423,6 +423,61 @@ curl -u orch-admin:s3cr3t! \
|
||||
|
||||
CLI clients configure these values via `Authority.BackfillReason` / `Authority.BackfillTicket` (environment variables `STELLAOPS_ORCH_BACKFILL_REASON` and `STELLAOPS_ORCH_BACKFILL_TICKET`). Tokens missing either field are rejected with `invalid_request`; audit events store the supplied values as `backfill.reason` and `backfill.ticket`.
|
||||
|
||||
### 7.4 Delegated service accounts
|
||||
|
||||
StellaOps Authority issues short-lived delegated tokens for service accounts so automation can operate on behalf of a tenant without sharing the underlying client identity.
|
||||
|
||||
**Configuration summary**
|
||||
|
||||
```yaml
|
||||
delegation:
|
||||
quotas:
|
||||
maxActiveTokens: 50
|
||||
serviceAccounts:
|
||||
- accountId: "svc-observer"
|
||||
tenant: "tenant-default"
|
||||
displayName: "Observability Exporter"
|
||||
description: "Delegated identity used by Export Center to read findings."
|
||||
enabled: true
|
||||
allowedScopes: [ "jobs:read", "findings:read" ]
|
||||
authorizedClients: [ "export-center-worker" ]
|
||||
|
||||
tenants:
|
||||
- name: "tenant-default"
|
||||
delegation:
|
||||
maxActiveTokens: 25
|
||||
```
|
||||
|
||||
* `delegation.quotas.maxActiveTokens` caps concurrent delegated tokens per tenant. Authority enforces both a tenant-wide ceiling and a per-account ceiling (the same value by default).
|
||||
* `serviceAccounts[].allowedScopes` lists scopes that the delegate may request. Requests for scopes outside this set return `invalid_scope`.
|
||||
* `serviceAccounts[].authorizedClients` restricts which OAuth clients may assume the delegate. Leave empty to allow any tenant client.
|
||||
* `tenants[].delegation.maxActiveTokens` optionally overrides the quota for a specific tenant.
|
||||
|
||||
**Requesting a delegated token**
|
||||
|
||||
```bash
|
||||
curl -u export-center-worker:s3cr3t \
|
||||
-d 'grant_type=client_credentials' \
|
||||
-d 'scope=jobs:read findings:read' \
|
||||
-d 'service_account=svc-observer' \
|
||||
https://authority.example.com/token
|
||||
```
|
||||
|
||||
Optional `delegation_actor` metadata appends an identity to the actor chain:
|
||||
|
||||
```bash
|
||||
-d 'delegation_actor=pipeline://exporter/step/42'
|
||||
```
|
||||
|
||||
**Token shape & observability**
|
||||
|
||||
* Access tokens include `stellaops:service_account` and an `act` claim describing the caller hierarchy (`client_id` ⇒ optional `delegation_actor`).
|
||||
* `authority_tokens` records `tokenKind = "service_account"`, the `serviceAccountId`, and the normalized `actorChain[]`.
|
||||
* Audit events (`authority.client_credentials.grant`) emit `delegation.service_account`, `delegation.actor`, and quota outcomes (`delegation.quota.exceeded = true/false`).
|
||||
* When quota limits are exceeded Authority returns `invalid_request` (`Delegation token quota exceeded for tenant/service account`) and annotates the audit log.
|
||||
|
||||
Delegated tokens still honour scope validation, tenant enforcement, sender constraints (DPoP/mTLS), and fresh-auth checks.
|
||||
|
||||
## 8. Offline & Sovereign Operation
|
||||
- **No outbound dependencies:** Authority only contacts MongoDB and local plugins. Discovery and JWKS are cached by clients with offline tolerances (`AllowOfflineCacheFallback`, `OfflineCacheTolerance`). Operators should mirror these responses for air-gapped use.
|
||||
- **Structured logging:** Every revocation export, signing rotation, bootstrap action, and token issuance emits structured logs with `traceId`, `client_id`, `subjectId`, and `network.remoteIp` where applicable. Mirror logs to your SIEM to retain audit trails without central connectivity.
|
||||
|
||||
@@ -29,7 +29,7 @@
|
||||
| DOCS-SCANNER-BENCH-62-004 | TODO | Docs Guild, Java Analyzer Guild | DOCS-SCANNER-BENCH-62-003 | Document Java lockfile ingestion plan and associated policy templates per `scanning-gaps-stella-misses-from-competitors.md`. | Draft guidance published; policy examples reviewed. |
|
||||
| DOCS-SCANNER-BENCH-62-005 | TODO | Docs Guild, Go Analyzer Guild | DOCS-SCANNER-BENCH-62-004 | Document Go stripped-binary fallback enrichment guidance once implementation lands. | Docs updated with inferred module policy patterns. |
|
||||
| DOCS-SCANNER-BENCH-62-006 | TODO | Docs Guild, Rust Analyzer Guild | DOCS-SCANNER-BENCH-62-005 | Document Rust fingerprint enrichment guidance and policy examples. | Docs cover heuristic vs authoritative crate handling. |
|
||||
| DOCS-SCANNER-BENCH-62-007 | TODO | Docs Guild, Security Guild | DOCS-SCANNER-BENCH-62-006 | Produce secret leak detection documentation (rules, policy templates) once implementation lands. | Docs include rule bundle guidance and policy patterns. |
|
||||
| DOCS-SCANNER-BENCH-62-007 | DOING (2025-11-02) | Docs Guild, Security Guild | DOCS-SCANNER-BENCH-62-006 | Produce secret leak detection documentation (rules, policy templates) once implementation lands. | Docs include rule bundle guidance and policy patterns. |
|
||||
| DOCS-SCANNER-BENCH-62-008 | TODO | Docs Guild, EntryTrace Guild | DOCS-SCANNER-BENCH-62-007 | Publish EntryTrace explain/heuristic maintenance guide per `scanning-gaps-stella-misses-from-competitors.md`. | Guide covers cadence, contribution workflow, and policy predicates. |
|
||||
| DOCS-SCANNER-BENCH-62-009 | DONE (2025-11-02) | Docs Guild, Ruby Analyzer Guild | DOCS-SCANNER-BENCH-62-008 | Extend Ruby ecosystem gap analysis in `scanning-gaps-stella-misses-from-competitors.md` with implementation notes, detection tables, and backlog mapping. | Ruby section updated with competitor techniques, task linkage, and scoring rationale. |
|
||||
| DOCS-SCANNER-BENCH-62-010 | DONE (2025-11-02) | Docs Guild, PHP Analyzer Guild | DOCS-SCANNER-BENCH-62-009 | Document PHP analyzer parity gaps with detection technique tables and policy hooks. | PHP section merged with plan references and backlog linkage. |
|
||||
|
||||
@@ -86,6 +86,23 @@ Follow the sprint files below in order. Update task status in both `SPRINTS` and
|
||||
> 2025-11-02: DOCS-SCANNER-BENCH-62-013 marked DONE (Docs Guild, Swift Analyzer Guild) – Swift analyzer roadmap captured with policy hooks.
|
||||
> 2025-11-02: DOCS-SCANNER-BENCH-62-014 marked DONE (Docs Guild, Runtime Guild) – Kubernetes/VM alignment section published.
|
||||
> 2025-11-02: DOCS-SCANNER-BENCH-62-015 marked DONE (Docs Guild, Export Center Guild) – DSSE/Rekor enablement guidance appended to gap doc.
|
||||
> 2025-11-02: SCANNER-ENG-0009 moved to DOING (Ruby Analyzer Guild) – drafting Ruby analyzer parity design package.
|
||||
> 2025-11-02: SCANNER-ENG-0016 added (Ruby Analyzer Guild) – implementing Ruby lock collector & vendor cache ingestion.
|
||||
> 2025-11-02: SCANNER-ENG-0016 moved to DOING (Ruby Analyzer Guild) – lockfile parser skeleton committed with initial Gemfile.lock parsing.
|
||||
> 2025-11-02: SCANNER-ENG-0017 added (Ruby Analyzer Guild) – building runtime require/autoload graph builder.
|
||||
> 2025-11-02: SCANNER-ENG-0018 added (Ruby Analyzer Guild) – emitting Ruby capability and framework signals.
|
||||
> 2025-11-02: SCANNER-ENG-0019 added (Ruby Analyzer Guild, CLI Guild) – delivering Ruby CLI verbs and Offline Kit packaging.
|
||||
> 2025-11-02: SCANNER-LIC-0001 added (Scanner Guild, Legal Guild) – vetting tree-sitter Ruby licensing/offline packaging.
|
||||
> 2025-11-02: SCANNER-LIC-0001 moved to DOING (Scanner Guild, Legal Guild) – SPDX review in progress.
|
||||
> 2025-11-02: SCANNER-POLICY-0001 added (Policy Guild, Ruby Analyzer Guild) – defining Ruby capability predicates in Policy Engine.
|
||||
> 2025-11-02: SCANNER-CLI-0001 added (CLI Guild, Ruby Analyzer Guild) – coordinating CLI UX/docs for Ruby verbs.
|
||||
> 2025-11-02: AIAI-31-011 moved to DOING (Advisory AI Guild) – implementing Excititor VEX document provider.
|
||||
> 2025-11-02: AIAI-31-011 marked DONE (Advisory AI Guild) – Excititor VEX provider + OpenVEX chunking shipped with tests.
|
||||
> 2025-11-02: AIAI-31-002 moved to DOING (Advisory AI Guild, SBOM Service Guild) – building SBOM context retriever for timelines/paths/blast radius.
|
||||
> 2025-11-02: AIAI-31-002 progressing – SBOM context models/tests landed; awaiting SBOM guild client hookup.
|
||||
|
||||
> 2025-11-02: AIAI-31-003 moved to DOING – kicking off deterministic tooling (comparators, dependency lookup). First drop covers semver range evaluator + RPM EVR comparator.
|
||||
|
||||
> 2025-11-02: AIAI-31-004 moved to DOING – starting deterministic orchestration pipeline (summary/conflict/remediation flow).
|
||||
|
||||
> 2025-11-02: ISSUER-30-006 moved to DOING (Issuer Directory Guild, DevOps Guild) – deployment manifests, backup/restore, secret handling, and offline kit docs in progress.
|
||||
|
||||
@@ -83,6 +83,7 @@ AUTH-POLICY-27-002 | DONE (2025-11-02) | Provide attestation signing service bin
|
||||
> 2025-11-02: Added interactive-only `policy:publish`/`policy:promote` scopes with metadata requirements (`policy_reason`, `policy_ticket`, `policy_digest`), fresh-auth validation, audit enrichment, and updated config/docs for operators.
|
||||
AUTH-POLICY-27-003 | DOING (2025-11-02) | Update Authority configuration/docs for Policy Studio roles, signing policies, approval workflows, and CLI integration; include compliance checklist. Dependencies: AUTH-POLICY-27-001, AUTH-POLICY-27-002. | Authority Core & Docs Guild (src/Authority/StellaOps.Authority/TASKS.md)
|
||||
AUTH-TEN-49-001 | DOING (2025-11-02) | Implement service accounts & delegation tokens (`act` chain), per-tenant quotas, audit stream of auth decisions, and revocation APIs. Dependencies: AUTH-TEN-47-001. | Authority Core & Security Guild (src/Authority/StellaOps.Authority/TASKS.md)
|
||||
> 2025-11-02: Service account store + configuration wired, delegation quotas enforced, token persistence extended with `serviceAccountId`/`tokenKind`/`actorChain`, docs & samples refreshed, and new tests cover delegated issuance/persistence.
|
||||
AUTH-VULN-29-001 | TODO | Define Vuln Explorer scopes/roles (`vuln:view`, `vuln:investigate`, `vuln:operate`, `vuln:audit`) with ABAC attributes (env, owner, business_tier) and update discovery metadata/offline kit defaults. Dependencies: AUTH-POLICY-27-001. | Authority Core & Security Guild (src/Authority/StellaOps.Authority/TASKS.md)
|
||||
AUTH-VULN-29-002 | TODO | Enforce CSRF/anti-forgery tokens for workflow actions, sign attachment tokens, and record audit logs with ledger event hashes. Dependencies: AUTH-VULN-29-001, LEDGER-29-002. | Authority Core & Security Guild (src/Authority/StellaOps.Authority/TASKS.md)
|
||||
AUTH-VULN-29-003 | TODO | Update security docs/config samples for Vuln Explorer roles, ABAC policies, attachment signing, and ledger verification guidance. Dependencies: AUTH-VULN-29-001..002. | Authority Core & Docs Guild (src/Authority/StellaOps.Authority/TASKS.md)
|
||||
@@ -103,7 +104,7 @@ ISSUER-30-002 | DONE (2025-11-01) | Implement key management endpoints (add/rota
|
||||
ISSUER-30-003 | DOING | Provide trust weight APIs and tenant overrides with validation (+/- bounds) and audit trails. Dependencies: ISSUER-30-001. | Issuer Directory Guild, Policy Guild (src/IssuerDirectory/StellaOps.IssuerDirectory/TASKS.md)
|
||||
ISSUER-30-004 | DONE (2025-11-01) | Integrate with VEX Lens and Excitator signature verification (client SDK, caching, retries). Dependencies: ISSUER-30-001..003. | Issuer Directory Guild, VEX Lens Guild (src/IssuerDirectory/StellaOps.IssuerDirectory/TASKS.md)
|
||||
ISSUER-30-005 | DONE (2025-11-01) | Instrument metrics/logs (issuer changes, key rotation, verification failures) and dashboards/alerts. Dependencies: ISSUER-30-001..004. | Issuer Directory Guild, Observability Guild (src/IssuerDirectory/StellaOps.IssuerDirectory/TASKS.md)
|
||||
ISSUER-30-006 | TODO | Provide deployment manifests, backup/restore, secure secret storage, and offline kit instructions. Dependencies: ISSUER-30-001..005. | Issuer Directory Guild, DevOps Guild (src/IssuerDirectory/StellaOps.IssuerDirectory/TASKS.md)
|
||||
ISSUER-30-006 | DOING (2025-11-02) | Provide deployment manifests, backup/restore, secure secret storage, and offline kit instructions. Dependencies: ISSUER-30-001..005. | Issuer Directory Guild, DevOps Guild (src/IssuerDirectory/StellaOps.IssuerDirectory/TASKS.md)
|
||||
|
||||
|
||||
[Identity & Signing] 100.D) __Libraries
|
||||
|
||||
@@ -7,8 +7,11 @@ Task ID | State | Task description | Owners (Source)
|
||||
--- | --- | --- | ---
|
||||
AIAI-31-001 | DONE (2025-11-02) | Implement structured and vector retrievers for advisories/VEX with paragraph anchors and citation metadata. Dependencies: CONCELIER-VULN-29-001, EXCITITOR-VULN-29-001. | Advisory AI Guild (src/AdvisoryAI/StellaOps.AdvisoryAI/TASKS.md)
|
||||
AIAI-31-002 | DOING | Build SBOM context retriever (purl version timelines, dependency paths, env flags, blast radius estimator). Dependencies: SBOM-VULN-29-001. | Advisory AI Guild, SBOM Service Guild (src/AdvisoryAI/StellaOps.AdvisoryAI/TASKS.md)
|
||||
AIAI-31-003 | TODO | Implement deterministic toolset (version comparators, range checks, dependency analysis, policy lookup) exposed via orchestrator. Dependencies: AIAI-31-001..002. | Advisory AI Guild (src/AdvisoryAI/StellaOps.AdvisoryAI/TASKS.md)
|
||||
AIAI-31-004 | TODO | Build orchestration pipeline for Summary/Conflict/Remediation tasks (prompt templates, tool calls, token budgets, caching). Dependencies: AIAI-31-001..003, AUTH-VULN-29-001. | Advisory AI Guild (src/AdvisoryAI/StellaOps.AdvisoryAI/TASKS.md)
|
||||
AIAI-31-003 | DOING | Implement deterministic toolset (version comparators, range checks, dependency analysis, policy lookup) exposed via orchestrator. Dependencies: AIAI-31-001..002. | Advisory AI Guild (src/AdvisoryAI/StellaOps.AdvisoryAI/TASKS.md)
|
||||
AIAI-31-004 | DOING | Build orchestration pipeline for Summary/Conflict/Remediation tasks (prompt templates, tool calls, token budgets, caching). Dependencies: AIAI-31-001..003, AUTH-VULN-29-001. | Advisory AI Guild (src/AdvisoryAI/StellaOps.AdvisoryAI/TASKS.md)
|
||||
AIAI-31-004A | TODO | Wire orchestrator into WebService/Worker, expose API + queue contract, emit metrics, stub cache. Dependencies: AIAI-31-004, AIAI-31-002. | Advisory AI Guild, Platform Guild (src/AdvisoryAI/StellaOps.AdvisoryAI/TASKS.md)
|
||||
AIAI-31-004B | TODO | Implement prompt assembler, guardrails, cache persistence, DSSE provenance, golden outputs. Dependencies: AIAI-31-004A, DOCS-AIAI-31-003, AUTH-AIAI-31-004. | Advisory AI Guild, Security Guild (src/AdvisoryAI/StellaOps.AdvisoryAI/TASKS.md)
|
||||
AIAI-31-004C | TODO | Deliver CLI `stella advise run` command, renderer, docs, CLI golden tests. Dependencies: AIAI-31-004B, CLI-AIAI-31-003. | Advisory AI Guild, CLI Guild (src/AdvisoryAI/StellaOps.AdvisoryAI/TASKS.md)
|
||||
AIAI-31-005 | TODO | Implement guardrails (redaction, injection defense, output validation, citation enforcement) and fail-safe handling. Dependencies: AIAI-31-004. | Advisory AI Guild, Security Guild (src/AdvisoryAI/StellaOps.AdvisoryAI/TASKS.md)
|
||||
AIAI-31-006 | TODO | Expose REST API endpoints (`/advisory/ai/*`) with RBAC, rate limits, OpenAPI schemas, and batching support. Dependencies: AIAI-31-004..005. | Advisory AI Guild (src/AdvisoryAI/StellaOps.AdvisoryAI/TASKS.md)
|
||||
AIAI-31-007 | TODO | Instrument metrics (`advisory_ai_latency`, `guardrail_blocks`, `validation_failures`, `citation_coverage`), logs, and traces; publish dashboards/alerts. Dependencies: AIAI-31-004..006. | Advisory AI Guild, Observability Guild (src/AdvisoryAI/StellaOps.AdvisoryAI/TASKS.md)
|
||||
@@ -17,6 +20,11 @@ AIAI-31-010 | DONE (2025-11-02) | Implement Concelier advisory raw document prov
|
||||
AIAI-31-011 | DONE (2025-11-02) | Implement Excititor VEX document provider to surface structured VEX statements for retrieval. Dependencies: EXCITITOR-LNM-21-201, EXCITITOR-CORE-AOC-19-002. | Advisory AI Guild (src/AdvisoryAI/StellaOps.AdvisoryAI/TASKS.md)
|
||||
AIAI-31-009 | TODO | Develop unit/golden/property/perf tests, injection harness, and regression suite; ensure determinism with seeded caches. Dependencies: AIAI-31-001..006. | Advisory AI Guild, QA Guild (src/AdvisoryAI/StellaOps.AdvisoryAI/TASKS.md)
|
||||
|
||||
|
||||
|
||||
> 2025-11-02: AIAI-31-004 kicked off orchestration pipeline design – establishing deterministic task sequence (summary/conflict/remediation) and cache key strategy.
|
||||
> 2025-11-02: AIAI-31-004 orchestration prerequisites documented in docs/modules/advisory-ai/orchestration-pipeline.md (tasks 004A/004B/004C).
|
||||
> 2025-11-02: AIAI-31-003 moved to DOING – beginning deterministic tooling (comparators, dependency analysis) while awaiting SBOM context client. Semantic & EVR comparators shipped; toolset interface published for orchestrator adoption.
|
||||
> 2025-11-02: Structured + vector retrievers landed with deterministic CSAF/OSV/Markdown chunkers, deterministic hash embeddings, and unit coverage for sample advisories.
|
||||
> 2025-11-02: SBOM context request/result models finalized; retriever tests now validate environment-flag toggles and dependency-path dedupe. SBOM guild to wire real context service client.
|
||||
|
||||
|
||||
@@ -211,7 +211,7 @@ DOCS-SCANNER-BENCH-62-003 | TODO | Capture Python lockfile/editable install requ
|
||||
DOCS-SCANNER-BENCH-62-004 | TODO | Document Java lockfile ingestion guidance and policy templates. | Docs Guild, Java Analyzer Guild (docs/TASKS.md)
|
||||
DOCS-SCANNER-BENCH-62-005 | TODO | Document Go stripped-binary fallback enrichment guidance once implementation lands. | Docs Guild, Go Analyzer Guild (docs/TASKS.md)
|
||||
DOCS-SCANNER-BENCH-62-006 | TODO | Document Rust fingerprint enrichment guidance and policy examples. | Docs Guild, Rust Analyzer Guild (docs/TASKS.md)
|
||||
DOCS-SCANNER-BENCH-62-007 | TODO | Produce secret leak detection documentation (rules, policy templates). | Docs Guild, Security Guild (docs/TASKS.md)
|
||||
DOCS-SCANNER-BENCH-62-007 | DOING (2025-11-02) | Produce secret leak detection documentation (rules, policy templates). | Docs Guild, Security Guild (docs/TASKS.md)
|
||||
DOCS-SCANNER-BENCH-62-008 | TODO | Publish EntryTrace explain/heuristic maintenance guide. | Docs Guild, EntryTrace Guild (docs/TASKS.md)
|
||||
DOCS-SCANNER-BENCH-62-009 | TODO | Produce SAST integration documentation (connector framework, policy templates). | Docs Guild, Policy Guild (docs/TASKS.md)
|
||||
DOCS-TEN-47-001 | TODO | Publish `/docs/security/tenancy-overview.md` and `/docs/security/scopes-and-roles.md` outlining scope grammar, tenant model, imposed rule reminder. | Docs Guild, Authority Core (docs/TASKS.md)
|
||||
|
||||
@@ -1,100 +1,115 @@
|
||||
# Advisory AI architecture
|
||||
|
||||
> Captures the retrieval, guardrail, and inference packaging requirements defined in the Advisory AI implementation plan and related module guides.
|
||||
|
||||
## 1) Goals
|
||||
|
||||
- Summarise advisories/VEX evidence into operator-ready briefs with citations.
|
||||
- Explain conflicting statements with provenance and trust weights (using VEX Lens & Excititor data).
|
||||
- Suggest remediation plans aligned with Offline Kit deployment models and scheduler follow-ups.
|
||||
- Operate deterministically where possible; cache generated artefacts with digests for audit.
|
||||
|
||||
## 2) Pipeline overview
|
||||
|
||||
```
|
||||
+---------------------+
|
||||
Concelier/VEX Lens | Evidence Retriever |
|
||||
Policy Engine ----> | (vector + keyword) | ---> Context Pack (JSON)
|
||||
Zastava runtime +---------------------+
|
||||
|
|
||||
v
|
||||
+-------------+
|
||||
| Prompt |
|
||||
| Assembler |
|
||||
+-------------+
|
||||
|
|
||||
v
|
||||
+-------------+
|
||||
| Guarded LLM |
|
||||
| (local/host)|
|
||||
+-------------+
|
||||
|
|
||||
v
|
||||
+-----------------+
|
||||
| Citation & |
|
||||
| Validation |
|
||||
+-----------------+
|
||||
|
|
||||
v
|
||||
+----------------+
|
||||
| Output cache |
|
||||
| (hash, bundle) |
|
||||
+----------------+
|
||||
```
|
||||
|
||||
## 3) Retrieval & context
|
||||
|
||||
- Hybrid search: vector embeddings (SBERT-compatible) + keyword filters for advisory IDs, PURLs, CVEs.
|
||||
- Context packs include:
|
||||
- Advisory raw excerpts with highlighted sections and source URLs.
|
||||
- VEX statements (normalized tuples + trust metadata).
|
||||
- Policy explain traces for the affected finding.
|
||||
- Runtime/impact hints from Zastava (exposure, entrypoints).
|
||||
- Export-ready remediation data (fixed versions, patches).
|
||||
|
||||
All context references include `content_hash` and `source_id` enabling verifiable citations.
|
||||
|
||||
## 4) Guardrails
|
||||
|
||||
- Prompt templates enforce structure: summary, conflicts, remediation, references.
|
||||
- Response validator ensures:
|
||||
- No hallucinated advisories (every fact must map to input context).
|
||||
- Citations follow `[n]` indexing referencing actual sources.
|
||||
- Remediation suggestions only cite policy-approved sources (fixed versions, vendor hotfixes).
|
||||
- Moderation/PII filters prevent leaking secrets; responses failing validation are rejected and logged.
|
||||
|
||||
## 5) Output persistence
|
||||
|
||||
- Cached artefacts stored in `advisory_ai_outputs` with fields:
|
||||
- `output_hash` (sha256 of JSON response).
|
||||
- `input_digest` (hash of context pack).
|
||||
- `summary`, `conflicts`, `remediation`, `citations`.
|
||||
- `generated_at`, `model_id`, `profile` (Sovereign/FIPS etc.).
|
||||
- `signatures` (optional DSSE if run in deterministic mode).
|
||||
- Offline bundle format contains `summary.md`, `citations.json`, `context_manifest.json`, `signatures/`.
|
||||
|
||||
## 6) Profiles & sovereignty
|
||||
|
||||
- **Profiles:** `default`, `fips-local` (FIPS-compliant local model), `gost-local`, `cloud-openai` (optional, disabled by default). Each profile defines allowed models, key management, and telemetry endpoints.
|
||||
- **CryptoProfile/RootPack integration:** generated artefacts can be signed using configured CryptoProfile to satisfy procurement/trust requirements.
|
||||
|
||||
## 7) APIs
|
||||
|
||||
- `POST /v1/advisory-ai/summaries` — generate (or retrieve cached) summary for `{advisoryKey, artifactId, policyVersion}`.
|
||||
- `POST /v1/advisory-ai/conflicts` — explain conflicting VEX statements with trust ranking.
|
||||
- `POST /v1/advisory-ai/remediation` — fetch remediation plan with target fix versions, prerequisites, verification steps.
|
||||
- `GET /v1/advisory-ai/outputs/{hash}` — retrieve cached artefact (used by CLI/Console/Export Center).
|
||||
|
||||
All endpoints accept `profile` parameter (default `fips-local`) and return `output_hash`, `input_digest`, and `citations` for verification.
|
||||
|
||||
## 8) Observability
|
||||
|
||||
- Metrics: `advisory_ai_requests_total{profile,type}`, `advisory_ai_latency_seconds`, `advisory_ai_validation_failures_total`.
|
||||
- Logs: include `output_hash`, `input_digest`, `profile`, `model_id`, `tenant`, `artifacts`. Sensitive context is not logged.
|
||||
- Traces: spans for retrieval, prompt assembly, model inference, validation, cache write.
|
||||
|
||||
## 9) Operational controls
|
||||
|
||||
- Feature flags per tenant (`ai.summary.enabled`, `ai.remediation.enabled`).
|
||||
- Rate limits (per tenant, per profile) enforced by Orchestrator to prevent runaway usage.
|
||||
- Offline/air-gapped deployments run local models packaged with Offline Kit; model weights validated via manifest digests.
|
||||
# Advisory AI architecture
|
||||
|
||||
> Captures the retrieval, guardrail, and inference packaging requirements defined in the Advisory AI implementation plan and related module guides.
|
||||
|
||||
## 1) Goals
|
||||
|
||||
- Summarise advisories/VEX evidence into operator-ready briefs with citations.
|
||||
- Explain conflicting statements with provenance and trust weights (using VEX Lens & Excititor data).
|
||||
- Suggest remediation plans aligned with Offline Kit deployment models and scheduler follow-ups.
|
||||
- Operate deterministically where possible; cache generated artefacts with digests for audit.
|
||||
|
||||
## 2) Pipeline overview
|
||||
|
||||
```
|
||||
+---------------------+
|
||||
Concelier/VEX Lens | Evidence Retriever |
|
||||
Policy Engine ----> | (vector + keyword) | ---> Context Pack (JSON)
|
||||
Zastava runtime +---------------------+
|
||||
|
|
||||
v
|
||||
+-------------+
|
||||
| Prompt |
|
||||
| Assembler |
|
||||
+-------------+
|
||||
|
|
||||
v
|
||||
+-------------+
|
||||
| Guarded LLM |
|
||||
| (local/host)|
|
||||
+-------------+
|
||||
|
|
||||
v
|
||||
+-----------------+
|
||||
| Citation & |
|
||||
| Validation |
|
||||
+-----------------+
|
||||
|
|
||||
v
|
||||
+----------------+
|
||||
| Output cache |
|
||||
| (hash, bundle) |
|
||||
+----------------+
|
||||
```
|
||||
|
||||
## 3) Retrieval & context
|
||||
|
||||
- Hybrid search: vector embeddings (SBERT-compatible) + keyword filters for advisory IDs, PURLs, CVEs.
|
||||
- Context packs include:
|
||||
- Advisory raw excerpts with highlighted sections and source URLs.
|
||||
- VEX statements (normalized tuples + trust metadata).
|
||||
- Policy explain traces for the affected finding.
|
||||
- Runtime/impact hints from Zastava (exposure, entrypoints).
|
||||
- Export-ready remediation data (fixed versions, patches).
|
||||
- **SBOM context retriever** (AIAI-31-002) hydrates:
|
||||
- Version timelines (first/last observed, status, fix availability).
|
||||
- Dependency paths (runtime vs build/test, deduped by coordinate chain).
|
||||
- Tenant environment flags (prod/stage toggles) with optional blast radius summary.
|
||||
- Service-side clamps: max 500 timeline entries, 200 dependency paths, with client-provided toggles for env/blast data.
|
||||
|
||||
Retriever requests and results are trimmed/normalized before hashing; metadata (counts, provenance keys) is returned for downstream guardrails. Unit coverage ensures deterministic ordering and flag handling.
|
||||
|
||||
All context references include `content_hash` and `source_id` enabling verifiable citations.
|
||||
|
||||
## 4) Guardrails
|
||||
|
||||
- Prompt templates enforce structure: summary, conflicts, remediation, references.
|
||||
- Response validator ensures:
|
||||
- No hallucinated advisories (every fact must map to input context).
|
||||
- Citations follow `[n]` indexing referencing actual sources.
|
||||
- Remediation suggestions only cite policy-approved sources (fixed versions, vendor hotfixes).
|
||||
- Moderation/PII filters prevent leaking secrets; responses failing validation are rejected and logged.
|
||||
|
||||
## 5) Deterministic tooling
|
||||
|
||||
- **Version comparators** — offline semantic version + RPM EVR parsers with range evaluators. Supports chained constraints (`>=`, `<=`, `!=`) used by remediation advice and blast radius calcs.
|
||||
- Registered via `AddAdvisoryDeterministicToolset` for reuse across orchestrator, CLI, and services.
|
||||
- **Orchestration pipeline** — see `orchestration-pipeline.md` for prerequisites, task breakdown, and cross-guild responsibilities before wiring the execution flows.
|
||||
- **Planned extensions** — NEVRA/EVR comparators, ecosystem-specific normalisers, dependency chain scorers (AIAI-31-003 scope).
|
||||
- Exposed via internal interfaces to allow orchestrator/toolchain reuse; all helpers stay side-effect free and deterministic for golden testing.
|
||||
|
||||
## 6) Output persistence
|
||||
|
||||
- Cached artefacts stored in `advisory_ai_outputs` with fields:
|
||||
- `output_hash` (sha256 of JSON response).
|
||||
- `input_digest` (hash of context pack).
|
||||
- `summary`, `conflicts`, `remediation`, `citations`.
|
||||
- `generated_at`, `model_id`, `profile` (Sovereign/FIPS etc.).
|
||||
- `signatures` (optional DSSE if run in deterministic mode).
|
||||
- Offline bundle format contains `summary.md`, `citations.json`, `context_manifest.json`, `signatures/`.
|
||||
|
||||
## 7) Profiles & sovereignty
|
||||
|
||||
- **Profiles:** `default`, `fips-local` (FIPS-compliant local model), `gost-local`, `cloud-openai` (optional, disabled by default). Each profile defines allowed models, key management, and telemetry endpoints.
|
||||
- **CryptoProfile/RootPack integration:** generated artefacts can be signed using configured CryptoProfile to satisfy procurement/trust requirements.
|
||||
|
||||
## 8) APIs
|
||||
|
||||
- `POST /v1/advisory-ai/summaries` — generate (or retrieve cached) summary for `{advisoryKey, artifactId, policyVersion}`.
|
||||
- `POST /v1/advisory-ai/conflicts` — explain conflicting VEX statements with trust ranking.
|
||||
- `POST /v1/advisory-ai/remediation` — fetch remediation plan with target fix versions, prerequisites, verification steps.
|
||||
- `GET /v1/advisory-ai/outputs/{hash}` — retrieve cached artefact (used by CLI/Console/Export Center).
|
||||
|
||||
All endpoints accept `profile` parameter (default `fips-local`) and return `output_hash`, `input_digest`, and `citations` for verification.
|
||||
|
||||
## 9) Observability
|
||||
|
||||
- Metrics: `advisory_ai_requests_total{profile,type}`, `advisory_ai_latency_seconds`, `advisory_ai_validation_failures_total`.
|
||||
- Logs: include `output_hash`, `input_digest`, `profile`, `model_id`, `tenant`, `artifacts`. Sensitive context is not logged.
|
||||
- Traces: spans for retrieval, prompt assembly, model inference, validation, cache write.
|
||||
|
||||
## 10) Operational controls
|
||||
|
||||
- Feature flags per tenant (`ai.summary.enabled`, `ai.remediation.enabled`).
|
||||
- Rate limits (per tenant, per profile) enforced by Orchestrator to prevent runaway usage.
|
||||
- Offline/air-gapped deployments run local models packaged with Offline Kit; model weights validated via manifest digests.
|
||||
|
||||
82
docs/modules/advisory-ai/orchestration-pipeline.md
Normal file
82
docs/modules/advisory-ai/orchestration-pipeline.md
Normal file
@@ -0,0 +1,82 @@
|
||||
# Advisory AI Orchestration Pipeline (Planning Notes)
|
||||
|
||||
> **Status:** Draft – prerequisite design for AIAI-31-004 integration work.
|
||||
> **Audience:** Advisory AI guild, WebService/Worker guilds, CLI guild, Docs/QA support teams.
|
||||
|
||||
## 1. Goal
|
||||
|
||||
Wire the deterministic pipeline (Summary / Conflict / Remediation flows) into the Advisory AI service, workers, and CLI with deterministic caching, prompt preparation, and guardrail fallback. This document captures the pre-integration checklist and task breakdown so each guild understands their responsibilities before coding begins.
|
||||
|
||||
## 2. Prerequisites
|
||||
|
||||
| Area | Requirement | Owner | Status |
|
||||
|------|-------------|-------|--------|
|
||||
| **Toolset** | Deterministic comparators, dependency analyzer (`IDeterministicToolset`, `AdvisoryPipelineOrchestrator`) | Advisory AI | ✅ landed (AIAI-31-003) |
|
||||
| **SBOM context** | Real SBOM context client delivering timelines + dependency paths | SBOM Service Guild | ⏳ pending (AIAI-31-002) |
|
||||
| **Prompt artifacts** | Liquid/Handlebars prompt templates for summary/conflict/remediation | Advisory AI Docs Guild | ⏳ authoring needed |
|
||||
| **Cache strategy** | Decision on DSSE or hash-only cache entries, TTLs, and eviction policy | Advisory AI + Platform | 🔲 define |
|
||||
| **Auth scopes** | Confirm service account scopes for new API endpoints/worker-to-service calls | Authority Guild | 🔲 define |
|
||||
|
||||
**Blocking risk:** SBOM client and prompt templates must exist (even stubbed) before the orchestrator can produce stable plans.
|
||||
|
||||
## 3. Integration plan (high-level)
|
||||
|
||||
1. **Service layer (WebService / Worker)**
|
||||
- Inject `IAdvisoryPipelineOrchestrator` via `AddAdvisoryPipeline`.
|
||||
- Define REST endpoint `POST /v1/advisories/{key}/pipeline/{task}` (task ∈ summary/conflict/remediation).
|
||||
- Worker consumes queue messages (`advisory.pipeline.execute`) -> fetches plan -> executes prompt -> persists output & provenance.
|
||||
- Add metrics: `advisory_pipeline_requests_total`, `advisory_pipeline_plan_cache_hits_total`, `advisory_pipeline_latency_seconds`.
|
||||
2. **CLI**
|
||||
- New command `stella advise run <task>` with flags for artifact id, profile, policy version, `--force-refresh`.
|
||||
- Render JSON/Markdown outputs; handle caching hints (print cache key, status).
|
||||
3. **Caching / storage**
|
||||
- Choose storage (Mongo collection vs existing DSSE output store).
|
||||
- Persist `AdvisoryTaskPlan` metadata + generated output keyed by cache key + policy version.
|
||||
- Expose TTL/force-refresh semantics.
|
||||
4. **Docs & QA**
|
||||
- Publish API spec (`docs/advisory-ai/api.md`) + CLI docs.
|
||||
- Add golden outputs for deterministic runs; property tests for cache key stability.
|
||||
|
||||
## 4. Task Breakdown
|
||||
|
||||
### AIAI-31-004A (Service orchestration wiring)
|
||||
|
||||
- **Scope:** WebService/Worker injection, REST/queue plumbing, metrics counters, basic cache stub.
|
||||
- **Dependencies:** `AddAdvisoryPipeline`, SBOM client stub.
|
||||
- **Exit:** API responds with plan metadata + queue message; worker logs execution attempt; metrics emitted.
|
||||
|
||||
### AIAI-31-004B (Prompt assembly & cache persistence)
|
||||
|
||||
- **Scope:** Implement prompt assembler, connect to guardrails, persist cache entries w/ DSSE metadata.
|
||||
- **Dependencies:** Prompt templates, cache storage decision, guardrail interface.
|
||||
- **Exit:** Deterministic outputs stored; force-refresh honoured; tests cover prompt assembly + caching.
|
||||
|
||||
### AIAI-31-004C (CLI integration & docs)
|
||||
|
||||
- **Scope:** CLI command + output renderer, docs updates, CLI tests (golden outputs).
|
||||
- **Dependencies:** Service endpoints stable, caching semantics documented.
|
||||
- **Exit:** CLI command produces deterministic output, docs updated, smoke tests recorded.
|
||||
|
||||
### Supporting tasks (other guilds)
|
||||
|
||||
- **AUTH-AIAI-31-004** – Update scopes and DSSE policy (Authority guild).
|
||||
- **DOCS-AIAI-31-003** – Publish API documentation, CLI guide updates (Docs guild).
|
||||
- **QA-AIAI-31-004** – Golden/properties/perf suite for pipeline (QA guild).
|
||||
|
||||
## 5. Acceptance checklist (per task)
|
||||
|
||||
| Item | Notes |
|
||||
|------|-------|
|
||||
| Cache key stability | `AdvisoryPipelineOrchestrator` hash must remain stable under re-run of identical inputs. |
|
||||
| Metrics & logging | Request id, cache key, task type, profile, latency; guardrail results logged without sensitive prompt data. |
|
||||
| Offline readiness | All prompt templates bundled with Offline Kit; CLI works in air-gapped mode with cached data. |
|
||||
| Policy awareness | Plans encode policy version used; outputs reference policy digest for audit. |
|
||||
| Testing | Unit tests (plan generation, cache keys, DI), integration (service endpoint, worker, CLI), deterministic golden outputs. |
|
||||
|
||||
## 6. Next steps
|
||||
|
||||
1. Finalize SBOM context client (AIAI-31-002) and prompt templates.
|
||||
2. Create queue schema spec (`docs/modules/advisory-ai/queue-contracts.md`) if not already available.
|
||||
3. Schedule cross-guild kickoff to agree on cache store & DSSE policy.
|
||||
|
||||
_Last updated: 2025-11-02_
|
||||
@@ -90,6 +90,11 @@ Payloads follow the contract in `Contracts/IssuerDtos.cs` and align with domain
|
||||
3. **SDK integration (ISSUER-30-004)** — supply cached issuer metadata to VEX Lens and Excititor clients.
|
||||
4. **Observability & Ops (ISSUER-30-005/006)** — metrics, dashboards, deployment automation, offline kit.
|
||||
|
||||
## 9. Operations & runbooks
|
||||
- [Deployment guide](operations/deployment.md)
|
||||
- [Backup & restore](operations/backup-restore.md)
|
||||
- [Offline kit notes](operations/offline-kit.md)
|
||||
|
||||
---
|
||||
|
||||
*Document owner: Issuer Directory Guild*
|
||||
|
||||
103
docs/modules/issuer-directory/operations/backup-restore.md
Normal file
103
docs/modules/issuer-directory/operations/backup-restore.md
Normal file
@@ -0,0 +1,103 @@
|
||||
# Issuer Directory Backup & Restore
|
||||
|
||||
## Scope
|
||||
- **Applies to:** Issuer Directory when deployed via Docker Compose (`deploy/compose/docker-compose.*.yaml`) or the Helm chart (`deploy/helm/stellaops`).
|
||||
- **Artifacts covered:** MongoDB database `issuer-directory`, service configuration (`etc/issuer-directory.yaml`), CSAF seed file (`data/csaf-publishers.json`), and secret material for the Mongo connection string.
|
||||
- **Frequency:** Take a hot backup before every upgrade and at least daily in production. Keep encrypted copies off-site/air-gapped according to your compliance program.
|
||||
|
||||
## Inventory checklist
|
||||
| Component | Location (Compose default) | Notes |
|
||||
| --- | --- | --- |
|
||||
| Mongo data | `mongo-data` volume (`/var/lib/docker/volumes/.../mongo-data`) | Contains `issuers`, `issuer_keys`, `issuer_trust_overrides`, and `issuer_audit` collections. |
|
||||
| Configuration | `etc/issuer-directory.yaml` | Mounted read-only at `/etc/issuer-directory.yaml` inside the container. |
|
||||
| CSAF seed file | `src/IssuerDirectory/StellaOps.IssuerDirectory/data/csaf-publishers.json` | Ensure customised seeds are part of the backup; regenerate if you ship regional overrides. |
|
||||
| Mongo secret | `.env` entry `ISSUER_DIRECTORY_MONGO_CONNECTION_STRING` or secret store export | Required to restore connectivity; treat as sensitive. |
|
||||
|
||||
> **Tip:** Export the secret via `kubectl get secret issuer-directory-secrets -o yaml` (sanitize before storage) or copy the Compose `.env` file into an encrypted vault.
|
||||
|
||||
## Hot backup (no downtime)
|
||||
1. **Create output directory**
|
||||
```bash
|
||||
BACKUP_DIR=backup/issuer-directory/$(date +%Y-%m-%dT%H%M%S)
|
||||
mkdir -p "$BACKUP_DIR"
|
||||
```
|
||||
2. **Dump Mongo collections**
|
||||
```bash
|
||||
docker compose -f deploy/compose/docker-compose.prod.yaml exec mongo \
|
||||
mongodump --archive=/dump/issuer-directory-$(date +%Y%m%dT%H%M%SZ).gz \
|
||||
--gzip --db issuer-directory
|
||||
|
||||
docker compose -f deploy/compose/docker-compose.prod.yaml cp \
|
||||
mongo:/dump/issuer-directory-$(date +%Y%m%dT%H%M%SZ).gz "$BACKUP_DIR/"
|
||||
```
|
||||
For Kubernetes, run the same `mongodump` command inside the `stellaops-mongo` pod and copy the archive via `kubectl cp`.
|
||||
3. **Capture configuration and seeds**
|
||||
```bash
|
||||
cp etc/issuer-directory.yaml "$BACKUP_DIR/"
|
||||
cp src/IssuerDirectory/StellaOps.IssuerDirectory/data/csaf-publishers.json "$BACKUP_DIR/"
|
||||
```
|
||||
4. **Capture secrets**
|
||||
```bash
|
||||
grep '^ISSUER_DIRECTORY_MONGO_CONNECTION_STRING=' dev.env > "$BACKUP_DIR/issuer-directory.mongo.secret"
|
||||
chmod 600 "$BACKUP_DIR/issuer-directory.mongo.secret"
|
||||
```
|
||||
5. **Generate checksums and encrypt**
|
||||
```bash
|
||||
(cd "$BACKUP_DIR" && sha256sum * > SHA256SUMS)
|
||||
tar czf "$BACKUP_DIR.tar.gz" -C "$BACKUP_DIR" .
|
||||
age -r you@example.org "$BACKUP_DIR.tar.gz" > "$BACKUP_DIR.tar.gz.age"
|
||||
```
|
||||
|
||||
## Cold backup (planned downtime)
|
||||
1. Notify stakeholders and pause automation calling the API.
|
||||
2. Stop services:
|
||||
```bash
|
||||
docker compose -f deploy/compose/docker-compose.prod.yaml down issuer-directory
|
||||
```
|
||||
(For Helm: `kubectl scale deploy stellaops-issuer-directory --replicas=0`.)
|
||||
3. Snapshot volumes:
|
||||
```bash
|
||||
docker run --rm -v mongo-data:/data \
|
||||
-v "$(pwd)":/backup busybox tar czf /backup/mongo-data-$(date +%Y%m%d).tar.gz -C /data .
|
||||
```
|
||||
4. Copy configuration, seeds, and secrets as in the hot backup.
|
||||
5. Restart services and confirm `/health/live` returns `200 OK`.
|
||||
|
||||
## Restore procedure
|
||||
1. **Provision clean volumes**
|
||||
- Compose: `docker volume rm mongo-data` (optional) then `docker compose up -d mongo`.
|
||||
- Helm: delete the Mongo PVC or attach a fresh volume snapshot.
|
||||
2. **Restore Mongo**
|
||||
```bash
|
||||
docker compose exec -T mongo \
|
||||
mongorestore --archive \
|
||||
--gzip --drop < issuer-directory-YYYYMMDDTHHMMSSZ.gz
|
||||
```
|
||||
3. **Restore configuration/secrets**
|
||||
- Copy `issuer-directory.yaml` into `etc/`.
|
||||
- Reapply the secret: `kubectl apply -f issuer-directory-secret.yaml` or repopulate `.env`.
|
||||
4. **Restore CSAF seeds** (optional)
|
||||
- If you maintain a customised seed file, copy it back before starting the container. Otherwise the bundled file will be used.
|
||||
5. **Start services**
|
||||
```bash
|
||||
docker compose up -d issuer-directory
|
||||
# or
|
||||
kubectl scale deploy stellaops-issuer-directory --replicas=1
|
||||
```
|
||||
6. **Validate**
|
||||
- `curl -fsSL https://localhost:8447/health/live`
|
||||
- Issue an access token and list issuers to confirm results.
|
||||
- Check Mongo counts match expectations (`db.issuers.countDocuments()`, etc.).
|
||||
|
||||
## Disaster recovery notes
|
||||
- **Retention:** Maintain 30 daily + 12 monthly archives. Store copies in geographically separate, access-controlled vaults.
|
||||
- **Audit reconciliation:** Ensure `issuer_audit` entries cover the restore window; export them for compliance.
|
||||
- **Seed replay:** If the CSAF seed file was lost, set `ISSUER_DIRECTORY_SEED_CSAF=true` for the first restart to rehydrate the global tenant.
|
||||
- **Testing:** Run quarterly restore drills in a staging environment to validate procedure drift.
|
||||
|
||||
## Verification checklist
|
||||
- [ ] `/health/live` returns `200 OK`.
|
||||
- [ ] Mongo collections (`issuers`, `issuer_keys`, `issuer_trust_overrides`) have expected counts.
|
||||
- [ ] `issuer_directory_changes_total` and `issuer_directory_key_operations_total` metrics resume within 1 minute.
|
||||
- [ ] Audit entries exist for post-restore CRUD activity.
|
||||
- [ ] Client integrations (VEX Lens, Excititor) resolve issuers successfully.
|
||||
100
docs/modules/issuer-directory/operations/deployment.md
Normal file
100
docs/modules/issuer-directory/operations/deployment.md
Normal file
@@ -0,0 +1,100 @@
|
||||
# Issuer Directory Deployment Guide
|
||||
|
||||
## Scope
|
||||
- **Applies to:** Issuer Directory WebService (`stellaops/issuer-directory-web`) running via the provided Docker Compose bundles (`deploy/compose/docker-compose.*.yaml`) or the Helm chart (`deploy/helm/stellaops`).
|
||||
- **Covers:** Environment prerequisites, secret handling, Compose + Helm rollout steps, and post-deploy verification.
|
||||
- **Audience:** Platform/DevOps engineers responsible for Identity & Signing sprint deliverables.
|
||||
|
||||
## 1 · Prerequisites
|
||||
- Authority must be running and reachable at the issuer URL you configure (default Compose host: `https://authority:8440`).
|
||||
- MongoDB 4.2+ with credentials for the `issuer-directory` database (Compose defaults to the root user defined in `.env`).
|
||||
- Network access to Authority, MongoDB, and (optionally) Prometheus if you scrape metrics.
|
||||
- Issuer Directory configuration file `etc/issuer-directory.yaml` checked and customised for your environment (tenant header, audiences, telemetry level, CSAF seed path).
|
||||
|
||||
> **Secrets:** Use `etc/secrets/issuer-directory.mongo.secret.example` as a template. Store the real connection string in an untracked file or secrets manager and reference it via environment variables (`ISSUER_DIRECTORY_MONGO_CONNECTION_STRING`) rather than committing credentials.
|
||||
|
||||
## 2 · Deploy with Docker Compose
|
||||
1. **Prepare environment variables**
|
||||
```bash
|
||||
cp deploy/compose/env/dev.env.example dev.env
|
||||
cp etc/secrets/issuer-directory.mongo.secret.example issuer-directory.mongo.env
|
||||
# Edit dev.env and issuer-directory.mongo.env with production-ready secrets.
|
||||
```
|
||||
|
||||
2. **Inspect the merged configuration**
|
||||
```bash
|
||||
docker compose \
|
||||
--env-file dev.env \
|
||||
--env-file issuer-directory.mongo.env \
|
||||
-f deploy/compose/docker-compose.dev.yaml config
|
||||
```
|
||||
The command confirms the new `issuer-directory` service resolves the port (`${ISSUER_DIRECTORY_PORT:-8447}`) and the Mongo connection string is in place.
|
||||
|
||||
3. **Launch the stack**
|
||||
```bash
|
||||
docker compose \
|
||||
--env-file dev.env \
|
||||
--env-file issuer-directory.mongo.env \
|
||||
-f deploy/compose/docker-compose.dev.yaml up -d issuer-directory
|
||||
```
|
||||
Compose automatically mounts `../../etc/issuer-directory.yaml` into the container at `/etc/issuer-directory.yaml`, seeds CSAF publishers, and exposes the API on `https://localhost:8447`.
|
||||
|
||||
4. **Smoke test**
|
||||
```bash
|
||||
curl -k https://localhost:8447/health/live
|
||||
stellaops-cli issuer-directory issuers list \
|
||||
--base-url https://localhost:8447 \
|
||||
--tenant demo \
|
||||
--access-token "$(stellaops-cli auth token issue --scope issuer-directory:read)"
|
||||
```
|
||||
|
||||
5. **Upgrade & rollback**
|
||||
- Update Compose images to the desired release manifest (`deploy/releases/*.yaml`), re-run `docker compose config`, then `docker compose up -d`.
|
||||
- Rollbacks follow the same steps with the previous manifest. Mongo collections are backwards compatible within `2025.10.x`.
|
||||
|
||||
## 3 · Deploy with Helm
|
||||
1. **Create or update the secret**
|
||||
```bash
|
||||
kubectl create secret generic issuer-directory-secrets \
|
||||
--from-literal=ISSUERDIRECTORY__MONGO__CONNECTIONSTRING='mongodb://stellaops:<password>@stellaops-mongo:27017' \
|
||||
--dry-run=client -o yaml | kubectl apply -f -
|
||||
```
|
||||
Add optional overrides (e.g. `ISSUERDIRECTORY__AUTHORITY__ISSUER`) if your Authority issuer differs from the default.
|
||||
|
||||
2. **Template for validation**
|
||||
```bash
|
||||
helm template issuer-directory deploy/helm/stellaops \
|
||||
-f deploy/helm/stellaops/values-prod.yaml \
|
||||
--set services.issuer-directory.env.ISSUERDIRECTORY__AUTHORITY__ISSUER=https://authority.prod.stella-ops.org \
|
||||
> /tmp/issuer-directory.yaml
|
||||
```
|
||||
|
||||
3. **Install / upgrade**
|
||||
```bash
|
||||
helm upgrade --install stellaops deploy/helm/stellaops \
|
||||
-f deploy/helm/stellaops/values-prod.yaml \
|
||||
--set services.issuer-directory.env.ISSUERDIRECTORY__AUTHORITY__ISSUER=https://authority.prod.stella-ops.org
|
||||
```
|
||||
The chart provisions:
|
||||
- ConfigMap `stellaops-issuer-directory-config` with `IssuerDirectory` settings.
|
||||
- Deployment `stellaops-issuer-directory` with readiness/liveness probes on `/health/live`.
|
||||
- Service on port `8080` (ClusterIP by default).
|
||||
|
||||
4. **Expose for operators (optional)**
|
||||
- Use an Ingress/HTTPRoute to publish `https://issuer-directory.<env>.stella-ops.org`.
|
||||
- Ensure the upstream includes DPoP headers if proxied through an API gateway.
|
||||
|
||||
5. **Post-deploy validation**
|
||||
```bash
|
||||
kubectl exec deploy/stellaops-issuer-directory -- \
|
||||
curl -sf http://127.0.0.1:8080/health/live
|
||||
kubectl logs deploy/stellaops-issuer-directory | grep 'IssuerDirectory Mongo connected'
|
||||
```
|
||||
Prometheus should begin scraping `issuer_directory_changes_total` and related metrics (labels: `tenant`, `issuer`, `action`).
|
||||
|
||||
## 4 · Operational checklist
|
||||
- **Secrets:** Connection strings live in `issuer-directory-secrets` (Helm) or an `.env` file stored in your secrets vault (Compose). Rotate credentials via secret update + pod restart.
|
||||
- **Audit streams:** Confirm `issuer_directory_audit` collection receives entries when CRUD operations run; export logs for compliance.
|
||||
- **Tenants:** The service enforces the `X-StellaOps-Tenant` header. For multi-tenant staging, configure the reverse proxy to inject the correct tenant or issue scoped tokens.
|
||||
- **CSAF seeds:** `ISSUER_DIRECTORY_SEED_CSAF=true` replays `data/csaf-publishers.json` on startup. Set to `false` once production tenants are fully managed, or override `csafSeedPath` with a curated bundle.
|
||||
- **Release alignment:** Before promotion, run `deploy/tools/validate-profiles.sh` to lint Compose/Helm bundles, then verify the new `issuer-directory-web` entry in `deploy/releases/2025.10-edge.yaml` (or the relevant manifest) matches the channel you intend to ship.
|
||||
71
docs/modules/issuer-directory/operations/offline-kit.md
Normal file
71
docs/modules/issuer-directory/operations/offline-kit.md
Normal file
@@ -0,0 +1,71 @@
|
||||
# Issuer Directory Offline Kit Notes
|
||||
|
||||
## Purpose
|
||||
Operators bundling Stella Ops for fully disconnected environments must include the Issuer Directory service so VEX Lens, Excititor, and Policy Engine can resolve trusted issuers without reaching external registries.
|
||||
|
||||
## 1 · Bundle contents
|
||||
Include the following artefacts in your Offline Update Kit staging tree:
|
||||
|
||||
| Path (within kit) | Source | Notes |
|
||||
| --- | --- | --- |
|
||||
| `images/issuer-directory-web.tar` | `registry.stella-ops.org/stellaops/issuer-directory-web` (digest from `deploy/releases/<channel>.yaml`) | Export with `crane pull --format=tar` or `skopeo copy docker://... oci:...`. |
|
||||
| `config/issuer-directory/issuer-directory.yaml` | `etc/issuer-directory.yaml` (customised) | Replace Authority issuer, tenant header, and log level as required. |
|
||||
| `config/issuer-directory/csaf-publishers.json` | `src/IssuerDirectory/StellaOps.IssuerDirectory/data/csaf-publishers.json` or regional override | Operators can edit before import to add private publishers. |
|
||||
| `secrets/issuer-directory/connection.env` | Secure secret store export (`ISSUER_DIRECTORY_MONGO_CONNECTION_STRING=`) | Encrypt at rest; Offline Kit importer places it in the Compose/Helm secret. |
|
||||
| `docs/issuer-directory/deployment.md` | `docs/modules/issuer-directory/operations/deployment.md` | Ship alongside kit documentation for operators. |
|
||||
|
||||
> **Image digests:** Update `deploy/releases/2025.10-edge.yaml` (or the relevant manifest) with the exact digest before building the kit so `offline-manifest.json` can assert integrity.
|
||||
|
||||
## 2 · Compose (air-gapped) deployment
|
||||
1. Load images locally on the target:
|
||||
```bash
|
||||
docker load < images/issuer-directory-web.tar
|
||||
```
|
||||
2. Copy Compose artefacts:
|
||||
```bash
|
||||
cp deploy/compose/docker-compose.airgap.yaml .
|
||||
cp deploy/compose/env/airgap.env.example airgap.env
|
||||
cp secrets/issuer-directory/connection.env issuer-directory.mongo.env
|
||||
```
|
||||
3. Update `airgap.env` with site-specific values (Authority issuer, tenant, ports) and remove outbound endpoints.
|
||||
4. Bring up the service:
|
||||
```bash
|
||||
docker compose \
|
||||
--env-file airgap.env \
|
||||
--env-file issuer-directory.mongo.env \
|
||||
-f docker-compose.airgap.yaml up -d issuer-directory
|
||||
```
|
||||
5. Verify via `curl -k https://issuer-directory.airgap.local:8447/health/live`.
|
||||
|
||||
## 3 · Kubernetes (air-gapped) deployment
|
||||
1. Pre-load the OCI image into your local registry mirror and update `values-airgap.yaml` to reference it.
|
||||
2. Apply the secret bundled in the kit:
|
||||
```bash
|
||||
kubectl apply -f secrets/issuer-directory/connection-secret.yaml
|
||||
```
|
||||
(Generate this file during packaging with `kubectl create secret generic issuer-directory-secrets ... --dry-run=client -o yaml`.)
|
||||
3. Install/upgrade the chart:
|
||||
```bash
|
||||
helm upgrade --install stellaops deploy/helm/stellaops \
|
||||
-f deploy/helm/stellaops/values-airgap.yaml \
|
||||
--set services.issuer-directory.env.ISSUERDIRECTORY__AUTHORITY__ISSUER=https://authority.airgap.local/realms/stellaops
|
||||
```
|
||||
4. Confirm `issuer_directory_changes_total` is visible in your offline Prometheus stack.
|
||||
|
||||
## 4 · Import workflow summary
|
||||
1. Run `ops/offline-kit/build_offline_kit.py` with the additional artefacts noted above.
|
||||
2. Sign the resulting tarball and manifest (Cosign) and record the SHA-256 in the release notes.
|
||||
3. At the destination:
|
||||
```bash
|
||||
stellaops-cli offline kit import \
|
||||
--bundle stella-ops-offline-kit-<version>-airgap.tar.gz \
|
||||
--destination /opt/stellaops/offline-kit
|
||||
```
|
||||
4. Follow the Compose or Helm path depending on your topology.
|
||||
|
||||
## 5 · Post-import validation
|
||||
- [ ] `docker images | grep issuer-directory` (Compose) or `kubectl get deploy stellaops-issuer-directory` (Helm) shows the expected version.
|
||||
- [ ] `csaf-publishers.json` in the container matches the offline bundle (hash check).
|
||||
- [ ] `/issuer-directory/issuers` returns global seed issuers (requires token with `issuer-directory:read` scope).
|
||||
- [ ] Audit collection receives entries when you create/update issuers offline.
|
||||
- [ ] Offline kit manifest (`offline-manifest.json`) lists `images/issuer-directory-web.tar` and `config/issuer-directory/issuer-directory.yaml` with SHA-256 values you recorded during packaging.
|
||||
@@ -8,13 +8,20 @@
|
||||
| SCANNER-DOCS-0002 | DONE (2025-11-02) | Docs Guild | Keep scanner benchmark comparisons (Trivy/Grype/Snyk) and deep-dive matrix current with source references. | Coordinate with docs/benchmarks owners |
|
||||
| SCANNER-DOCS-0003 | TODO | Docs Guild, Product Guild | Gather Windows/macOS analyzer demand signals and record findings in `docs/benchmarks/scanner/windows-macos-demand.md`. | Coordinate with Product Marketing & Sales enablement |
|
||||
| SCANNER-ENG-0008 | TODO | EntryTrace Guild, QA Guild | Maintain EntryTrace heuristic cadence per `docs/benchmarks/scanner/scanning-gaps-stella-misses-from-competitors.md`. | Include quarterly pattern review + explain trace updates |
|
||||
| SCANNER-ENG-0009 | TODO | Ruby Analyzer Guild | SCANNER-ANALYZERS-RUBY-28-001..012 | Deliver Ruby analyzer parity and observation pipeline per gap doc (lockfiles, runtime graph, policy signals). | Design complete; fixtures published; CLI/Offline docs updated. |
|
||||
| SCANNER-ENG-0009 | DOING (2025-11-02) | Ruby Analyzer Guild | SCANNER-ANALYZERS-RUBY-28-001..012 | Deliver Ruby analyzer parity and observation pipeline per gap doc (lockfiles, runtime graph, policy signals). | Design complete; fixtures published; CLI/Offline docs updated. |
|
||||
| SCANNER-ENG-0010 | TODO | PHP Analyzer Guild | SCANNER-ANALYZERS-PHP-27-001..012 | Ship PHP analyzer pipeline (composer lock, autoload graph, capability signals) to close comparison gaps. | Analyzer + policy integration merged; fixtures + docs aligned. |
|
||||
| SCANNER-ENG-0011 | TODO | Language Analyzer Guild | — | Scope Deno runtime analyzer (lockfile resolver, import graphs) based on competitor techniques. | Design doc approved; backlog split into analyzer/runtime work. |
|
||||
| SCANNER-ENG-0012 | TODO | Language Analyzer Guild | — | Evaluate Dart analyzer requirements (pubspec parsing, AOT artifacts) to restore parity. | Investigation summary + task split filed with Dart guild. |
|
||||
| SCANNER-ENG-0013 | TODO | Swift Analyzer Guild | — | Plan Swift Package Manager coverage (Package.resolved, xcframeworks, runtime hints) with policy hooks. | Design brief approved; backlog seeded with analyzer tasks. |
|
||||
| SCANNER-ENG-0014 | TODO | Runtime Guild, Zastava Guild | — | Align Kubernetes/VM target coverage roadmap between Scanner and Zastava per comparison findings. | Joint roadmap doc approved; cross-guild tasks opened. |
|
||||
| SCANNER-ENG-0015 | TODO | Export Center Guild, Scanner Guild | — | Document DSSE/Rekor operator enablement guidance and rollout levers surfaced in gap analysis. | Playbook drafted; Export Center backlog updated. |
|
||||
| SCANNER-ENG-0016 | DOING (2025-11-02) | Ruby Analyzer Guild (Lockfile Squad) | Implement `RubyLockCollector` and vendor cache ingestion per design §4.1–4.3. | Coordinate fixtures under `fixtures/lang/ruby/lockfiles`; target alpha by Sprint 21. |
|
||||
| SCANNER-ENG-0017 | TODO | Ruby Analyzer Guild (Runtime Squad) | Build runtime require/autoload graph builder with tree-sitter Ruby per design §4.4. | Deliver edges with reason codes and integrate EntryTrace hints. |
|
||||
| SCANNER-ENG-0018 | TODO | Ruby Analyzer Guild (Capability Squad) | Emit Ruby capability and framework surface signals as defined in design §4.5. | Policy predicates prototyped; capability records available in SBOM overlays. |
|
||||
| SCANNER-ENG-0019 | TODO | Ruby Analyzer Guild, CLI Guild | Ship Ruby CLI verbs (`stella ruby inspect|resolve`) and Offline Kit packaging per design §4.6. | CLI commands documented; offline manifest updated; e2e tests pass. |
|
||||
| SCANNER-LIC-0001 | DOING (2025-11-02) | Scanner Guild, Legal Guild | Vet tree-sitter Ruby licensing and Offline Kit packaging requirements. | SPDX review complete; packaging plan approved. |
|
||||
| SCANNER-POLICY-0001 | TODO | Policy Guild, Ruby Analyzer Guild | Define Policy Engine predicates for Ruby groups/capabilities and align lattice weights. | Policy schema merged; tests cover new predicates. |
|
||||
| SCANNER-CLI-0001 | TODO | CLI Guild, Ruby Analyzer Guild | Coordinate CLI UX/help text for new Ruby verbs and update CLI docs. | CLI help + docs updated; golden outputs recorded. |
|
||||
| SCANNER-ENG-0002 | TODO | Scanner Guild, CLI Guild | Design Node.js lockfile collector/CLI validator per `docs/benchmarks/scanner/scanning-gaps-stella-misses-from-competitors.md`. | Capture Surface & policy requirements before implementation |
|
||||
| SCANNER-ENG-0003 | TODO | Python Analyzer Guild, CLI Guild | Design Python lockfile/editable install parity checks per `docs/benchmarks/scanner/scanning-gaps-stella-misses-from-competitors.md`. | Include policy predicates & CLI story in design |
|
||||
| SCANNER-ENG-0004 | TODO | Java Analyzer Guild, CLI Guild | Design Java lockfile ingestion & validation per `docs/benchmarks/scanner/scanning-gaps-stella-misses-from-competitors.md`. | Cover Gradle/SBT collectors, CLI verb, policy hooks |
|
||||
|
||||
137
docs/modules/scanner/design/ruby-analyzer.md
Normal file
137
docs/modules/scanner/design/ruby-analyzer.md
Normal file
@@ -0,0 +1,137 @@
|
||||
# Ruby Analyzer Parity Design (SCANNER-ENG-0009)
|
||||
|
||||
**Status:** Draft • Owner: Ruby Analyzer Guild • Updated: 2025-11-02
|
||||
|
||||
## 1. Goals & Non-Goals
|
||||
- **Goals**
|
||||
- Deterministically catalogue Ruby application dependencies (Gemfile/Gemfile.lock, vendored specs, .gem archives) for container layers and local workspaces.
|
||||
- Build runtime usage graphs (require/require_relative, Zeitwerk autoloads, Rack boot chains, Sidekiq/ActiveJob schedulers).
|
||||
- Emit capability signals (exec/fs/net/serialization, framework fingerprints, job schedulers) consumable by Policy Engine and explain traces.
|
||||
- Provide CLI verbs (`stella ruby inspect`, `stella ruby resolve`) and Offline Kit parity for air-gapped deployments.
|
||||
- **Non-Goals**
|
||||
- Shipping dynamic runtime profilers (log-based or APM) in this iteration.
|
||||
- Implementing UI changes beyond exposing explain traces the Policy/UI guilds already support.
|
||||
|
||||
## 2. Scope & Inputs
|
||||
| Input | Location | Notes |
|
||||
|-------|----------|-------|
|
||||
| Gemfile / Gemfile.lock | Source tree, layer filesystem | Handle multiple apps per repo; honour Bundler groups. |
|
||||
| Vendor bundles (`vendor/bundle`, `.bundle/config`) | Layer filesystem | Needed for offline/built images; avoid double-counting platform-specific gems. |
|
||||
| `.gemspec` files / cached specs | `~/.bundle/cache`, `vendor/cache`, gems in layers | Support deterministic parsing without executing gem metadata. |
|
||||
| Framework configs | `config/application.rb`, `config/routes.rb`, `config/sidekiq.yml`, etc. | Feed framework surface mapper. |
|
||||
| Container metadata | Layer digests via RustFS CAS | Support incremental composition per layer. |
|
||||
|
||||
## 3. High-Level Architecture
|
||||
```
|
||||
┌─────────────────────────┐ ┌────────────────────┐
|
||||
│ Bundler Lock Collector │───────▶│ Package Graph │
|
||||
└─────────────────────────┘ │ Aggregator │
|
||||
└─────────┬──────────┘
|
||||
┌─────────────────────────┐ │
|
||||
│ Gemspec Inspector │───────────────▶│
|
||||
└─────────────────────────┘ │
|
||||
▼
|
||||
┌────────────────────┐
|
||||
┌─────────────────────────┐ │ Runtime Graph │
|
||||
│ Require/Autoload Scan │───────▶│ Builder (Zeitwerk) │
|
||||
└─────────────────────────┘ └─────────┬──────────┘
|
||||
│
|
||||
▼
|
||||
┌────────────────────┐
|
||||
│ Capability Emitter │
|
||||
└─────────┬──────────┘
|
||||
│
|
||||
▼
|
||||
┌────────────────────┐
|
||||
│ SBOM Writer │
|
||||
│ + Policy Signals │
|
||||
└────────────────────┘
|
||||
```
|
||||
|
||||
## 4. Detailed Components
|
||||
### 4.1 Bundler Lock Collector
|
||||
- Parse `Gemfile.lock` deterministically (no network) using new `RubyLockCollector` under `StellaOps.Scanner.Analyzers.Lang.Ruby`.
|
||||
- Support alternative manifests (`gems.rb`, `gems.locked`) and workspace overrides.
|
||||
- Emit package nodes with fields: `name`, `version`, `source` (path/git/rubygems), `bundlerGroup[]`, `platform`, `declaredOnly` flag.
|
||||
- Implementation:
|
||||
- Reuse parsing strategy from Trivy (`pkg/fanal/analyzer/language/ruby/bundler`) but port to C# with streaming reader and stable ordering.
|
||||
- Integrate with Surface.Validation to enforce size limits and tenant allowlists for git/path sources.
|
||||
|
||||
### 4.2 Gemspec Inspector
|
||||
- Scan cached specs under `vendor/cache`, `.bundle/cache`, and gem directories to pick up transitive packages when lockfiles missing.
|
||||
- Parse without executing Ruby by using a deterministic DSL subset (similar to Trivy gemspec parser).
|
||||
- Link results to lockfile entries by `<name, version, platform>`; create new records flagged `InferredFromSpec` when lockfile absent.
|
||||
|
||||
### 4.3 Package Aggregator
|
||||
- New orchestrator `RubyPackageAggregator` merges lock and gemspec data with installed gems from container layers (once runtime analyzer ships).
|
||||
- Precedence: Installed > Lockfile > Gemspec.
|
||||
- Deduplicate by package key (name+version+platform) and attach provenance bits for Policy Engine.
|
||||
|
||||
### 4.4 Runtime Graph Builder
|
||||
- Static analysis for `require`, `require_relative`, `autoload`, Zeitwerk conventions, and Rails initialisers.
|
||||
- Implementation phases:
|
||||
1. Parse AST using tree-sitter Ruby embedded under `StellaOps.Scanner.Analyzers.Lang.Ruby.Syntax` with deterministic bindings.
|
||||
2. Generate edges `entrypoint -> file` and `file -> package` with reason codes (`require-static`, `autoload-zeitwerk`, `autoload-const_missing`).
|
||||
3. Identify framework entrypoints (Rails controllers, Rack middleware, Sidekiq workers) via heuristics defined in `SCANNER-ANALYZERS-RUBY-28-*` tasks.
|
||||
- Output merges with EntryTrace usage hints to support runtime filtering in Policy Engine.
|
||||
|
||||
### 4.5 Capability & Surface Signals
|
||||
- Emit evidence documents for:
|
||||
- Process/exec usage (`Kernel.system`, `` `cmd` ``, `Open3`).
|
||||
- Network clients (`Net::HTTP`, `Faraday`, `Redis`, `ActiveRecord::Base.establish_connection`).
|
||||
- Serialization sinks (`Marshal.load`, `YAML.load`, `Oj.load`).
|
||||
- Job schedulers (Sidekiq, Resque, ActiveJob, Whenever, Clockwork) with schedule metadata.
|
||||
- Capability records flow to Policy Engine under `capability.ruby.*` namespaces to allow gating on dangerous constructs.
|
||||
|
||||
### 4.6 CLI & Offline Integration
|
||||
- Add CLI verbs:
|
||||
- `stella ruby inspect <path>` – runs collector locally, outputs JSON summary with provenance.
|
||||
- `stella ruby resolve --image <ref>` – fetches scan artifacts, prints dependency graph grouped by bundler group/platform.
|
||||
- Ship analyzer DLLs and rules in Offline Kit manifest; include autoload/zeitwerk fingerprints and heuristics hashed for determinism.
|
||||
|
||||
## 5. Data Contracts
|
||||
| Artifact | Shape | Consumer |
|
||||
|----------|-------|----------|
|
||||
| `ruby_packages.json` | Array `{id, name, version, source, provenance, groups[], platform}` | SBOM Composer, Policy Engine |
|
||||
| `ruby_runtime_edges.json` | Edges `{from, to, reason, confidence}` | EntryTrace overlay, Policy explain traces |
|
||||
| `ruby_capabilities.json` | Capability `{kind, location, evidenceHash, params}` | Policy Engine (capability predicates) |
|
||||
|
||||
All records follow AOC appender rules (immutable, tenant-scoped) and include `hash`, `layerDigest`, and `timestamp` normalized to UTC ISO-8601.
|
||||
|
||||
## 6. Testing Strategy
|
||||
- **Fixtures**: Extend `fixtures/lang/ruby` with Rails, Sinatra, Sidekiq, Rack, container images (with/without vendor cache).
|
||||
- **Determinism**: Golden snapshots for package lists and capability outputs across repeated runs.
|
||||
- **Integration**: Worker e2e to ensure per-layer aggregation; CLI golden outputs (`stella ruby inspect`).
|
||||
- **Policy**: Unit tests verifying new predicates (`ruby.group`, `ruby.capability.exec`, etc.) in Policy Engine test suite.
|
||||
|
||||
## 7. Rollout Plan & Dependencies
|
||||
1. Implement collectors and aggregators (SCANNER-ANALYZERS-RUBY-28-001..004).
|
||||
2. Add capability analyzer and observations (SCANNER-ANALYZERS-RUBY-28-005..008).
|
||||
3. Wire CLI commands and Offline Kit packaging (SCANNER-ANALYZERS-RUBY-28-011).
|
||||
4. Update docs (DOCS-SCANNER-BENCH-62-009 follow-up) once analyzer alpha ready.
|
||||
|
||||
**Dependencies**
|
||||
- Tree-sitter Ruby grammar inclusion (needs Offline Kit packaging and licensing check).
|
||||
- Policy Engine support for new predicates and capability schemas.
|
||||
- Surface.Validation updates for git/path gem sources and secret resolution.
|
||||
|
||||
## 8. Open Questions
|
||||
- Do we require dynamic runtime logs (e.g., `ActiveSupport::Notifications`) for confidence boosts? (defer to future iteration)
|
||||
- Should we enforce signed gem provenance in MVP? Pending Product decision.
|
||||
- Need alignment with Export Center on Ruby-specific manifest emissions.
|
||||
|
||||
## 9. Licensing & Offline Packaging (SCANNER-LIC-0001)
|
||||
- **License**: tree-sitter core and `tree-sitter-ruby` grammar are MIT licensed (confirmed via upstream LICENSE files retrieved 2025-11-02).
|
||||
- **Obligations**:
|
||||
1. Include both MIT license texts in `/third-party-licenses/` and in Offline Kit manifests.
|
||||
2. Update `NOTICE.md` to acknowledge embedded grammars per company policy.
|
||||
3. Record the grammar commit hashes in build metadata; regenerate generated C/WASM artifacts deterministically.
|
||||
4. Ensure build pipeline uses `tree-sitter-cli` only as a build-time tool (not redistributed) to avoid extra licensing obligations.
|
||||
- **Deliverables**:
|
||||
- SCANNER-LIC-0001 to capture Legal sign-off and update packaging scripts.
|
||||
- Export Center to mirror license files into Offline Kit bundle.
|
||||
|
||||
---
|
||||
*References:*
|
||||
- Trivy: `pkg/fanal/analyzer/language/ruby/bundler`, `pkg/fanal/analyzer/language/ruby/gemspec`
|
||||
- Gap analysis: `docs/benchmarks/scanner/scanning-gaps-stella-misses-from-competitors.md#ruby-analyzer-parity-trivy-grype-snyk`
|
||||
@@ -285,6 +285,31 @@ clients:
|
||||
serviceIdentity: cartographer
|
||||
```
|
||||
|
||||
### 3.3 Delegated service accounts
|
||||
|
||||
Add delegated service accounts when automation needs scoped tokens with shorter lifetimes:
|
||||
|
||||
```yaml
|
||||
delegation:
|
||||
quotas:
|
||||
maxActiveTokens: 50
|
||||
serviceAccounts:
|
||||
- accountId: "svc-observer"
|
||||
tenant: "tenant-default"
|
||||
allowedScopes: [ "jobs:read", "findings:read" ]
|
||||
authorizedClients: [ "export-center-worker" ]
|
||||
|
||||
tenants:
|
||||
- name: "tenant-default"
|
||||
delegation:
|
||||
maxActiveTokens: 25
|
||||
```
|
||||
|
||||
- Clients request delegated tokens by supplying `service_account=<accountId>` (and optional `delegation_actor`) alongside the usual client-credentials payload.
|
||||
- Authority enforces both tenant and service-account quotas. Exceeding either returns `invalid_request` and records `delegation.quota.exceeded` in audit events.
|
||||
- Only scopes listed in `allowedScopes` are granted; `authorizedClients` restricts which OAuth clients may impersonate the delegate.
|
||||
- Delegated tokens include `stellaops:service_account` and an `act` claim. The token store persists `tokenKind = "service_account"`, `serviceAccountId`, and the normalized actor chain for offline auditing.
|
||||
|
||||
---
|
||||
|
||||
## 4 · Operational safeguards
|
||||
@@ -323,10 +348,12 @@ clients:
|
||||
- [ ] Claim transforms enforce `serviceIdentity` for `effective:write`.
|
||||
- [ ] Claim transforms enforce `serviceIdentity` for `graph:write`.
|
||||
- [ ] Concelier/Excititor smoke tests cover missing tenant rejection.
|
||||
- [ ] Delegation quotas configured (`delegation.quotas.maxActiveTokens`, `tenants[].delegation.maxActiveTokens` where required).
|
||||
- [ ] Service account seeds (`delegation.serviceAccounts`) reviewed for allowed scopes and authorized clients; audit dashboards show `delegation.service_account` usage.
|
||||
- [ ] Offline kit credentials reviewed for least privilege.
|
||||
- [ ] Audit/monitoring guidance validated with Observability Guild.
|
||||
- [ ] Authority Core sign-off recorded (owner: @authority-core, due 2025-10-28).
|
||||
|
||||
---
|
||||
|
||||
*Last updated: 2025-10-27 (Sprint 19).*
|
||||
*Last updated: 2025-11-02 (Sprint 19).*
|
||||
|
||||
Reference in New Issue
Block a user