166 lines
9.5 KiB
Markdown
166 lines
9.5 KiB
Markdown
# Developer Tools Architecture
|
|
|
|
> Standalone CLI utilities for development, testing, and CI support workflows.
|
|
|
|
## Overview
|
|
|
|
The Tools directory contains a set of independent CLI applications, each with its own `Program.cs` entry point. These tools are not deployed as services -- they are invoked locally by developers or executed in CI pipelines. Each tool is narrowly scoped to a single responsibility, from fixture management to workflow generation.
|
|
|
|
## Components
|
|
|
|
```
|
|
src/Tools/
|
|
FixtureUpdater/ # Golden fixture refresh from live APIs
|
|
Program.cs
|
|
GoldenPairs/ # SBOM/advisory corpus management
|
|
Program.cs
|
|
PolicyDslValidator/ # Policy DSL file validation
|
|
Program.cs
|
|
PolicySchemaExporter/ # JSON schema export for IDE support
|
|
Program.cs
|
|
PolicySimulationSmoke/ # Policy simulation smoke tests
|
|
Program.cs
|
|
LanguageAnalyzerSmoke/ # Language detection accuracy tests
|
|
Program.cs
|
|
RustFsMigrator/ # RustFS data migration between schema versions
|
|
Program.cs
|
|
WorkflowGenerator/ # CI workflow generation (F# DSL)
|
|
Program.fs
|
|
StellaOps.Bench/ # Performance benchmarks (absorbed from src/Bench/)
|
|
LinkNotMerge/ # Link-not-merge linkset benchmarks
|
|
LinkNotMerge.Vex/ # VEX-enriched linkset benchmarks
|
|
Notify/ # Notification pipeline benchmarks
|
|
PolicyEngine/ # Policy engine evaluation benchmarks
|
|
Scanner.Analyzers/ # Language analyzer benchmarks
|
|
Determinism/ # Determinism fixture inputs/configs
|
|
Graph/ # Graph benchmark scenarios and results
|
|
StellaOps.Verifier/ # Standalone offline bundle verifier (absorbed from src/Verifier/)
|
|
__Tests/
|
|
StellaOps.Sdk.Generator/ # Multi-language SDK code generation (absorbed from src/Sdk/)
|
|
go/ java/ python/ ts/
|
|
postprocess/
|
|
StellaOps.Sdk.Release/ # SDK release automation (absorbed from src/Sdk/)
|
|
StellaOps.DevPortal.Site/ # Developer portal static site (absorbed from src/DevPortal/)
|
|
src/ public/ scripts/
|
|
tests/supply-chain/ # Deterministic supply-chain hardening lanes
|
|
01-jcs-property/
|
|
02-schema-fuzz/
|
|
03-rekor-neg/
|
|
04-big-dsse-referrers/
|
|
05-corpus/
|
|
```
|
|
|
|
## Tool Descriptions
|
|
|
|
### FixtureUpdater
|
|
|
|
Pulls latest test data from running Stella Ops services and updates frozen golden fixtures deterministically. Ensures test suites use realistic, version-controlled data without manual fixture authoring.
|
|
|
|
### GoldenPairs
|
|
|
|
Manages SBOM/advisory pairs used for testing. Provides version tracking and diff tools for the test corpus, ensuring changes to upstream advisory formats are detected and accommodated.
|
|
|
|
### PolicyDslValidator
|
|
|
|
Validates policy DSL files against the current schema. Used in CI gates to catch policy syntax errors before merge.
|
|
|
|
### PolicySchemaExporter
|
|
|
|
Exports the Policy DSL schema to JSON format for documentation and IDE autocomplete support. Enables policy authors to get inline validation and completion in their editors.
|
|
|
|
### PolicySimulationSmoke
|
|
|
|
Runs end-to-end policy simulation smoke tests against a configured Policy Engine instance. Validates that policy evaluation produces expected verdicts for a known set of inputs.
|
|
|
|
### LanguageAnalyzerSmoke
|
|
|
|
Tests the language analyzer's detection accuracy against a curated set of source files. Reports precision and recall metrics for supported languages.
|
|
|
|
### RustFsMigrator
|
|
|
|
Migrates data stored in RustFS (S3-compatible object storage) between schema versions. Handles object key transformations and metadata updates required during platform upgrades.
|
|
|
|
### WorkflowGenerator
|
|
|
|
Generates GitHub Actions and .NET test workflow definitions from an F# DSL. Ensures CI workflow files are consistent, auditable, and derived from a single source of truth rather than hand-edited YAML.
|
|
|
|
### StellaOps.Bench (Performance Benchmarks)
|
|
|
|
Collected from the former `src/Bench/` directory. Contains performance benchmarks for multiple subsystems:
|
|
|
|
- **LinkNotMerge** -- Benchmarks for linkset aggregation and link-not-merge scenario execution, with baseline tracking and Prometheus-compatible metrics export.
|
|
- **LinkNotMerge.Vex** -- VEX-enriched variant of link-not-merge benchmarks.
|
|
- **Notify** -- Benchmarks for the notification pipeline against `StellaOps.Notify.Models`.
|
|
- **PolicyEngine** -- Benchmarks for policy engine evaluation against `StellaOps.Policy`.
|
|
- **Scanner.Analyzers** -- Benchmarks for language analyzers (Node, Go, Java, Python, .NET, Bun) against `StellaOps.Scanner.Analyzers.Lang.*`.
|
|
- **Determinism** -- Frozen fixture inputs and configurations for deterministic benchmark reproducibility.
|
|
|
|
Each benchmark subproject can be published as an independent `dotnet tool`.
|
|
|
|
### StellaOps.Verifier (Standalone Bundle Verifier)
|
|
|
|
Absorbed from the former `src/Verifier/` directory. A standalone CLI tool for offline verification of Stella Ops evidence bundles in air-gapped environments. Publishes as a single-file, self-contained executable (`stella-verifier`) targeting multiple RIDs: `win-x64`, `linux-x64`, `linux-musl-x64`, `osx-x64`, `osx-arm64`. Has no framework dependencies -- designed for minimal footprint in restricted environments.
|
|
|
|
### StellaOps.Sdk.Generator (SDK Code Generation)
|
|
|
|
Absorbed from the former `src/Sdk/StellaOps.Sdk.Generator/` directory. Generates typed SDK clients for Go, Java, Python, and TypeScript from the Stella Ops OpenAPI specification. Includes per-language `config.yaml`, generation scripts, postprocessing templates, and deterministic toolchain lockfile (`toolchain.lock.yaml`).
|
|
|
|
### StellaOps.Sdk.Release (SDK Release Automation)
|
|
|
|
Absorbed from the former `src/Sdk/StellaOps.Sdk.Release/` directory. Automates the release pipeline for generated SDK packages.
|
|
|
|
### StellaOps.DevPortal.Site (Developer Portal)
|
|
|
|
Absorbed from the former `src/DevPortal/` directory. An Astro-based static site providing interactive API documentation, SDK quickstarts, code examples, and a "Try It" console backed by RapiDoc. Includes offline build support (`build-offline.mjs`), link checking, accessibility audits, and performance checks. Not a .NET project -- uses Node.js/npm.
|
|
|
|
### Supply-Chain Hardening Suite
|
|
|
|
Deterministic multi-lane harness validating canonicalization invariants, mutation-fuzz crash resistance, Rekor negative-path diagnostics, and oversized DSSE/referrer reject behavior.
|
|
|
|
## Data Flow
|
|
|
|
Tools are consumers and producers of artifacts:
|
|
|
|
1. **FixtureUpdater** and **GoldenPairs** pull data from live services or local corpora and write deterministic fixture files to the repository.
|
|
2. **PolicyDslValidator** and **PolicySchemaExporter** read policy definitions and produce validation results or schema files.
|
|
3. **PolicySimulationSmoke** and **LanguageAnalyzerSmoke** execute tests against upstream services/libraries and produce pass/fail reports.
|
|
4. **RustFsMigrator** reads from and writes to S3-compatible storage.
|
|
5. **WorkflowGenerator** reads F# DSL definitions and writes CI workflow YAML files.
|
|
6. **Supply-chain hardening suite** reads frozen corpus fixtures, executes deterministic lanes, and emits replay-ready artifacts for CI and local triage.
|
|
7. **StellaOps.Bench** reads frozen inputs (SBOMs, VEX, graphs) and produces benchmark reports with latency/throughput metrics.
|
|
8. **StellaOps.Verifier** reads evidence bundles from disk and produces pass/fail verification results for offline audit.
|
|
9. **StellaOps.Sdk.Generator** reads the OpenAPI spec and produces typed SDK source code for Go, Java, Python, and TypeScript.
|
|
10. **StellaOps.DevPortal.Site** reads the OpenAPI spec and SDK documentation, producing a static developer portal site.
|
|
|
|
## Database Schema
|
|
|
|
Not applicable. Tools are CLI utilities with no persistent database.
|
|
|
|
## Endpoints
|
|
|
|
Not applicable. Tools are client-side CLI applications with no HTTP endpoints.
|
|
|
|
## Dependencies
|
|
|
|
| Library/Tool | Purpose |
|
|
|--------------------------|------------------------------------------------------|
|
|
| Policy Engine libs | Policy DSL parsing, schema definitions |
|
|
| Scanner libs | Language analyzer, SBOM processing |
|
|
| F# compiler | WorkflowGenerator DSL compilation |
|
|
| DotNet.Glob | File pattern matching in fixture tools |
|
|
| AWS SDK (S3) | RustFsMigrator object storage access |
|
|
| StellaOps.Policy | Bench.PolicyEngine benchmark target |
|
|
| StellaOps.Scanner.Analyzers.Lang.* | Bench.ScannerAnalyzers benchmark targets |
|
|
| StellaOps.Notify.Models | Bench.Notify benchmark target |
|
|
| StellaOps.TestKit | Shared test infrastructure for benchmark tests |
|
|
| System.CommandLine | Verifier CLI argument parsing |
|
|
| Astro + Starlight | DevPortal static site generation |
|
|
| openapi-generator-cli | Sdk.Generator multi-language code generation |
|
|
|
|
## Security Considerations
|
|
|
|
- **No network listeners**: Tools do not expose HTTP endpoints or accept inbound connections.
|
|
- **Credential handling**: Tools that connect to live services (FixtureUpdater, PolicySimulationSmoke) use the same Authority-issued tokens as other Stella Ops services. Credentials are never embedded in tool binaries or fixture files.
|
|
- **Deterministic output**: FixtureUpdater and GoldenPairs produce deterministic output to ensure reproducible test runs and prevent fixture drift.
|
|
- **CI isolation**: Tools run in isolated CI containers with scoped permissions; they do not have access to production secrets.
|