Refactor code structure and optimize performance across multiple modules
This commit is contained in:
StellaOps Bot
532
docs/operations/configuration-guide.md
Normal file
532
docs/operations/configuration-guide.md
Normal file
@@ -0,0 +1,532 @@
|
||||
# StellaOps Configuration Guide
|
||||
|
||||
This document describes the consolidated configuration structure for StellaOps deployments.
|
||||
|
||||
## Directory Structure Overview
|
||||
|
||||
All configuration lives under `etc/` at the repository root. This provides a single source of truth for all service configurations, trust anchors, crypto profiles, and plugin manifests.
|
||||
|
||||
```
|
||||
etc/
|
||||
├── authority/ # Authentication & authorization
|
||||
├── certificates/ # Trust anchors and signing keys
|
||||
├── concelier/ # Advisory ingestion
|
||||
├── crypto/ # Regional cryptographic profiles
|
||||
├── env/ # Environment-specific profiles
|
||||
├── llm-providers/ # AI/LLM provider configurations
|
||||
├── notify/ # Notification service & templates
|
||||
├── plugins/ # Plugin manifests (NOT binaries)
|
||||
├── policy/ # Policy engine & packs
|
||||
├── router/ # Transport router
|
||||
├── scanner/ # Container scanning
|
||||
├── scheduler/ # Job scheduling
|
||||
├── scm-connectors/ # Source control integrations
|
||||
├── secrets/ # Development secrets (NEVER production)
|
||||
├── signals/ # Runtime signals
|
||||
├── vex/ # VEX processing
|
||||
└── README.md # This file
|
||||
```
|
||||
|
||||
## Configuration Precedence
|
||||
|
||||
Configuration values are resolved in the following order (highest priority first):
|
||||
|
||||
1. **Command-line flags** - `--config-key=value`
|
||||
2. **Environment variables** - `STELLAOPS_<SERVICE>__<KEY>=value`
|
||||
3. **Active config file** - `etc/<service>/<service>.yaml`
|
||||
4. **Default values** - Built into the application
|
||||
|
||||
### Environment Variable Naming
|
||||
|
||||
Environment variables use double underscore (`__`) to represent nested configuration:
|
||||
|
||||
```bash
|
||||
# Translates to: { "Scanner": { "Concurrency": { "MaxParallel": 8 } } }
|
||||
STELLAOPS_SCANNER__CONCURRENCY__MAXPARALLEL=8
|
||||
```
|
||||
|
||||
## Service Configuration
|
||||
|
||||
### File Naming Convention
|
||||
|
||||
| File Pattern | Purpose |
|
||||
|--------------|---------|
|
||||
| `<service>.yaml` | Active configuration (git-ignored in production) |
|
||||
| `<service>.yaml.sample` | Documented template with all options |
|
||||
| `<service>.<profile>.yaml` | Profile-specific configuration |
|
||||
|
||||
### Creating Active Configuration
|
||||
|
||||
```bash
|
||||
# Copy sample to create active config
|
||||
cp etc/scanner/scanner.yaml.sample etc/scanner/scanner.yaml
|
||||
|
||||
# Edit for your environment
|
||||
vi etc/scanner/scanner.yaml
|
||||
```
|
||||
|
||||
## Directory Reference
|
||||
|
||||
### `etc/authority/` - Authentication & Authorization
|
||||
|
||||
```
|
||||
etc/authority/
|
||||
├── authority.yaml.sample # Main authority service config
|
||||
└── plugins/ # Auth provider plugin configs
|
||||
├── ldap.yaml.sample # LDAP/Active Directory
|
||||
├── oidc.yaml.sample # OpenID Connect
|
||||
└── saml.yaml.sample # SAML 2.0
|
||||
```
|
||||
|
||||
**Key settings:**
|
||||
- Token signing algorithms and key rotation
|
||||
- OAuth2/OIDC client registration
|
||||
- DPoP (Demonstrating Proof of Possession) settings
|
||||
- Session management
|
||||
|
||||
### `etc/certificates/` - Trust Anchors & Signing
|
||||
|
||||
```
|
||||
etc/certificates/
|
||||
├── trust-roots/ # CA certificate bundles
|
||||
│ ├── globalsign.pem # Commercial CA bundle
|
||||
│ ├── russian-trusted.pem # Russian Federation roots
|
||||
│ └── README.md # Certificate provenance
|
||||
└── signing/ # Signing keys (dev/sample)
|
||||
└── authority-signing-2025-dev.pem
|
||||
```
|
||||
|
||||
**Usage:**
|
||||
- Mount `trust-roots/` to `/etc/ssl/certs/stellaops/` in containers
|
||||
- Production signing keys should come from HSM or Vault, not this directory
|
||||
- Development keys are clearly marked with `-dev` suffix
|
||||
|
||||
### `etc/concelier/` - Advisory Ingestion
|
||||
|
||||
```
|
||||
etc/concelier/
|
||||
├── concelier.yaml.sample # Main concelier config
|
||||
└── sources/ # Advisory source configurations
|
||||
├── nist-nvd.yaml.sample # NVD API configuration
|
||||
├── github-advisory.yaml.sample
|
||||
├── oval-debian.yaml.sample
|
||||
└── oval-rhel.yaml.sample
|
||||
```
|
||||
|
||||
**Key settings:**
|
||||
- Advisory source endpoints and credentials
|
||||
- Merge strategy and precedence rules
|
||||
- Rate limiting and retry policies
|
||||
- Offline mode configuration
|
||||
|
||||
### `etc/crypto/` - Regional Cryptographic Profiles
|
||||
|
||||
```
|
||||
etc/crypto/
|
||||
├── crypto.yaml.sample # Global crypto settings
|
||||
└── profiles/
|
||||
├── cn/ # China - GM/T 0003/0004 (SM2/SM3/SM4)
|
||||
│ ├── crypto.profile.yaml
|
||||
│ ├── env.sample
|
||||
│ └── pq-vectors.txt # Post-quantum test vectors
|
||||
├── eu/ # EU - eIDAS qualified signatures
|
||||
│ ├── crypto.profile.yaml
|
||||
│ └── env.sample
|
||||
├── kr/ # Korea - KCMVP
|
||||
│ ├── crypto.profile.yaml
|
||||
│ └── env.sample
|
||||
├── ru/ # Russia - GOST R 34.10/34.11/34.12
|
||||
│ ├── crypto.profile.yaml
|
||||
│ └── env.sample
|
||||
└── us-fips/ # USA - FIPS 140-3
|
||||
├── crypto.profile.yaml
|
||||
└── env.sample
|
||||
```
|
||||
|
||||
**Crypto profile structure:**
|
||||
```yaml
|
||||
# crypto.profile.yaml
|
||||
region: us-fips
|
||||
compliance:
|
||||
standard: "FIPS 140-3"
|
||||
level: 1
|
||||
providers:
|
||||
preferred: ["BouncyCastle-FIPS", "OpenSSL-FIPS"]
|
||||
fallback: ["BouncyCastle"]
|
||||
algorithms:
|
||||
signing: ["RSA-PSS-SHA384", "ECDSA-P384-SHA384"]
|
||||
hashing: ["SHA-384", "SHA-512"]
|
||||
encryption: ["AES-256-GCM"]
|
||||
keyExchange: ["ECDH-P384", "ML-KEM-768"] # Hybrid PQ
|
||||
```
|
||||
|
||||
**Activation:**
|
||||
```bash
|
||||
# Via environment variable
|
||||
export STELLAOPS_CRYPTO_PROFILE=us-fips
|
||||
|
||||
# Via Docker Compose
|
||||
docker compose -f docker-compose.yml -f docker-compose.fips.yml up
|
||||
```
|
||||
|
||||
### `etc/env/` - Environment Profiles
|
||||
|
||||
```
|
||||
etc/env/
|
||||
├── dev.env.sample # Development defaults
|
||||
├── stage.env.sample # Staging environment
|
||||
├── prod.env.sample # Production hardened
|
||||
└── airgap.env.sample # Air-gapped deployment
|
||||
```
|
||||
|
||||
**Environment profile contents:**
|
||||
```bash
|
||||
# dev.env.sample
|
||||
STELLAOPS_LOG_LEVEL=Debug
|
||||
STELLAOPS_TELEMETRY_ENABLED=true
|
||||
STELLAOPS_TELEMETRY_ENDPOINT=http://localhost:4317
|
||||
POSTGRES_HOST=localhost
|
||||
POSTGRES_DB=stellaops_dev
|
||||
```
|
||||
|
||||
**Usage with Docker Compose:**
|
||||
```bash
|
||||
cp etc/env/dev.env.sample .env
|
||||
docker compose up
|
||||
```
|
||||
|
||||
### `etc/llm-providers/` - AI/LLM Configuration
|
||||
|
||||
```
|
||||
etc/llm-providers/
|
||||
├── claude.yaml.sample # Anthropic Claude
|
||||
├── ollama.yaml.sample # Local Ollama server
|
||||
├── openai.yaml.sample # OpenAI API
|
||||
└── llama-server.yaml.sample # llama.cpp server
|
||||
```
|
||||
|
||||
**Provider configuration:**
|
||||
```yaml
|
||||
# claude.yaml.sample
|
||||
provider: claude
|
||||
endpoint: https://api.anthropic.com
|
||||
model: claude-sonnet-4-20250514
|
||||
# API key via environment: STELLAOPS_LLM_APIKEY
|
||||
options:
|
||||
maxTokens: 4096
|
||||
temperature: 0.1
|
||||
```
|
||||
|
||||
**Offline/air-gapped deployments** should use `ollama.yaml.sample` or `llama-server.yaml.sample` with local model bundles.
|
||||
|
||||
### `etc/notify/` - Notification Service
|
||||
|
||||
```
|
||||
etc/notify/
|
||||
├── notify.yaml.sample # Main notify config
|
||||
└── templates/ # Notification templates
|
||||
├── vex-decision.html # VEX decision notification
|
||||
├── scan-complete.html # Scan completion
|
||||
├── policy-violation.html # Policy gate failure
|
||||
└── alert.html # Generic alert
|
||||
```
|
||||
|
||||
**Template variables:**
|
||||
```html
|
||||
<!-- vex-decision.html -->
|
||||
<h2>VEX Decision: {{.Decision}}</h2>
|
||||
<p>Vulnerability: {{.VulnId}}</p>
|
||||
<p>Justification: {{.Justification}}</p>
|
||||
<p>Decided by: {{.DecidedBy}} at {{.Timestamp}}</p>
|
||||
```
|
||||
|
||||
### `etc/plugins/` - Plugin Manifests
|
||||
|
||||
Plugin manifests define available plugins. **Compiled binaries** live in `plugins/` at root.
|
||||
|
||||
```
|
||||
etc/plugins/
|
||||
├── notify/
|
||||
│ ├── email.yaml # SMTP email plugin
|
||||
│ ├── slack.yaml # Slack webhook
|
||||
│ ├── teams.yaml # Microsoft Teams
|
||||
│ └── webhook.yaml # Generic webhook
|
||||
└── scanner/
|
||||
├── lang/ # Language ecosystem analyzers
|
||||
│ ├── dotnet.yaml
|
||||
│ ├── go.yaml
|
||||
│ ├── java.yaml
|
||||
│ ├── node.yaml
|
||||
│ ├── python.yaml
|
||||
│ ├── ruby.yaml
|
||||
│ └── rust.yaml
|
||||
└── os/ # OS package analyzers
|
||||
├── apk.yaml # Alpine
|
||||
├── dpkg.yaml # Debian/Ubuntu
|
||||
└── rpm.yaml # RHEL/Fedora
|
||||
```
|
||||
|
||||
**Manifest structure:**
|
||||
```yaml
|
||||
# etc/plugins/scanner/lang/java.yaml
|
||||
id: stellaops.scanner.analyzer.java
|
||||
name: Java Ecosystem Analyzer
|
||||
version: 1.0.0
|
||||
assembly: StellaOps.Scanner.Analyzers.Lang.Java.dll
|
||||
capabilities:
|
||||
- maven
|
||||
- gradle
|
||||
- sbt
|
||||
filePatterns:
|
||||
- "pom.xml"
|
||||
- "build.gradle"
|
||||
- "build.gradle.kts"
|
||||
- "build.sbt"
|
||||
```
|
||||
|
||||
### `etc/policy/` - Policy Engine & Packs
|
||||
|
||||
```
|
||||
etc/policy/
|
||||
├── policy-engine.yaml.sample # Engine configuration
|
||||
├── policy-gates.yaml.sample # Gate definitions
|
||||
├── packs/ # Policy pack bundles
|
||||
│ ├── starter-day1.yaml # Starter pack for new deployments
|
||||
│ └── enterprise.yaml.sample # Enterprise compliance pack
|
||||
└── schemas/
|
||||
└── policy-pack.schema.json # JSON Schema for validation
|
||||
```
|
||||
|
||||
**Policy gate example:**
|
||||
```yaml
|
||||
# policy-gates.yaml.sample
|
||||
gates:
|
||||
- id: no-critical-unfixed
|
||||
name: "No Critical Unfixed Vulnerabilities"
|
||||
condition: |
|
||||
count(findings where severity == "CRITICAL" and fixAvailable == true) == 0
|
||||
action: block
|
||||
|
||||
- id: sbom-required
|
||||
name: "SBOM Must Be Present"
|
||||
condition: |
|
||||
sbom != null and sbom.components.length > 0
|
||||
action: warn
|
||||
```
|
||||
|
||||
### `etc/scanner/` - Container Scanning
|
||||
|
||||
```
|
||||
etc/scanner/
|
||||
├── scanner.yaml.sample # Main scanner config
|
||||
└── poe.yaml.sample # Proof-of-exploit configuration
|
||||
```
|
||||
|
||||
**Key settings:**
|
||||
```yaml
|
||||
# scanner.yaml.sample
|
||||
scanner:
|
||||
concurrency:
|
||||
maxParallel: 4
|
||||
maxMemoryMb: 4096
|
||||
analyzers:
|
||||
enabled:
|
||||
- lang/*
|
||||
- os/*
|
||||
disabled: []
|
||||
sbom:
|
||||
formats: [spdx-3.0.1-json, cyclonedx-1.6-json]
|
||||
includeFiles: true
|
||||
evidence:
|
||||
generateAttestations: true
|
||||
signAttestations: true
|
||||
```
|
||||
|
||||
### `etc/vex/` - VEX Processing
|
||||
|
||||
```
|
||||
etc/vex/
|
||||
├── excititor.yaml.sample # VEX ingestion config
|
||||
├── vexlens.yaml.sample # Consensus computation
|
||||
└── trust-lattice.yaml.sample # Issuer trust configuration
|
||||
```
|
||||
|
||||
**Trust lattice example:**
|
||||
```yaml
|
||||
# trust-lattice.yaml.sample
|
||||
lattice:
|
||||
trustLevels:
|
||||
- id: vendor
|
||||
weight: 100
|
||||
description: "Vendor/upstream maintainer"
|
||||
- id: coordinator
|
||||
weight: 80
|
||||
description: "Security coordinator (CERT, etc.)"
|
||||
- id: community
|
||||
weight: 40
|
||||
description: "Community contributor"
|
||||
precedenceRules:
|
||||
- higher_trust_wins
|
||||
- more_recent_wins_on_tie
|
||||
```
|
||||
|
||||
## Directories Outside `etc/`
|
||||
|
||||
### `plugins/` - Compiled Plugin Binaries
|
||||
|
||||
Runtime artifacts, **not configuration**. Built during CI/CD.
|
||||
|
||||
```
|
||||
plugins/
|
||||
├── scanner/
|
||||
│ ├── analyzers/
|
||||
│ │ ├── lang/ # Language analyzers (.dll, .pdb)
|
||||
│ │ └── os/ # OS analyzers
|
||||
│ ├── buildx/ # BuildX SBOM plugin
|
||||
│ └── entrytrace/ # Binary tracing plugin
|
||||
└── notify/
|
||||
└── ...
|
||||
```
|
||||
|
||||
### `opt/` - Optional Vendor Packages
|
||||
|
||||
Customer-provided packages for specific crypto providers:
|
||||
|
||||
```
|
||||
opt/
|
||||
└── cryptopro/
|
||||
└── downloads/ # CryptoPro CSP packages (Russia)
|
||||
```
|
||||
|
||||
### `offline/` - Air-Gap Operational Data
|
||||
|
||||
Runtime state for air-gapped deployments:
|
||||
|
||||
```
|
||||
offline/
|
||||
├── feeds/ # Cached vulnerability feeds
|
||||
├── packages/ # Cached package metadata
|
||||
└── advisory-ai/ # Offline AI model bundles
|
||||
```
|
||||
|
||||
## Docker Compose Integration
|
||||
|
||||
### Volume Mounts
|
||||
|
||||
```yaml
|
||||
# docker-compose.yml
|
||||
services:
|
||||
scanner:
|
||||
volumes:
|
||||
# Configuration (read-only)
|
||||
- ./etc/scanner:/app/etc/scanner:ro
|
||||
- ./etc/certificates/trust-roots:/etc/ssl/certs/stellaops:ro
|
||||
|
||||
# Plugin binaries (read-only)
|
||||
- ./plugins/scanner:/app/plugins/scanner:ro
|
||||
|
||||
# Runtime data (read-write)
|
||||
- scanner-data:/var/lib/stellaops/scanner
|
||||
```
|
||||
|
||||
### Environment File
|
||||
|
||||
```yaml
|
||||
# docker-compose.yml
|
||||
services:
|
||||
scanner:
|
||||
env_file:
|
||||
- ./etc/env/dev.env # Copied from dev.env.sample
|
||||
```
|
||||
|
||||
### Crypto Profile Overlays
|
||||
|
||||
```bash
|
||||
# FIPS deployment
|
||||
docker compose -f docker-compose.yml -f devops/compose/docker-compose.fips.yml up
|
||||
|
||||
# eIDAS deployment
|
||||
docker compose -f docker-compose.yml -f devops/compose/docker-compose.eidas.yml up
|
||||
|
||||
# Air-gapped with Russian crypto
|
||||
docker compose -f docker-compose.yml \
|
||||
-f devops/compose/docker-compose.airgap.yml \
|
||||
-f devops/compose/docker-compose.russia.yml up
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
### 1. Initialize Configuration
|
||||
|
||||
```bash
|
||||
# Clone sample configs
|
||||
./devops/scripts/init-config.sh dev
|
||||
|
||||
# This copies all .sample files to active configs for development
|
||||
```
|
||||
|
||||
### 2. Customize for Environment
|
||||
|
||||
```bash
|
||||
# Edit main service configs
|
||||
vi etc/scanner/scanner.yaml
|
||||
vi etc/authority/authority.yaml
|
||||
|
||||
# Set environment-specific values
|
||||
vi etc/env/dev.env
|
||||
```
|
||||
|
||||
### 3. Select Crypto Profile (if needed)
|
||||
|
||||
```bash
|
||||
# For US/FIPS compliance
|
||||
cp etc/crypto/profiles/us-fips/env.sample etc/crypto/profiles/us-fips/env
|
||||
export STELLAOPS_CRYPTO_PROFILE=us-fips
|
||||
```
|
||||
|
||||
### 4. Start Services
|
||||
|
||||
```bash
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
## Configuration Validation
|
||||
|
||||
```bash
|
||||
# Validate all configuration files
|
||||
./devops/scripts/validate-config.sh
|
||||
|
||||
# Validate specific service
|
||||
./devops/scripts/validate-config.sh scanner
|
||||
|
||||
# Validate policy packs against schema
|
||||
./devops/scripts/validate-policy-packs.sh
|
||||
```
|
||||
|
||||
## Migration from Legacy Structure
|
||||
|
||||
If upgrading from a deployment with the legacy multi-directory structure:
|
||||
|
||||
| Legacy Location | New Location |
|
||||
|-----------------|--------------|
|
||||
| `certificates/` | `etc/certificates/` |
|
||||
| `config/env/.env.*` | `etc/crypto/profiles/*/env.sample` |
|
||||
| `config/crypto-profiles.sample.json` | `etc/crypto/crypto.yaml.sample` |
|
||||
| `policies/` | `etc/policy/` |
|
||||
| `etc/rootpack/` | `etc/crypto/profiles/` |
|
||||
|
||||
See `docs/operations/configuration-migration.md` for detailed migration steps.
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **Never commit active configs** - `.gitignore` excludes `*.yaml` (only `*.yaml.sample` committed)
|
||||
2. **Secrets via environment** - Use `STELLAOPS_*` env vars or external secret managers
|
||||
3. **Development secrets are clearly marked** - `etc/secrets/` contains only dev/sample keys
|
||||
4. **Production signing keys** - Should come from HSM, Vault, or KMS - never from files
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [PostgreSQL Operations Guide](./postgresql-guide.md)
|
||||
- [Air-Gap Deployment](../24_OFFLINE_KIT.md)
|
||||
- [Crypto Profile Reference](../modules/cryptography/architecture.md)
|
||||
- [Policy Engine Guide](../modules/policy/architecture.md)
|
||||
Reference in New Issue
Block a user