Refactor code structure and optimize performance across multiple modules
This commit is contained in:
StellaOps Bot
166
devops/docs/README.md
Normal file
166
devops/docs/README.md
Normal file
@@ -0,0 +1,166 @@
|
||||
# DevOps Infrastructure
|
||||
|
||||
This directory contains operational tooling, deployment configurations, and CI/CD support for StellaOps.
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
devops/
|
||||
├── ansible/ # Ansible playbooks for deployment automation
|
||||
├── compose/ # Docker Compose configurations
|
||||
├── database/ # Database schemas and migrations
|
||||
│ ├── mongo/ # MongoDB (deprecated)
|
||||
│ └── postgres/ # PostgreSQL schemas
|
||||
├── docker/ # Dockerfiles and container build scripts
|
||||
│ ├── Dockerfile.ci # CI runner environment
|
||||
│ └── base/ # Base images
|
||||
├── docs/ # This documentation
|
||||
├── gitlab/ # GitLab CI templates (legacy)
|
||||
├── helm/ # Helm charts for Kubernetes deployment
|
||||
├── logging/ # Logging configuration templates
|
||||
│ ├── serilog.json.template # Serilog config for .NET services
|
||||
│ ├── filebeat.yml # Filebeat for log shipping
|
||||
│ └── logrotate.conf # Log rotation configuration
|
||||
├── observability/ # Monitoring, metrics, and tracing
|
||||
├── offline/ # Air-gap deployment support
|
||||
│ ├── airgap/ # Air-gap bundle scripts
|
||||
│ └── kit/ # Offline installation kit
|
||||
├── releases/ # Release artifacts and manifests
|
||||
├── scripts/ # Operational scripts
|
||||
├── services/ # Per-service operational configs
|
||||
├── telemetry/ # OpenTelemetry and metrics configs
|
||||
└── tools/ # DevOps tooling
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Local CI Environment
|
||||
|
||||
Build and run the CI Docker environment locally:
|
||||
|
||||
```bash
|
||||
# Build the CI image
|
||||
docker build -f devops/docker/Dockerfile.ci -t stellaops-ci:local .
|
||||
|
||||
# Run tests in CI environment
|
||||
docker run --rm -v $(pwd):/workspace stellaops-ci:local \
|
||||
dotnet test --filter "Category=Unit"
|
||||
```
|
||||
|
||||
### Local Testing
|
||||
|
||||
```bash
|
||||
# Run all PR-gating tests
|
||||
./devops/scripts/test-local.sh
|
||||
|
||||
# Validate compose configurations
|
||||
./devops/scripts/validate-compose.sh
|
||||
|
||||
# Validate Helm charts
|
||||
./.gitea/scripts/validate/validate-helm.sh
|
||||
```
|
||||
|
||||
### Logging Configuration
|
||||
|
||||
The `logging/` directory contains templates for centralized logging:
|
||||
|
||||
1. **Serilog** (`serilog.json.template`) - Structured logging for .NET services
|
||||
- Console and file sinks
|
||||
- Rolling files with 14-day retention
|
||||
- 100MB file size limit with roll-over
|
||||
- Environment-variable templating
|
||||
|
||||
2. **Filebeat** (`filebeat.yml`) - Log shipping to Elasticsearch/Logstash
|
||||
- JSON log parsing from Serilog output
|
||||
- Container log support
|
||||
- Kubernetes metadata enrichment
|
||||
- Air-gap fallback to file output
|
||||
|
||||
3. **Logrotate** (`logrotate.conf`) - System-level log rotation
|
||||
- Daily rotation with 14-day retention
|
||||
- Compression with delay
|
||||
- Service-specific overrides for high-volume services
|
||||
|
||||
To use:
|
||||
|
||||
```bash
|
||||
# Copy template and customize
|
||||
cp devops/logging/serilog.json.template /etc/stellaops/serilog.json
|
||||
|
||||
# Set service name
|
||||
export STELLAOPS_SERVICE_NAME=scanner
|
||||
|
||||
# Install filebeat config (requires root)
|
||||
sudo cp devops/logging/filebeat.yml /etc/filebeat/filebeat.yml
|
||||
|
||||
# Install logrotate config (requires root)
|
||||
sudo cp devops/logging/logrotate.conf /etc/logrotate.d/stellaops
|
||||
```
|
||||
|
||||
## Compose Profiles
|
||||
|
||||
The `compose/` directory contains Docker Compose configurations with profiles:
|
||||
|
||||
| Profile | Description |
|
||||
|---------|-------------|
|
||||
| `core` | Essential services (PostgreSQL, Router, Authority) |
|
||||
| `scanner` | Vulnerability scanning services |
|
||||
| `full` | All services for complete deployment |
|
||||
| `dev` | Development profile with hot-reload |
|
||||
| `test` | Testing profile with test containers |
|
||||
|
||||
```bash
|
||||
# Start core services
|
||||
docker compose --profile core up -d
|
||||
|
||||
# Start full stack
|
||||
docker compose --profile full up -d
|
||||
```
|
||||
|
||||
## Helm Charts
|
||||
|
||||
The `helm/` directory contains Helm charts for Kubernetes:
|
||||
|
||||
```bash
|
||||
# Lint charts
|
||||
helm lint devops/helm/stellaops
|
||||
|
||||
# Template with values
|
||||
helm template stellaops devops/helm/stellaops -f values.yaml
|
||||
|
||||
# Install
|
||||
helm install stellaops devops/helm/stellaops -n stellaops --create-namespace
|
||||
```
|
||||
|
||||
## Release Process
|
||||
|
||||
See [RELEASE_PROCESS.md](../../docs/releases/RELEASE_PROCESS.md) for the complete release workflow.
|
||||
|
||||
Quick release commands:
|
||||
|
||||
```bash
|
||||
# Dry-run release build
|
||||
python devops/release/build_release.py --version 2026.04.0 --dry-run
|
||||
|
||||
# Verify release artifacts
|
||||
python devops/release/verify_release.py --release-dir out/release
|
||||
```
|
||||
|
||||
## Air-Gap / Offline Deployment
|
||||
|
||||
The `offline/` directory contains tools for air-gapped environments:
|
||||
|
||||
```bash
|
||||
# Create offline bundle
|
||||
./devops/offline/airgap/create-bundle.sh --version 2026.04
|
||||
|
||||
# Import on air-gapped system
|
||||
./devops/offline/kit/import-bundle.sh stellaops-2026.04-bundle.tar.gz
|
||||
```
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Release Engineering Playbook](../../docs/13_RELEASE_ENGINEERING_PLAYBOOK.md)
|
||||
- [Versioning Strategy](../../docs/releases/VERSIONING.md)
|
||||
- [Offline Kit Guide](../../docs/24_OFFLINE_KIT.md)
|
||||
- [CI/CD Workflows](../../.gitea/workflows/README.md)
|
||||
Reference in New Issue
Block a user