# StellaOps Developer Onboarding Guide > **Target Audience:** DevOps operators with developer knowledge who need to understand, deploy, and debug the StellaOps platform. ## Table of Contents 1. [Architecture Overview](#architecture-overview) 2. [Prerequisites](#prerequisites) 3. [Quick Start - Full Platform in Docker](#quick-start) 4. [Hybrid Debugging Workflow](#hybrid-debugging-workflow) 5. [Service-by-Service Debugging Guide](#service-by-service-debugging-guide) 6. [Configuration Deep Dive](#configuration-deep-dive) 7. [Common Development Workflows](#common-development-workflows) 8. [Troubleshooting](#troubleshooting) --- ## Architecture Overview StellaOps is a deterministic SBOM + VEX platform built as a microservices architecture with 36+ services organized into functional domains. ### Runtime Topology - High-Level ``` ┌─────────────────────────────────────────────────────────────────────┐ │ INFRASTRUCTURE LAYER │ │ ┌──────────────────┐ ┌──────────────────┐ ┌─────────────────┐ │ │ │ PostgreSQL │ │ Valkey │ │ RustFS │ │ │ │ (v16+ ONLY) │ │ (Redis-compat) │ │ (S3-like API) │ │ │ │ │ │ - Caching │ │ - Artifacts │ │ │ │ All services use │ │ - DPoP nonces │ │ - SBOMs │ │ │ │ PostgreSQL for │ │ - Event queues │ │ - Signatures │ │ │ │ persistent data │ │ - Rate limiting│ │ │ │ │ └──────────────────┘ └──────────────────┘ └─────────────────┘ │ │ │ │ ┌──────────────────────────────────────────────────────────────┐ │ │ │ Optional: NATS JetStream (alternative transport for queues) │ │ │ │ Only used if explicitly configured in appsettings │ │ │ └──────────────────────────────────────────────────────────────┘ │ └─────────────────────────────────────────────────────────────────────┘ ▲ │ ┌─────────────────────────────────────────────────────────────────────┐ │ AUTHENTICATION & SIGNING │ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │ │ Authority │─▶│ Signer │─▶│ Attestor │ │ │ │ (OAuth2/OIDC)│ │(DSSE/PKIX) │ │(in-toto/DSSE)│ │ │ └──────────────┘ └──────────────┘ └──────────────┘ │ └─────────────────────────────────────────────────────────────────────┘ ▲ │ ┌─────────────────────────────────────────────────────────────────────┐ │ INGESTION & AGGREGATION │ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │ │ Concelier │ │ Excititor │ │IssuerDirectry│ │ │ │(Advisories) │ │ (VEX) │ │(CSAF Pubshrs)│ │ │ └──────────────┘ └──────────────┘ └──────────────┘ │ └─────────────────────────────────────────────────────────────────────┘ ▲ │ ┌─────────────────────────────────────────────────────────────────────┐ │ SCANNING & ANALYSIS │ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │ │Scanner.Web │ │Scanner.Worker│ │ AdvisoryAI │ │ │ │(API/Control) │ │(Analyzers) │ │(ML Analysis) │ │ │ └──────────────┘ └──────────────┘ └──────────────┘ │ │ ┌──────────────┐ ┌──────────────┐ │ │ │ RiskEngine │ │ Policy │ │ │ │ (Scoring) │ │ (Engine) │ │ │ └──────────────┘ └──────────────┘ │ └─────────────────────────────────────────────────────────────────────┘ ▲ │ ┌─────────────────────────────────────────────────────────────────────┐ │ ORCHESTRATION & WORKFLOW │ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │ │ Scheduler │ │ Orchestrator │ │ TaskRunner │ │ │ │(Job Sched) │ │(Coordinator) │ │(Executor) │ │ │ └──────────────┘ └──────────────┘ └──────────────┘ │ └─────────────────────────────────────────────────────────────────────┘ ▲ │ ┌─────────────────────────────────────────────────────────────────────┘ │ EVENTS & NOTIFICATIONS │ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │ │ Notify │ │ Notifier │ │TimelineIndex │ │ │ │(Slack/Teams) │ │ (Advanced) │ │ (Events) │ │ │ └──────────────┘ └──────────────┘ └──────────────┘ │ └─────────────────────────────────────────────────────────────────────┘ ▲ │ ┌─────────────────────────────────────────────────────────────────────┐ │ DATA & EXPORT │ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │ │ExportCenter │ │EvidenceLocker│ │FindingsLedger│ │ │ │(SARIF/SBOM) │ │(Artifacts) │ │(Audit Trail) │ │ │ └──────────────┘ └──────────────┘ └──────────────┘ │ └─────────────────────────────────────────────────────────────────────┘ ▲ │ ┌─────────────────────────────────────────────────────────────────────┐ │ USER EXPERIENCE │ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │ │ Gateway │ │ Web (UI) │ │ CLI │ │ │ │ (API Router) │ │ (Angular v17)│ │(Multi-plat) │ │ │ └──────────────┘ └──────────────┘ └──────────────┘ │ └─────────────────────────────────────────────────────────────────────┘ ``` ### Detailed Request Flow - Scan Execution Example This diagram shows a complete scan request lifecycle with detailed routing through services: ``` ┌──────────────────────────────────────────────────────────────────────────────────┐ │ 1. CLIENT REQUEST (CLI or Web UI) │ │ $ stella scan docker://alpine:latest --sbom-format=spdx │ └───────────────────────────────────┬──────────────────────────────────────────────┘ │ HTTPS ▼ ┌──────────────────────────────────────────────────────────────────────────────────┐ │ 2. GATEWAY (API Router) │ │ - Terminates TLS │ │ - Routes to appropriate backend service │ │ - Load balancing (if multiple instances) │ └───────────────────────────────────┬──────────────────────────────────────────────┘ │ HTTP (internal) ▼ ┌──────────────────────────────────────────────────────────────────────────────────┐ │ 3. AUTHORITY (Authentication) │ │ - Validates OAuth2 access token (DPoP-bound) │ │ - Checks DPoP proof against Valkey nonce cache │ │ - Returns user identity and scopes │ │ │ │ ┌─────────────┐ │ │ │ Valkey │◀── DPoP nonce validation (GET/SET) │ │ │ (Cache) │ │ │ └─────────────┘ │ │ ┌─────────────┐ │ │ │ PostgreSQL │◀── User/client lookup (SELECT) │ │ └─────────────┘ │ └───────────────────────────────────┬──────────────────────────────────────────────┘ │ Authenticated request ▼ ┌──────────────────────────────────────────────────────────────────────────────────┐ │ 4. SCANNER.WEB (Scan API Controller) │ │ - Validates scan request parameters │ │ - Creates scan job record in PostgreSQL │ │ - Enqueues scan job to Valkey queue (default) or NATS (if configured) │ │ │ │ ┌─────────────┐ │ │ │ PostgreSQL │◀── INSERT scan_jobs (job_id, image_ref, status='pending') │ │ └─────────────┘ │ │ ┌─────────────┐ │ │ │ Valkey │◀── XADD scanner:jobs (enqueue job message) │ │ │ (Queue) │ │ │ └─────────────┘ │ │ │ │ Returns: HTTP 202 Accepted { "job_id": "scan-abc123", "status": "queued" } │ └───────────────────────────────────┬──────────────────────────────────────────────┘ │ │ (Client polls for status) │ ┌──────────────────────────────────────────────────────────────────────────────────┐ │ 5. SCANNER.WORKER (Background Processor) │ │ - Consumes job from Valkey queue (XREADGROUP scanner:jobs) │ │ - Updates job status to 'running' │ │ - Downloads container image from registry │ │ - Executes analyzers (OS packages, language deps, files) │ │ - Generates SBOM (SPDX/CycloneDX) │ │ - Stores artifacts to RustFS │ │ │ │ ┌─────────────┐ │ │ │ PostgreSQL │◀── UPDATE scan_jobs SET status='running' │ │ │ │◀── INSERT sbom_documents, packages, vulnerabilities │ │ │ │◀── UPDATE scan_jobs SET status='completed' │ │ └─────────────┘ │ │ ┌─────────────┐ │ │ │ RustFS │◀── PUT /artifacts/scan-abc123/sbom.spdx.json │ │ │ (S3 API) │◀── PUT /artifacts/scan-abc123/image-layers.tar.gz │ │ └─────────────┘ │ │ ┌─────────────┐ │ │ │ Valkey │◀── XADD scanner:events (publish scan.completed event) │ │ │(Event Stream│ │ │ └─────────────┘ │ └───────────────────────────────────┬──────────────────────────────────────────────┘ │ Event published ▼ ┌──────────────────────────────────────────────────────────────────────────────────┐ │ 6. EVENT PROPAGATION (Valkey Streams) │ │ │ │ ┌───────────────────────────────────────────────────────────────────┐ │ │ │ Valkey Event Stream: "scanner:events" │ │ │ │ Event: { "type": "scan.completed", "job_id": "scan-abc123", ... } │ │ │ └───────────────────────────────────────────────────────────────────┘ │ │ │ │ │ ┌──────────────┼──────────────┬───────────────┐ │ │ ▼ ▼ ▼ ▼ │ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │ │ Notify │ │Timeline │ │ Policy │ │ Export │ │ │ │ Worker │ │ Indexer │ │ Engine │ │ Center │ │ │ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │ │ │ │ │ │ │ │ │ (all subscribe to scanner:events via XREADGROUP) │ └─────────┼───────────────┼──────────────┼───────────────┼──────────────────────┘ │ │ │ │ ▼ ▼ ▼ ▼ ┌──────────────┐ ┌──────────────┐ ┌──────────┐ ┌──────────────┐ │ 7a. NOTIFY │ │ 7b. TIMELINE │ │ 7c.POLICY│ │ 7d. EXPORT │ │ │ │ INDEXER │ │ ENGINE │ │ CENTER │ │ - Query scan │ │ │ │ │ │ │ │ results │ │ - Index event│ │ - Eval │ │ - Generate │ │ - Check user │ │ timeline │ │ policy │ │ SARIF │ │ notif prefs│ │ - Store in │ │ rules │ │ - Export to │ │ - Send Slack │ │ PostgreSQL │ │ - Block/ │ │ external │ │ message │ │ │ │ Allow │ │ systems │ │ │ │ │ │ │ │ │ │ PostgreSQL ◀─┤ │ PostgreSQL ◀─┤ │PostgreSQL│ │ RustFS ◀─┤ │ (user prefs) │ │ (timeline) │ │(policies)│ │ (exports) │ └──────────────┘ └──────────────┘ └──────────┘ └──────────────┘ ``` ### Export Flow - SBOM Distribution ``` ┌──────────────────────────────────────────────────────────────────────────────────┐ │ EXPORT REQUEST: GET /api/v1/scans/{scan_id}/export?format=spdx │ └───────────────────────────────────┬──────────────────────────────────────────────┘ │ ▼ ┌──────────────────────────────────────────────────────────────────────────────────┐ │ SCANNER.WEB or EXPORT CENTER │ │ │ │ 1. Query scan metadata from PostgreSQL │ │ ┌─────────────┐ │ │ │ PostgreSQL │◀── SELECT * FROM scan_jobs WHERE job_id = $1 │ │ │ │◀── SELECT * FROM sbom_documents WHERE scan_id = $1 │ │ └─────────────┘ │ │ │ │ 2. Retrieve SBOM artifact from RustFS │ │ ┌─────────────┐ │ │ │ RustFS │◀── GET /artifacts/scan-abc123/sbom.spdx.json │ │ └─────────────┘ │ │ │ │ 3. Sign SBOM with Signer service │ │ ┌─────────────┐ │ │ │ Signer │◀── POST /api/v1/sign (SBOM payload) │ │ │ │──▶ Returns: DSSE envelope with signature │ │ └─────────────┘ │ │ │ │ 4. Create in-toto attestation with Attestor │ │ ┌─────────────┐ │ │ │ Attestor │◀── POST /api/v1/attest (signed SBOM) │ │ │ │──▶ Returns: in-toto attestation bundle │ │ └─────────────┘ │ │ │ │ 5. Store final bundle to RustFS │ │ ┌─────────────┐ │ │ │ RustFS │◀── PUT /artifacts/scan-abc123/bundle.jsonl │ │ └─────────────┘ │ │ │ │ 6. Return signed bundle to client │ │ Returns: HTTP 200 OK (application/vnd.in-toto+json) │ └──────────────────────────────────────────────────────────────────────────────────┘ ``` ### Notification Flow - Vulnerability Alert ``` ┌──────────────────────────────────────────────────────────────────────────────────┐ │ TRIGGER: New critical CVE detected in existing scan │ │ Source: Concelier advisory ingestion │ └───────────────────────────────────┬──────────────────────────────────────────────┘ │ ▼ ┌──────────────────────────────────────────────────────────────────────────────────┐ │ CONCELIER.WORKER (Advisory Processor) │ │ │ │ 1. Ingest new advisory from NVD/OSV/CSAF │ │ ┌─────────────┐ │ │ │ PostgreSQL │◀── INSERT INTO advisories (cve_id, severity, ...) │ │ └─────────────┘ │ │ │ │ 2. Match advisory against existing scans (PURL/CPE matching) │ │ ┌─────────────┐ │ │ │ PostgreSQL │◀── SELECT scans WHERE package_purl IN (affected_purls) │ │ └─────────────┘ │ │ │ │ 3. Publish drift event to Valkey │ │ ┌─────────────┐ │ │ │ Valkey │◀── XADD concelier:drift (new vulnerability found) │ │ └─────────────┘ │ └───────────────────────────────────┬──────────────────────────────────────────────┘ │ Event published ▼ ┌──────────────────────────────────────────────────────────────────────────────────┐ │ NOTIFY.WORKER (Notification Processor) │ │ │ │ 1. Consume drift event from Valkey stream │ │ ┌─────────────┐ │ │ │ Valkey │◀── XREADGROUP concelier:drift notify-workers │ │ └─────────────┘ │ │ │ │ 2. Query user notification preferences │ │ ┌─────────────┐ │ │ │ PostgreSQL │◀── SELECT * FROM user_notification_preferences │ │ │ │ WHERE user_id = scan_owner AND channel = 'slack' │ │ └─────────────┘ │ │ │ │ 3. Render notification template │ │ Template: "🚨 New critical CVE-2024-1234 affects alpine:latest scan" │ │ │ │ 4. Deliver notification via configured channels │ │ ┌─────────────────────────────────────────────────────────┐ │ │ │ External APIs │ │ │ │ - POST https://hooks.slack.com/services/T00/B00/xxx │ │ │ │ - POST https://graph.microsoft.com/v1.0/teams/channels │ │ │ │ - SMTP send (email) │ │ │ └─────────────────────────────────────────────────────────┘ │ │ │ │ 5. Store delivery receipt in PostgreSQL │ │ ┌─────────────┐ │ │ │ PostgreSQL │◀── INSERT INTO notification_deliveries (status, ...) │ │ └─────────────┘ │ └──────────────────────────────────────────────────────────────────────────────────┘ ``` ### Key Architectural Principles 1. **Deterministic Evidence** - Same inputs always produce same outputs 2. **VEX-First Decisioning** - Policy decisions based on OpenVEX statements 3. **Offline-First** - Full air-gap operation supported 4. **Plugin Architecture** - Extensible connectors for advisories, analyzers, auth 5. **Sovereign Crypto** - FIPS, eIDAS, GOST, SM support 6. **Schema Isolation** - Per-module PostgreSQL schemas ### Service Categories | Category | Services | Purpose | |----------|----------|---------| | **Infrastructure** | PostgreSQL v16+, Valkey 8.0, RustFS, NATS (optional) | Database, cache/messaging, object storage, optional queue transport | | **Auth & Signing** | Authority, Signer, Attestor | OAuth2/OIDC with DPoP, cryptographic signing, in-toto attestations | | **Ingestion** | Concelier, Excititor, IssuerDirectory | Advisory/VEX ingestion, normalization, merging, CSAF publisher discovery | | **Scanning** | Scanner.Web, Scanner.Worker, AdvisoryAI | Container scanning, SBOM generation (SPDX/CDX), ML vulnerability analysis | | **Policy & Risk** | Policy Engine, RiskEngine | OPA/Rego policy evaluation, risk scoring, exploitability assessment | | **Orchestration** | Scheduler, Orchestrator, TaskRunner | Job scheduling, workflow coordination, distributed task execution | | **Notifications** | Notify, Notifier, TimelineIndexer | Event delivery (Slack/Teams/Email), notification management, timeline tracking | | **Data & Export** | ExportCenter, EvidenceLocker, FindingsLedger | SARIF/SBOM export, evidence storage, immutable audit trail | | **User Experience** | Gateway, Web UI, CLI | API routing, Angular v17 UI, multi-platform command-line tools | --- ## Prerequisites ### Required Software 1. **Docker Desktop** (Windows/Mac) or **Docker Engine + Docker Compose** (Linux) - Version: 20.10+ recommended - Enable WSL2 backend (Windows) 2. **.NET 10 SDK** - Download: https://dotnet.microsoft.com/download/dotnet/10.0 - Verify: `dotnet --version` (should show 10.0.x) 3. **Visual Studio 2022** (v17.12+) or **Visual Studio Code** - Workload: ASP.NET and web development - Workload: .NET desktop development - Extension (VS Code): C# Dev Kit 4. **Git** - Version: 2.30+ recommended ### Optional Tools - **PostgreSQL Client** (psql, pgAdmin, DBeaver) - for database inspection - **Redis Insight** or **Another Redis Desktop Manager** - for Valkey inspection (Valkey is Redis-compatible) - **Postman/Insomnia** - for API testing - **AWS CLI or s3cmd** - for RustFS (S3-compatible) inspection ### System Requirements - **RAM:** 16 GB minimum, 32 GB recommended - **Disk:** 50 GB free space (for Docker images, volumes, build artifacts) - **CPU:** 4 cores minimum, 8 cores recommended --- ## Quick Start ### Step 1: Clone the Repository ```bash cd C:\dev\ git clone https://git.stella-ops.org/stella-ops.org/git.stella-ops.org cd git.stella-ops.org ``` ### Step 2: Prepare Environment Configuration ```bash # Copy the development environment template cd deploy\compose copy env\dev.env.example .env # Edit .env with your preferred text editor notepad .env ``` **Key settings to configure:** ```bash # PostgreSQL Database POSTGRES_USER=stellaops POSTGRES_PASSWORD=your_secure_password_here POSTGRES_DB=stellaops_platform POSTGRES_PORT=5432 # Valkey (Redis-compatible cache and messaging) VALKEY_PORT=6379 # RustFS Object Storage RUSTFS_HTTP_PORT=8080 # Service ports (adjust if conflicts exist) AUTHORITY_PORT=8440 SIGNER_PORT=8441 ATTESTOR_PORT=8442 CONCELIER_PORT=8445 SCANNER_WEB_PORT=8444 NOTIFY_WEB_PORT=8446 # Scanner configuration (Valkey default, can switch to NATS if needed) SCANNER_EVENTS_DRIVER=valkey SCANNER_EVENTS_DSN=valkey:6379 # Scheduler configuration (Valkey default, can switch to NATS if needed) SCHEDULER_QUEUE_KIND=Valkey SCHEDULER_QUEUE_VALKEY_URL=valkey:6379 # Authority configuration AUTHORITY_ISSUER=https://authority:8440 SIGNER_POE_INTROSPECT_URL=https://www.stella-ops.org/license/introspect ``` ### Step 3: Start the Full Platform ```bash # From deploy/compose directory docker compose -f docker-compose.dev.yaml up -d ``` **This will start all infrastructure and services:** - PostgreSQL v16+ (port 5432) - Primary database for all services - Valkey 8.0 (port 6379) - Cache, DPoP nonces, event streams, rate limiting - RustFS (port 8080) - S3-compatible object storage for artifacts/SBOMs - NATS JetStream (port 4222) - Optional transport (only if configured) - Authority (port 8440) - OAuth2/OIDC authentication - Signer (port 8441) - Cryptographic signing - Attestor (port 8442) - in-toto attestation generation - Scanner.Web (port 8444) - Scan API - Concelier (port 8445) - Advisory ingestion - And 30+ more services... ### Step 4: Verify Services Are Running ```bash # Check all services are up docker compose -f docker-compose.dev.yaml ps # Check logs for a specific service docker compose -f docker-compose.dev.yaml logs -f scanner-web # Check infrastructure health docker compose -f docker-compose.dev.yaml logs postgres docker compose -f docker-compose.dev.yaml logs valkey docker compose -f docker-compose.dev.yaml logs rustfs ``` ### Step 5: Access the Platform Open your browser and navigate to: - **RustFS:** http://localhost:8080 (S3-compatible object storage) - **Scanner API:** http://localhost:8444/swagger (if Swagger enabled) - **Concelier API:** http://localhost:8445/swagger - **Authority:** http://localhost:8440/.well-known/openid-configuration (OIDC discovery) --- ## Hybrid Debugging Workflow The hybrid workflow allows you to: 1. Run infrastructure (databases, queues) in Docker 2. Run most services in Docker 3. **Selectively debug** one or two services in Visual Studio ### Workflow Overview ``` ┌────────────────────────────────────────────────────────────┐ │ DOCKER ENVIRONMENT │ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │ │PostgreSQL│ │ Valkey │ │ RustFS │ │ │ │ (DB) │ │(Cache/Msg│ │(Storage) │ │ │ └──────────┘ └──────────┘ └──────────┘ │ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │ │ Authority│ │ Signer │ │ Attestor │ │ │ └──────────┘ └──────────┘ └──────────┘ │ │ ┌──────────┐ ┌──────────┐ │ │ │Concelier │ │ Excititor│ ← Running normally │ │ └──────────┘ └──────────┘ │ └────────────────────────────────────────────────────────────┘ ▲ │ HTTP calls + Valkey streams ▼ ┌────────────────────────────────────────────────────────────┐ │ VISUAL STUDIO (F5) │ │ ┌────────────────────────────────────────────┐ │ │ │ Scanner.WebService │ │ │ │ Running on http://localhost:5210 │ │ │ │ (Breakpoints, hot reload, debugging) │ ← YOU DEBUG HERE │ └────────────────────────────────────────────┘ │ └────────────────────────────────────────────────────────────┘ ``` ### Step-by-Step: Debug Scanner.WebService #### 1. Stop the Docker Container for Scanner ```bash cd deploy\compose docker compose -f docker-compose.dev.yaml stop scanner-web ``` **Verify it's stopped:** ```bash docker compose -f docker-compose.dev.yaml ps scanner-web # Should show: State = "exited" ``` #### 2. Configure Local Development Settings Create or modify the service's `appsettings.Development.json`: ```bash cd C:\dev\New folder\git.stella-ops.org\src\Scanner\StellaOps.Scanner.WebService ``` **Create `appsettings.Development.json`:** ```json { "Logging": { "LogLevel": { "Default": "Information", "Microsoft.AspNetCore": "Warning", "StellaOps": "Debug" } }, "ConnectionStrings": { "DefaultConnection": "Host=localhost;Port=5432;Database=stellaops_platform;Username=stellaops;Password=your_password_here;Include Error Detail=true" }, "Scanner": { "Storage": { "Mongo": { "ConnectionString": "mongodb://stellaops:your_password_here@localhost:27017" } }, "ArtifactStore": { "Driver": "rustfs", "Endpoint": "http://localhost:8080/api/v1", "Bucket": "scanner-artifacts", "TimeoutSeconds": 30 }, "Queue": { "Broker": "nats://localhost:4222" }, "Events": { "Enabled": false } }, "Authority": { "Issuer": "https://localhost:8440", "BaseUrl": "https://localhost:8440", "BypassNetworks": ["127.0.0.1", "::1"] } } ``` **Note:** Adjust connection strings to match your Docker infrastructure ports. If PostgreSQL is on Docker's bridge network, you may need to expose it on `localhost:5432`. #### 3. Expose Docker Services to localhost For services running in Docker to be accessible from your host machine, ensure ports are mapped in `docker-compose.dev.yaml`: ```yaml # Already configured in docker-compose.dev.yaml postgres: ports: - "${POSTGRES_PORT:-5432}:5432" mongo: ports: - "27017:27017" nats: ports: - "${NATS_CLIENT_PORT:-4222}:4222" rustfs: ports: - "${RUSTFS_HTTP_PORT:-8080}:8080" ``` **Verify connectivity:** ```bash # Test PostgreSQL psql -h localhost -U stellaops -d stellaops_platform # Test NATS telnet localhost 4222 # Test RustFS curl http://localhost:8080/health ``` #### 4. Open Solution in Visual Studio ```bash # Open the solution cd C:\dev\New folder\git.stella-ops.org start src\StellaOps.sln ``` **In Visual Studio:** 1. Right-click `StellaOps.Scanner.WebService` project 2. Select **"Set as Startup Project"** 3. Press **F5** to start debugging **Expected output:** ``` info: Microsoft.Hosting.Lifetime[14] Now listening on: http://localhost:5210 info: Microsoft.Hosting.Lifetime[0] Application started. Press Ctrl+C to shut down. ``` #### 5. Update Other Services to Call localhost Since you're running Scanner.WebService on `localhost:5210` instead of `scanner-web:8444`, you need to update any services that call it. **Option A: Environment Variables (Docker containers)** Update `.env` file: ```bash # Use host.docker.internal to reach host machine from Docker SCANNER_WEB_BASEURL=http://host.docker.internal:5210 ``` Restart dependent services: ```bash docker compose -f docker-compose.dev.yaml restart scheduler-web ``` **Option B: Modify docker-compose.dev.yaml** ```yaml scheduler-web: environment: SCHEDULER__WORKER__RUNNER__SCANNER__BASEADDRESS: "http://host.docker.internal:5210" ``` Then restart: ```bash docker compose -f docker-compose.dev.yaml up -d scheduler-web ``` #### 6. Set Breakpoints and Debug 1. Navigate to `Program.cs` in Scanner.WebService 2. Set a breakpoint on a line in a controller or service method 3. Trigger the endpoint using: - Swagger UI (if enabled): http://localhost:5210/swagger - Postman/curl - CLI command **Example curl:** ```bash curl -X POST http://localhost:5210/api/scans \ -H "Content-Type: application/json" \ -d '{"imageRef": "alpine:latest"}' ``` Your breakpoint should hit, and you can step through code. #### 7. Return to Docker Mode When you're done debugging: ```bash # Stop Visual Studio debugger (Shift+F5) # Restart the Docker container cd deploy\compose docker compose -f docker-compose.dev.yaml start scanner-web # Verify it's running docker compose -f docker-compose.dev.yaml ps scanner-web ``` --- ## Service-by-Service Debugging Guide ### Authority (OAuth2/OIDC Provider) **Project:** `src/Authority/StellaOps.Authority/StellaOps.Authority.csproj` **Stop Docker container:** ```bash docker compose -f docker-compose.dev.yaml stop authority ``` **Configuration:** `appsettings.Development.json` ```json { "ConnectionStrings": { "DefaultConnection": "Host=localhost;Port=5432;Database=stellaops_platform;Username=stellaops;Password=your_password" }, "StellaOps_Authority": { "Issuer": "https://localhost:5001", "Mongo": { "ConnectionString": "mongodb://stellaops:your_password@localhost:27017" } }, "Kestrel": { "Endpoints": { "Https": { "Url": "https://localhost:5001" } } } } ``` **Run in Visual Studio:** F5 on `StellaOps.Authority` project **Default URL:** https://localhost:5001 **Update dependent services:** ```bash # In .env AUTHORITY_ISSUER=https://host.docker.internal:5001 ``` ### Concelier (Advisory Ingestion) **Project:** `src/Concelier/StellaOps.Concelier.WebService/StellaOps.Concelier.WebService.csproj` **Stop Docker:** ```bash docker compose -f docker-compose.dev.yaml stop concelier ``` **Configuration:** `appsettings.Development.json` ```json { "ConnectionStrings": { "DefaultConnection": "Host=localhost;Port=5432;Database=stellaops_platform;Username=stellaops;Password=your_password" }, "Concelier": { "Storage": { "Mongo": { "ConnectionString": "mongodb://stellaops:your_password@localhost:27017" }, "S3": { "Endpoint": "http://localhost:9000", "AccessKeyId": "stellaops", "SecretAccessKey": "your_password" } }, "Authority": { "BaseUrl": "https://localhost:8440" } } } ``` **Run:** F5 on `StellaOps.Concelier.WebService` **Default URL:** http://localhost:5000 ### Scanner.Worker (Background Analyzer) **Project:** `src/Scanner/StellaOps.Scanner.Worker/StellaOps.Scanner.Worker.csproj` **Stop Docker:** ```bash docker compose -f docker-compose.dev.yaml stop scanner-worker ``` **Configuration:** Same as Scanner.WebService (shares settings) **Run:** F5 on `StellaOps.Scanner.Worker` **Note:** Worker has no HTTP endpoint - it consumes from NATS queue ### Scheduler.WebService **Project:** `src/Scheduler/StellaOps.Scheduler.WebService/StellaOps.Scheduler.WebService.csproj` **Stop Docker:** ```bash docker compose -f docker-compose.dev.yaml stop scheduler-web ``` **Configuration:** `appsettings.Development.json` ```json { "ConnectionStrings": { "DefaultConnection": "Host=localhost;Port=5432;Database=stellaops_orchestrator;Username=stellaops;Password=your_password" }, "Scheduler": { "Queue": { "Kind": "Nats", "Nats": { "Url": "nats://localhost:4222" } }, "Worker": { "Runner": { "Scanner": { "BaseAddress": "http://localhost:5210" } } } } } ``` **Run:** F5 on `StellaOps.Scheduler.WebService` ### Notify.WebService **Project:** `src/Notify/StellaOps.Notify.WebService/StellaOps.Notify.WebService.csproj` **Stop Docker:** ```bash docker compose -f docker-compose.dev.yaml stop notify-web ``` **Configuration:** Uses `etc/notify.dev.yaml` **Run:** F5 on `StellaOps.Notify.WebService` --- ## Configuration Deep Dive ### Configuration Hierarchy All services follow this configuration priority (highest to lowest): 1. **Environment Variables** - `STELLAOPS__` or `__` 2. **appsettings.{Environment}.json** - `appsettings.Development.json`, `appsettings.Production.json` 3. **appsettings.json** - Base configuration 4. **YAML files** - `../etc/.yaml`, `../etc/.local.yaml` ### Common Configuration Patterns #### PostgreSQL Connection Strings ```json { "ConnectionStrings": { "DefaultConnection": "Host=localhost;Port=5432;Database=;Username=stellaops;Password=;Pooling=true;Minimum Pool Size=1;Maximum Pool Size=100;Command Timeout=60" } } ``` **Database names by service:** - Scanner: `stellaops_platform` or `scanner_*` - Orchestrator: `stellaops_orchestrator` - Authority: `stellaops_platform` (shared, schema-isolated) - Concelier: `stellaops_platform` (vuln schema) - Notify: `stellaops_platform` (notify schema) #### Valkey Configuration (Default Transport) ```json { "Scanner": { "Events": { "Driver": "valkey", "Dsn": "localhost:6379" }, "Cache": { "Redis": { "ConnectionString": "localhost:6379" } } }, "Scheduler": { "Queue": { "Kind": "Valkey", "Valkey": { "Url": "localhost:6379" } } } } ``` #### NATS Queue Configuration (Optional Alternative Transport) ```json { "Scanner": { "Events": { "Driver": "nats", "Dsn": "nats://localhost:4222" } }, "Scheduler": { "Queue": { "Kind": "Nats", "Nats": { "Url": "nats://localhost:4222" } } } } ``` #### RustFS Configuration (S3-Compatible Object Storage) ```json { "Scanner": { "Storage": { "RustFS": { "Endpoint": "http://localhost:8080", "AccessKeyId": "stellaops", "SecretAccessKey": "your_password", "BucketName": "scanner-artifacts", "Region": "us-east-1", "ForcePathStyle": true } } } } ``` #### RustFS Configuration ```json { "Scanner": { "ArtifactStore": { "Driver": "rustfs", "Endpoint": "http://localhost:8080/api/v1", "Bucket": "scanner-artifacts", "TimeoutSeconds": 30 } } } ``` ### Environment Variable Mapping ASP.NET Core uses `__` (double underscore) for nested configuration: ```bash # This JSON configuration: { "Scanner": { "Queue": { "Broker": "nats://localhost:4222" } } } # Can be set via environment variable: SCANNER__QUEUE__BROKER=nats://localhost:4222 # Or with STELLAOPS_ prefix: STELLAOPS_SCANNER__QUEUE__BROKER=nats://localhost:4222 ``` --- ## Common Development Workflows ### Workflow 1: Debug a Single Service with Full Stack **Scenario:** You need to debug Scanner.WebService while all other services run normally. ```bash # 1. Start full platform cd deploy\compose docker compose -f docker-compose.dev.yaml up -d # 2. Stop the service you want to debug docker compose -f docker-compose.dev.yaml stop scanner-web # 3. Open Visual Studio cd C:\dev\New folder\git.stella-ops.org start src\StellaOps.sln # 4. Set Scanner.WebService as startup project and F5 # 5. Test the service curl -X POST http://localhost:5210/api/scans -H "Content-Type: application/json" -d '{"imageRef":"alpine:latest"}' # 6. When done, stop VS debugger and restart Docker container docker compose -f docker-compose.dev.yaml start scanner-web ``` ### Workflow 2: Debug Multiple Services Together **Scenario:** Debug Scanner.WebService and Scanner.Worker together. ```bash # 1. Stop both containers docker compose -f docker-compose.dev.yaml stop scanner-web scanner-worker # 2. In Visual Studio, configure multiple startup projects: # - Right-click solution > Properties # - Set "Multiple startup projects" # - Select Scanner.WebService: Start # - Select Scanner.Worker: Start # 3. Press F5 to debug both simultaneously ``` ### Workflow 3: Test Integration with Modified Code **Scenario:** You modified Concelier and want to test how Scanner integrates with it. ```bash # 1. Build Concelier locally cd src\Concelier\StellaOps.Concelier.WebService dotnet build # 2. Stop Docker Concelier cd ..\..\..\deploy\compose docker compose -f docker-compose.dev.yaml stop concelier # 3. Run Concelier in Visual Studio (F5) # 4. Keep Scanner in Docker, but point it to localhost Concelier # Update .env: CONCELIER_BASEURL=http://host.docker.internal:5000 # 5. Restart Scanner to pick up new config docker compose -f docker-compose.dev.yaml restart scanner-web ``` ### Workflow 4: Reset Database State **Scenario:** You need a clean database to test migrations or start fresh. ```bash # 1. Stop all services docker compose -f docker-compose.dev.yaml down # 2. Remove database volumes docker volume rm compose_postgres-data docker volume rm compose_mongo-data # 3. Restart platform (will recreate volumes and databases) docker compose -f docker-compose.dev.yaml up -d # 4. Wait for migrations to run docker compose -f docker-compose.dev.yaml logs -f postgres # Look for migration completion messages ``` ### Workflow 5: Test Offline/Air-Gap Mode **Scenario:** Test the platform in offline mode. ```bash # 1. Use the air-gap compose profile cd deploy\compose docker compose -f docker-compose.airgap.yaml up -d # 2. Verify no external network calls docker compose -f docker-compose.airgap.yaml logs | grep -i "external\|outbound\|internet" ``` --- ## Troubleshooting ### Common Issues #### 1. Port Already in Use **Error:** ``` Error starting userland proxy: listen tcp 0.0.0.0:5432: bind: address already in use ``` **Solutions:** **Option A: Change the port in .env** ```bash # Edit .env POSTGRES_PORT=5433 # Use a different port ``` **Option B: Stop the conflicting process** ```bash # Windows netstat -ano | findstr :5432 taskkill /PID /F # Linux/Mac lsof -i :5432 kill -9 ``` #### 2. Cannot Connect to PostgreSQL from Visual Studio **Error:** ``` Npgsql.NpgsqlException: Connection refused ``` **Solutions:** 1. **Verify PostgreSQL is accessible from host:** ```bash psql -h localhost -U stellaops -d stellaops_platform ``` 2. **Check Docker network:** ```bash docker network inspect compose_stellaops # Ensure your service has "host.docker.internal" DNS resolution ``` 3. **Update connection string:** ```json { "ConnectionStrings": { "DefaultConnection": "Host=localhost;Port=5432;Database=stellaops_platform;Username=stellaops;Password=your_password;Include Error Detail=true" } } ``` #### 3. NATS Connection Refused **Error:** ``` NATS connection error: connection refused ``` **Solution:** By default, services use **Valkey** for messaging, not NATS. Ensure Valkey is running: ```bash docker compose -f docker-compose.dev.yaml ps valkey # Should show: State = "Up" # Test connectivity telnet localhost 6379 ``` Update configuration to use Valkey (default): ```json { "Scanner": { "Events": { "Driver": "valkey", "Dsn": "localhost:6379" } }, "Scheduler": { "Queue": { "Kind": "Valkey", "Valkey": { "Url": "localhost:6379" } } } } ``` **If you explicitly want to use NATS** (optional): ```bash docker compose -f docker-compose.dev.yaml ps nats # Ensure NATS is running # Update appsettings.Development.json: { "Scanner": { "Events": { "Driver": "nats", "Dsn": "nats://localhost:4222" } } } ``` #### 4. Valkey Connection Refused **Error:** ``` StackExchange.Redis.RedisConnectionException: It was not possible to connect to the redis server(s) ``` **Solutions:** 1. **Check Valkey is running:** ```bash docker compose -f docker-compose.dev.yaml ps valkey # Should show: State = "Up" # Check logs docker compose -f docker-compose.dev.yaml logs valkey ``` 2. **Reset Valkey:** ```bash docker compose -f docker-compose.dev.yaml stop valkey docker volume rm compose_valkey-data docker compose -f docker-compose.dev.yaml up -d valkey ``` #### 5. Service Cannot Reach host.docker.internal **Error:** ``` Could not resolve host: host.docker.internal ``` **Solution (Windows/Mac):** Should work automatically with Docker Desktop. **Solution (Linux):** Add to docker-compose.dev.yaml: ```yaml services: scanner-web: extra_hosts: - "host.docker.internal:host-gateway" ``` Or use the host's IP address: ```bash # Find host IP ip addr show docker0 # Use that IP instead of host.docker.internal ``` #### 6. Certificate Validation Errors (Authority/HTTPS) **Error:** ``` The SSL connection could not be established ``` **Solution:** For development, disable certificate validation: ```json { "Authority": { "ValidateCertificate": false } } ``` Or trust the development certificate: ```bash dotnet dev-certs https --trust ``` #### 7. Build Errors - Missing SDK **Error:** ``` error MSB4236: The SDK 'Microsoft.NET.Sdk.Web' specified could not be found ``` **Solution:** Install .NET 10 SDK: ```bash # Verify installation dotnet --list-sdks # Should show: # 10.0.xxx [C:\Program Files\dotnet\sdk] ``` #### 8. Hot Reload Not Working **Symptom:** Changes in code don't reflect when running in Visual Studio. **Solutions:** 1. Ensure Hot Reload is enabled: Tools > Options > Debugging > .NET Hot Reload > Enable Hot Reload 2. Rebuild the project: Ctrl+Shift+B 3. Restart debugging session: Shift+F5, then F5 #### 9. Docker Compose Fails to Parse .env **Error:** ``` invalid interpolation format ``` **Solution:** Ensure no spaces around `=` in .env: ```bash # Wrong POSTGRES_USER = stellaops # Correct POSTGRES_USER=stellaops ``` #### 10. Volume Permission Issues (Linux) **Error:** ``` Permission denied writing to /data/db ``` **Solution:** ```bash # Fix permissions on volume directories sudo chown -R $USER:$USER ./volumes # Or run Docker as root (not recommended for production) sudo docker compose -f docker-compose.dev.yaml up -d ``` --- ## Next Steps ### Learning Path 1. **Week 1: Infrastructure** - Understand PostgreSQL schema isolation (all services use PostgreSQL) - Learn Valkey streams for event queuing and caching - Study RustFS S3-compatible object storage - Optional: NATS JetStream as alternative transport 2. **Week 2: Core Services** - Deep dive into Scanner architecture (analyzers, workers, caching) - Understand Concelier advisory ingestion and merging - Study VEX workflow in Excititor 3. **Week 3: Authentication & Security** - Master OAuth2/OIDC flow in Authority - Understand signing flow (Signer → Attestor → Rekor) - Study policy evaluation engine 4. **Week 4: Integration** - Build end-to-end scan workflow - Implement custom Concelier connector - Create custom notification rules ### Key Documentation - **Architecture:** `docs/07_HIGH_LEVEL_ARCHITECTURE.md` - **Build Commands:** `CLAUDE.md` - **Database Spec:** `docs/db/SPECIFICATION.md` - **API Reference:** `docs/09_API_CLI_REFERENCE.md` - **Module Architecture:** `docs/modules//architecture.md` ### Support - **Issues:** https://git.stella-ops.org/stella-ops.org/git.stella-ops.org/issues - **Discussions:** Internal team channels - **Documentation:** `docs/` directory in the repository --- ## Quick Reference Card ### Essential Commands ```bash # Start full platform cd deploy\compose docker compose -f docker-compose.dev.yaml up -d # Stop a specific service for debugging docker compose -f docker-compose.dev.yaml stop # View logs docker compose -f docker-compose.dev.yaml logs -f # Restart a service docker compose -f docker-compose.dev.yaml restart # Stop all services docker compose -f docker-compose.dev.yaml down # Stop all services and remove volumes (DESTRUCTIVE) docker compose -f docker-compose.dev.yaml down -v # Build the solution cd C:\dev\New folder\git.stella-ops.org dotnet build src\StellaOps.sln # Run tests dotnet test src\StellaOps.sln # Run a specific project cd src\Scanner\StellaOps.Scanner.WebService dotnet run ``` ### Service Default Ports | Service | Port | URL | Notes | |---------|------|-----|-------| | **Infrastructure** | | PostgreSQL | 5432 | `localhost:5432` | Primary database (REQUIRED) | | Valkey | 6379 | `localhost:6379` | Cache/events/queues (REQUIRED) | | RustFS | 8080 | http://localhost:8080 | S3-compatible storage (REQUIRED) | | NATS | 4222 | `nats://localhost:4222` | Optional alternative transport | | **Services** | | Authority | 8440 | https://localhost:8440 | OAuth2/OIDC auth | | Signer | 8441 | https://localhost:8441 | Cryptographic signing | | Attestor | 8442 | https://localhost:8442 | in-toto attestations | | Scanner.Web | 8444 | http://localhost:8444 | Scan API | | Concelier | 8445 | http://localhost:8445 | Advisory ingestion | | Notify | 8446 | http://localhost:8446 | Notifications | | IssuerDirectory | 8447 | http://localhost:8447 | CSAF publisher discovery | ### Visual Studio Shortcuts | Action | Shortcut | |--------|----------| | Start Debugging | F5 | | Start Without Debugging | Ctrl+F5 | | Stop Debugging | Shift+F5 | | Step Over | F10 | | Step Into | F11 | | Step Out | Shift+F11 | | Toggle Breakpoint | F9 | | Build Solution | Ctrl+Shift+B | | Rebuild Solution | Ctrl+Shift+F5 | --- **Document Version:** 1.0 **Last Updated:** 2025-12-22 **Maintained By:** StellaOps Development Team