git.stella-ops.org/docs/integrations/LOCAL_SERVICES.md

# Local Integration Services

This guide covers the third-party services available for local integration testing with Stella Ops.

## Architecture Overview

```
                          stellaops network
  +------------------------------------------------------------------+
  |                                                                    |
  |  STELLA OPS CORE                  INTEGRATION SERVICES             |
  |  (docker-compose.stella-ops.yml)  (docker-compose.integrations.yml)|
  |                                                                    |
  |  router-gateway ─────────> gitea        (SCM)       127.1.2.1:3000 |
  |  concelier      ─────────> jenkins      (CI/CD)     127.1.2.2:8080 |
  |  integrations   ─────────> nexus        (Registry)  127.1.2.3:8081 |
  |  scanner        ─────────> vault        (Secrets)   127.1.2.4:8200 |
  |  evidence-locker ────────> docker-reg   (Registry)  127.1.2.5:5000 |
  |  airgap-controller ──────> minio        (S3)        127.1.2.6:9000 |
  |                            gitlab [heavy](All-in-1) 127.1.2.7:8929 |
  |                                                                    |
  |  MOCK FIXTURES (docker-compose.integration-fixtures.yml)           |
  |  harbor-fixture   (Registry mock)   127.1.1.6:80                   |
  |  github-app-fixture (SCM mock)      127.1.1.7:80                   |
  |  advisory-fixture (Advisory mock)   127.1.1.8:80                   |
  +------------------------------------------------------------------+
```

## Quick Start

### Prerequisites
- Docker Desktop with 8 GB+ RAM allocated
- The main Stella Ops stack running (`docker-compose.stella-ops.yml`)
- Hosts file entries (see below)

### 1. Add hosts file entries

Add to `C:\Windows\System32\drivers\etc\hosts`:

```
127.1.2.1  gitea.stella-ops.local
127.1.2.2  jenkins.stella-ops.local
127.1.2.3  nexus.stella-ops.local
127.1.2.4  vault.stella-ops.local
127.1.2.5  registry.stella-ops.local
127.1.2.6  minio.stella-ops.local
127.1.2.7  gitlab.stella-ops.local
127.1.2.8  consul.stella-ops.local
```

### 2. Start services

```bash
cd devops/compose

# Start the default low-idle services (recommended)
docker compose -f docker-compose.integrations.yml up -d

# Or start specific services only
docker compose -f docker-compose.integrations.yml up -d gitea vault jenkins

# Start WITH mock fixtures (for full integration testing)
docker compose \
  -f docker-compose.integrations.yml \
  -f docker-compose.integration-fixtures.yml \
  up -d

# Start Consul only when validating the Consul connector
docker compose -f docker-compose.integrations.yml --profile consul up -d consul

# Start GitLab CE (heavy, 4 GB+ RAM, ~3 min startup)
docker compose -f docker-compose.integrations.yml --profile heavy up -d gitlab

# Re-enable GitLab registry/package surfaces for registry-specific tests
GITLAB_ENABLE_REGISTRY=true GITLAB_ENABLE_PACKAGES=true \
  docker compose -f docker-compose.integrations.yml --profile heavy up -d gitlab
```

### 3. Verify services

```bash
# Quick health check for all services
docker compose -f docker-compose.integrations.yml ps

# Gitea is only complete once the container is healthy
docker compose -f docker-compose.integrations.yml ps gitea
```

### 4. Register the local integration catalog

After the core stack plus the local provider lanes are running, there are two
supported local operator paths.

Browser-driven Integrations Hub path:

```powershell
node src/Web/StellaOps.Web/scripts/live-integrations-ui-bootstrap.mjs
```

- Drives the live browser through `/setup/integrations/onboarding/*`.
- Persists evidence to `src/Web/StellaOps.Web/output/playwright/live-integrations-ui-bootstrap.json`.
- The harness now supports inline GitLab secret staging through the browser when
  `STELLAOPS_UI_BOOTSTRAP_GITLAB_ACCESS_TOKEN` and
  `STELLAOPS_UI_BOOTSTRAP_GITLAB_REGISTRY_BASIC` are supplied.
- The separate first-run setup wizard (`/setup-wizard/wizard`) now reaches the
  Platform setup API through the frontdoor and uses persisted,
  installation-scoped setup sessions for the five truthful control-plane steps.

Scripted convergence path:

```powershell
powershell -ExecutionPolicy Bypass -File scripts/register-local-integrations.ps1 `
  -Tenant demo-prod
```

This converges the default local-ready lane to 13 healthy entries:
- Harbor fixture
- Docker Registry
- Nexus
- GitHub App fixture
- Gitea
- Jenkins
- Vault
- Consul
- eBPF runtime-host fixture
- MinIO (`S3Compatible`)
- StellaOps mirror
- NVD mirror
- OSV mirror

Optional GitLab providers require Vault-backed credentials. The recommended
local flow is:

```powershell
# Reuse or rotate the local GitLab bootstrap PAT and write it to Vault.
powershell -ExecutionPolicy Bypass -File scripts/bootstrap-local-gitlab-secrets.ps1 `
  -VerifyRegistry

# Register SCM + CI using the bootstrapped authref://vault/gitlab#access-token
powershell -ExecutionPolicy Bypass -File scripts/register-local-integrations.ps1 `
  -Tenant demo-prod `
  -IncludeGitLab

# Also requires GitLab registry enabled; uses authref://vault/gitlab#registry-basic
powershell -ExecutionPolicy Bypass -File scripts/register-local-integrations.ps1 `
  -Tenant demo-prod `
  -IncludeGitLab `
  -IncludeGitLabRegistry

# Or do the GitLab-backed registration in one step
powershell -ExecutionPolicy Bypass -File scripts/register-local-integrations.ps1 `
  -Tenant demo-prod `
  -IncludeGitLab `
  -IncludeGitLabRegistry `
  -BootstrapGitLabSecrets
```

For a repeatable browser-driven proof of the same Integrations Hub path, run:

```powershell
node src/Web/StellaOps.Web/scripts/live-integrations-ui-bootstrap.mjs
```

It authenticates against `https://stella-ops.local`, creates integrations
through the onboarding UI routes, and records the final catalog plus health
results in
`src/Web/StellaOps.Web/output/playwright/live-integrations-ui-bootstrap.json`.

`docker-compose.testing.yml` is the separate infrastructure-test lane. It starts `postgres-test`, `valkey-test`, mocks, and an isolated Gitea profile on different ports; it does not start Consul or GitLab.

---

## Service Reference

### Gitea (SCM)

| Property | Value |
|----------|-------|
| URL | http://gitea.stella-ops.local:3000 |
| API | http://gitea.stella-ops.local:3000/api/v1 |
| SSH | gitea.stella-ops.local:2222 |
| Admin | `stellaops` / `Stella2026!` on fresh volumes |
| Bootstrap | Container entrypoint seeds the repo root and first admin before health goes green |
| Swagger | http://gitea.stella-ops.local:3000/api/swagger |
| Integration type | SCM (Gitea provider) |
| Docker DNS | `gitea.stella-ops.local` |

**Stella Ops integration config:**
- Endpoint: `http://gitea.stella-ops.local:3000`
- AuthRef: `authref://vault/gitea#api-token`
- Organization: *(your Gitea org name)*

**Create an API token:**
1. Log in to Gitea as `stellaops` / `Stella2026!` on a fresh volume, or use the existing admin user if this environment was already initialized.
2. Settings > Applications > Generate Token
3. Store in Vault at `secret/gitea` with key `api-token`

> The previous local-service flow was contradictory: the compose profile marked Gitea as install-locked while the docs still described a manual first-login admin creation path. The compose bootstrap now makes the service deterministic and leaves only PAT creation as a manual step.

---

### Jenkins (CI/CD)

| Property | Value |
|----------|-------|
| URL | http://jenkins.stella-ops.local:8080 |
| API | http://jenkins.stella-ops.local:8080/api/json |
| Admin | Setup wizard disabled; create user via script console |
| Agent port | 127.1.2.2:50000 |
| Integration type | CI/CD (Jenkins provider) |
| Docker DNS | `jenkins.stella-ops.local` |

**Stella Ops integration config:**
- Endpoint: `http://jenkins.stella-ops.local:8080`
- AuthRef: `authref://vault/jenkins#api-token`

**Create an API token:**
1. Open Jenkins > Manage Jenkins > Users > admin > Configure
2. Add API Token
3. Store in Vault at `secret/jenkins` with key `api-token`

---

### Nexus (Repository Manager)

| Property | Value |
|----------|-------|
| URL | http://nexus.stella-ops.local:8081 |
| API | http://nexus.stella-ops.local:8081/service/rest/v1/status |
| Docker hosted | nexus.stella-ops.local:8082 |
| Docker proxy | nexus.stella-ops.local:8083 |
| Admin | admin / *(see `/nexus-data/admin.password` on first run)* |
| Integration type | Registry (Nexus provider) |
| Docker DNS | `nexus.stella-ops.local` |

**Get initial admin password:**
```bash
docker exec stellaops-nexus cat /nexus-data/admin.password
```

**Stella Ops integration config:**
- Endpoint: `http://nexus.stella-ops.local:8081`
- AuthRef: `authref://vault/nexus#admin-password`

**Setup Docker hosted repository:**
1. Login to Nexus UI
2. Server Administration > Repositories > Create > docker (hosted)
3. HTTP port: 8082, Allow redeploy: true
4. Create a docker (proxy) repository pointing to Docker Hub, HTTP port: 8083

---

### HashiCorp Vault (Secrets)

| Property | Value |
|----------|-------|
| URL | http://vault.stella-ops.local:8200 |
| API | http://vault.stella-ops.local:8200/v1/sys/health |
| Root token | `stellaops-dev-root-token-2026` |
| Mode | Dev server (in-memory, unsealed) |
| Integration type | Secrets (Vault provider) |
| Docker DNS | `vault.stella-ops.local` |

**Stella Ops integration config:**
- Endpoint: `http://vault.stella-ops.local:8200`
- AuthRef: (Vault is the auth provider itself)

**Store integration credentials in Vault:**
```bash
# Enable KV v2 engine (already enabled in dev mode at secret/)
export VAULT_ADDR=http://vault.stella-ops.local:8200
export VAULT_TOKEN=stellaops-dev-root-token-2026

# Store Harbor credentials
vault kv put secret/harbor robot-account="harbor-robot-token"

# Store GitHub App credentials
vault kv put secret/github app-private-key="-----BEGIN RSA PRIVATE KEY-----..."

# Store Gitea API token
vault kv put secret/gitea api-token="your-gitea-token"

# Store Jenkins API token
vault kv put secret/jenkins api-token="your-jenkins-token"

# Store Nexus admin password
vault kv put secret/nexus admin-password="your-nexus-password"

# Store GitLab PATs for API and registry access (manual override path)
vault kv put secret/gitlab access-token="glpat-your-token" registry-basic="root:glpat-your-token"
```

Inline secret staging no longer requires a manual Vault write for GitLab-class
providers:

```bash
stella config integrations secrets targets
stella config integrations secrets upsert-bundle --bundle gitlab-server --target <vault-integration-id> --path gitlab/server --entry access-token=glpat-...
```

---

### Consul (Optional KV / Settings Store)

| Property | Value |
|----------|-------|
| URL | http://consul.stella-ops.local:8500 |
| API | http://consul.stella-ops.local:8500/v1/status/leader |
| Auth | None (single-node local server) |
| Start mode | `--profile consul` only |
| Integration type | Settings / KV (`Consul` provider) |
| Docker DNS | `consul.stella-ops.local` |

**Start Consul only when needed:**
```bash
docker compose -f docker-compose.integrations.yml --profile consul up -d consul
```

**Why opt-in:** even in its lower-idle local mode, Consul is still an extra control-plane service that most local connector checks do not need. The default integration lane keeps it off unless you are explicitly validating the Consul connector.

**Runtime mode:** the local compose profile now runs Consul as a persistent single-node server with the UI enabled instead of `agent -dev`. That preserves the HTTP KV surface while materially lowering idle CPU.

**Stella Ops integration config:**
- Endpoint: `http://consul.stella-ops.local:8500`
- AuthRef: *(none required in local mode)*

---

### Docker Registry (OCI v2)

| Property | Value |
|----------|-------|
| URL | http://registry.stella-ops.local:5000 |
| API | http://registry.stella-ops.local:5000/v2/ |
| Auth | None (open dev registry) |
| Integration type | Registry (generic OCI) |
| Docker DNS | `registry.stella-ops.local` |

**Push a test image:**
```bash
docker tag alpine:latest registry.stella-ops.local:5000/test/alpine:latest
docker push registry.stella-ops.local:5000/test/alpine:latest

# List repositories
curl http://registry.stella-ops.local:5000/v2/_catalog
```

**Stella Ops integration config:**
- Endpoint: `http://registry.stella-ops.local:5000`
- AuthRef: *(none required for dev)*

---

### MinIO (S3 Storage)

| Property | Value |
|----------|-------|
| Console | http://minio.stella-ops.local:9001 |
| S3 API | http://minio.stella-ops.local:9000 |
| Access key | `stellaops` |
| Secret key | `Stella2026!` |
| Integration type | Object Storage (`S3Compatible` provider) |
| Docker DNS | `minio.stella-ops.local` |

**Create buckets for Stella Ops:**
```bash
# Install mc CLI
docker exec stellaops-minio mc alias set local http://localhost:9000 stellaops Stella2026!

# Create buckets
docker exec stellaops-minio mc mb local/evidence-locker
docker exec stellaops-minio mc mb local/airgap-bundles
docker exec stellaops-minio mc mb local/scan-results
docker exec stellaops-minio mc mb local/sbom-archive
```

**Stella Ops integration config:**
- Endpoint: `http://minio.stella-ops.local:9000`
- Type: `ObjectStorage`
- Provider: `S3Compatible`
- AuthRef: optional for the default local health probe

---

### GitLab CE (Heavy, Optional)

| Property | Value |
|----------|-------|
| URL | http://gitlab.stella-ops.local:8929 |
| Admin | root / `Stella2026!` |
| SSH | gitlab.stella-ops.local:2224 |
| Container Registry | gitlab.stella-ops.local:5050 (`GITLAB_ENABLE_REGISTRY=true` only) |
| RAM required | 4 GB+ |
| Startup time | ~3-5 minutes |
| Integration type | SCM + CI/CD + Registry |
| Docker DNS | `gitlab.stella-ops.local` |

**Start GitLab (uses `heavy` profile):**
```bash
docker compose -f docker-compose.integrations.yml --profile heavy up -d gitlab
```

**Default local tuning:**
- SCM/API coverage stays available.
- Registry and package surfaces are disabled by default to reduce idle CPU.
- Puma and Sidekiq run in reduced-concurrency mode for local connector checks.

**Enable registry/package coverage explicitly when needed:**
```bash
GITLAB_ENABLE_REGISTRY=true GITLAB_ENABLE_PACKAGES=true \
  docker compose -f docker-compose.integrations.yml --profile heavy up -d gitlab
```

**Stella Ops integration config (SCM / CI):**
- Endpoint: `http://gitlab.stella-ops.local:8929`
- AuthRef: `authref://vault/gitlab#access-token`
- Bootstrap helper: `powershell -ExecutionPolicy Bypass -File scripts/bootstrap-local-gitlab-secrets.ps1`

**Stella Ops integration config (Registry):**
- Endpoint: `http://gitlab.stella-ops.local:5050`
- AuthRef: `authref://vault/gitlab#registry-basic`
- Secret format: `username:personal-access-token` (local default: `root:<token>`)
- The Docker registry connector follows GitLab's `WWW-Authenticate: Bearer` challenge and exchanges this basic secret against `/jwt/auth` before retrying catalog and tag probes.
- `scripts/bootstrap-local-gitlab-secrets.ps1 -VerifyRegistry` reuses a valid local Vault secret when possible and otherwise rotates the local `stella-local-integration` PAT before writing both authrefs.

---

## Mock Fixtures

In addition to real services, lightweight nginx-based fixtures provide deterministic mock APIs for UI testing.

| Fixture | Mocks | Address | Compose file |
|---------|-------|---------|-------------|
| harbor-fixture | Harbor v2 API | 127.1.1.6:80 | docker-compose.integration-fixtures.yml |
| github-app-fixture | GitHub App API | 127.1.1.7:80 | docker-compose.integration-fixtures.yml |
| advisory-fixture | CERT-In, FSTEC, VEX Hub, StellaOps Mirror, etc. | 127.1.1.8:80 | docker-compose.integration-fixtures.yml |

```bash
# Start fixtures only
docker compose -f docker-compose.integration-fixtures.yml up -d
```

---

## IP Address Map

| IP | Service | Port(s) |
|----|---------|---------|
| 127.1.0.1 | stella-ops.local (gateway) | 443 |
| 127.1.0.4 | authority (OIDC) | 80 |
| 127.1.1.1 | postgres | 5432 |
| 127.1.1.2 | valkey | 6379 |
| 127.1.1.6 | harbor-fixture | 80 |
| 127.1.1.7 | github-app-fixture | 80 |
| 127.1.1.8 | advisory-fixture | 80 |
| 127.1.2.1 | gitea | 3000, 2222 |
| 127.1.2.2 | jenkins | 8080, 50000 |
| 127.1.2.3 | nexus | 8081, 8082, 8083 |
| 127.1.2.4 | vault | 8200 |
| 127.1.2.5 | docker-registry | 5000 |
| 127.1.2.6 | minio | 9000, 9001 |
| 127.1.2.7 | gitlab (heavy) | 8929, 2224, 5050 |

---

## Volumes

All service data persists in named Docker volumes. To reset a service:

```bash
# Stop and remove a specific service + its volume
docker compose -f docker-compose.integrations.yml down -v nexus
docker volume rm stellaops-nexus-data

# Reset ALL integration services
docker compose -f docker-compose.integrations.yml down -v
```

---

## Integration Matrix

| Stella Ops Category | Provider | Local Service | Status |
|---------------------|----------|---------------|--------|
| **Registry** | Harbor | harbor-fixture (mock) | Ready |
| **Registry** | Docker Hub / OCI | docker-registry | Ready |
| **Registry** | Nexus | nexus | Ready |
| **Registry** | GitLab Registry | gitlab (heavy) | Ready when `GITLAB_ENABLE_REGISTRY=true` and `authref://vault/gitlab#registry-basic` is populated |
| **SCM** | GitHub App | github-app-fixture (mock) | Ready |
| **SCM** | Gitea | gitea | Ready |
| **SCM** | GitLab Server | gitlab (heavy) | Ready with Vault-backed PAT |
| **CI/CD** | Jenkins | jenkins | Ready |
| **CI/CD** | GitLab CI | gitlab (heavy) | Ready with reduced local concurrency |
| **Secrets** | Vault | vault | Ready |
| **Secrets** | Consul | consul | Opt-in (`--profile consul`) |
| **Runtime Host** | eBPF Agent | runtime-host-fixture (mock) | Ready |
| **Feed Mirror** | StellaOps / NVD / OSV mirror | concelier | Ready |
| **Storage** | S3-compatible (MinIO) | minio | Ready |
| **Advisory & VEX** | 74 sources | advisory-fixture + live | 74/74 healthy |

> **Current provider list:** the local Integrations service currently reports connector plugins for Harbor,
> Docker Registry, GitLab Container Registry, Nexus, GitHub App, Gitea, GitLab Server,
> GitLab CI, Jenkins, Vault, Consul, eBPF Agent, the `S3Compatible` object-storage provider, the feed mirror providers
> (`StellaOpsMirror`, `NvdMirror`, `OsvMirror`), and the test-only InMemory provider.
>
> **Storage note:** the `S3Compatible` connector defaults to probing `/minio/health/live`
> when the configured endpoint is the service root, which matches the local MinIO fixture.
>
> **Auth caveat:** several connector plugins validate public health/version endpoints only. A green connection
> test proves reachability and the plugin wiring, but it does not guarantee that privileged API operations are
> fully configured unless you also provision the corresponding secret material in Vault.