up
Some checks failed
Build Test Deploy / build-test (push) Has been cancelled
Build Test Deploy / authority-container (push) Has been cancelled
Build Test Deploy / docs (push) Has been cancelled
Build Test Deploy / deploy (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled
Some checks failed
Build Test Deploy / build-test (push) Has been cancelled
Build Test Deploy / authority-container (push) Has been cancelled
Build Test Deploy / docs (push) Has been cancelled
Build Test Deploy / deploy (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled
This commit is contained in:
root
250
AGENTS.md
250
AGENTS.md
@@ -1,125 +1,125 @@
|
||||
# 1) What is StellaOps?
|
||||
|
||||
**StellaOps** an open, sovereign, modular container-security toolkit built for high-speed, offline operation, released under AGPL-3.0-or-later.
|
||||
|
||||
It follows an SBOM-first model—analyzing each container layer or ingesting existing CycloneDX/SPDX SBOMs, then enriching them with vulnerability, licence, secret-leak, and misconfiguration data to produce cryptographically signed reports.
|
||||
|
||||
Vulnerability detection maps OS and language dependencies to sources such as NVD, GHSA, OSV, ENISA.
|
||||
Secrets sweep flags exposed credentials or keys in files or environment variables.
|
||||
Licence audit identifies potential conflicts, especially copyleft obligations.
|
||||
Misconfiguration checks detect unsafe Dockerfile patterns (root user, latest tags, permissive modes).
|
||||
Provenance features include in-toto/SLSA attestations signed with cosign for supply-chain trust.
|
||||
|
||||
| Guiding principle | What it means for Feedser |
|
||||
|-------------------|---------------------------|
|
||||
| **SBOM-first ingest** | Prefer signed SBOMs or reproducible layer diffs before falling back to raw scraping; connectors treat source docs as provenance, never as mutable truth. |
|
||||
| **Deterministic outputs** | Same inputs yield identical canonical advisories and exported JSON/Trivy DB artefacts; merge hashes and export manifests are reproducible across machines. |
|
||||
| **Restart-time plug-ins only** | Connector/exporter plug-ins load at service start, keeping runtime sandboxing simple and avoiding hot-patch attack surface. |
|
||||
| **Sovereign/offline-first** | No mandatory outbound calls beyond allow-listed advisories; Offline Kit bundles Mongo snapshots and exporter artefacts for air-gapped installs. |
|
||||
| **Operational transparency** | Every stage logs structured events (fetch, parse, merge, export) with correlation IDs so parallel agents can debug without shared state. |
|
||||
|
||||
Performance: warm scans < 5 s, cold scans < 30 s on a 4 vCPU runner.
|
||||
Deployment: entirely SaaS-free, suitable for air-gapped or on-prem use through its Offline Kit.
|
||||
Policy: anonymous users → 33 scans/day; verified → 333 /day; nearing 90 % quota triggers throttling but never full blocks.
|
||||
|
||||
More documention is available ./docs/*.md files. Read `docs/README.md` to gather information about the available documentation. You could inquiry specific documents as your work requires it
|
||||
|
||||
---
|
||||
|
||||
# 3) Practices
|
||||
|
||||
## 3.1) Naming
|
||||
All modules are .NET projects based on .NET 10 (preview). Exclussion is the UI. It is based on Angular
|
||||
All modules are contained by one or more projects. Each project goes in its dedicated folder. Each project starts with StellaOps.<ModuleName>. In case it is common for for all StellaOps modules it is library or plugin and it is named StellaOps.<LibraryOrPlugin>.
|
||||
|
||||
## 3.2) Key technologies & integrations
|
||||
|
||||
- **Runtime**: .NET 10 (`net10.0`) preview SDK; C# latest preview features.
|
||||
- **Data**: MongoDB (canonical store and job/export state).
|
||||
- **Observability**: structured logs, counters, and (optional) OpenTelemetry traces.
|
||||
- **Ops posture**: offline‑first, allowlist for remote hosts, strict schema validation, gated LLM fallback (only where explicitly configured).
|
||||
|
||||
# 4) Modules
|
||||
StellaOps is contained by different modules installable via docker containers
|
||||
- Feedser. Responsible for aggregation and delivery of vulnerability database
|
||||
- Cli. Command line tool to unlock full potential - request database operations, install scanner, request scan, configure backend
|
||||
- Backend. Configures and Manages scans
|
||||
- UI. UI to access the backend (and scanners)
|
||||
- Agent. Installable daemon that does the scanning
|
||||
- Zastava. Realtime monitor for allowed (verified) installations.
|
||||
|
||||
## 4.1) Feedser
|
||||
It is webservice based module that is responsible for aggregating vulnerabilities information from various sources, parsing and normalizing them into a canonical shape, merging and deduplicating the results in one place, with export capabilities to Json and TrivyDb. It supports init and resume for all of the sources, parse/normalize and merge/deduplication operations, plus export. Export supports delta exports—similarly to full and incremential database backups.
|
||||
|
||||
### 4.1.1) Usage
|
||||
It supports operations to be started by cmd line:
|
||||
# stella db [fetch|merge|export] [init|resume <point>]
|
||||
or
|
||||
api available on https://db.stella-ops.org
|
||||
|
||||
### 4.1.2) Data flow (end‑to‑end)
|
||||
|
||||
1. **Fetch**: connectors request source windows with retries/backoff, persist raw documents with SHA256/ETag metadata.
|
||||
2. **Parse & Normalize**: validate to DTOs (schema-checked), quarantine failures, normalize to canonical advisories (aliases, affected ranges with NEVRA/EVR/SemVer, references, provenance).
|
||||
3. **Merge & Deduplicate**: enforce precedence, build/maintain alias graphs, compute deterministic hashes, and eliminate duplicates before persisting to MongoDB.
|
||||
4. **Export**: JSON tree and/or Trivy DB; package and (optionally) push; write export state.
|
||||
|
||||
### 4.1.3) Architecture
|
||||
For more information of the architecture see `./docs/ARCHITECTURE_FEEDSER.md`.
|
||||
|
||||
---
|
||||
|
||||
### 4.1.4) Glossary (quick)
|
||||
|
||||
- **OVAL** — Vendor/distro security definition format; authoritative for OS packages.
|
||||
- **NEVRA / EVR** — RPM and Debian version semantics for OS packages.
|
||||
- **PURL / SemVer** — Coordinates and version semantics for OSS ecosystems.
|
||||
- **KEV** — Known Exploited Vulnerabilities (flag only).
|
||||
|
||||
---
|
||||
# 5) Your role as StellaOps contributor
|
||||
|
||||
You acting as information technology engineer that will take different type of roles in goal achieving StellaOps production implementation
|
||||
In order you to work - you have to be supplied with directory that contains `AGENTS.md`,`TASKS.md` files. There will you have more information about the role you have, the scope of your work and the tasks you will have.
|
||||
|
||||
Boundaries:
|
||||
- You operate only in the working directories I gave you, unless there is dependencies that makes you to work on dependency in shared directory. Then you ask for confirmation.
|
||||
|
||||
You main characteristics:
|
||||
- Keep endpoints small, deterministic, and cancellation-aware.
|
||||
- Improve logs/metrics as per tasks.
|
||||
- Update `TASKS.md` when moving tasks forward.
|
||||
- When you are done with all task you state explicitly you are done.
|
||||
- Impersonate the role described on working directory `AGENTS.md` you will read, if role is not available - take role of the CTO of the StellaOps in early stages.
|
||||
- You always strive for best practices
|
||||
- You always strive for re-usability
|
||||
- When in doubt of design decision - you ask then act
|
||||
- You are autonomus - meaning that you will work for long time alone and achieve maximum without stopping for stupid questions
|
||||
- You operate on the same directory where other agents will work. In case you need to work on directory that is dependency on provided `AGENTS.md`,`TASKS.md` files you have to ask for confirmation first.
|
||||
|
||||
## 5.1) Type of contributions
|
||||
|
||||
- **BE‑Base (Platform & Pipeline)**
|
||||
Owns DI, plugin host, job scheduler/coordinator, configuration binding, minimal API endpoints, and Mongo bootstrapping.
|
||||
- **BE‑Conn‑X (Connectors)**
|
||||
One agent per source family (NVD, Red Hat, Ubuntu, Debian, SUSE, GHSA, OSV, PSIRTs, CERTs, KEV, ICS). Implements fetch/parse/map with incremental watermarks.
|
||||
- **BE‑Merge (Canonical Merge & Dedupe)**
|
||||
Identity graph, precedence policies, canonical JSON serializer, and deterministic hashing (`merge_event`).
|
||||
- **BE‑Export (JSON & Trivy DB)**
|
||||
Deterministic export trees, Trivy DB packaging, optional ORAS push, and offline bundle.
|
||||
- **QA (Validation & Observability)**
|
||||
Schema tests, fixture goldens, determinism checks, metrics/logs/traces, e2e reproducibility runs.
|
||||
- **DevEx/Docs**
|
||||
Maintains this agent framework, templates, and per‑directory guides; assists parallelization and reviews.
|
||||
|
||||
|
||||
## 5.2) Work-in-parallel rules (important)
|
||||
|
||||
- **Directory ownership**: Each agent works **only inside its module directory**. Cross‑module edits require a brief handshake in issues/PR description.
|
||||
- **Scoping**: Use each module’s `AGENTS.md` and `TASKS.md` to plan; autonomous agents must read `src/AGENTS.md` and the module docs before acting.
|
||||
- **Determinism**: Sort keys, normalize timestamps to UTC ISO‑8601, avoid non‑deterministic data in exports and tests.
|
||||
- **Status tracking**: Update your module’s `TASKS.md` as you progress (TODO → DOING → DONE/BLOCKED).
|
||||
- **Tests**: Add/extend fixtures and unit tests per change; never regress determinism or precedence.
|
||||
- **Test layout**: Use module-specific projects in `StellaOps.Feedser.<Component>.Tests`; shared fixtures/harnesses live in `StellaOps.Feedser.Testing`.
|
||||
|
||||
---
|
||||
# 1) What is StellaOps?
|
||||
|
||||
**StellaOps** an open, sovereign, modular container-security toolkit built for high-speed, offline operation, released under AGPL-3.0-or-later.
|
||||
|
||||
It follows an SBOM-first model—analyzing each container layer or ingesting existing CycloneDX/SPDX SBOMs, then enriching them with vulnerability, licence, secret-leak, and misconfiguration data to produce cryptographically signed reports.
|
||||
|
||||
Vulnerability detection maps OS and language dependencies to sources such as NVD, GHSA, OSV, ENISA.
|
||||
Secrets sweep flags exposed credentials or keys in files or environment variables.
|
||||
Licence audit identifies potential conflicts, especially copyleft obligations.
|
||||
Misconfiguration checks detect unsafe Dockerfile patterns (root user, latest tags, permissive modes).
|
||||
Provenance features include in-toto/SLSA attestations signed with cosign for supply-chain trust.
|
||||
|
||||
| Guiding principle | What it means for Feedser |
|
||||
|-------------------|---------------------------|
|
||||
| **SBOM-first ingest** | Prefer signed SBOMs or reproducible layer diffs before falling back to raw scraping; connectors treat source docs as provenance, never as mutable truth. |
|
||||
| **Deterministic outputs** | Same inputs yield identical canonical advisories and exported JSON/Trivy DB artefacts; merge hashes and export manifests are reproducible across machines. |
|
||||
| **Restart-time plug-ins only** | Connector/exporter plug-ins load at service start, keeping runtime sandboxing simple and avoiding hot-patch attack surface. |
|
||||
| **Sovereign/offline-first** | No mandatory outbound calls beyond allow-listed advisories; Offline Kit bundles Mongo snapshots and exporter artefacts for air-gapped installs. |
|
||||
| **Operational transparency** | Every stage logs structured events (fetch, parse, merge, export) with correlation IDs so parallel agents can debug without shared state. |
|
||||
|
||||
Performance: warm scans < 5 s, cold scans < 30 s on a 4 vCPU runner.
|
||||
Deployment: entirely SaaS-free, suitable for air-gapped or on-prem use through its Offline Kit.
|
||||
Policy: anonymous users → 33 scans/day; verified → 333 /day; nearing 90 % quota triggers throttling but never full blocks.
|
||||
|
||||
More documention is available ./docs/*.md files. Read `docs/README.md` to gather information about the available documentation. You could inquiry specific documents as your work requires it
|
||||
|
||||
---
|
||||
|
||||
# 3) Practices
|
||||
|
||||
## 3.1) Naming
|
||||
All modules are .NET projects based on .NET 10 (preview). Exclussion is the UI. It is based on Angular
|
||||
All modules are contained by one or more projects. Each project goes in its dedicated folder. Each project starts with StellaOps.<ModuleName>. In case it is common for for all StellaOps modules it is library or plugin and it is named StellaOps.<LibraryOrPlugin>.
|
||||
|
||||
## 3.2) Key technologies & integrations
|
||||
|
||||
- **Runtime**: .NET 10 (`net10.0`) preview SDK; C# latest preview features.
|
||||
- **Data**: MongoDB (canonical store and job/export state).
|
||||
- **Observability**: structured logs, counters, and (optional) OpenTelemetry traces.
|
||||
- **Ops posture**: offline‑first, allowlist for remote hosts, strict schema validation, gated LLM fallback (only where explicitly configured).
|
||||
|
||||
# 4) Modules
|
||||
StellaOps is contained by different modules installable via docker containers
|
||||
- Feedser. Responsible for aggregation and delivery of vulnerability database
|
||||
- Cli. Command line tool to unlock full potential - request database operations, install scanner, request scan, configure backend
|
||||
- Backend. Configures and Manages scans
|
||||
- UI. UI to access the backend (and scanners)
|
||||
- Agent. Installable daemon that does the scanning
|
||||
- Zastava. Realtime monitor for allowed (verified) installations.
|
||||
|
||||
## 4.1) Feedser
|
||||
It is webservice based module that is responsible for aggregating vulnerabilities information from various sources, parsing and normalizing them into a canonical shape, merging and deduplicating the results in one place, with export capabilities to Json and TrivyDb. It supports init and resume for all of the sources, parse/normalize and merge/deduplication operations, plus export. Export supports delta exports—similarly to full and incremential database backups.
|
||||
|
||||
### 4.1.1) Usage
|
||||
It supports operations to be started by cmd line:
|
||||
# stella db [fetch|merge|export] [init|resume <point>]
|
||||
or
|
||||
api available on https://db.stella-ops.org
|
||||
|
||||
### 4.1.2) Data flow (end‑to‑end)
|
||||
|
||||
1. **Fetch**: connectors request source windows with retries/backoff, persist raw documents with SHA256/ETag metadata.
|
||||
2. **Parse & Normalize**: validate to DTOs (schema-checked), quarantine failures, normalize to canonical advisories (aliases, affected ranges with NEVRA/EVR/SemVer, references, provenance).
|
||||
3. **Merge & Deduplicate**: enforce precedence, build/maintain alias graphs, compute deterministic hashes, and eliminate duplicates before persisting to MongoDB.
|
||||
4. **Export**: JSON tree and/or Trivy DB; package and (optionally) push; write export state.
|
||||
|
||||
### 4.1.3) Architecture
|
||||
For more information of the architecture see `./docs/ARCHITECTURE_FEEDSER.md`.
|
||||
|
||||
---
|
||||
|
||||
### 4.1.4) Glossary (quick)
|
||||
|
||||
- **OVAL** — Vendor/distro security definition format; authoritative for OS packages.
|
||||
- **NEVRA / EVR** — RPM and Debian version semantics for OS packages.
|
||||
- **PURL / SemVer** — Coordinates and version semantics for OSS ecosystems.
|
||||
- **KEV** — Known Exploited Vulnerabilities (flag only).
|
||||
|
||||
---
|
||||
# 5) Your role as StellaOps contributor
|
||||
|
||||
You acting as information technology engineer that will take different type of roles in goal achieving StellaOps production implementation
|
||||
In order you to work - you have to be supplied with directory that contains `AGENTS.md`,`TASKS.md` files. There will you have more information about the role you have, the scope of your work and the tasks you will have.
|
||||
|
||||
Boundaries:
|
||||
- You operate only in the working directories I gave you, unless there is dependencies that makes you to work on dependency in shared directory. Then you ask for confirmation.
|
||||
|
||||
You main characteristics:
|
||||
- Keep endpoints small, deterministic, and cancellation-aware.
|
||||
- Improve logs/metrics as per tasks.
|
||||
- Update `TASKS.md` when moving tasks forward.
|
||||
- When you are done with all task you state explicitly you are done.
|
||||
- Impersonate the role described on working directory `AGENTS.md` you will read, if role is not available - take role of the CTO of the StellaOps in early stages.
|
||||
- You always strive for best practices
|
||||
- You always strive for re-usability
|
||||
- When in doubt of design decision - you ask then act
|
||||
- You are autonomus - meaning that you will work for long time alone and achieve maximum without stopping for stupid questions
|
||||
- You operate on the same directory where other agents will work. In case you need to work on directory that is dependency on provided `AGENTS.md`,`TASKS.md` files you have to ask for confirmation first.
|
||||
|
||||
## 5.1) Type of contributions
|
||||
|
||||
- **BE‑Base (Platform & Pipeline)**
|
||||
Owns DI, plugin host, job scheduler/coordinator, configuration binding, minimal API endpoints, and Mongo bootstrapping.
|
||||
- **BE‑Conn‑X (Connectors)**
|
||||
One agent per source family (NVD, Red Hat, Ubuntu, Debian, SUSE, GHSA, OSV, PSIRTs, CERTs, KEV, ICS). Implements fetch/parse/map with incremental watermarks.
|
||||
- **BE‑Merge (Canonical Merge & Dedupe)**
|
||||
Identity graph, precedence policies, canonical JSON serializer, and deterministic hashing (`merge_event`).
|
||||
- **BE‑Export (JSON & Trivy DB)**
|
||||
Deterministic export trees, Trivy DB packaging, optional ORAS push, and offline bundle.
|
||||
- **QA (Validation & Observability)**
|
||||
Schema tests, fixture goldens, determinism checks, metrics/logs/traces, e2e reproducibility runs.
|
||||
- **DevEx/Docs**
|
||||
Maintains this agent framework, templates, and per‑directory guides; assists parallelization and reviews.
|
||||
|
||||
|
||||
## 5.2) Work-in-parallel rules (important)
|
||||
|
||||
- **Directory ownership**: Each agent works **only inside its module directory**. Cross‑module edits require a brief handshake in issues/PR description.
|
||||
- **Scoping**: Use each module’s `AGENTS.md` and `TASKS.md` to plan; autonomous agents must read `src/AGENTS.md` and the module docs before acting.
|
||||
- **Determinism**: Sort keys, normalize timestamps to UTC ISO‑8601, avoid non‑deterministic data in exports and tests.
|
||||
- **Status tracking**: Update your module’s `TASKS.md` as you progress (TODO → DOING → DONE/BLOCKED).
|
||||
- **Tests**: Add/extend fixtures and unit tests per change; never regress determinism or precedence.
|
||||
- **Test layout**: Use module-specific projects in `StellaOps.Feedser.<Component>.Tests`; shared fixtures/harnesses live in `StellaOps.Feedser.Testing`.
|
||||
|
||||
---
|
||||
|
||||
Reference in New Issue
Block a user