Refactor and update test projects, remove obsolete tests, and upgrade dependencies

- Deleted obsolete test files for SchedulerAuditService and SchedulerMongoSessionFactory.
- Removed unused TestDataFactory class.
- Updated project files for Mongo.Tests to remove references to deleted files.
- Upgraded BouncyCastle.Cryptography package to version 2.6.2 across multiple projects.
- Replaced Microsoft.Extensions.Http.Polly with Microsoft.Extensions.Http.Resilience in Zastava.Webhook project.
- Updated NetEscapades.Configuration.Yaml package to version 3.1.0 in Configuration library.
- Upgraded Pkcs11Interop package to version 5.1.2 in Cryptography libraries.
- Refactored Argon2idPasswordHasher to use BouncyCastle for hashing instead of Konscious.
- Updated JsonSchema.Net package to version 7.3.2 in Microservice project.
- Updated global.json to use .NET SDK version 10.0.101.
This commit is contained in:
master
2025-12-10 19:13:29 +02:00
parent a3c7fe5e88
commit b7059d523e
369 changed files with 11125 additions and 14245 deletions

View File

@@ -1,6 +1,6 @@
# Local PostgreSQL for StellaOps (Scheduler focus)
# Local PostgreSQL for StellaOps
This doc describes how to bring up a local PostgreSQL 17 instance for Scheduler development and tests.
This doc describes how to bring up a local PostgreSQL 17 instance for development and tests.
## Quick start (Docker)
@@ -15,10 +15,17 @@ Defaults:
- Password: `stella`
- Database: `stella`
Features enabled:
- `pg_stat_statements` for query performance analysis
- Pre-created schemas: authority, vuln, vex, scheduler, notify, policy, concelier, audit
- Extensions: pg_trgm, btree_gin, pgcrypto
Verify:
```bash
docker ps --filter name=stella-postgres
docker exec -it stella-postgres psql -U stella -d stella -c 'select version();'
docker exec -it stella-postgres psql -U stella -d stella -c 'SELECT version();'
docker exec -it stella-postgres psql -U stella -d stella -c '\dn' # List schemas
docker exec -it stella-postgres psql -U stella -d stella -c '\dx' # List extensions
```
Stop/cleanup:
@@ -39,37 +46,16 @@ docker volume rm stella-postgres-data
- `PGPASSWORD=stella`
- `PGDATABASE=stella`
## Using with Scheduler Postgres storage
- Scheduler Postgres repositories connect via `SchedulerDataSource` using tenant-aware connections; for local work set your appsettings or environment to the connection string above.
- Integration tests currently rely on Testcontainers; if Docker is available the tests will spin up their own isolated container. When Docker is unavailable, run against this local instance by exporting the variables above and disabling Testcontainers in your local run configuration if supported.
## Using with module storage
- Module repositories connect via their respective DataSource types using tenant-aware connections; for local work set your appsettings or environment to the connection string above.
- Integration tests rely on Testcontainers; if Docker is available the tests will spin up their own isolated container. When Docker is unavailable, run against this local instance by exporting the environment variables above.
## Notes
- Image: `postgres:17` (latest GA at time of writing).
- Healthcheck is built into the compose service; wait for `healthy` before running tests.
- Keep volumes deterministic: the compose file names the volume `stella-postgres-data`.
- Schemas are pre-created via init scripts in `ops/devops/local-postgres/init/`.
## Scheduler Mongo → Postgres backfill
## Operations guide
Use the new `Scheduler.Backfill` tool to copy Scheduler data from MongoDB into the Postgres schema.
```bash
dotnet run \
--project src/Scheduler/Tools/Scheduler.Backfill/Scheduler.Backfill.csproj \
--mongo "${MONGO_CONNECTION_STRING:-mongodb://localhost:27017}" \
--mongo-db "${MONGO_DATABASE:-stellaops_scheduler}" \
--pg "Host=localhost;Port=5432;Username=stella;Password=stella;Database=stella" \
--batch 500
```
Flags:
- `--dry-run` to validate without writing.
- `--batch` to tune insert batch size (defaults to 500).
What it does:
- Reads `schedules` and `runs` collections.
- Serialises documents with `CanonicalJsonSerializer` for deterministic JSON.
- Upserts into `scheduler.schedules` and `scheduler.runs` tables (created by migration `001_initial_schema.sql`).
Verification tips:
- Compare counts after backfill: `select count(*) from scheduler.schedules;` and `...runs;`.
- Spot-check next-fire timing by comparing `cron_expression` and `timezone` with the Mongo source; deterministic ordering is preserved via canonical JSON.
For production deployment, performance tuning, monitoring, and backup/restore procedures, see [`docs/operations/postgresql-guide.md`](../operations/postgresql-guide.md).