audit work, fixed StellaOps.sln warnings/errors, fixed tests, sprints work, new advisories
This commit is contained in:
@@ -686,3 +686,91 @@ stella config get mode # Check current mode
|
||||
| Version | Date | Author | Changes |
|
||||
|---------|------|--------|---------|
|
||||
| 1.0.0 | 2025-12-20 | Agent | Initial release |
|
||||
| 1.1.0 | 2026-01-07 | Agent | Added HLC job sync section |
|
||||
|
||||
---
|
||||
|
||||
## Appendix D: HLC-Based Job Synchronization
|
||||
|
||||
> **Sprint**: SPRINT_20260105_002_003_ROUTER
|
||||
> **Added**: 2026-01-07
|
||||
|
||||
### Overview
|
||||
|
||||
When running in air-gap mode, scheduled jobs are queued locally using Hybrid Logical Clocks (HLC). Upon reconnection or physical transfer, these jobs can be merged deterministically with the central scheduler while preserving global ordering.
|
||||
|
||||
### Key Concepts
|
||||
|
||||
| Term | Description |
|
||||
|------|-------------|
|
||||
| **HLC Timestamp** | Tuple of (PhysicalTime, LogicalCounter, NodeId) for total ordering |
|
||||
| **Job Log** | Append-only log of jobs enqueued while offline |
|
||||
| **Chain Link** | Cryptographic hash linking each job to its predecessor |
|
||||
| **Air-Gap Bundle** | Exportable package containing job logs from one or more offline nodes |
|
||||
|
||||
### Exporting Jobs from Offline Node
|
||||
|
||||
```bash
|
||||
# Export all pending jobs to a bundle
|
||||
stella airgap export \
|
||||
--output /media/usb/jobs-bundle.json \
|
||||
--tenant <tenant-id>
|
||||
|
||||
# Export with DSSE signature (requires signing key)
|
||||
stella airgap export \
|
||||
--output /media/usb/jobs-bundle.json \
|
||||
--tenant <tenant-id> \
|
||||
--sign
|
||||
```
|
||||
|
||||
### Importing Jobs to Central Scheduler
|
||||
|
||||
```bash
|
||||
# Import bundle and merge jobs
|
||||
stella airgap import \
|
||||
--bundle /media/usb/jobs-bundle.json
|
||||
|
||||
# Import with signature verification
|
||||
stella airgap import \
|
||||
--bundle /media/usb/jobs-bundle.json \
|
||||
--verify-signature
|
||||
|
||||
# Dry-run to preview merge results
|
||||
stella airgap import \
|
||||
--bundle /media/usb/jobs-bundle.json \
|
||||
--dry-run
|
||||
```
|
||||
|
||||
### Merge Behavior
|
||||
|
||||
1. **Total Ordering**: Jobs are merged by HLC order key (PhysicalTime, LogicalCounter, NodeId, JobId)
|
||||
2. **Duplicate Detection**: Jobs with the same JobId are deduplicated
|
||||
3. **Chain Verification**: Each job's chain link is verified against its predecessor
|
||||
4. **Conflict Resolution**: Same-payload jobs from different nodes are kept; conflicting payloads are flagged
|
||||
|
||||
### Monitoring Metrics
|
||||
|
||||
| Metric | Description |
|
||||
|--------|-------------|
|
||||
| `airgap_bundles_exported_total` | Total bundles exported |
|
||||
| `airgap_bundles_imported_total` | Total bundles imported |
|
||||
| `airgap_jobs_synced_total` | Total jobs synchronized |
|
||||
| `airgap_merge_conflicts_total` | Merge conflicts by type |
|
||||
| `airgap_sync_duration_seconds` | Sync operation duration |
|
||||
|
||||
### Troubleshooting
|
||||
|
||||
**Bundle signature verification fails**
|
||||
1. Check that signing key is correctly configured
|
||||
2. Verify bundle was not modified during transfer
|
||||
3. Check key ID matches expected signer
|
||||
|
||||
**Merge conflicts detected**
|
||||
1. Review conflicts in import output
|
||||
2. Check for clock skew between offline nodes
|
||||
3. Use `--conflict-strategy` flag to auto-resolve
|
||||
|
||||
**Jobs missing after import**
|
||||
1. Check for duplicate JobIds (same job, same node)
|
||||
2. Verify chain integrity with `stella airgap verify --bundle <path>`
|
||||
3. Review import logs for skipped entries
|
||||
|
||||
Reference in New Issue
Block a user