Add signal contracts for reachability, exploitability, trust, and unknown symbols
- Introduced `ReachabilityState`, `RuntimeHit`, `ExploitabilitySignal`, `ReachabilitySignal`, `SignalEnvelope`, `SignalType`, `TrustSignal`, and `UnknownSymbolSignal` records to define various signal types and their properties. - Implemented JSON serialization attributes for proper data interchange. - Created project files for the new signal contracts library and corresponding test projects. - Added deterministic test fixtures for micro-interaction testing. - Included cryptographic keys for secure operations with cosign.
This commit is contained in:
StellaOps Bot
451
docs/implplan/UNBLOCK_IMPLEMENTATION_PLAN.md
Normal file
451
docs/implplan/UNBLOCK_IMPLEMENTATION_PLAN.md
Normal file
@@ -0,0 +1,451 @@
|
||||
# Blocker Unblock Implementation Plan
|
||||
|
||||
> **Created:** 2025-12-04
|
||||
> **Purpose:** Step-by-step implementation plan to unblock remaining ~14 tasks
|
||||
> **Estimated Effort:** 16-22 developer-days
|
||||
|
||||
## Executive Summary
|
||||
|
||||
After creating 11 specification contracts that unblocked ~61 tasks, we have **14 remaining blocked tasks** that require actual implementation work (not just specs). This plan outlines the implementation roadmap.
|
||||
|
||||
---
|
||||
|
||||
## Remaining Blockers Analysis
|
||||
|
||||
| Blocker | Tasks Blocked | Type | Complexity |
|
||||
|---------|--------------|------|------------|
|
||||
| WEB-POLICY-20-004 (Rate Limiting) | 6 | Code Implementation | SIMPLE |
|
||||
| Shared Signals Library | 5+ | New Library | MODERATE |
|
||||
| Postgres Repositories | 5 | Code Implementation | MODERATE |
|
||||
| Test Infrastructure | N/A | Infrastructure | MODERATE |
|
||||
| PGMI0101 Staffing | 3 | Human Decision | N/A |
|
||||
|
||||
---
|
||||
|
||||
## Implementation Phases
|
||||
|
||||
### Phase 1: Policy Engine Rate Limiting (WEB-POLICY-20-004)
|
||||
|
||||
**Duration:** 1-2 days
|
||||
**Unblocks:** 6 tasks (WEB-POLICY-20-004 chain)
|
||||
**Dependencies:** None
|
||||
|
||||
#### 1.1 Create Rate Limit Options
|
||||
|
||||
**File:** `src/Policy/StellaOps.Policy.Engine/Options/PolicyEngineRateLimitOptions.cs`
|
||||
|
||||
```csharp
|
||||
namespace StellaOps.Policy.Engine.Options;
|
||||
|
||||
public sealed class PolicyEngineRateLimitOptions
|
||||
{
|
||||
public const string SectionName = "RateLimiting";
|
||||
|
||||
/// <summary>Default permits per window for simulation endpoints</summary>
|
||||
public int SimulationPermitLimit { get; set; } = 100;
|
||||
|
||||
/// <summary>Window duration in seconds</summary>
|
||||
public int WindowSeconds { get; set; } = 60;
|
||||
|
||||
/// <summary>Queue limit for pending requests</summary>
|
||||
public int QueueLimit { get; set; } = 10;
|
||||
|
||||
/// <summary>Enable tenant-aware partitioning</summary>
|
||||
public bool TenantPartitioning { get; set; } = true;
|
||||
}
|
||||
```
|
||||
|
||||
#### 1.2 Register Rate Limiter in Program.cs
|
||||
|
||||
Add to `src/Policy/StellaOps.Policy.Engine/Program.cs`:
|
||||
|
||||
```csharp
|
||||
// Rate limiting configuration
|
||||
var rateLimitOptions = builder.Configuration
|
||||
.GetSection(PolicyEngineRateLimitOptions.SectionName)
|
||||
.Get<PolicyEngineRateLimitOptions>() ?? new();
|
||||
|
||||
builder.Services.AddRateLimiter(options =>
|
||||
{
|
||||
options.RejectionStatusCode = StatusCodes.Status429TooManyRequests;
|
||||
|
||||
options.AddTokenBucketLimiter("policy-simulation", limiterOptions =>
|
||||
{
|
||||
limiterOptions.TokenLimit = rateLimitOptions.SimulationPermitLimit;
|
||||
limiterOptions.ReplenishmentPeriod = TimeSpan.FromSeconds(rateLimitOptions.WindowSeconds);
|
||||
limiterOptions.TokensPerPeriod = rateLimitOptions.SimulationPermitLimit;
|
||||
limiterOptions.QueueLimit = rateLimitOptions.QueueLimit;
|
||||
limiterOptions.QueueProcessingOrder = QueueProcessingOrder.OldestFirst;
|
||||
});
|
||||
|
||||
options.OnRejected = async (context, cancellationToken) =>
|
||||
{
|
||||
PolicyEngineTelemetry.RateLimitExceededCounter.Add(1);
|
||||
context.HttpContext.Response.StatusCode = StatusCodes.Status429TooManyRequests;
|
||||
await context.HttpContext.Response.WriteAsJsonAsync(new
|
||||
{
|
||||
error = "ERR_POL_007",
|
||||
message = "Rate limit exceeded. Please retry after the reset window.",
|
||||
retryAfterSeconds = rateLimitOptions.WindowSeconds
|
||||
}, cancellationToken);
|
||||
};
|
||||
});
|
||||
```
|
||||
|
||||
#### 1.3 Apply to Simulation Endpoints
|
||||
|
||||
Modify `src/Policy/StellaOps.Policy.Engine/Endpoints/RiskSimulationEndpoints.cs`:
|
||||
|
||||
```csharp
|
||||
group.MapPost("/simulate", SimulateRisk)
|
||||
.RequireRateLimiting("policy-simulation") // ADD THIS
|
||||
.WithName("SimulateRisk");
|
||||
```
|
||||
|
||||
#### 1.4 Add Telemetry Counter
|
||||
|
||||
Add to `src/Policy/StellaOps.Policy.Engine/Telemetry/PolicyEngineTelemetry.cs`:
|
||||
|
||||
```csharp
|
||||
public static readonly Counter<long> RateLimitExceededCounter =
|
||||
Meter.CreateCounter<long>(
|
||||
"policy_rate_limit_exceeded_total",
|
||||
unit: "requests",
|
||||
description: "Total requests rejected due to rate limiting");
|
||||
```
|
||||
|
||||
#### 1.5 Configuration Sample
|
||||
|
||||
Add to `etc/policy-engine.yaml.sample`:
|
||||
|
||||
```yaml
|
||||
RateLimiting:
|
||||
SimulationPermitLimit: 100
|
||||
WindowSeconds: 60
|
||||
QueueLimit: 10
|
||||
TenantPartitioning: true
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Phase 2: Shared Signals Contracts Library
|
||||
|
||||
**Duration:** 3-4 days
|
||||
**Unblocks:** 5+ modules (Concelier, Scanner, Policy, Signals, Authority)
|
||||
**Dependencies:** None
|
||||
|
||||
#### 2.1 Create Project Structure
|
||||
|
||||
```
|
||||
src/__Libraries/StellaOps.Signals.Contracts/
|
||||
├── StellaOps.Signals.Contracts.csproj
|
||||
├── AGENTS.md
|
||||
├── Models/
|
||||
│ ├── SignalEnvelope.cs
|
||||
│ ├── SignalType.cs
|
||||
│ ├── ReachabilitySignal.cs
|
||||
│ ├── EntropySignal.cs
|
||||
│ ├── ExploitabilitySignal.cs
|
||||
│ ├── TrustSignal.cs
|
||||
│ └── UnknownSymbolSignal.cs
|
||||
├── Abstractions/
|
||||
│ ├── ISignalEmitter.cs
|
||||
│ ├── ISignalConsumer.cs
|
||||
│ └── ISignalContext.cs
|
||||
└── Extensions/
|
||||
└── ServiceCollectionExtensions.cs
|
||||
```
|
||||
|
||||
#### 2.2 Core Models
|
||||
|
||||
**SignalEnvelope.cs:**
|
||||
```csharp
|
||||
namespace StellaOps.Signals.Contracts;
|
||||
|
||||
public sealed record SignalEnvelope(
|
||||
string SignalKey,
|
||||
SignalType SignalType,
|
||||
object Value,
|
||||
DateTimeOffset ComputedAt,
|
||||
string SourceService,
|
||||
string? TenantId = null,
|
||||
string? CorrelationId = null,
|
||||
string? ProvenanceDigest = null);
|
||||
```
|
||||
|
||||
**SignalType.cs:**
|
||||
```csharp
|
||||
namespace StellaOps.Signals.Contracts;
|
||||
|
||||
public enum SignalType
|
||||
{
|
||||
Reachability,
|
||||
Entropy,
|
||||
Exploitability,
|
||||
Trust,
|
||||
UnknownSymbol,
|
||||
Custom
|
||||
}
|
||||
```
|
||||
|
||||
#### 2.3 Signal Models
|
||||
|
||||
Each signal type gets a dedicated record:
|
||||
|
||||
- `ReachabilitySignal` - package reachability from callgraph
|
||||
- `EntropySignal` - code complexity/risk metrics
|
||||
- `ExploitabilitySignal` - KEV status, exploit availability
|
||||
- `TrustSignal` - reputation, chain of custody scores
|
||||
- `UnknownSymbolSignal` - unresolved dependencies
|
||||
|
||||
#### 2.4 Abstractions
|
||||
|
||||
```csharp
|
||||
public interface ISignalEmitter
|
||||
{
|
||||
ValueTask EmitAsync(SignalEnvelope signal, CancellationToken ct = default);
|
||||
ValueTask EmitBatchAsync(IEnumerable<SignalEnvelope> signals, CancellationToken ct = default);
|
||||
}
|
||||
|
||||
public interface ISignalConsumer
|
||||
{
|
||||
IAsyncEnumerable<SignalEnvelope> ConsumeAsync(
|
||||
SignalType? filterType = null,
|
||||
CancellationToken ct = default);
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Phase 3: Postgres Repositories
|
||||
|
||||
**Duration:** 4-5 days
|
||||
**Unblocks:** Persistence for new features
|
||||
**Dependencies:** SQL migrations
|
||||
|
||||
#### 3.1 Repository Interfaces
|
||||
|
||||
Create in `src/Policy/__Libraries/StellaOps.Policy.Storage.Postgres/Repositories/`:
|
||||
|
||||
| Interface | Methods |
|
||||
|-----------|---------|
|
||||
| `ISnapshotRepository` | Create, GetById, List, Delete |
|
||||
| `IViolationEventRepository` | Append, GetById, List (immutable) |
|
||||
| `IWorkerResultRepository` | Create, GetById, List, Update |
|
||||
| `IConflictRepository` | Create, GetById, List, Resolve |
|
||||
| `ILedgerExportRepository` | Create, GetById, List, GetByDigest |
|
||||
|
||||
#### 3.2 SQL Migrations
|
||||
|
||||
Create migrations for tables:
|
||||
|
||||
```sql
|
||||
-- policy.snapshots
|
||||
CREATE TABLE policy.snapshots (
|
||||
id UUID PRIMARY KEY,
|
||||
tenant_id TEXT NOT NULL,
|
||||
policy_id UUID NOT NULL,
|
||||
version INTEGER NOT NULL,
|
||||
content_digest TEXT NOT NULL,
|
||||
metadata JSONB,
|
||||
created_by TEXT NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- policy.violation_events (append-only)
|
||||
CREATE TABLE policy.violation_events (
|
||||
id UUID PRIMARY KEY,
|
||||
tenant_id TEXT NOT NULL,
|
||||
policy_id UUID NOT NULL,
|
||||
rule_id TEXT NOT NULL,
|
||||
severity TEXT NOT NULL,
|
||||
subject_purl TEXT,
|
||||
details JSONB,
|
||||
occurred_at TIMESTAMPTZ NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- Similar for conflicts, worker_results, ledger_exports
|
||||
```
|
||||
|
||||
#### 3.3 Implementation Pattern
|
||||
|
||||
Follow `RiskProfileRepository.cs` pattern:
|
||||
|
||||
```csharp
|
||||
public sealed class SnapshotRepository : RepositoryBase<PolicyDataSource>, ISnapshotRepository
|
||||
{
|
||||
public SnapshotRepository(PolicyDataSource dataSource, ILogger<SnapshotRepository> logger)
|
||||
: base(dataSource, logger) { }
|
||||
|
||||
public async Task<SnapshotEntity> CreateAsync(SnapshotEntity entity, CancellationToken ct)
|
||||
{
|
||||
const string sql = """
|
||||
INSERT INTO policy.snapshots
|
||||
(id, tenant_id, policy_id, version, content_digest, metadata, created_by)
|
||||
VALUES (@Id, @TenantId, @PolicyId, @Version, @ContentDigest, @Metadata::jsonb, @CreatedBy)
|
||||
RETURNING *
|
||||
""";
|
||||
|
||||
return await ExecuteScalarAsync<SnapshotEntity>(sql, entity, ct);
|
||||
}
|
||||
|
||||
// ... other CRUD methods
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Phase 4: Test Infrastructure
|
||||
|
||||
**Duration:** 2-3 days
|
||||
**Unblocks:** Validation before merge
|
||||
**Dependencies:** Phase 3
|
||||
|
||||
#### 4.1 Postgres Test Fixture
|
||||
|
||||
```csharp
|
||||
public sealed class PostgresFixture : IAsyncLifetime
|
||||
{
|
||||
private TestcontainersContainer? _container;
|
||||
public string ConnectionString { get; private set; } = string.Empty;
|
||||
|
||||
public async Task InitializeAsync()
|
||||
{
|
||||
_container = new TestcontainersBuilder<TestcontainersContainer>()
|
||||
.WithImage("postgres:16-alpine")
|
||||
.WithEnvironment("POSTGRES_PASSWORD", "test")
|
||||
.WithPortBinding(5432, true)
|
||||
.Build();
|
||||
|
||||
await _container.StartAsync();
|
||||
ConnectionString = $"Host=localhost;Port={_container.GetMappedPublicPort(5432)};...";
|
||||
|
||||
// Run migrations
|
||||
await MigrationRunner.RunAsync(ConnectionString);
|
||||
}
|
||||
|
||||
public async Task DisposeAsync() => await _container?.DisposeAsync();
|
||||
}
|
||||
```
|
||||
|
||||
#### 4.2 Test Classes
|
||||
|
||||
- `RateLimitingTests.cs` - quota exhaustion, recovery, tenant partitioning
|
||||
- `SnapshotRepositoryTests.cs` - CRUD operations
|
||||
- `ViolationEventRepositoryTests.cs` - append-only semantics
|
||||
- `ConflictRepositoryTests.cs` - resolution workflow
|
||||
- `SignalEnvelopeTests.cs` - serialization, validation
|
||||
|
||||
---
|
||||
|
||||
### Phase 5: New Endpoints
|
||||
|
||||
**Duration:** 2-3 days
|
||||
**Unblocks:** API surface completion
|
||||
**Dependencies:** Phase 3
|
||||
|
||||
#### 5.1 Endpoint Groups
|
||||
|
||||
| Path | Operations | Auth |
|
||||
|------|------------|------|
|
||||
| `/api/policy/snapshots` | GET, POST, DELETE | `policy:read`, `policy:author` |
|
||||
| `/api/policy/violations` | GET | `policy:read` |
|
||||
| `/api/policy/conflicts` | GET, POST (resolve) | `policy:read`, `policy:review` |
|
||||
| `/api/policy/exports` | GET, POST | `policy:read`, `policy:archive` |
|
||||
|
||||
---
|
||||
|
||||
## Execution Order
|
||||
|
||||
```
|
||||
Day 1-2: Phase 1 (Rate Limiting)
|
||||
└── WEB-POLICY-20-004 ✓ UNBLOCKED
|
||||
|
||||
Day 3-5: Phase 2 (Signals Library)
|
||||
└── Concelier, Scanner, Policy, Signals, Authority ✓ ENABLED
|
||||
|
||||
Day 6-9: Phase 3 (Repositories)
|
||||
└── Persistence layer ✓ COMPLETE
|
||||
|
||||
Day 10-12: Phase 4 (Tests)
|
||||
└── Validation ✓ READY
|
||||
|
||||
Day 13-15: Phase 5 (Endpoints)
|
||||
└── API surface ✓ COMPLETE
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Files to Create/Modify Summary
|
||||
|
||||
### New Files (22 files)
|
||||
|
||||
```
|
||||
src/Policy/StellaOps.Policy.Engine/Options/
|
||||
└── PolicyEngineRateLimitOptions.cs
|
||||
|
||||
src/__Libraries/StellaOps.Signals.Contracts/
|
||||
├── StellaOps.Signals.Contracts.csproj
|
||||
├── AGENTS.md
|
||||
├── Models/SignalEnvelope.cs
|
||||
├── Models/SignalType.cs
|
||||
├── Models/ReachabilitySignal.cs
|
||||
├── Models/EntropySignal.cs
|
||||
├── Models/ExploitabilitySignal.cs
|
||||
├── Models/TrustSignal.cs
|
||||
├── Models/UnknownSymbolSignal.cs
|
||||
├── Abstractions/ISignalEmitter.cs
|
||||
├── Abstractions/ISignalConsumer.cs
|
||||
└── Extensions/ServiceCollectionExtensions.cs
|
||||
|
||||
src/Policy/__Libraries/StellaOps.Policy.Storage.Postgres/Repositories/
|
||||
├── ISnapshotRepository.cs
|
||||
├── SnapshotRepository.cs
|
||||
├── IViolationEventRepository.cs
|
||||
├── ViolationEventRepository.cs
|
||||
├── IConflictRepository.cs
|
||||
├── ConflictRepository.cs
|
||||
├── ILedgerExportRepository.cs
|
||||
└── LedgerExportRepository.cs
|
||||
```
|
||||
|
||||
### Files to Modify (5 files)
|
||||
|
||||
```
|
||||
src/Policy/StellaOps.Policy.Engine/Program.cs
|
||||
src/Policy/StellaOps.Policy.Engine/Telemetry/PolicyEngineTelemetry.cs
|
||||
src/Policy/StellaOps.Policy.Engine/Endpoints/RiskSimulationEndpoints.cs
|
||||
src/Policy/StellaOps.Policy.Engine/Endpoints/PathScopeSimulationEndpoint.cs
|
||||
etc/policy-engine.yaml.sample
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] Rate limiting returns 429 when quota exceeded
|
||||
- [ ] Signals library compiles and referenced by 5+ modules
|
||||
- [ ] All 5 repositories pass CRUD tests
|
||||
- [ ] Endpoints return proper responses with auth
|
||||
- [ ] Telemetry metrics visible in dashboards
|
||||
- [ ] No regression in existing tests
|
||||
|
||||
---
|
||||
|
||||
## Risk Mitigation
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|------------|
|
||||
| Breaking existing endpoints | Feature flag rate limiting |
|
||||
| Signal library circular deps | Careful namespace isolation |
|
||||
| Migration failures | Test migrations in isolated DB first |
|
||||
| Test flakiness | Use deterministic test data |
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. **Start Phase 1** - Implement rate limiting (simplest, immediate impact)
|
||||
2. **Parallel Phase 2** - Create Signals.Contracts scaffolding
|
||||
3. **Review** - Get feedback before Phase 3
|
||||
Reference in New Issue
Block a user