git.stella-ops.org/docs/testing/webservice-test-rollout-plan.md

WebService Test Rollout Plan

This document defines the rollout plan for applying the WebService test discipline to all StellaOps web services.

Overview

Following the pilot implementation on Scanner.WebService (Sprint 5100.0007.0006), this plan defines the order and timeline for rolling out comprehensive WebService tests to all remaining services.

---

Service Inventory

Service	Module Path	Priority	Status	Sprint
Scanner.WebService	src/Scanner/StellaOps.Scanner.WebService	P0 (Pilot)	✅ Existing tests	5100.0007.0006
Concelier.WebService	src/Concelier/StellaOps.Concelier.WebService	P1	Pending	TBD
Excititor.WebService	src/Excititor/StellaOps.Excititor.WebService	P1	Pending	TBD
Policy.Engine	src/Policy/StellaOps.Policy.Engine	P1	Pending	TBD
Scheduler.WebService	src/Scheduler/StellaOps.Scheduler.WebService	P2	Pending	TBD
Notify.WebService	src/Notify/StellaOps.Notify.WebService	P2	Pending	TBD
Authority	src/Authority/StellaOps.Authority	P2	Pending	TBD
Signer	src/Signer/StellaOps.Signer	P3	Pending	TBD
Attestor	src/Attestor/StellaOps.Attestor	P3	Pending	TBD
ExportCenter.WebService	src/ExportCenter/StellaOps.ExportCenter.WebService	P3	Pending	TBD
Registry.TokenService	src/Registry/StellaOps.Registry.TokenService	P3	Pending	TBD
VulnExplorer.Api	src/VulnExplorer/StellaOps.VulnExplorer.Api	P3	Pending	TBD
Graph.Api	src/Graph/StellaOps.Graph.Api	P3	Pending	TBD
Orchestrator	src/Orchestrator/StellaOps.Orchestrator	P4	Pending	TBD

---

Rollout Phases

Phase 1: Core Data Flow Services (P1)

Timeline: Sprint 5100.0008.* (Q1 2026)

Services:

• Concelier.WebService — Primary advisory ingestion service
• Excititor.WebService — Enrichment and correlation service
• Policy.Engine — Policy evaluation service

Rationale: These services form the core data flow pipeline. They have high traffic, complex contracts, and critical security boundaries.

Test Requirements:

Test Type	Concelier	Excititor	Policy
Contract (OpenAPI)	Required	Required	Required
OTel traces	Required	Required	Required
Negative tests	Required	Required	Required
Auth/AuthZ	Required	Required	Required
Tenant isolation	Required	Required	Required

---

Phase 2: Scheduling & Notification Services (P2)

Timeline: Sprint 5100.0009.* (Q2 2026)

Services:

• Scheduler.WebService — Job scheduling and orchestration
• Notify.WebService — Notification dispatch
• Authority — Authentication/authorization service

Rationale: These services support operational workflows. Authority is critical for security testing of all other services.

Test Requirements:

Test Type	Scheduler	Notify	Authority
Contract (OpenAPI)	Required	Required	Required
OTel traces	Required	Required	Required
Negative tests	Required	Required	Required
Auth/AuthZ	N/A (system)	Required	N/A (self)
Token issuance	N/A	N/A	Required

---

Phase 3: Signing & Attestation Services (P3)

Timeline: Sprint 5100.0010.* (Q2-Q3 2026)

Services:

• Signer — Cryptographic signing service
• Attestor — Attestation generation/verification
• ExportCenter.WebService — Report export service
• Registry.TokenService — OCI registry token service
• VulnExplorer.Api — Vulnerability exploration API
• Graph.Api — Graph query API

Rationale: These services have specialized contracts and lower traffic. They require careful security testing due to cryptographic operations.

Test Requirements:

Test Type	Signer	Attestor	Others
Contract (OpenAPI)	Required	Required	Required
OTel traces	Required	Required	Required
Negative tests	Required	Required	Required
Crypto validation	Required	Required	N/A

---

Phase 4: Orchestration Services (P4)

Timeline: Sprint 5100.0011.* (Q3 2026)

Services:

• Orchestrator — Workflow orchestration

Rationale: Orchestrator is a meta-service that coordinates other services. Testing depends on other services being testable first.

---

Test Coverage Targets

Minimum Requirements (PR-Gating)

Test Category	Min Coverage	Lane
Contract (OpenAPI)	100% of public endpoints	Contract
Negative (4xx errors)	100% of error codes	Unit/Security
Auth/AuthZ	100% of protected endpoints	Security

Recommended (Quality Gate)

Test Category	Target Coverage	Lane
OTel traces	80% of endpoints	Integration
Tenant isolation	100% of data endpoints	Security
Performance baselines	Key endpoints	Performance

---

Implementation Checklist per Service

## <Service Name> WebService Tests

### Setup
- [ ] Create `<Service>ApplicationFactory` (WebApplicationFactory)
- [ ] Create `<Service>TestFixture` if custom setup needed
- [ ] Add test project: `StellaOps.<Service>.WebService.Tests`
- [ ] Add reference to `StellaOps.TestKit`

### Contract Tests
- [ ] Extract OpenAPI schema snapshot (`Snapshots/<service>-v1.json`)
- [ ] Add schema stability test
- [ ] Add response contract tests for key endpoints

### OTel Tests
- [ ] Add trace assertion tests for key endpoints
- [ ] Verify required tags (tenant.id, request.id, http.route)

### Negative Tests
- [ ] Malformed content type → 415
- [ ] Oversized payload → 413
- [ ] Method mismatch → 405
- [ ] Missing required field → 400
- [ ] Invalid field value → 400

### Auth Tests
- [ ] Anonymous request → 401
- [ ] Expired token → 401
- [ ] Missing scope → 403
- [ ] Cross-tenant access → 404/403

### CI Integration
- [ ] Verify traits assigned: Contract, Security, Integration, Unit
- [ ] PR passes all lanes
- [ ] Add to TEST_COVERAGE_MATRIX.md

---

Sprint Planning Template

When creating sprints for new service tests:

# Sprint 5100.XXXX.YYYY - <Service> WebService Tests

## Topic & Scope
- Apply WebService test discipline to <Service>.WebService
- Contract tests, OTel traces, negative tests, auth tests
- **Working directory:** `src/<Module>/__Tests/StellaOps.<Module>.WebService.Tests`

## Delivery Tracker
| # | Task ID | Status | Task Definition |
|---|---------|--------|-----------------|
| 1 | WEBSVC-XXXX-001 | TODO | Create <Service>ApplicationFactory |
| 2 | WEBSVC-XXXX-002 | TODO | Add OpenAPI contract tests |
| 3 | WEBSVC-XXXX-003 | TODO | Add OTel trace tests |
| 4 | WEBSVC-XXXX-004 | TODO | Add negative tests (4xx) |
| 5 | WEBSVC-XXXX-005 | TODO | Add auth/authz tests |
| 6 | WEBSVC-XXXX-006 | TODO | Update TEST_COVERAGE_MATRIX.md |

---

Success Metrics

Metric	Target	Measurement
Services with contract tests	100%	Count of services with OpenAPI snapshot tests
Services with auth tests	100%	Count of services with auth boundary tests
Contract test failures in production	0	Breaking changes detected in staging
Security test coverage	100% of auth endpoints	Audit of protected routes vs tests

---

Risks & Mitigations

Risk	Impact	Mitigation
Services lack OpenAPI spec	Cannot do contract testing	Generate spec via Swashbuckle/NSwag
OTel not configured in service	Cannot verify traces	Add OTel middleware as prerequisite
Auth disabled in test mode	False confidence	Test with auth enabled, use test tokens
Test fixtures are slow	CI timeout	Share fixtures, use in-memory providers

---

References

• WebService Test Discipline
• Test Coverage Matrix
• CI Lane Filters
• Testing Strategy Models

---

Last updated: 2025-06-30 · Sprint 5100.0007.0006