232 lines
10 KiB
Markdown
232 lines
10 KiB
Markdown
# Doctor Check Evidence Schemas
|
|
|
|
This document defines the standardized evidence schemas for all Doctor health checks. These schemas enable AdvisoryAI to understand field meanings, expected ranges, and root cause differentiation.
|
|
|
|
> **Sprint:** SPRINT_20260118_015_Doctor_check_quality_improvements
|
|
> **Task:** DQUAL-006 - Standardize evidence schema documentation
|
|
|
|
---
|
|
|
|
## Evidence Schema Conventions
|
|
|
|
### Field Naming
|
|
- Use `snake_case` for all field names
|
|
- Boolean fields: `is_*`, `has_*`, `*_enabled`, `*_available`
|
|
- Timestamp fields: `*_utc` suffix, ISO8601 format
|
|
- Duration fields: `*_ms` or `*_seconds` suffix
|
|
- Status fields: lowercase string enums
|
|
|
|
### Value Types
|
|
- `string`: UTF-8 text
|
|
- `int`: 64-bit signed integer
|
|
- `float`: 64-bit floating point
|
|
- `bool`: `true` or `false` (lowercase in JSON)
|
|
- `list<T>`: JSON array of type T
|
|
- `ISO8601`: timestamp string in ISO8601 format
|
|
|
|
---
|
|
|
|
## Policy Engine Checks
|
|
|
|
### check.policy.engine
|
|
|
|
**Description:** Verify policy engine compilation, evaluation, and storage health
|
|
|
|
**Evidence Fields:**
|
|
|
|
| Field | Type | Description | Expected Range |
|
|
|-------|------|-------------|----------------|
|
|
| `engine_type` | string | Policy engine type | `opa`, `rego`, `custom`, `unknown` |
|
|
| `engine_version` | string | Engine version string | Semantic version or `unknown` |
|
|
| `engine_url` | string | Policy engine base URL | Valid HTTP(S) URL |
|
|
| `compilation_status` | string | Compilation health | `OK`, `FAILED` |
|
|
| `evaluation_status` | string | Evaluation health | `OK`, `FAILED` |
|
|
| `storage_status` | string | Storage health | `OK`, `FAILED` |
|
|
| `policy_count` | int | Number of loaded policies | ≥ 0 |
|
|
| `compilation_time_ms` | int | Compilation latency | 0-10000 (typical < 100) |
|
|
| `evaluation_latency_p50_ms` | int | Median evaluation time | 0-5000 (typical < 50) |
|
|
| `cache_hit_ratio` | float | Policy cache efficiency | 0.0-1.0 |
|
|
| `last_compilation_error` | string? | Most recent compilation error | null or error message |
|
|
| `evaluation_error` | string? | Most recent evaluation error | null or error message |
|
|
| `storage_error` | string? | Most recent storage error | null or error message |
|
|
|
|
**Likely Cause Differentiation:**
|
|
|
|
| Evidence Pattern | Likely Cause |
|
|
|-----------------|--------------|
|
|
| `compilation_status=FAILED` | OPA/Rego syntax error or engine unavailable |
|
|
| `evaluation_status=FAILED` | Policy evaluation timeout or runtime error |
|
|
| `storage_status=FAILED` | PostgreSQL connection issue or disk full |
|
|
| `evaluation_latency_p50_ms > 100` | Complex policies or cold cache |
|
|
| `cache_hit_ratio < 0.5` | Cache not warmed or policies changing frequently |
|
|
|
|
---
|
|
|
|
## Authentication Checks
|
|
|
|
### check.auth.oidc
|
|
|
|
**Description:** Verify connectivity to configured OIDC provider and discovery endpoint
|
|
|
|
**Evidence Fields:**
|
|
|
|
| Field | Type | Description | Expected Range |
|
|
|-------|------|-------------|----------------|
|
|
| `issuer_url` | string | OIDC issuer URL | Valid HTTPS URL |
|
|
| `discovery_reachable` | bool | Can reach discovery endpoint | `true` or `false` |
|
|
| `discovery_response_ms` | int | Discovery fetch latency | 0-10000 (typical < 500) |
|
|
| `authorization_endpoint_present` | bool | Has authorization endpoint | `true` |
|
|
| `token_endpoint_present` | bool | Has token endpoint | `true` |
|
|
| `jwks_uri_present` | bool | Has JWKS URI | `true` |
|
|
| `jwks_key_count` | int | Number of signing keys | ≥ 1 |
|
|
| `jwks_fetch_ms` | int | JWKS fetch latency | 0-10000 (typical < 500) |
|
|
| `http_status_code` | int? | HTTP response code | null or 100-599 |
|
|
| `error_message` | string? | Error details | null or error string |
|
|
| `connection_error_type` | string? | Error classification | `ssl_error`, `dns_failure`, `refused`, `timeout`, `connection_failed` |
|
|
|
|
**Likely Cause Differentiation:**
|
|
|
|
| Evidence Pattern | Likely Cause |
|
|
|-----------------|--------------|
|
|
| `discovery_reachable=false`, `connection_error_type=dns_failure` | DNS resolution failure |
|
|
| `discovery_reachable=false`, `connection_error_type=ssl_error` | TLS certificate issue |
|
|
| `discovery_reachable=false`, `connection_error_type=refused` | OIDC provider down or firewall |
|
|
| `discovery_reachable=true`, `authorization_endpoint_present=false` | Malformed discovery document |
|
|
| `jwks_key_count=0` | JWKS endpoint error or key rotation in progress |
|
|
|
|
---
|
|
|
|
## Cryptography Checks
|
|
|
|
### check.crypto.fips
|
|
|
|
**Description:** Verify FIPS 140-2 mode is enabled when required by crypto profile
|
|
|
|
**Evidence Fields:**
|
|
|
|
| Field | Type | Description | Expected Range |
|
|
|-------|------|-------------|----------------|
|
|
| `fips_mode_enabled` | bool | System FIPS mode active | `true` or `false` |
|
|
| `platform` | string | Operating system platform | `windows`, `linux`, `macos`, `unknown` |
|
|
| `crypto_provider` | string | Cryptographic provider | `bcrypt`, `openssl`, `managed`, `unknown` |
|
|
| `openssl_fips_module_loaded` | bool | OpenSSL FIPS module status | `true` or `false` |
|
|
| `crypto_profile` | string | Configured crypto profile | Profile name from config |
|
|
| `algorithms_tested` | string | Comma-separated algorithm list | Algorithm names |
|
|
| `algorithms_available` | string | Algorithms that passed testing | Algorithm names |
|
|
| `algorithms_missing` | string | Algorithms that failed testing | Algorithm names or `none` |
|
|
| `status` | string | Overall compliance status | `compliant`, `non-compliant` |
|
|
| `test_aes_256` | string | AES-256 test result | `pass` or `fail: <error>` |
|
|
| `test_sha_256` | string | SHA-256 test result | `pass` or `fail: <error>` |
|
|
| `test_sha_384` | string | SHA-384 test result | `pass` or `fail: <error>` |
|
|
| `test_sha_512` | string | SHA-512 test result | `pass` or `fail: <error>` |
|
|
| `test_rsa_2048` | string | RSA-2048 test result | `pass` or `fail: <error>` |
|
|
| `test_ecdsa_p256` | string | ECDSA-P256 test result | `pass` or `fail: <error>` |
|
|
|
|
**Likely Cause Differentiation:**
|
|
|
|
| Evidence Pattern | Likely Cause |
|
|
|-----------------|--------------|
|
|
| `fips_mode_enabled=false`, `platform=linux` | FIPS mode not enabled via fips-mode-setup |
|
|
| `fips_mode_enabled=false`, `platform=windows` | FIPS Group Policy not configured |
|
|
| `openssl_fips_module_loaded=false` | OpenSSL FIPS provider not installed |
|
|
| `algorithms_missing` contains values | Crypto provider missing FIPS-validated algorithms |
|
|
|
|
---
|
|
|
|
## Attestation Checks
|
|
|
|
### check.attestation.clock.skew
|
|
|
|
**Description:** Verify system clock is synchronized for attestation validity
|
|
|
|
**Evidence Fields:**
|
|
|
|
| Field | Type | Description | Expected Range |
|
|
|-------|------|-------------|----------------|
|
|
| `local_time_utc` | ISO8601 | System time | Valid timestamp |
|
|
| `server_time_utc` | ISO8601 | Reference server time | Valid timestamp |
|
|
| `skew_seconds` | float | Clock difference (positive = ahead) | -300 to 300 (typical < 5) |
|
|
| `max_allowed_skew` | int | Threshold in seconds | Default: 5 |
|
|
| `ntp_daemon_running` | bool | NTP service active | `true` or `false` |
|
|
| `ntp_daemon_type` | string | NTP daemon type | `chronyd`, `ntpd`, `systemd-timesyncd`, `w32time`, `unknown` |
|
|
| `ntp_servers_configured` | string | Comma-separated NTP servers | Server hostnames |
|
|
| `last_sync_time_utc` | ISO8601? | Last successful sync | Timestamp or `null` |
|
|
| `sync_age_seconds` | int? | Seconds since last sync | ≥ 0 or `null` |
|
|
| `is_virtual_machine` | bool | Running in VM | `true` or `false` |
|
|
| `vm_type` | string | VM hypervisor type | `vmware`, `hyper-v`, `kvm`, `xen`, `container`, `none` |
|
|
| `vm_clock_sync_enabled` | bool | VM time sync tools enabled | `true` or `false` |
|
|
| `connection_error_type` | string? | Network error type | `ssl_error`, `dns_failure`, `refused`, `timeout`, `connection_failed` |
|
|
|
|
**Likely Cause Differentiation:**
|
|
|
|
| Evidence Pattern | Likely Cause |
|
|
|-----------------|--------------|
|
|
| `ntp_daemon_running=false` | NTP service not started |
|
|
| `ntp_daemon_running=true`, `sync_age_seconds > 3600` | NTP server unreachable |
|
|
| `is_virtual_machine=true`, `vm_clock_sync_enabled=false` | VM clock drift without sync |
|
|
| `skew_seconds > 0` (large positive) | System clock set to future |
|
|
| `skew_seconds < 0` (large negative) | System clock set to past |
|
|
|
|
### check.attestation.transparency.consistency
|
|
|
|
**Description:** Verify stored log checkpoints match remote transparency log
|
|
|
|
**Evidence Fields:**
|
|
|
|
| Field | Type | Description | Expected Range |
|
|
|-------|------|-------------|----------------|
|
|
| `checkpoint_path` | string | Local checkpoint file path | Filesystem path |
|
|
| `stored_tree_size` | int | Local tree size | ≥ 0 |
|
|
| `remote_tree_size` | int | Remote tree size | ≥ stored_tree_size |
|
|
| `stored_root_hash` | string | Local root hash | Hex string |
|
|
| `remote_root_hash` | string | Remote root hash | Hex string |
|
|
| `entries_behind` | int | Entries to catch up | ≥ 0 |
|
|
| `checkpoint_age` | ISO8601 | Checkpoint last update | Valid timestamp |
|
|
| `consistency_verified` | bool | Log is consistent | `true` or `false` |
|
|
|
|
**Likely Cause Differentiation:**
|
|
|
|
| Evidence Pattern | Likely Cause |
|
|
|-----------------|--------------|
|
|
| `remote_tree_size < stored_tree_size` | **CRITICAL:** Possible log rollback/tampering |
|
|
| `stored_root_hash != remote_root_hash` at same size | **CRITICAL:** Possible log modification |
|
|
| `entries_behind > 10000` | Checkpoint very stale, needs sync |
|
|
| Checkpoint file parse error | Corrupted checkpoint file |
|
|
|
|
---
|
|
|
|
## Remediation Step Properties
|
|
|
|
All remediation steps now include safety annotations:
|
|
|
|
| Property | Type | Description |
|
|
|----------|------|-------------|
|
|
| `Order` | int | Step sequence (1-based) |
|
|
| `Description` | string | Human-readable description |
|
|
| `Command` | string | Command to execute |
|
|
| `CommandType` | enum | `Shell`, `Sql`, `Api`, `Manual`, `Comment` |
|
|
| `IsDestructive` | bool | Step modifies/deletes data |
|
|
| `DryRunVariant` | string? | Safe preview command |
|
|
| `Placeholders` | dict? | User-supplied values needed |
|
|
|
|
**AdvisoryAI Integration:**
|
|
- Commands with `IsDestructive=true` must NOT be auto-executed
|
|
- Always prefer `DryRunVariant` before suggesting destructive commands
|
|
- `CommandType.Manual` requires human confirmation
|
|
|
|
---
|
|
|
|
## Adding New Check Schemas
|
|
|
|
When adding a new Doctor check:
|
|
|
|
1. Define evidence fields in the check implementation
|
|
2. Add schema documentation to this file
|
|
3. Include "Likely Cause Differentiation" table
|
|
4. Test evidence output matches schema
|
|
5. Update AdvisoryAI prompt if needed
|
|
|
|
---
|
|
|
|
*Last updated: 2026-01-18 (SPRINT_20260118_015)*
|