synergy moats product advisory implementations
This commit is contained in:
189
docs/operations/runbooks/policy-compilation-failed.md
Normal file
189
docs/operations/runbooks/policy-compilation-failed.md
Normal file
@@ -0,0 +1,189 @@
|
||||
# Runbook: Policy Engine - Rego Compilation Errors
|
||||
|
||||
> **Sprint:** SPRINT_20260117_029_DOCS_runbook_coverage
|
||||
> **Task:** RUN-003 - Policy Engine Runbooks
|
||||
|
||||
## Metadata
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **Component** | Policy Engine |
|
||||
| **Severity** | High |
|
||||
| **On-call scope** | Platform team |
|
||||
| **Last updated** | 2026-01-17 |
|
||||
| **Doctor check** | `check.policy.compilation-health` |
|
||||
|
||||
---
|
||||
|
||||
## Symptoms
|
||||
|
||||
- [ ] Policy deployment failing with "compilation error"
|
||||
- [ ] Alert `PolicyCompilationFailed` firing
|
||||
- [ ] Error: "rego_parse_error" or "rego_type_error"
|
||||
- [ ] New policies not taking effect
|
||||
- [ ] OPA rejecting policy bundle
|
||||
|
||||
---
|
||||
|
||||
## Impact
|
||||
|
||||
| Impact Type | Description |
|
||||
|-------------|-------------|
|
||||
| **User-facing** | New policies cannot be deployed; using stale policies |
|
||||
| **Data integrity** | Existing policies continue to work; new rules not enforced |
|
||||
| **SLA impact** | Policy updates blocked; security posture may be outdated |
|
||||
|
||||
---
|
||||
|
||||
## Diagnosis
|
||||
|
||||
### Quick checks
|
||||
|
||||
1. **Check Doctor diagnostics:**
|
||||
```bash
|
||||
stella doctor --check check.policy.compilation-health
|
||||
```
|
||||
|
||||
2. **Check policy compilation status:**
|
||||
```bash
|
||||
stella policy status --compilation
|
||||
```
|
||||
|
||||
3. **Validate specific policy:**
|
||||
```bash
|
||||
stella policy validate --file <policy-file>
|
||||
```
|
||||
|
||||
### Deep diagnosis
|
||||
|
||||
1. **Get detailed compilation errors:**
|
||||
```bash
|
||||
stella policy compile --verbose
|
||||
```
|
||||
Look for: Line numbers, error types, undefined references
|
||||
|
||||
2. **Check for syntax errors:**
|
||||
```bash
|
||||
stella policy lint --file <policy-file>
|
||||
```
|
||||
|
||||
3. **Check for type errors:**
|
||||
```bash
|
||||
stella policy typecheck --file <policy-file>
|
||||
```
|
||||
|
||||
4. **Check OPA version compatibility:**
|
||||
```bash
|
||||
stella policy opa version
|
||||
stella policy check-compat --file <policy-file>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Resolution
|
||||
|
||||
### Immediate mitigation
|
||||
|
||||
1. **Rollback to last working policy:**
|
||||
```bash
|
||||
stella policy rollback --to-last-good
|
||||
```
|
||||
|
||||
2. **Disable the failing policy:**
|
||||
```bash
|
||||
stella policy disable <policy-id>
|
||||
stella policy reload
|
||||
```
|
||||
|
||||
3. **Use previous bundle:**
|
||||
```bash
|
||||
stella policy bundle load --version <previous-version>
|
||||
```
|
||||
|
||||
### Root cause fix
|
||||
|
||||
**If syntax error:**
|
||||
|
||||
1. Get exact error location:
|
||||
```bash
|
||||
stella policy validate --file <policy-file> --show-line
|
||||
```
|
||||
|
||||
2. Common syntax issues:
|
||||
- Missing brackets or braces
|
||||
- Invalid rule head syntax
|
||||
- Incorrect import statements
|
||||
|
||||
3. Fix and re-validate:
|
||||
```bash
|
||||
stella policy validate --file <fixed-policy.rego>
|
||||
```
|
||||
|
||||
**If undefined reference:**
|
||||
|
||||
1. Check for missing imports:
|
||||
```bash
|
||||
stella policy analyze --file <policy-file> --show-imports
|
||||
```
|
||||
|
||||
2. Verify data references exist:
|
||||
```bash
|
||||
stella policy data show
|
||||
```
|
||||
|
||||
3. Add missing imports or data definitions
|
||||
|
||||
**If type error:**
|
||||
|
||||
1. Check type mismatches:
|
||||
```bash
|
||||
stella policy typecheck --file <policy-file> --verbose
|
||||
```
|
||||
|
||||
2. Common type issues:
|
||||
- Comparing incompatible types
|
||||
- Invalid function arguments
|
||||
- Missing type annotations
|
||||
|
||||
**If OPA version incompatibility:**
|
||||
|
||||
1. Check Rego version features used:
|
||||
```bash
|
||||
stella policy analyze --file <policy-file> --show-features
|
||||
```
|
||||
|
||||
2. Update policy to use compatible features or upgrade OPA
|
||||
|
||||
### Verification
|
||||
|
||||
```bash
|
||||
# Validate fixed policy
|
||||
stella policy validate --file <fixed-policy.rego>
|
||||
|
||||
# Test policy compilation
|
||||
stella policy compile --file <fixed-policy.rego>
|
||||
|
||||
# Deploy policy
|
||||
stella policy deploy --file <fixed-policy.rego>
|
||||
|
||||
# Test policy evaluation
|
||||
stella policy evaluate --test
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Prevention
|
||||
|
||||
- [ ] **CI/CD:** Add policy validation to CI pipeline before deployment
|
||||
- [ ] **Linting:** Run `stella policy lint` on all policy changes
|
||||
- [ ] **Testing:** Write unit tests for policies with `stella policy test`
|
||||
- [ ] **Staging:** Deploy to staging environment before production
|
||||
|
||||
---
|
||||
|
||||
## Related Resources
|
||||
|
||||
- **Architecture:** `docs/modules/policy/architecture.md`
|
||||
- **Related runbooks:** `policy-opa-crash.md`, `policy-evaluation-slow.md`
|
||||
- **Rego reference:** https://www.openpolicyagent.org/docs/latest/policy-language/
|
||||
- **Policy testing:** `docs/modules/policy/testing.md`
|
||||
Reference in New Issue
Block a user