# Competitor Ingest Error Taxonomy (CM10)

Status: Draft · Date: 2025-12-04
Scope: Standardize retry/backoff/error taxonomy for the competitor ingest pipeline with deterministic diagnostics.

## Objectives

- Define comprehensive error taxonomy for ingest failures.
- Specify retry behavior for transient errors.
- Enable deterministic error diagnostics.
- Support offline error handling.

## Error Categories

### Retryable Errors

| Code | Category | Description | Max Retries | Backoff |
|------|----------|-------------|-------------|---------|
| `E1001` | `network_error` | Network connectivity failure | 3 | Exponential |
| `E1002` | `network_timeout` | Connection/read timeout | 3 | Exponential |
| `E1003` | `rate_limit` | Rate limit exceeded (429) | 5 | Linear |
| `E1004` | `service_unavailable` | Service temporarily unavailable (503) | 3 | Exponential |
| `E1005` | `transient_io` | Temporary I/O error | 2 | Fixed |
| `E1006` | `lock_contention` | Resource lock conflict | 3 | Exponential |

### Non-Retryable Errors

| Code | Category | Description | Action |
|------|----------|-------------|--------|
| `E2001` | `signature_invalid` | Signature verification failed | Reject |
| `E2002` | `signature_expired` | Signature validity exceeded | Reject |
| `E2003` | `key_unknown` | Signing key not found | Reject |
| `E2004` | `key_expired` | Signing key validity exceeded | Reject |
| `E2005` | `key_revoked` | Signing key revoked | Reject |
| `E2006` | `alg_unsupported` | Unsupported algorithm | Reject |
| `E2007` | `hash_mismatch` | Content hash mismatch | Reject |
| `E2008` | `schema_invalid` | Input doesn't match schema | Reject |
| `E2009` | `version_unsupported` | Tool version not supported | Reject |
| `E2010` | `no_evidence` | No acceptable evidence found | Reject |

### Warning Conditions

| Code | Category | Description | Action |
|------|----------|-------------|--------|
| `W3001` | `provenance_unknown` | Provenance not verifiable | Accept with warning |
| `W3002` | `degraded_confidence` | Low confidence result | Accept with warning |
| `W3003` | `stale_data` | Data exceeds freshness threshold | Accept with warning |
| `W3004` | `partial_mapping` | Some fields couldn't be mapped | Accept with warning |
| `W3005` | `deprecated_format` | Using deprecated format | Accept with warning |

## Error Format

### Standard Error Response

```json
{
  "error": {
    "code": "E2001",
    "category": "signature_invalid",
    "message": "DSSE signature verification failed",
    "details": {
      "reason": "Key ID not found in trusted keyring",
      "keyId": "unknown-key-12345",
      "algorithm": "ecdsa-p256"
    },
    "retryable": false,
    "diagnostics": {
      "timestamp": "2025-12-04T12:00:00Z",
      "traceId": "abc123...",
      "inputHash": "b3:...",
      "stage": "signature_verification"
    }
  }
}
```

### Warning Response

```json
{
  "result": {
    "status": "accepted_with_warnings",
    "warnings": [
      {
        "code": "W3001",
        "category": "provenance_unknown",
        "message": "SBOM accepted without verified provenance",
        "details": {
          "reason": "No signature present",
          "fallbackLevel": 2
        }
      }
    ],
    "output": {...}
  }
}
```

## Retry Configuration

### Backoff Strategies

```json
{
  "backoff": {
    "exponential": {
      "description": "2^attempt * base, capped at max",
      "config": {
        "base": 1000,
        "factor": 2,
        "max": 60000,
        "jitter": 0.1
      },
      "example": [1000, 2000, 4000, 8000, 16000, 32000, 60000]
    },
    "linear": {
      "description": "initial + (attempt * increment), capped at max",
      "config": {
        "initial": 1000,
        "increment": 1000,
        "max": 30000
      },
      "example": [1000, 2000, 3000, 4000, 5000]
    },
    "fixed": {
      "description": "Constant delay between retries",
      "config": {
        "delay": 5000
      },
      "example": [5000, 5000, 5000]
    }
  }
}
```

### Retry Decision Logic

```python
def should_retry(error: IngestError, attempt: int) -> RetryDecision:
    if error.code.startswith('E2'):
        return RetryDecision(retry=False, reason="Non-retryable error")

    config = RETRY_CONFIG.get(error.category)
    if not config:
        return RetryDecision(retry=False, reason="Unknown error category")

    if attempt >= config.max_retries:
        return RetryDecision(retry=False, reason="Max retries exceeded")

    delay = calculate_backoff(config, attempt)
    return RetryDecision(retry=True, delay=delay)
```

## Diagnostic Output

### Deterministic Diagnostics

All error diagnostics must be deterministic and reproducible:

```json
{
  "diagnostics": {
    "timestamp": "2025-12-04T12:00:00Z",
    "traceId": "deterministic-trace-id",
    "inputHash": "b3:...",
    "stage": "normalization",
    "context": {
      "tool": "syft",
      "toolVersion": "1.0.0",
      "adapterVersion": "1.0.0",
      "inputSize": 12345,
      "componentCount": 42
    },
    "errorChain": [
      {
        "stage": "schema_validation",
        "error": "Missing required field: artifacts[0].purl",
        "path": "/artifacts/0"
      }
    ]
  }
}
```

### Offline Diagnostics

For offline mode, diagnostics include:
- No timestamps that depend on wall clock
- Deterministic trace IDs (based on input hash)
- All context from bundled metadata

```json
{
  "diagnostics": {
    "mode": "offline",
    "kitVersion": "1.0.0",
    "traceId": "b3:input-hash-derived-trace-id",
    "kitHash": "b3:...",
    "trustRootHash": "b3:..."
  }
}
```

## Error Handling Workflow

```
┌─────────────┐
│   Ingest    │
│   Request   │
└─────────────┘
       │
       ▼
┌─────────────┐
│  Validate   │──Error──► E2008 schema_invalid
└─────────────┘
       │
       ▼
┌─────────────┐
│   Verify    │──Error──► E2001-E2007 signature errors
│  Signature  │
└─────────────┘
       │
       ▼
┌─────────────┐
│  Normalize  │──Error──► E2008-E2009 format errors
└─────────────┘
       │
       ▼
┌─────────────┐
│   Store     │──Error──► E1001-E1006 retryable errors
└─────────────┘
       │
       ▼
┌─────────────┐
│   Success   │
│  or Warn    │
└─────────────┘
```

## API Error Responses

### HTTP Status Mapping

| Error Category | HTTP Status | Response Body |
|----------------|-------------|---------------|
| Retryable (E1xxx) | 503 | Error with Retry-After header |
| Rate limit (E1003) | 429 | Error with Retry-After header |
| Signature (E2001-E2007) | 400 | Error with details |
| Schema (E2008) | 400 | Error with validation details |
| Version (E2009) | 400 | Error with supported versions |
| No evidence (E2010) | 400 | Error with fallback options |

### Example Error Response

```http
HTTP/1.1 400 Bad Request
Content-Type: application/json
X-Stellaops-Error-Code: E2001
X-Stellaops-Trace-Id: abc123...

{
  "error": {
    "code": "E2001",
    "category": "signature_invalid",
    "message": "DSSE signature verification failed",
    "details": {...},
    "retryable": false,
    "diagnostics": {...}
  }
}
```

### Retry Response

```http
HTTP/1.1 503 Service Unavailable
Content-Type: application/json
Retry-After: 5
X-Stellaops-Error-Code: E1004
X-Stellaops-Retry-Attempt: 1

{
  "error": {
    "code": "E1004",
    "category": "service_unavailable",
    "message": "Upstream service temporarily unavailable",
    "retryable": true,
    "retryAfter": 5000,
    "attempt": 1,
    "maxAttempts": 3
  }
}
```

## Logging

### Error Log Format

```json
{
  "level": "error",
  "timestamp": "2025-12-04T12:00:00Z",
  "logger": "stellaops.scanner.ingest",
  "message": "Ingest failed: signature_invalid",
  "error": {
    "code": "E2001",
    "category": "signature_invalid"
  },
  "context": {
    "traceId": "abc123...",
    "tool": "syft",
    "inputHash": "b3:..."
  }
}
```

## Links

- Sprint: `docs/implplan/SPRINT_0186_0001_0001_record_deterministic_execution.md` (CM10)
- Normalization: `docs/modules/scanner/design/competitor-ingest-normalization.md` (CM1)
- Verification: `docs/modules/scanner/design/competitor-signature-verification.md` (CM2)
- Offline Kit: `docs/modules/scanner/design/competitor-offline-ingest-kit.md` (CM5)