sprints completion. new product advisories prepared
This commit is contained in:
master
@@ -945,19 +945,239 @@ Binary extraction and fingerprint generation MUST run with:
|
||||
|
||||
### 7.3 Ops Endpoints
|
||||
|
||||
> **Sprint:** SPRINT_20260112_007_BINIDX_binaryindex_user_config
|
||||
|
||||
BinaryIndex exposes read-only ops endpoints for health, bench, cache, and effective configuration:
|
||||
|
||||
- GET `/api/v1/ops/binaryindex/health` -> BinaryIndexOpsHealthResponse
|
||||
- POST `/api/v1/ops/binaryindex/bench/run` -> BinaryIndexBenchResponse
|
||||
- GET `/api/v1/ops/binaryindex/cache` -> BinaryIndexFunctionCacheStats
|
||||
- GET `/api/v1/ops/binaryindex/config` -> BinaryIndexEffectiveConfig
|
||||
| Endpoint | Method | Response Schema | Description |
|
||||
|----------|--------|-----------------|-------------|
|
||||
| `/api/v1/ops/binaryindex/health` | GET | `BinaryIndexOpsHealthResponse` | Health status, lifter warmness per ISA, cache availability |
|
||||
| `/api/v1/ops/binaryindex/bench/run` | POST | `BinaryIndexBenchResponse` | Run latency benchmark, return min/max/mean/p50/p95/p99 stats |
|
||||
| `/api/v1/ops/binaryindex/cache` | GET | `BinaryIndexFunctionCacheStats` | Function cache hit/miss/eviction statistics |
|
||||
| `/api/v1/ops/binaryindex/config` | GET | `BinaryIndexEffectiveConfig` | Effective configuration with secrets redacted |
|
||||
|
||||
#### 7.3.1 Response Schemas
|
||||
|
||||
**BinaryIndexOpsHealthResponse:**
|
||||
```json
|
||||
{
|
||||
"status": "healthy",
|
||||
"timestamp": "2026-01-16T12:00:00Z",
|
||||
"components": {
|
||||
"lifterPool": { "status": "healthy", "message": null },
|
||||
"functionCache": { "status": "healthy", "message": null },
|
||||
"persistence": { "status": "healthy", "message": null }
|
||||
},
|
||||
"lifterWarmness": {
|
||||
"intel-64": { "isa": "intel-64", "warm": true, "poolSize": 4, "acquireTimeMs": 12 },
|
||||
"armv8-64": { "isa": "armv8-64", "warm": true, "poolSize": 2, "acquireTimeMs": 8 }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**BinaryIndexBenchResponse:**
|
||||
```json
|
||||
{
|
||||
"timestamp": "2026-01-16T12:00:00Z",
|
||||
"sampleSize": 100,
|
||||
"latencySummary": {
|
||||
"minMs": 5.2,
|
||||
"maxMs": 142.8,
|
||||
"meanMs": 28.4,
|
||||
"p50Ms": 22.1,
|
||||
"p95Ms": 78.3,
|
||||
"p99Ms": 121.5
|
||||
},
|
||||
"operations": [
|
||||
{ "operation": "lifterAcquire", "samples": 100, "meanMs": 12.4 },
|
||||
{ "operation": "irNormalization", "samples": 100, "meanMs": 8.7 },
|
||||
{ "operation": "cacheLookup", "samples": 100, "meanMs": 1.2 }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**BinaryIndexFunctionCacheStats:**
|
||||
```json
|
||||
{
|
||||
"enabled": true,
|
||||
"backend": "valkey",
|
||||
"hits": 15234,
|
||||
"misses": 892,
|
||||
"evictions": 45,
|
||||
"hitRate": 0.944,
|
||||
"keyPrefix": "stellaops:binidx:funccache:",
|
||||
"cacheTtlSeconds": 14400,
|
||||
"estimatedEntries": 12500,
|
||||
"estimatedMemoryBytes": 52428800
|
||||
}
|
||||
```
|
||||
|
||||
**BinaryIndexEffectiveConfig:**
|
||||
```json
|
||||
{
|
||||
"b2r2Pool": {
|
||||
"maxPoolSizePerIsa": 4,
|
||||
"warmPreload": ["intel-64", "armv8-64"],
|
||||
"acquireTimeoutMs": 5000,
|
||||
"enableMetrics": true
|
||||
},
|
||||
"semanticLifting": {
|
||||
"b2r2Version": "1.5.0",
|
||||
"normalizationRecipeVersion": "2024.1",
|
||||
"maxInstructionsPerFunction": 10000,
|
||||
"maxFunctionsPerBinary": 5000,
|
||||
"functionLiftTimeoutMs": 30000,
|
||||
"enableDeduplication": true
|
||||
},
|
||||
"functionCache": {
|
||||
"connectionString": "********",
|
||||
"keyPrefix": "stellaops:binidx:funccache:",
|
||||
"cacheTtlSeconds": 14400,
|
||||
"maxTtlSeconds": 86400,
|
||||
"earlyExpiryPercent": 0.1,
|
||||
"maxEntrySizeBytes": 1048576
|
||||
},
|
||||
"persistence": {
|
||||
"schema": "binaries",
|
||||
"minPoolSize": 5,
|
||||
"maxPoolSize": 20,
|
||||
"commandTimeoutSeconds": 30,
|
||||
"retryOnFailure": true,
|
||||
"batchSize": 100
|
||||
},
|
||||
"backendVersions": {
|
||||
"b2r2": "1.5.0",
|
||||
"valkey": "7.2.0",
|
||||
"postgres": "15.4"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### 7.3.2 Rate Limiting
|
||||
|
||||
The `/bench/run` endpoint is rate-limited to prevent load spikes:
|
||||
- Default: 5 requests per minute per tenant
|
||||
- Configurable via `BinaryIndex:Ops:BenchRateLimitPerMinute`
|
||||
|
||||
#### 7.3.3 Secret Redaction
|
||||
|
||||
The config endpoint automatically redacts sensitive keys:
|
||||
|
||||
| Redacted Keys | Pattern |
|
||||
|---------------|---------|
|
||||
| `connectionString` | Replaced with `********` |
|
||||
| `password` | Replaced with `********` |
|
||||
| `secret*` | Any key starting with "secret" |
|
||||
| `apiKey` | Replaced with `********` |
|
||||
| `token` | Replaced with `********` |
|
||||
|
||||
Redaction is applied recursively to nested objects.
|
||||
|
||||
---
|
||||
|
||||
## 8. Configuration
|
||||
|
||||
> **Sprint:** SPRINT_20260112_007_BINIDX_binaryindex_user_config
|
||||
|
||||
### 8.1 Configuration Sections
|
||||
|
||||
All configuration is under the `BinaryIndex` section in `appsettings.yaml` or environment variables with `BINARYINDEX__` prefix.
|
||||
|
||||
#### 8.1.1 B2R2 Lifter Pool (`BinaryIndex:B2R2Pool`)
|
||||
|
||||
| Key | Type | Default | Description |
|
||||
|-----|------|---------|-------------|
|
||||
| `MaxPoolSizePerIsa` | int | 4 | Maximum lifter instances per ISA |
|
||||
| `WarmPreload` | string[] | ["intel-64", "armv8-64"] | ISAs to warm on startup |
|
||||
| `AcquireTimeoutMs` | int | 5000 | Timeout for lifter acquisition |
|
||||
| `EnableMetrics` | bool | true | Emit Prometheus metrics for pool |
|
||||
|
||||
```yaml
|
||||
# binaryindex.yaml
|
||||
BinaryIndex:
|
||||
B2R2Pool:
|
||||
MaxPoolSizePerIsa: 4
|
||||
WarmPreload:
|
||||
- intel-64
|
||||
- armv8-64
|
||||
AcquireTimeoutMs: 5000
|
||||
EnableMetrics: true
|
||||
```
|
||||
|
||||
#### 8.1.2 Semantic Lifting (`BinaryIndex:SemanticLifting`)
|
||||
|
||||
| Key | Type | Default | Description |
|
||||
|-----|------|---------|-------------|
|
||||
| `B2R2Version` | string | "1.5.0" | B2R2 disassembler version |
|
||||
| `NormalizationRecipeVersion` | string | "2024.1" | IR normalization recipe version |
|
||||
| `MaxInstructionsPerFunction` | int | 10000 | Max instructions to lift per function |
|
||||
| `MaxFunctionsPerBinary` | int | 5000 | Max functions to process per binary |
|
||||
| `FunctionLiftTimeoutMs` | int | 30000 | Timeout for lifting single function |
|
||||
| `EnableDeduplication` | bool | true | Deduplicate IR before fingerprinting |
|
||||
|
||||
```yaml
|
||||
BinaryIndex:
|
||||
SemanticLifting:
|
||||
MaxInstructionsPerFunction: 10000
|
||||
MaxFunctionsPerBinary: 5000
|
||||
FunctionLiftTimeoutMs: 30000
|
||||
EnableDeduplication: true
|
||||
```
|
||||
|
||||
#### 8.1.3 Function Cache (`BinaryIndex:FunctionCache`)
|
||||
|
||||
| Key | Type | Default | Description |
|
||||
|-----|------|---------|-------------|
|
||||
| `ConnectionString` | string | — | Valkey connection string (secret) |
|
||||
| `KeyPrefix` | string | "stellaops:binidx:funccache:" | Cache key prefix |
|
||||
| `CacheTtlSeconds` | int | 14400 | Default cache TTL (4 hours) |
|
||||
| `MaxTtlSeconds` | int | 86400 | Maximum TTL (24 hours) |
|
||||
| `EarlyExpiryPercent` | decimal | 0.1 | Early expiry jitter (10%) |
|
||||
| `MaxEntrySizeBytes` | int | 1048576 | Max entry size (1 MB) |
|
||||
|
||||
```yaml
|
||||
BinaryIndex:
|
||||
FunctionCache:
|
||||
ConnectionString: ${VALKEY_CONNECTION} # from env
|
||||
KeyPrefix: "stellaops:binidx:funccache:"
|
||||
CacheTtlSeconds: 14400
|
||||
MaxEntrySizeBytes: 1048576
|
||||
```
|
||||
|
||||
#### 8.1.4 Persistence (`Postgres:BinaryIndex`)
|
||||
|
||||
| Key | Type | Default | Description |
|
||||
|-----|------|---------|-------------|
|
||||
| `Schema` | string | "binaries" | PostgreSQL schema name |
|
||||
| `MinPoolSize` | int | 5 | Minimum connection pool size |
|
||||
| `MaxPoolSize` | int | 20 | Maximum connection pool size |
|
||||
| `CommandTimeoutSeconds` | int | 30 | Command execution timeout |
|
||||
| `RetryOnFailure` | bool | true | Retry transient failures |
|
||||
| `BatchSize` | int | 100 | Batch insert size |
|
||||
|
||||
```yaml
|
||||
Postgres:
|
||||
BinaryIndex:
|
||||
Schema: binaries
|
||||
MinPoolSize: 5
|
||||
MaxPoolSize: 20
|
||||
CommandTimeoutSeconds: 30
|
||||
RetryOnFailure: true
|
||||
BatchSize: 100
|
||||
```
|
||||
|
||||
#### 8.1.5 Ops Configuration (`BinaryIndex:Ops`)
|
||||
|
||||
| Key | Type | Default | Description |
|
||||
|-----|------|---------|-------------|
|
||||
| `EnableHealthEndpoint` | bool | true | Enable /health endpoint |
|
||||
| `EnableBenchEndpoint` | bool | true | Enable /bench/run endpoint |
|
||||
| `BenchRateLimitPerMinute` | int | 5 | Rate limit for bench endpoint |
|
||||
| `RedactedKeys` | string[] | See 7.3.3 | Keys to redact in config output |
|
||||
|
||||
### 8.2 Legacy Configuration
|
||||
|
||||
```yaml
|
||||
# binaryindex.yaml (corpus configuration)
|
||||
binaryindex:
|
||||
enabled: true
|
||||
|
||||
@@ -995,12 +1215,6 @@ binaryindex:
|
||||
rustfs_bucket: stellaops/binaryindex
|
||||
```
|
||||
|
||||
Additional appsettings sections (case-insensitive):
|
||||
- `BinaryIndex:B2R2Pool` - lifter pool sizing and warm ISA list.
|
||||
- `BinaryIndex:SemanticLifting` - LowUIR enablement and deterministic controls.
|
||||
- `BinaryIndex:FunctionCache` - Valkey function cache configuration.
|
||||
- `Postgres:BinaryIndex` - persistence for canonical IR fingerprints.
|
||||
|
||||
---
|
||||
|
||||
## 9. Testing Strategy
|
||||
|
||||
Reference in New Issue
Block a user