# Concelier Backfill & Rollback Plan (STORE-AOC-19-005-DEV)

## Objective
Prepare and rehearse the raw-linkset backfill/rollback so Concelier Mongo reflects Link-Not-Merge data deterministically across dev/stage. This runbook unblocks STORE-AOC-19-005-DEV.

## Inputs
- Source dataset: staging export tarball `linksets-stage-backfill.tar.zst`.
- Expected placement: `out/linksets/linksets-stage-backfill.tar.zst`.
- Hash: record SHA-256 in this file once available (example below).

Example hash capture (replace with real):
```
$ sha256sum out/linksets/linksets-stage-backfill.tar.zst
3ac7d1c8f4f7b5c5b27c1c7ac6d6e9b2a2d6d7a1a1c3f4e5b6c7d8e9f0a1b2c3  out/linksets/linksets-stage-backfill.tar.zst
```

## Preflight
- Environment variables:
  - `CONCELIER_MONGO_URI` pointing to the target (dev or staging) Mongo.
  - `CONCELIER_DB` (default `concelier`).
- Take a snapshot of affected collections:
  ```
  mongodump --uri "$CONCELIER_MONGO_URI" --db "$CONCELIER_DB" --collection linksets --collection advisory_chunks --out out/backups/pre-run
  ```
- Ensure write lock is acceptable for the maintenance window.

## Backfill steps
1) Extract dataset:
   ```
   mkdir -p out/linksets/extracted
   tar -xf out/linksets/linksets-stage-backfill.tar.zst -C out/linksets/extracted
   ```
2) Import linksets + chunks (bypass validation to preserve upstream IDs):
   ```
   mongoimport --uri "$CONCELIER_MONGO_URI" --db "$CONCELIER_DB" \
     --collection linksets --file out/linksets/extracted/linksets.ndjson --mode=upsert --upsertFields=_id

   mongoimport --uri "$CONCELIER_MONGO_URI" --db "$CONCELIER_DB" \
     --collection advisory_chunks --file out/linksets/extracted/advisory_chunks.ndjson --mode=upsert --upsertFields=_id
   ```
3) Verify counts vs manifest:
   ```
   jq '.' out/linksets/extracted/manifest.json
   mongo --quiet "$CONCELIER_MONGO_URI/$CONCELIER_DB" --eval "db.linksets.countDocuments()"
   mongo --quiet "$CONCELIER_MONGO_URI/$CONCELIER_DB" --eval "db.advisory_chunks.countDocuments()"
   ```
4) Dry-run rollback marker (no-op unless `ENABLE_ROLLBACK=1` set):
   ```
   ENABLE_ROLLBACK=0 python scripts/concelier/backfill/rollback.py --manifest out/linksets/extracted/manifest.json
   ```

## Rollback procedure
- If validation fails, restore from preflight dump:
  ```
  mongorestore --uri "$CONCELIER_MONGO_URI" --drop out/backups/pre-run
  ```
- If partial write detected, rerun mongoimport for the affected collection only with `--mode=upsert`.

## Validation checklist
- Hash of tarball matches recorded SHA-256.
- Post-import counts align with `manifest.json`.
- Linkset cursor pagination smoke test:
  ```
  dotnet test src/Concelier/StellaOps.Concelier.WebService.Tests --filter LinksetsEndpoint_SupportsCursorPagination
  ```
- Storage metrics (if enabled) show non-zero `concelier_storage_import_total` for this window.

## Artefacts to record
- Tarball SHA-256 and size.
- `manifest.json` copy stored alongside tarball.
- Import log (`out/linksets/import.log`) and validation results.
- Decision: maintenance window and rollback outcome.

## Owners
- Concelier Storage Guild (Mongo)
- AirGap/Backfill reviewers for sign-off