up
This commit is contained in:
StellaOps Bot
305
docs/db/tasks/PHASE_7_CLEANUP.md
Normal file
305
docs/db/tasks/PHASE_7_CLEANUP.md
Normal file
@@ -0,0 +1,305 @@
|
||||
# Phase 7: Cleanup & Optimization
|
||||
|
||||
**Sprint:** 11
|
||||
**Duration:** 1 sprint
|
||||
**Status:** TODO
|
||||
**Dependencies:** All previous phases completed
|
||||
|
||||
---
|
||||
|
||||
## Objectives
|
||||
|
||||
1. Remove MongoDB dependencies from converted modules
|
||||
2. Archive MongoDB data
|
||||
3. Optimize PostgreSQL performance
|
||||
4. Update documentation
|
||||
5. Update air-gap kit
|
||||
|
||||
---
|
||||
|
||||
## Deliverables
|
||||
|
||||
| Deliverable | Acceptance Criteria |
|
||||
|-------------|---------------------|
|
||||
| Code cleanup | MongoDB code removed from converted modules |
|
||||
| Data archive | MongoDB data archived and documented |
|
||||
| Performance tuning | Query times within acceptable range |
|
||||
| Documentation | All docs updated for PostgreSQL |
|
||||
| Air-gap kit | PostgreSQL support added |
|
||||
|
||||
---
|
||||
|
||||
## Task Breakdown
|
||||
|
||||
### T7.1: Remove MongoDB Dependencies
|
||||
|
||||
**Status:** TODO
|
||||
**Estimate:** 2 days
|
||||
|
||||
**Description:**
|
||||
Remove MongoDB storage projects and references from converted modules.
|
||||
|
||||
**Subtasks:**
|
||||
- [ ] T7.1.1: Remove `StellaOps.Authority.Storage.Mongo` project
|
||||
- [ ] T7.1.2: Remove `StellaOps.Scheduler.Storage.Mongo` project
|
||||
- [ ] T7.1.3: Remove `StellaOps.Notify.Storage.Mongo` project
|
||||
- [ ] T7.1.4: Remove `StellaOps.Policy.Storage.Mongo` project
|
||||
- [ ] T7.1.5: Remove `StellaOps.Concelier.Storage.Mongo` project
|
||||
- [ ] T7.1.6: Remove `StellaOps.Excititor.Storage.Mongo` project
|
||||
- [ ] T7.1.7: Update solution files
|
||||
- [ ] T7.1.8: Remove dual-write wrappers
|
||||
- [ ] T7.1.9: Remove MongoDB configuration options
|
||||
- [ ] T7.1.10: Run full build to verify no broken references
|
||||
|
||||
**Verification:**
|
||||
- [ ] Solution builds without MongoDB packages
|
||||
- [ ] No MongoDB references in converted modules
|
||||
- [ ] All tests pass
|
||||
|
||||
---
|
||||
|
||||
### T7.2: Archive MongoDB Data
|
||||
|
||||
**Status:** TODO
|
||||
**Estimate:** 1 day
|
||||
|
||||
**Description:**
|
||||
Archive MongoDB databases for historical reference.
|
||||
|
||||
**Subtasks:**
|
||||
- [ ] T7.2.1: Take final MongoDB backup
|
||||
- [ ] T7.2.2: Export to BSON/JSON archives
|
||||
- [ ] T7.2.3: Store archives in secure location
|
||||
- [ ] T7.2.4: Document archive contents and structure
|
||||
- [ ] T7.2.5: Set retention policy for archives
|
||||
- [ ] T7.2.6: Schedule MongoDB cluster decommission
|
||||
|
||||
**Archive Structure:**
|
||||
```
|
||||
archives/
|
||||
├── mongodb-authority-2025-XX-XX.bson.gz
|
||||
├── mongodb-scheduler-2025-XX-XX.bson.gz
|
||||
├── mongodb-notify-2025-XX-XX.bson.gz
|
||||
├── mongodb-policy-2025-XX-XX.bson.gz
|
||||
├── mongodb-concelier-2025-XX-XX.bson.gz
|
||||
├── mongodb-excititor-2025-XX-XX.bson.gz
|
||||
└── ARCHIVE_MANIFEST.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### T7.3: PostgreSQL Performance Optimization
|
||||
|
||||
**Status:** TODO
|
||||
**Estimate:** 2 days
|
||||
|
||||
**Description:**
|
||||
Analyze and optimize PostgreSQL performance.
|
||||
|
||||
**Subtasks:**
|
||||
- [ ] T7.3.1: Enable `pg_stat_statements` extension
|
||||
- [ ] T7.3.2: Identify slow queries
|
||||
- [ ] T7.3.3: Analyze query plans with EXPLAIN ANALYZE
|
||||
- [ ] T7.3.4: Add missing indexes
|
||||
- [ ] T7.3.5: Remove unused indexes
|
||||
- [ ] T7.3.6: Tune PostgreSQL configuration
|
||||
- [ ] T7.3.7: Set up query monitoring dashboard
|
||||
- [ ] T7.3.8: Document performance baselines
|
||||
|
||||
**Configuration Tuning:**
|
||||
```ini
|
||||
# postgresql.conf optimizations
|
||||
shared_buffers = 25% of RAM
|
||||
effective_cache_size = 75% of RAM
|
||||
work_mem = 64MB
|
||||
maintenance_work_mem = 512MB
|
||||
random_page_cost = 1.1 # for SSD
|
||||
effective_io_concurrency = 200 # for SSD
|
||||
max_parallel_workers_per_gather = 4
|
||||
```
|
||||
|
||||
**Monitoring Queries:**
|
||||
```sql
|
||||
-- Top slow queries
|
||||
SELECT query, calls, mean_time, total_time
|
||||
FROM pg_stat_statements
|
||||
ORDER BY mean_time DESC
|
||||
LIMIT 20;
|
||||
|
||||
-- Unused indexes
|
||||
SELECT schemaname, tablename, indexname
|
||||
FROM pg_stat_user_indexes
|
||||
WHERE idx_scan = 0;
|
||||
|
||||
-- Table bloat
|
||||
SELECT schemaname, tablename,
|
||||
pg_size_pretty(pg_total_relation_size(schemaname || '.' || tablename)) as size
|
||||
FROM pg_stat_user_tables
|
||||
ORDER BY pg_total_relation_size(schemaname || '.' || tablename) DESC;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### T7.4: Update Documentation
|
||||
|
||||
**Status:** TODO
|
||||
**Estimate:** 1.5 days
|
||||
|
||||
**Description:**
|
||||
Update all documentation to reflect PostgreSQL as the primary database.
|
||||
|
||||
**Subtasks:**
|
||||
- [ ] T7.4.1: Update `docs/07_HIGH_LEVEL_ARCHITECTURE.md`
|
||||
- [ ] T7.4.2: Update module architecture docs
|
||||
- [ ] T7.4.3: Update deployment guides
|
||||
- [ ] T7.4.4: Update operations runbooks
|
||||
- [ ] T7.4.5: Update troubleshooting guides
|
||||
- [ ] T7.4.6: Update `CLAUDE.md` technology stack
|
||||
- [ ] T7.4.7: Create PostgreSQL operations guide
|
||||
- [ ] T7.4.8: Document backup/restore procedures
|
||||
- [ ] T7.4.9: Document scaling recommendations
|
||||
|
||||
**New Documents:**
|
||||
- `docs/operations/postgresql-guide.md`
|
||||
- `docs/operations/postgresql-backup-restore.md`
|
||||
- `docs/operations/postgresql-troubleshooting.md`
|
||||
|
||||
---
|
||||
|
||||
### T7.5: Update Air-Gap Kit
|
||||
|
||||
**Status:** TODO
|
||||
**Estimate:** 1 day
|
||||
|
||||
**Description:**
|
||||
Update offline/air-gap kit to include PostgreSQL.
|
||||
|
||||
**Subtasks:**
|
||||
- [ ] T7.5.1: Add PostgreSQL container image to kit
|
||||
- [ ] T7.5.2: Update kit scripts for PostgreSQL setup
|
||||
- [ ] T7.5.3: Include schema migrations in kit
|
||||
- [ ] T7.5.4: Update kit documentation
|
||||
- [ ] T7.5.5: Test kit installation in air-gapped environment
|
||||
- [ ] T7.5.6: Update `docs/24_OFFLINE_KIT.md`
|
||||
|
||||
**Air-Gap Kit Structure:**
|
||||
```
|
||||
offline-kit/
|
||||
├── images/
|
||||
│ ├── postgres-16-alpine.tar
|
||||
│ └── stellaops-*.tar
|
||||
├── schemas/
|
||||
│ ├── authority.sql
|
||||
│ ├── vuln.sql
|
||||
│ ├── vex.sql
|
||||
│ ├── scheduler.sql
|
||||
│ ├── notify.sql
|
||||
│ └── policy.sql
|
||||
├── scripts/
|
||||
│ ├── setup-postgres.sh
|
||||
│ ├── run-migrations.sh
|
||||
│ └── import-data.sh
|
||||
└── docs/
|
||||
└── OFFLINE_SETUP.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### T7.6: Final Verification
|
||||
|
||||
**Status:** TODO
|
||||
**Estimate:** 1 day
|
||||
|
||||
**Description:**
|
||||
Run final verification of all systems.
|
||||
|
||||
**Subtasks:**
|
||||
- [ ] T7.6.1: Run full integration test suite
|
||||
- [ ] T7.6.2: Run performance benchmark suite
|
||||
- [ ] T7.6.3: Verify all modules on PostgreSQL
|
||||
- [ ] T7.6.4: Verify determinism tests pass
|
||||
- [ ] T7.6.5: Verify air-gap kit works
|
||||
- [ ] T7.6.6: Generate final verification report
|
||||
- [ ] T7.6.7: Get sign-off from stakeholders
|
||||
|
||||
---
|
||||
|
||||
### T7.7: Decommission MongoDB
|
||||
|
||||
**Status:** TODO
|
||||
**Estimate:** 0.5 days
|
||||
|
||||
**Description:**
|
||||
Final decommission of MongoDB infrastructure.
|
||||
|
||||
**Subtasks:**
|
||||
- [ ] T7.7.1: Verify no services using MongoDB
|
||||
- [ ] T7.7.2: Stop MongoDB instances
|
||||
- [ ] T7.7.3: Archive final state
|
||||
- [ ] T7.7.4: Remove MongoDB from infrastructure
|
||||
- [ ] T7.7.5: Update monitoring/alerting
|
||||
- [ ] T7.7.6: Update cost projections
|
||||
|
||||
---
|
||||
|
||||
## Exit Criteria
|
||||
|
||||
- [ ] All MongoDB code removed from converted modules
|
||||
- [ ] MongoDB data archived
|
||||
- [ ] PostgreSQL performance optimized
|
||||
- [ ] All documentation updated
|
||||
- [ ] Air-gap kit updated and tested
|
||||
- [ ] Final verification report approved
|
||||
- [ ] MongoDB infrastructure decommissioned
|
||||
|
||||
---
|
||||
|
||||
## Post-Conversion Monitoring
|
||||
|
||||
### First Week
|
||||
|
||||
- Monitor error rates closely
|
||||
- Track query performance
|
||||
- Watch for any data inconsistencies
|
||||
- Have rollback plan ready (restore MongoDB)
|
||||
|
||||
### First Month
|
||||
|
||||
- Review query statistics weekly
|
||||
- Optimize any slow queries found
|
||||
- Monitor storage growth
|
||||
- Adjust vacuum settings if needed
|
||||
|
||||
### Ongoing
|
||||
|
||||
- Regular performance reviews
|
||||
- Index maintenance
|
||||
- Backup verification
|
||||
- Capacity planning
|
||||
|
||||
---
|
||||
|
||||
## Rollback Considerations
|
||||
|
||||
**Note:** After Phase 7 completion, rollback to MongoDB becomes significantly more complex. Ensure all stakeholders understand:
|
||||
|
||||
1. MongoDB archives are read-only backup
|
||||
2. Any new data created after cutover is PostgreSQL-only
|
||||
3. Full rollback would require data export/import
|
||||
|
||||
---
|
||||
|
||||
## Success Metrics
|
||||
|
||||
| Metric | Target | Measurement |
|
||||
|--------|--------|-------------|
|
||||
| Query latency (p95) | < 100ms | pg_stat_statements |
|
||||
| Error rate | < 0.01% | Application logs |
|
||||
| Storage efficiency | < 120% of MongoDB | Disk usage |
|
||||
| Test coverage | 100% | CI reports |
|
||||
| Documentation coverage | 100% | Manual review |
|
||||
|
||||
---
|
||||
|
||||
*Phase Version: 1.0.0*
|
||||
*Last Updated: 2025-11-28*
|
||||
Reference in New Issue
Block a user