|
1 | | -PR: Centralized Configuration Management Service (microservices/config-service) |
| 1 | +PR: feat(config-service): add centralized configuration management service |
2 | 2 |
|
3 | 3 | Summary |
4 | 4 | ------- |
5 | | -Adds a new standalone NestJS microservice, `config-service`, providing centralized configuration, environment management, encrypted secrets, webhook-based real-time updates, caching, versioning, and audit logging for the monorepo. |
| 5 | +This PR adds a new, standalone NestJS microservice at `microservices/config-service` that centralizes runtime configuration, environment-scoped settings, feature flags, and encrypted secret management for the platform. |
| 6 | + |
| 7 | +Primary capabilities |
| 8 | +-------------------- |
| 9 | +- Typed key/value configurations with environment scoping and versioning |
| 10 | +- Encrypted secret storage and rotation (AES-256-CBC) |
| 11 | +- In-memory config caching with TTL and invalidation on update |
| 12 | +- Real-time update propagation via webhook subscriptions (HMAC-SHA256 signed) |
| 13 | +- Comprehensive audit logging for create/update/delete/rotate events |
| 14 | +- Docker + `docker-compose` dev setup and TypeORM integration |
6 | 15 |
|
7 | 16 | Why |
8 | 17 | --- |
9 | | -Centralize management of environment variables, feature flags, and secrets to simplify configuration drift, enable runtime updates, centralize audit trails, and standardize secret rotation across services. |
| 18 | +Centralized configuration simplifies operations, reduces environment drift, enables runtime toggles and feature flags, standardizes secret encryption and rotation, and gives a single audit trail for config changes. |
10 | 19 |
|
11 | | -Scope / Files Changed |
| 20 | +Files added / changed |
12 | 21 | --------------------- |
13 | | -New service added at: `microservices/config-service` |
14 | | -Key files and folders (high-level): |
15 | | -- `microservices/config-service/package.json` |
16 | | -- `microservices/config-service/Dockerfile` |
17 | | -- `microservices/config-service/docker-compose.yml` |
| 22 | +All changes live under `microservices/config-service` (new folder). Highlights: |
| 23 | + |
| 24 | +- `microservices/config-service/package.json` (scripts, deps) |
| 25 | +- `microservices/config-service/Dockerfile` and `.dockerignore` |
| 26 | +- `microservices/config-service/docker-compose.yml` (service + Postgres) |
18 | 27 | - `microservices/config-service/.env.example` |
19 | | -- `microservices/config-service/src/app.module.ts` |
20 | | -- `microservices/config-service/src/main.ts` |
| 28 | +- `microservices/config-service/src/config/orm-config.ts` |
21 | 29 | - `microservices/config-service/src/entities/*` (Config, Environment, Secret, AuditLog, WebhookSubscription) |
22 | | -- `microservices/config-service/src/modules/*` (configuration, secret, environment, audit, webhook modules) |
| 30 | +- `microservices/config-service/src/modules/*` (configuration, secret, environment, audit, webhook) |
23 | 31 | - `microservices/config-service/src/common/*` (encryption, validation, DTOs) |
24 | | -- `microservices/config-service/README.md` and related docs |
| 32 | +- `microservices/config-service/src/scripts/seed-environments.ts` (seed script) |
| 33 | +- `microservices/config-service/README.md` and docs |
25 | 34 | - Tests: `microservices/config-service/test/*` |
26 | 35 |
|
27 | | -Implementation Details |
28 | | ----------------------- |
29 | | -- Database: PostgreSQL via TypeORM (entities + orm-config) |
30 | | -- Secrets: AES-256-CBC encryption with IV; encrypted values stored in DB, rotation support |
31 | | -- Configs: key/value store, typed (string|number|boolean|json), environment-scoped, versioned |
32 | | -- Caching: in-memory cache (CacheManager) with configurable TTL and invalidation on updates |
33 | | -- Webhooks: subscription model, HMAC-SHA256 signing, retry logic with backoff |
34 | | -- Audit log: stores CREATE/UPDATE/DELETE/ROTATE events with metadata |
35 | | -- API docs: Swagger available at `/api` |
36 | | -- Docker: Dockerfile and docker-compose (includes PostgreSQL) for local/dev runs |
37 | | - |
38 | | -Database Migrations |
| 36 | +Database / Migrations |
| 37 | +--------------------- |
| 38 | +- Development: TypeORM `synchronize` is enabled when `NODE_ENV !== 'production'` for convenience. |
| 39 | +- Production: generate migrations and run them as part of deployment pipeline. DO NOT rely on `synchronize` in production. |
| 40 | + |
| 41 | +How to run locally |
39 | 42 | ------------------- |
40 | | -- Entities are set to `synchronize` when `NODE_ENV !== 'production'`. |
41 | | -- For production, run migrations generated from entities. |
42 | 43 |
|
43 | | -Commands |
44 | | --------- |
45 | | -Install and run locally: |
| 44 | +1) Quick start using Docker (recommended): |
| 45 | + |
| 46 | +```bash |
| 47 | +cd microservices/config-service |
| 48 | +docker-compose up -d |
| 49 | +# Service will be available at http://localhost:3020 |
| 50 | +``` |
| 51 | + |
| 52 | +2) Run locally against a Postgres instance: |
| 53 | + |
46 | 54 | ```bash |
47 | 55 | cd microservices/config-service |
48 | 56 | npm install |
49 | 57 | cp .env.example .env |
50 | | -# Edit .env => set ENCRYPTION_KEY and DB credentials |
51 | | -npm run migration:run # if using migrations |
| 58 | +# Edit .env: set ENCRYPTION_KEY (secure), DB_HOST, DB_USER, DB_PASSWORD |
| 59 | +npm run seed:environments |
52 | 60 | npm run start:dev |
53 | 61 | ``` |
54 | 62 |
|
55 | | -Docker (recommended for quick local setup): |
| 63 | +API surface (selected endpoints) |
| 64 | +------------------------------- |
| 65 | +- `GET /health` — health check |
| 66 | +- `GET /api` — Swagger UI |
| 67 | +- `POST /environments` — create environment |
| 68 | +- `GET /environments` — list environments |
| 69 | +- `POST /configurations` — create config (key/value) |
| 70 | +- `GET /configurations/key/:key` — get config by key |
| 71 | +- `POST /configurations/:id/increment-version` — bump version |
| 72 | +- `POST /secrets` — create secret (stored encrypted) |
| 73 | +- `GET /secrets/:id/value` — get decrypted secret value (requires auth in prod) |
| 74 | +- `POST /webhooks` — create webhook subscription |
| 75 | +- `POST /webhooks/:id/trigger` — trigger webhook (for testing) |
| 76 | +- `GET /audit-logs` — view recent audit logs |
| 77 | + |
| 78 | +Security |
| 79 | +-------- |
| 80 | +- Secrets are encrypted using AES-256-CBC. The `ENCRYPTION_KEY` env var must be populated in production from a secure store (Vault/KMS). |
| 81 | +- Webhook payloads are signed with HMAC-SHA256. Consumers MUST verify `X-Webhook-Signature` (or `X-Webhook-Signature` header). |
| 82 | +- Audit endpoints should be protected in production (RBAC / internal network only). |
| 83 | + |
| 84 | +Backwards compatibility |
| 85 | +----------------------- |
| 86 | +This change is additive: it introduces a new service and does not modify existing services. Consumer services must opt-in to use the config service by fetching configuration on startup and/or subscribing to webhooks. |
| 87 | + |
| 88 | +Manual verification (smoke tests) |
| 89 | +-------------------------------- |
| 90 | +1. Health check: |
| 91 | + |
56 | 92 | ```bash |
57 | | -cd microservices/config-service |
58 | | -docker-compose up -d |
59 | | -# Access: http://localhost:3020 |
| 93 | +curl http://localhost:3020/health |
| 94 | +``` |
| 95 | + |
| 96 | +2. Create environments (seed script also available): |
| 97 | + |
| 98 | +```bash |
| 99 | +curl -X POST http://localhost:3020/environments \ |
| 100 | + -H "Content-Type: application/json" \ |
| 101 | + -d '{"name":"development","displayName":"Development"}' |
| 102 | +``` |
| 103 | + |
| 104 | +3. Create a configuration and retrieve it: |
| 105 | + |
| 106 | +```bash |
| 107 | +curl -X POST http://localhost:3020/configurations \ |
| 108 | + -H "Content-Type: application/json" \ |
| 109 | + -d '{"key":"FEATURE_X_ENABLED","value":"true","type":"boolean","description":"Toggle for feature X"}' |
| 110 | + |
| 111 | +curl http://localhost:3020/configurations/key/FEATURE_X_ENABLED |
| 112 | +``` |
| 113 | + |
| 114 | +4. Create and read a secret (decrypted): |
| 115 | + |
| 116 | +```bash |
| 117 | +curl -X POST http://localhost:3020/secrets \ |
| 118 | + -H "Content-Type: application/json" \ |
| 119 | + -d '{"name":"DB_PASSWORD","value":"s3cr3t"}' |
| 120 | + |
| 121 | +# then |
| 122 | +curl http://localhost:3020/secrets/<secret-id>/value |
| 123 | +``` |
| 124 | + |
| 125 | +5. Subscribe a webhook and trigger an event (consumer must validate signature): |
| 126 | + |
| 127 | +```bash |
| 128 | +curl -X POST http://localhost:3020/webhooks \ |
| 129 | + -H "Content-Type: application/json" \ |
| 130 | + -d '{"serviceName":"payment-service","webhookUrl":"http://payment-service:3019/webhooks/config-update","events":["CONFIG_UPDATED"],"secret":"payment-secret"}' |
| 131 | + |
| 132 | +# Trigger update |
| 133 | +curl -X POST http://localhost:3020/webhooks/<id>/trigger -d '{"event":"CONFIG_UPDATED","data":{}}' |
60 | 134 | ``` |
61 | 135 |
|
62 | 136 | Testing |
63 | 137 | ------- |
64 | 138 | - Unit tests: `npm test` |
65 | 139 | - E2E tests: `npm run test:e2e` |
66 | | -- Basic unit and e2e tests are included; CI should run these on PR. |
| 140 | +- Include CI jobs to run both unit and E2E tests (E2E should provision Postgres or use Docker Compose) |
67 | 141 |
|
68 | | -Rollout & Migration Plan |
| 142 | +Seed and migration notes |
69 | 143 | ------------------------ |
70 | | -1. Deploy `config-service` to staging with production-like env vars (ensure `ENCRYPTION_KEY` is set and secure). |
71 | | -2. Run DB migrations against staging database. |
72 | | -3. Create initial environments (`development`, `staging`, `production`) via API or seed script. |
73 | | -4. Add initial configurations and secrets required by services. |
74 | | -5. For each dependent service: |
75 | | - - Add `CONFIG_SERVICE_URL` and `WEBHOOK_URL` env vars. |
76 | | - - Add startup logic to fetch required configs on boot (examples provided in `CONFIG_SERVICE_INTEGRATION.md`). |
77 | | - - Optionally subscribe service webhook endpoints to `config-service` for real-time updates. |
78 | | -6. Deploy one consumer service to staging and verify config fetch and webhook behavior. |
79 | | -7. Monitor audit logs and webhook deliveries. |
80 | | - |
81 | | -Rollback Plan |
| 144 | +- `src/scripts/seed-environments.ts` seeds `development`, `staging`, and `production`. |
| 145 | +- Before production deploy: run `npm run migration:generate` (or create a migration by hand) and apply with `npm run migration:run`. |
| 146 | + |
| 147 | +Rollout plan |
| 148 | +------------ |
| 149 | +1. Deploy `config-service` to staging with secure environment variables (set `ENCRYPTION_KEY` via vault/KMS). |
| 150 | +2. Run DB migrations against staging DB. |
| 151 | +3. Run `npm run seed:environments` on staging to create base environments. |
| 152 | +4. Update a single consumer service to fetch config on startup and optionally subscribe to webhooks; deploy and verify behavior. |
| 153 | +5. Monitor audit logs and webhook delivery. If stable, onboard additional services in waves. |
| 154 | + |
| 155 | +Rollback plan |
82 | 156 | ------------- |
83 | | -- If `config-service` causes issues, remove or disable webhook subscriptions from consumer services and revert consumer service config to local environment-based values. |
84 | | -- Restore DB from backup prior to deploy if schema or data corruption occurs. |
85 | | -- Redeploy previous version of `config-service` image. |
86 | | - |
87 | | -Secrets & Rotation |
88 | | ------------------- |
89 | | -- Secrets stored encrypted (DB: `encryptedValue`, `iv`). |
90 | | -- Rotation API available: `POST /secrets/:id/rotate`. |
91 | | -- Rotation detection task exists (check endpoints `GET /secrets/rotation/check`). |
92 | | -- Ensure `ENCRYPTION_KEY` is stored securely in production (vault, KMS). |
93 | | - |
94 | | -Security Considerations |
95 | | ------------------------ |
96 | | -- Do not commit `.env` or secret values. |
97 | | -- Use a secure `ENCRYPTION_KEY` (32+ chars) in production and rotate as needed. |
98 | | -- Webhook requests signed with HMAC-SHA256; consumers must verify signatures. |
99 | | -- Audit logs store changes; restrict access to audit endpoints. |
100 | | - |
101 | | -Testing & Verification Checklist (for reviewer) |
102 | | ------------------------------------------------ |
103 | | -- [ ] Service builds successfully: `npm run build` |
104 | | -- [ ] Unit tests pass: `npm test` |
105 | | -- [ ] E2E tests pass: `npm run test:e2e` |
106 | | -- [ ] Docker compose starts services and PostgreSQL |
107 | | -- [ ] Can create environment, config, and secret via API |
108 | | -- [ ] Secrets are stored encrypted (DB) and `GET /secrets/:id/value` returns decrypted value |
109 | | -- [ ] Webhook delivery works and signature verification can be validated by consumer |
110 | | -- [ ] Audit logs contain CREATE/UPDATE/DELETE events |
111 | | -- [ ] Config caching is invalidated on update |
112 | | -- [ ] Version increment endpoint works (`POST /configurations/:id/increment-version`) |
113 | | - |
114 | | -Notes / Known Limitations |
115 | | -------------------------- |
116 | | -- `synchronize` is enabled in non-production by default; production should use migrations. |
117 | | -- Secret encryption uses a symmetric key from env; for stronger security consider integration with KMS (AWS KMS, HashiCorp Vault). |
118 | | -- Scaling: current cache is in-memory; for multi-instance deployments use Redis-backed caching for shared cache invalidation. |
119 | | - |
120 | | -Suggested Reviewers |
121 | | -------------------- |
122 | | -- Backend/Platform: @backend-team |
123 | | -- Security: @security-team |
124 | | -- DevOps: @devops-team |
125 | | - |
126 | | -Labels |
127 | | ------- |
128 | | -- feature |
129 | | -- service |
130 | | -- infra |
131 | | - |
132 | | -Release Notes |
133 | | -------------- |
134 | | -Adds a new centralized configuration management service for the platform providing environment-scoped configs, encrypted secrets with rotation, webhooks for real-time updates, in-memory caching, audit logs, and Docker deployment. |
| 157 | +- Disable webhook subscriptions for consumers and revert them to local environment variables. |
| 158 | +- Revert `config-service` deployment to previous image. |
| 159 | +- If DB schema or data is damaged, restore DB from pre-deploy backup. |
| 160 | + |
| 161 | +Review checklist (for the PR) |
| 162 | +--------------------------- |
| 163 | +- [ ] Build passes: `npm run build` in `microservices/config-service` |
| 164 | +- [ ] Unit tests: `npm test` |
| 165 | +- [ ] E2E tests: `npm run test:e2e` |
| 166 | +- [ ] Docker compose starts without errors |
| 167 | +- [ ] Seed script creates environments: `npm run seed:environments` |
| 168 | +- [ ] Configs and secrets can be created and retrieved via API |
| 169 | +- [ ] Webhook delivery and signature verification verified by a consumer service |
| 170 | +- [ ] Audit logs contain expected events |
| 171 | + |
| 172 | +Suggested reviewers and labels |
| 173 | +----------------------------- |
| 174 | +- Reviewers: backend/platform, security, devops |
| 175 | +- Labels: `feature`, `service`, `infra` |
| 176 | + |
| 177 | +Notes / Known limitations |
| 178 | +------------------------ |
| 179 | +- In-memory cache is not clustered; for multi-instance deployments use Redis. |
| 180 | +- `ENCRYPTION_KEY` currently comes from env — consider KMS integration. |
| 181 | +- `synchronize` MUST be disabled for production; use explicit migrations. |
135 | 182 |
|
136 | | -Next Steps |
137 | | ----------- |
138 | | -- Consider adding a DB migration and seed script for initial environments. |
139 | | -- Optionally integrate with a KMS for encryption key management. |
| 183 | +Next steps (post-merge) |
| 184 | +---------------------- |
| 185 | +- Add a production migration and include it in the deployment pipeline. |
| 186 | +- Add Redis-backed cache and update invalidation strategy for multi-instance clusters. |
| 187 | +- Integrate `ENCRYPTION_KEY` with a managed KMS and remove raw key usage in env for production. |
140 | 188 |
|
141 | | -File: `PR_CONFIG_SERVICE.md` created at repo root. Review and let me know if you want this copied to a GitHub PULL_REQUEST_TEMPLATE or a different format/branch ready for a PR. |
| 189 | +PR body created at `PR_CONFIG_SERVICE.md` — review and let me know if you want this formatted differently for GitHub or shortened for a release note. |
0 commit comments