Skip to content

Commit d61d0af

Browse files
Shubham4026Shubham4026
authored
Merge pull request #738 from tekdi/sdbv_rbac_changes
eat: Search by name is supported in cohort member api
2 parents df8e7bc + c5ef222 commit d61d0af

3 files changed

Lines changed: 365 additions & 0 deletions

File tree

‎docs/superpowers/specs/2026-05-07-pii-encryption-design.md‎

Lines changed: 348 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,348 @@
1+
# PII Encryption Design — User Microservice
2+
**Date:** 2026-05-07
3+
**Compliance:** DPDPA (Digital Personal Data Protection Act)
4+
**Scope:** `Users` table — Phase 1
5+
**Status:** Decisions finalised ✅
6+
7+
---
8+
9+
## 1. Objective
10+
11+
Encrypt all Personally Identifiable Information (PII) stored in the `Users` table at rest. Only ciphertext will persist in the DB. Encryption/decryption happens transparently in the service layer using AES-256 with a key from an environment variable (`PII_ENCRYPTION_KEY`).
12+
13+
---
14+
15+
## 2. PII Fields in Scope (Users Table)
16+
17+
Under DPDPA, **name is classified as Personal Data** — `firstName`, `lastName`, `middleName` are all required to be encrypted.
18+
19+
| Column | DB Column Name (after migration) | Encryption Mode | Searchable in DB? |
20+
|---|---|---|---|
21+
| `email` | `email` (same, value encrypted) | Deterministic (AES-256-CBC) | YES |
22+
| `mobile` | `mobile` (same, value encrypted) | Deterministic (AES-256-CBC) | YES |
23+
| `dob` | `dob` (same, value encrypted) | Random-IV (AES-256-GCM) | NO |
24+
| `firstName` | `firstName` (same, value encrypted) | Deterministic (AES-256-CBC) | YES |
25+
| `lastName` | `lastName` (same, value encrypted) | Deterministic (AES-256-CBC) | YES |
26+
| `middleName` | `middleName` (same, value encrypted) | Random-IV (AES-256-GCM) | NO |
27+
| `address` | `address` (same, value encrypted) | Random-IV (AES-256-GCM) | NO |
28+
| `pincode` | `pincode` (same, value encrypted) | Random-IV (AES-256-GCM) | NO |
29+
30+
---
31+
32+
## 3. Column Strategy — Keep Same Names, Store Encrypted Values ✅
33+
34+
**Decision:** Keep existing column names (`email`, `mobile`, `dob`, `firstName`, etc.) in the `Users` table. The encrypted ciphertext is stored directly in these columns. The separate `encryptedEmail`, `encryptedMobile`, `encryptedDob` columns (currently in DB) will be **dropped**.
35+
36+
**Why this is better:**
37+
- No entity renames or response field mapping changes needed
38+
- Reporting/analytics tools that query the DB directly will receive encrypted values — which is the correct DPDPA behaviour (they should not see plaintext PII)
39+
- Simpler migration — encrypt in-place, drop the redundant `encryptedXxx` columns
40+
41+
### Final DB Column State After Migration
42+
43+
| Column | Type | Value |
44+
|---|---|---|
45+
| `email` | `TEXT` | AES encrypted ciphertext |
46+
| `mobile` | `TEXT` | AES encrypted ciphertext |
47+
| `dob` | `TEXT` | AES encrypted ciphertext |
48+
| `firstName` | `TEXT` | AES encrypted ciphertext |
49+
| `lastName` | `TEXT` | AES encrypted ciphertext |
50+
| `middleName` | `TEXT` | AES encrypted ciphertext |
51+
| `address` | `TEXT` | AES encrypted ciphertext (was already TEXT) |
52+
| `pincode` | `TEXT` | AES encrypted ciphertext |
53+
| `encryptedEmail` | — | **DROPPED** |
54+
| `encryptedMobile` | — | **DROPPED** |
55+
| `encryptedDob` | — | **DROPPED** |
56+
57+
---
58+
59+
## 4. Encryption Approach
60+
61+
### Two Modes
62+
63+
**Deterministic (AES-256-CBC, fixed IV derived from key)**
64+
- Used for: `email`, `mobile`, `firstName`, `lastName`
65+
- Same plaintext → same ciphertext every time
66+
- Allows DB-level `WHERE email = encryptDeterministic(input)`
67+
- Slightly lower security than random-IV (no IV randomness) — acceptable trade-off for searchability
68+
69+
**Random-IV (AES-256-GCM, authenticated)**
70+
- Used for: `dob`, `middleName`, `address`, `pincode`
71+
- Same plaintext → different ciphertext each time
72+
- Maximum security, tamper-proof (GCM auth tag)
73+
- Not searchable at DB level — no need to search by these fields
74+
75+
### Key Configuration
76+
- **Env var:** `PII_ENCRYPTION_KEY` — 32-byte hex string (256-bit)
77+
- **Deterministic IV:** first 16 bytes of `SHA-256(PII_ENCRYPTION_KEY)` — fixed, derived from key
78+
- **Random IV:** `crypto.randomBytes(12)` — regenerated per encryption
79+
- **Storage format (GCM):** `iv_base64:authTag_base64:ciphertext_base64`
80+
- **Storage format (CBC):** `ciphertext_base64` (IV is fixed/derived, no need to store)
81+
- **Key Rotation:** requires a data migration script to re-encrypt all rows
82+
83+
---
84+
85+
## 5. Database Migration Steps
86+
87+
```
88+
Step 1: Widen column types to TEXT (to fit ciphertext)
89+
Step 2: Encrypt all existing rows in-place (via migration script)
90+
Step 3: Drop redundant encryptedXxx columns
91+
Step 4: Deploy new service code
92+
```
93+
94+
**Step 1 — SQL:**
95+
```sql
96+
ALTER TABLE "Users" ALTER COLUMN "email" TYPE TEXT;
97+
ALTER TABLE "Users" ALTER COLUMN "mobile" TYPE TEXT;
98+
ALTER TABLE "Users" ALTER COLUMN "dob" TYPE TEXT;
99+
ALTER TABLE "Users" ALTER COLUMN "firstName" TYPE TEXT;
100+
ALTER TABLE "Users" ALTER COLUMN "lastName" TYPE TEXT;
101+
ALTER TABLE "Users" ALTER COLUMN "middleName" TYPE TEXT;
102+
ALTER TABLE "Users" ALTER COLUMN "pincode" TYPE TEXT;
103+
-- address is already TEXT
104+
```
105+
106+
**Step 3 — SQL (after data encryption script completes):**
107+
```sql
108+
ALTER TABLE "Users" DROP COLUMN IF EXISTS "encryptedEmail";
109+
ALTER TABLE "Users" DROP COLUMN IF EXISTS "encryptedMobile";
110+
ALTER TABLE "Users" DROP COLUMN IF EXISTS "encryptedDob";
111+
```
112+
113+
> **Critical:** Steps 1-3 must complete and be verified **before** deploying new service code. The service code assumes all PII columns contain ciphertext.
114+
115+
---
116+
117+
## 6. New Service: `EncryptionService`
118+
119+
**Location:** `src/common/services/encryption.service.ts`
120+
121+
```typescript
122+
@Injectable()
123+
export class EncryptionService {
124+
// AES-256-CBC — fixed IV derived from key. Same input → same output.
125+
encryptDeterministic(value: string): string
126+
127+
// AES-256-GCM — random IV. Same input → different output each time.
128+
encryptRandom(value: string): string
129+
130+
// Detects mode from ciphertext format and decrypts accordingly.
131+
decrypt(value: string): string
132+
133+
// Convenience: encrypts all PII fields on a user object before DB write.
134+
encryptUserPII(user: Partial<User>): Partial<User>
135+
136+
// Convenience: decrypts all PII fields on a user object after DB read.
137+
decryptUserPII(user: Partial<User>): Partial<User>
138+
}
139+
```
140+
141+
**PII field → encryption mode mapping (inside EncryptionService):**
142+
```
143+
email → encryptDeterministic
144+
mobile → encryptDeterministic
145+
firstName → encryptDeterministic
146+
lastName → encryptDeterministic
147+
dob → encryptRandom
148+
middleName → encryptRandom
149+
address → encryptRandom
150+
pincode → encryptRandom
151+
```
152+
153+
---
154+
155+
## 7. API Impact Analysis
156+
157+
### 7.1 CREATE — `POST /user/create`
158+
159+
| Aspect | Change |
160+
|---|---|
161+
| **Request body** | No change — consumer sends plaintext |
162+
| **Service layer** | Call `encryptUserPII()` before `usersRepository.save()` |
163+
| **Keycloak** | Receives plaintext before encryption — no change |
164+
| **Response** | Call `decryptUserPII()` before returning |
165+
166+
---
167+
168+
### 7.2 GET — `GET /user/read/:userId`
169+
170+
| Aspect | Change |
171+
|---|---|
172+
| **DB fetch** | Fetches ciphertext from all PII columns |
173+
| **Service layer** | Call `decryptUserPII()` after fetch |
174+
| **Response** | Plaintext PII returned to consumer — field names unchanged |
175+
176+
---
177+
178+
### 7.3 UPDATE — `PATCH /user/update/:userid`
179+
180+
| Aspect | Change |
181+
|---|---|
182+
| **Request body** | No change — consumer sends plaintext |
183+
| **Service layer** | Call `encryptUserPII()` on updated fields before save |
184+
| **Keycloak** | Receives plaintext — no change |
185+
| **Response** | `decryptUserPII()` before returning |
186+
187+
---
188+
189+
### 7.4 LIST / SEARCH — `POST /user/list` ⚠️ MOST IMPACTED
190+
191+
Current PII filters:
192+
```json
193+
{ "filters": { "email": ["user@example.com"], "mobile": "9876543210" } }
194+
```
195+
196+
**Changes:**
197+
| Filter | Old Behaviour | New Behaviour |
198+
|---|---|---|
199+
| `email` | `WHERE email ILIKE ?` | `encryptDeterministic(input)` → `WHERE email = ?` (exact match) |
200+
| `mobile` | `WHERE mobile = ?` | `encryptDeterministic(input)` → `WHERE mobile = ?` (exact match) |
201+
| `firstName` | `WHERE firstName ILIKE ?` | `encryptDeterministic(input)` → `WHERE firstName = ?` (exact match) |
202+
| `lastName` | `WHERE lastName ILIKE ?` | `encryptDeterministic(input)` → `WHERE lastName = ?` (exact match) |
203+
204+
> **Breaking change for partial/wildcard search:** `ILIKE '%rahul%'` is no longer possible. Search must be exact. This is an accepted trade-off for DPDPA compliance.
205+
206+
After DB fetch → call `decryptUserPII()` on each result before building response.
207+
208+
---
209+
210+
### 7.5 CHECK USER — `POST /user/check`
211+
212+
`ExistUserDto` has `email`, `mobile`, `firstName`, `lastName`.
213+
214+
| Field | Change |
215+
|---|---|
216+
| `email` | `encryptDeterministic(input)` → `WHERE email = ?` |
217+
| `mobile` | `encryptDeterministic(input)` → `WHERE mobile = ?` |
218+
| `firstName` | `encryptDeterministic(input)` → `WHERE firstName = ?` |
219+
| `lastName` | `encryptDeterministic(input)` → `WHERE lastName = ?` |
220+
221+
---
222+
223+
### 7.6 FORGOT PASSWORD — `POST /user/forgot-password`
224+
225+
| Aspect | Change |
226+
|---|---|
227+
| Input email lookup | `encryptDeterministic(email)` → `WHERE email = ?` |
228+
| Response | No PII exposed |
229+
230+
---
231+
232+
### 7.7 SEND OTP — `POST /user/send-otp`
233+
234+
| Aspect | Change |
235+
|---|---|
236+
| Mobile lookup | `encryptDeterministic(mobile)` → `WHERE mobile = ?` |
237+
| Sending SMS | `decrypt(user.mobile)` → pass plaintext to SMS provider |
238+
239+
---
240+
241+
### 7.8 SEND OTP ON MAIL — `POST /user/send-otp-mail`
242+
243+
| Aspect | Change |
244+
|---|---|
245+
| Email lookup | `encryptDeterministic(email)` → `WHERE email = ?` |
246+
| Sending email | `decrypt(user.email)` → pass plaintext to mail provider |
247+
248+
---
249+
250+
### 7.9 PASSWORD RESET OTP — `POST /user/password-reset-otp`
251+
252+
Same pattern as 7.7 / 7.8 — encrypt input for lookup, decrypt for sending.
253+
254+
---
255+
256+
### 7.10 USER HIERARCHY VIEW — `POST /user/hierarchy-view`
257+
258+
| Aspect | Change |
259+
|---|---|
260+
| Email lookup | `encryptDeterministic(email)` → `WHERE email = ?` |
261+
| Domain extraction | `decrypt(user.email)` → then extract domain |
262+
263+
---
264+
265+
### 7.11 DELETE — `DELETE /user/delete/:userId`
266+
267+
Lookup by UUID — **no impact.** Encrypted row deleted with no special handling.
268+
269+
---
270+
271+
### 7.12 SUGGEST USERNAME — `POST /user/suggestUsername`
272+
273+
Input `firstName`/`lastName` come from request body, not DB — **no impact on input.**
274+
If fetching existing users for uniqueness: call `decryptUserPII()` on fetched rows.
275+
276+
---
277+
278+
### 7.13 PASSWORD RESET LINK — `POST /user/password-reset-link`
279+
280+
| Aspect | Change |
281+
|---|---|
282+
| Fetching user email to send link | `decrypt(user.email)` → pass to mail provider |
283+
284+
---
285+
286+
## 8. Response Shape — No Contract Change
287+
288+
API consumers always receive **decrypted plaintext**. Field names are unchanged.
289+
290+
```json
291+
{
292+
"userId": "uuid",
293+
"firstName": "Rahul",
294+
"lastName": "Sharma",
295+
"email": "rahul@example.com",
296+
"mobile": "9876543210",
297+
"dob": "1995-04-12",
298+
"address": "123 Main St",
299+
"pincode": "110001"
300+
}
301+
```
302+
303+
The DB stores ciphertext in these columns. The service decrypts transparently before responding.
304+
305+
---
306+
307+
## 9. Files to Create / Modify
308+
309+
| File | Action | What Changes |
310+
|---|---|---|
311+
| `src/common/services/encryption.service.ts` | **CREATE** | Core AES-256 service with deterministic + random-IV modes |
312+
| `src/common/services/encryption.service.spec.ts` | **CREATE** | Unit tests for both encryption modes |
313+
| `src/user/entities/user-entity.ts` | **MODIFY** | Remove `encryptedEmail/Mobile/Dob` columns; change PII column types to `text` |
314+
| `src/user/user.service.ts` | **MODIFY** | Inject `EncryptionService`; encrypt on write, decrypt on read; update all search filter logic |
315+
| `src/user/dto/user-search.dto.ts` | **MODIFY** | Remove `@IsEmail` from `filters.email` (string only now) |
316+
| `src/user/user.module.ts` | **MODIFY** | Add `EncryptionService` to providers |
317+
| `src/common/common.module.ts` | **MODIFY** | Export `EncryptionService` |
318+
| `migrations/YYYYMMDD-encrypt-pii-users.ts` | **CREATE** | Widen columns, encrypt existing rows, drop `encryptedXxx` columns |
319+
| `.env` / `.env.example` | **MODIFY** | Add `PII_ENCRYPTION_KEY=<32-byte-hex>` |
320+
321+
---
322+
323+
## 10. What Does NOT Change
324+
325+
| Field | Reason |
326+
|---|---|
327+
| `username` | Not PII — used for login, stays plaintext |
328+
| `enrollmentId` | Unique identifier — stays plaintext |
329+
| `gender` | Not sensitive personal data under DPDPA |
330+
| `status`, `createdAt`, `updatedAt`, `createdBy`, `updatedBy` | Operational metadata |
331+
| `userId` | Primary key UUID |
332+
| Keycloak data | Outside service boundary — Keycloak stores its own copy |
333+
| Non-PII filters in `/user/list` | `status`, `role`, `cohortId`, `tenantId` — no change |
334+
335+
---
336+
337+
## 11. Decisions Log
338+
339+
| # | Decision | Rationale |
340+
|---|---|---|
341+
| 1 | Keep same column names, store encrypted values in-place | Cleaner — no entity renames, no response mapping changes, reporting tools correctly see ciphertext |
342+
| 2 | Drop `encryptedEmail`, `encryptedMobile`, `encryptedDob` columns | Redundant after in-place encryption strategy |
343+
| 3 | `firstName`, `lastName` are PII and must be encrypted | DPDPA classifies full name as Personal Data |
344+
| 4 | `firstName`, `lastName`, `email`, `mobile` use deterministic encryption | Required for DB-level search (exact match) |
345+
| 5 | `dob`, `middleName`, `address`, `pincode` use random-IV | No search requirement — maximum security |
346+
| 6 | `ILIKE` / partial search on PII fields removed | Impossible after encryption — exact match only |
347+
| 7 | Data export (Phase 2) | Not in scope for Phase 1 |
348+
| 8 | `FieldValues` encryption | Phase 2 — after Phase 1 is stable |

‎src/cohortMembers/cohortMembers.service.ts‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -345,6 +345,9 @@ ON CM."userId" = U."userId" ${whereCase}`;
345345
"userId",
346346
"role",
347347
"name",
348+
"firstName",
349+
"middleName",
350+
"lastName",
348351
"status",
349352
"cohortAcademicYearId",
350353
"cohortMemberRole",
@@ -622,6 +625,15 @@ ON CM."userId" = U."userId" ${whereCase}`;
622625
case "firstName": {
623626
return `U."firstName" ILIKE '%${value}%'`;
624627
}
628+
case "middleName": {
629+
return `U."middleName" ILIKE '%${value}%'`;
630+
}
631+
case "lastName": {
632+
return `U."lastName" ILIKE '%${value}%'`;
633+
}
634+
case "name": {
635+
return `U."name" ILIKE '%${value}%'`;
636+
}
625637
case "cohortAcademicYearId": {
626638
const cohortIdAcademicYear = Array.isArray(value)
627639
? value.map((id) => `'${id}'`).join(", ")

‎src/cohortMembers/dto/cohortMembers-search.dto.ts‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,11 @@ class FiltersDto {
5151
@IsString()
5252
lastName?: string;
5353

54+
@ApiPropertyOptional({ type: String, description: "Partial match on firstName" })
55+
@IsOptional()
56+
@IsString()
57+
name?: string;
58+
5459
@ApiPropertyOptional({ type: Array, description: "Status", example: [] })
5560
@IsOptional()
5661
@IsArray()

0 commit comments

Comments
 (0)