Skip to content

Commit 8918f17

Browse files
committed
feat: add gcp-migration skill and fix reviewdog ci errors
1 parent 0a1ff7b commit 8918f17

4 files changed

Lines changed: 520 additions & 341 deletions

File tree

Lines changed: 237 additions & 57 deletions
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,24 @@
11
---
22
name: GCP Project Migration
3-
description: GCP 프로젝트 간 마이그레이션 (DB, Storage, Container Images, Terraform)
3+
description: Migration between GCP Projects (DB, Storage, Container Images, Terraform)
44
---
55

66
# GCP Project Migration Skill
77

8-
이 스킬은 GCP 프로젝트를 한 프로젝트에서 다른 프로젝트로 완전 이전하는 작업을 안내합니다.
8+
This skill guides you through the process of fully migrating from one GCP project to another.
99

10-
## 사전 조건
10+
## Prerequisites
1111

12-
1. ** GCP 프로젝트** 모두에 대한 Owner/Editor 권한
13-
2. **gcloud CLI** 설치 및 인증됨
14-
3. **Docker** 설치됨 (컨테이너 이미지 이전용)
15-
4. 새 프로젝트에 **Terraform 인프라가 먼저 배포**되어 있어야 함
12+
1. Owner/Editor permissions for **both GCP projects**
13+
2. **gcloud CLI** installed and authenticated
14+
3. **Docker** installed (for container image migration)
15+
4. **Terraform infrastructure must be deployed** to the new project first
1616

17-
## 마이그레이션 순서
17+
## Migration Steps
1818

19-
### Phase 1: API 활성화
19+
### Phase 1: Enable APIs
2020

21-
새 프로젝트에서 필요한 API들을 활성화합니다:
21+
Enable required APIs in the new project:
2222

2323
```bash
2424
gcloud services enable \
@@ -35,131 +35,311 @@ gcloud services enable \
3535
--project=NEW_PROJECT_ID
3636
```
3737

38-
### Phase 2: 데이터베이스 마이그레이션
38+
### Phase 2: Database Migration
3939

4040
1. **Export from old project**
4141
```bash
42-
# 구 프로젝트 계정으로 전환
42+
# Switch to old project account
4343
gcloud config set account OLD_ACCOUNT@gmail.com
4444

45-
# Cloud SQL 서비스 계정에 버킷 권한 부여
45+
# Grant bucket permissions to Cloud SQL service account
4646
OLD_SA=$(gcloud sql instances describe OLD_INSTANCE --project=OLD_PROJECT --format="value(serviceAccountEmailAddress)")
4747
gcloud storage buckets add-iam-policy-binding gs://MIGRATION_BUCKET \
4848
--member="serviceAccount:$OLD_SA" \
4949
--role="roles/storage.objectAdmin"
5050

51-
# DB 덤프
51+
# Dump DB
5252
gcloud sql export sql OLD_INSTANCE gs://MIGRATION_BUCKET/dump.sql \
53-
--database=pic2cook --project=OLD_PROJECT
53+
--database=database_name --project=OLD_PROJECT
5454
```
5555

5656
2. **Import to new project**
5757
```bash
58-
# 신 프로젝트 계정으로 전환
58+
# Switch to new project account
5959
gcloud config set account NEW_ACCOUNT@gmail.com
6060

61-
# 권한 부여 및 Import
61+
# Grant permissions and Import
6262
NEW_SA=$(gcloud sql instances describe NEW_INSTANCE --project=NEW_PROJECT --format="value(serviceAccountEmailAddress)")
6363
gcloud storage buckets add-iam-policy-binding gs://MIGRATION_BUCKET \
6464
--member="serviceAccount:$NEW_SA" \
6565
--role="roles/storage.objectViewer"
6666

6767
gcloud sql import sql NEW_INSTANCE gs://MIGRATION_BUCKET/dump.sql \
68-
--database=pic2cook --user=postgres --project=NEW_PROJECT
68+
--database=database_name --user=db_username --project=NEW_PROJECT
6969
```
7070

71-
### Phase 3: GCS 버킷 마이그레이션
71+
### Phase 3: GCS Bucket Migration
7272

7373
```bash
74-
# 구 계정에서 신 계정에 읽기 권한 부여
74+
# Grant read permission to new account from old account
7575
gcloud storage buckets add-iam-policy-binding gs://OLD_BUCKET \
7676
--member="user:NEW_ACCOUNT@gmail.com" \
7777
--role="roles/storage.objectViewer"
7878

79-
# 신 계정에서 버킷 생성 및 동기화
79+
# Create and sync bucket in new account
8080
gcloud storage buckets create gs://NEW_BUCKET --location=REGION --project=NEW_PROJECT
8181
gcloud storage rsync -r gs://OLD_BUCKET gs://NEW_BUCKET
8282
```
8383

84-
### Phase 4: 컨테이너 이미지 마이그레이션
84+
### Phase 4: Container Image Migration
8585

8686
```bash
87-
# Docker 인증
87+
# Docker Authentication
8888
gcloud auth configure-docker OLD_REGION-docker.pkg.dev,NEW_REGION-docker.pkg.dev
8989

90-
# 이미지 Pull -> Tag -> Push
90+
# Image Pull -> Tag -> Push
9191
docker pull OLD_REGION-docker.pkg.dev/OLD_PROJECT/REPO/IMAGE:TAG
9292
docker tag OLD_REGION-docker.pkg.dev/OLD_PROJECT/REPO/IMAGE:TAG \
9393
NEW_REGION-docker.pkg.dev/NEW_PROJECT/REPO/IMAGE:TAG
9494
docker push NEW_REGION-docker.pkg.dev/NEW_PROJECT/REPO/IMAGE:TAG
9595
```
9696

97-
### Phase 5: Terraform 변수 수정
97+
### Phase 5: Modify Terraform Variables
9898

99-
`apps/infra/variables.tf` 파일 상단의 변수만 수정:
99+
Modify only the variables at the top of the `apps/infra/variables.tf` file:
100100

101101
```hcl
102102
variable "project_id" {
103-
default = "NEW_PROJECT_ID" # ← 변경
103+
default = "NEW_PROJECT_ID" # ← Change
104104
}
105105
106106
variable "region" {
107-
default = "asia-northeast1" # ← 필요 시 변경
107+
default = "your_region" # ← Change if necessary
108108
}
109109
```
110110

111-
**수동 변경 필요**:
112-
- `provider.tf``backend.bucket` (Terraform 제약)
111+
**Manual Changes Required**:
112+
- `backend.bucket` in `provider.tf` (Terraform limitation)
113113

114-
### Phase 6: GitHub Secrets 업데이트
114+
### Phase 6: Update GitHub Secrets
115115

116-
Repository Settings > Secrets and variables > Actions에서:
116+
In Repository Settings > Secrets and variables > Actions:
117117

118-
| Secret | |
118+
| Secret | Value |
119119
| -------------------------------- | -------------------------------------------------------------------------------------------------------- |
120-
| `GCP_WORKLOAD_IDENTITY_PROVIDER` | `projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/pic2cook-pool/providers/github-provider` |
121-
| `GCP_SERVICE_ACCOUNT` | `pic2cook-deployer@NEW_PROJECT.iam.gserviceaccount.com` |
120+
| `GCP_WORKLOAD_IDENTITY_PROVIDER` | `projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/your-pool/providers/github-provider` |
121+
| `GCP_SERVICE_ACCOUNT` | `your-deployer@NEW_PROJECT.iam.gserviceaccount.com` |
122122

123-
값 조회:
123+
Retrieve Values:
124124
```bash
125125
# Project Number
126126
gcloud projects describe NEW_PROJECT --format="value(projectNumber)"
127127

128128
# WIF Provider
129129
gcloud iam workload-identity-pools providers describe github-provider \
130-
--workload-identity-pool=pic2cook-pool --location=global --project=NEW_PROJECT --format="value(name)"
130+
--workload-identity-pool=your-pool --location=global --project=NEW_PROJECT --format="value(name)"
131131
```
132132

133-
### Phase 7: 코드 변경 및 배포
133+
### Phase 7: Code Changes and Deployment
134134

135-
1. `apps/api/src/lib/config.py``gcs_bucket_name` 기본값 변경
136-
2. 커밋 및 푸시
137-
3. GitHub Actions가 자동 배포
135+
1. Change `gcs_bucket_name` default value in `apps/api/src/lib/config.py`
136+
2. Commit and Push
137+
3. GitHub Actions will auto-deploy
138138

139-
## 자동화 스크립트
139+
## Automation Script
140140

141-
전체 데이터 마이그레이션을 한 번에 실행:
141+
Run full data migration at once:
142142

143143
```bash
144144
./.agent/skills/gcp-migration/scripts/migrate-gcp-project.sh \
145145
--old-project OLD_PROJECT_ID \
146146
--new-project NEW_PROJECT_ID \
147-
--old-region us-central1 \
148-
--new-region asia-northeast1
147+
--old-region your_old_region \
148+
--new-region your_new_region
149149
```
150150

151-
## 참조 문서
151+
## References
152152

153-
- [마이그레이션 가이드](docs/gcp-migration-guide.md)
154-
- [마이그레이션 스크립트](.agent/skills/gcp-migration/scripts/migrate-gcp-project.sh)
153+
- [Migration Guide](references/gcp-migration-guide.md)
154+
- [Migration Script](scripts/migrate-gcp-project.sh)
155155

156-
## 체크리스트
156+
## Troubleshooting
157157

158-
- [ ] API 활성화 완료
159-
- [ ] DB 마이그레이션 완료
160-
- [ ] GCS 버킷 마이그레이션 완료
161-
- [ ] 컨테이너 이미지 마이그레이션 완료
162-
- [ ] Terraform variables.tf 수정
163-
- [ ] GitHub Secrets 업데이트
164-
- [ ] config.py 수정 및 푸시
165-
- [ ] 배포 확인 및 테스트
158+
### Cloud Run Job fails with TCP Connection Error
159+
160+
**Symptoms**:
161+
```
162+
Is the server running on that host and accepting TCP/IP connections?
163+
```
164+
165+
**Cause**: A VPC Connector is required for Cloud Run Jobs to access Cloud SQL Private IP.
166+
167+
**Resolution**:
168+
```bash
169+
# 1. Enable VPC Access API
170+
gcloud services enable vpcaccess.googleapis.com --project=NEW_PROJECT_ID
171+
172+
# 2. Check VPC Connector
173+
gcloud compute networks vpc-access connectors list \
174+
--region=REGION \
175+
--project=NEW_PROJECT_ID
176+
177+
# 3. Create VPC Connector (if missing, use same network as Cloud SQL)
178+
gcloud compute networks vpc-access connectors create CONNECTOR_NAME \
179+
--region=REGION \
180+
--network=NETWORK_NAME \
181+
--range=10.9.0.0/28 \
182+
--project=NEW_PROJECT_ID
183+
184+
# 4. Attach VPC Connector to Cloud Run Job
185+
gcloud run jobs update JOB_NAME \
186+
--region=REGION \
187+
--project=NEW_PROJECT_ID \
188+
--vpc-connector=CONNECTOR_NAME \
189+
--vpc-egress=private-ranges-only
190+
```
191+
192+
### Direct VPC and VPC Connector Conflict
193+
194+
**Symptoms**:
195+
```
196+
VPC connector and direct VPC can not be used together
197+
```
198+
199+
**Resolution**: Remove Direct VPC first, then add VPC Connector:
200+
```bash
201+
# Remove Direct VPC
202+
gcloud run jobs update JOB_NAME \
203+
--region=REGION \
204+
--project=NEW_PROJECT_ID \
205+
--clear-network
206+
207+
# Add VPC Connector
208+
gcloud run jobs update JOB_NAME \
209+
--region=REGION \
210+
--project=NEW_PROJECT_ID \
211+
--vpc-connector=CONNECTOR_NAME \
212+
--vpc-egress=private-ranges-only
213+
```
214+
215+
### Password Authentication Failed
216+
217+
**Symptoms**:
218+
```
219+
FATAL: password authentication failed for user "username"
220+
```
221+
222+
**Resolution**:
223+
```bash
224+
# Check Cloud SQL user list
225+
gcloud sql users list \
226+
--instance=INSTANCE_NAME \
227+
--project=NEW_PROJECT_ID
228+
229+
# Reset password (if needed)
230+
gcloud sql users set-password USERNAME \
231+
--instance=INSTANCE_NAME \
232+
--project=NEW_PROJECT_ID \
233+
--password="NEW_PASSWORD"
234+
```
235+
236+
### Missing Required Environment Variables
237+
238+
**Symptoms**:
239+
```
240+
pydantic_core._pydantic_core.ValidationError: Field required
241+
jwt_secret_key
242+
```
243+
244+
**Resolution**: Environment variables required for Migration Job:
245+
- `DATABASE_URL` - PostgreSQL connection string
246+
- `JWT_SECRET_KEY` - JWT signing secret
247+
248+
**IMPORTANT**: Use `--update-env-vars` to preserve existing environment variables:
249+
```bash
250+
# WRONG: Overwrites all existing environment variables
251+
gcloud run jobs update JOB_NAME --set-env-vars="KEY=value"
252+
253+
# CORRECT: Adds/Updates while keeping existing environment variables
254+
gcloud run jobs update JOB_NAME --update-env-vars="KEY=value"
255+
```
256+
257+
### Manual Migration Job Fix (Full Steps)
258+
259+
Full process for manually fixing the Migration Job:
260+
261+
```bash
262+
# 1. Enable VPC Access API
263+
gcloud services enable vpcaccess.googleapis.com --project=NEW_PROJECT_ID
264+
265+
# 2. Check Cloud SQL Private IP
266+
DB_IP=$(gcloud sql instances describe INSTANCE_NAME \
267+
--project=NEW_PROJECT_ID \
268+
--format="value(ipAddresses[0].ipAddress)")
269+
echo "Cloud SQL Private IP: $DB_IP"
270+
271+
# 3. Create VPC Connector (same network as Cloud SQL)
272+
gcloud compute networks vpc-access connectors create default-connector \
273+
--region=REGION \
274+
--network=default \
275+
--range=10.9.0.0/28 \
276+
--project=NEW_PROJECT_ID
277+
278+
# 4. Remove Direct VPC (if exists)
279+
gcloud run jobs update JOB_NAME \
280+
--region=REGION \
281+
--project=NEW_PROJECT_ID \
282+
--clear-network
283+
284+
# 5. Connect VPC Connector + Set Env Vars
285+
gcloud run jobs update JOB_NAME \
286+
--region=REGION \
287+
--project=NEW_PROJECT_ID \
288+
--vpc-connector=default-connector \
289+
--vpc-egress=private-ranges-only \
290+
--update-env-vars="DATABASE_URL=postgresql+asyncpg://USER:PASS@${DB_IP}:5432/DBNAME,JWT_SECRET_KEY=YOUR_JWT_SECRET"
291+
292+
# 6. Execute Migration
293+
gcloud run jobs execute JOB_NAME \
294+
--region=REGION \
295+
--project=NEW_PROJECT_ID
296+
297+
# 7. Check Status
298+
EXECUTION_ID=$(gcloud run jobs executions list \
299+
--job=JOB_NAME \
300+
--region=REGION \
301+
--project=NEW_PROJECT_ID \
302+
--limit=1 \
303+
--format="value(name)")
304+
305+
gcloud run jobs executions describe $EXECUTION_ID \
306+
--region=REGION \
307+
--project=NEW_PROJECT_ID \
308+
--format="value(status.conditions[0].status,status.conditions[0].message)"
309+
```
310+
311+
### Verify Cloud Run Job Configuration
312+
313+
```bash
314+
# Check current Job configuration
315+
gcloud run jobs describe JOB_NAME \
316+
--region=REGION \
317+
--project=NEW_PROJECT_ID \
318+
--format="yaml(spec.template.spec.template.spec)"
319+
320+
# Check only VPC configuration
321+
gcloud run jobs describe JOB_NAME \
322+
--region=REGION \
323+
--project=NEW_PROJECT_ID \
324+
--format="yaml(spec.template.metadata.annotations)"
325+
326+
# Check environment variables
327+
gcloud run jobs describe JOB_NAME \
328+
--region=REGION \
329+
--project=NEW_PROJECT_ID \
330+
--format="yaml(spec.template.spec.template.spec.containers[0].env)"
331+
```
332+
333+
## Checklist
334+
335+
- [ ] APIs Enabled
336+
- [ ] Database Migration Completed
337+
- [ ] GCS Bucket Migration Completed
338+
- [ ] Container Image Migration Completed
339+
- [ ] Terraform variables.tf Modified
340+
- [ ] GitHub Secrets Updated
341+
- [ ] config.py Modified and Pushed
342+
- [ ] VPC Connector Created and Attached
343+
- [ ] Migration Job Env Vars Set (DATABASE_URL, JWT_SECRET_KEY)
344+
- [ ] Migration Job Executed Successfully
345+
- [ ] Deployment Verified and Tested

0 commit comments

Comments
 (0)