Skip to content

Latest commit

 

History

History
218 lines (75 loc) · 3.34 KB

File metadata and controls

218 lines (75 loc) · 3.34 KB

Data Migration from S3 to EFS through DataSync-AWS

## 📜 **DEPLOYMENT_STEPS.md**

\# Deployment Steps – AWS S3 to EFS Data Migration using DataSync



This document provides a detailed guide to deploy and verify the \*\*S3 → EFS migration\*\* using AWS services.



---



\## Step 1: Create an S3 Bucket

1\. Go to \*\*AWS Console → S3 → Create Bucket\*\*.

2\. Enter a unique name (e.g., `s3-datasync-source-bucket`).

3\. Keep defaults and click \*\*Create Bucket\*\*.

4\. Upload files or folders that you want to migrate.



---



\## Step 2: Create an EFS File System

1\. Navigate to \*\*AWS Console → EFS → Create File System\*\*.

2\. Choose:

   - \*\*VPC:\*\* Same VPC where your EC2 will be launched.

   - \*\*Mount Targets:\*\* Create for each Availability Zone (if needed).

3\. Note down the \*\*EFS DNS name\*\* (e.g., `fs-xxxxxx.efs.ap-south-1.amazonaws.com`).



---



\## Step 3: Create AWS DataSync Locations

1\. Go to \*\*AWS Console → DataSync → Locations → Create location\*\*.

2\. \*\*For Source:\*\*

   - Select \*\*Amazon S3\*\*.

   - Choose your S3 bucket.

3\. \*\*For Destination:\*\*

   - Select \*\*Amazon EFS\*\*.

   - Choose your EFS file system.



---



\## Step 4: Create and Run a DataSync Task

1\. Go to \*\*DataSync → Create Task\*\*.

2\. Select \*\*S3 (source)\*\* and \*\*EFS (destination)\*\*.

3\. Configure:

   - \*\*Task Name:\*\* `s3-to-efs-datasync`

   - \*\*Bandwidth limit:\*\* Default

   - \*\*Task execution role:\*\* Automatically created

4\. Review and click \*\*Create Task\*\*.

5\. Once created, click \*\*Start Task\*\*\*\*Verify transfer status\*\* (should show “Success”).



---



\## Step 5: Launch EC2 Instance using AWS CloudShell

1\. Open \*\*CloudShell\*\* in AWS Console.

2\. Run the command:

   ```bash

-> aws ssm get-parameter --name /aws/service/datasync/ami --region your-region



-> https://console.aws.amazon.com/ec2/v2/home?region=your-region#LaunchInstanceWizard:ami=ami-id



3\. Once launched copy the public IP address of EC2 instance.





\## Step 6: Connect to EC2 using MobaXterm

1. Open MobaXterm → Session → SSH.

2\. Enter:

Remote Host: <EC2 Public IP>

Username: ec2-user

Key File: my-keypair.pem

3\. Click OK to connect.





\## Step 7: Install NFS Utils and Mount EFS

1.Inside EC2 terminal:



sudo yum update -y

sudo yum install -y nfs-utils

sudo mkdir /mnt/efs

sudo mount -t nfs4 -o nfsvers=4.1 fs-xxxxxx.efs.ap-south-1.amazonaws.com:/ /mnt/efs



\## Step 8: Verify Migrated Data

1. Run:

ls /mnt/efs



You should see all files that were originally in the S3 bucket.





\## Outcome:



1. DataSync Task: Shows “Successful” status.

2\. EFS Mounted: Verified under /mnt/efs.

3\. File Consistency: Source and destination data match.





\## Clean-Up (Optional)



After verification:



1.Stop and terminate EC2 instance.

2.Delete DataSync task.

3.Remove S3 bucket or EFS if no longer needed.





\## 👩‍💻 Author

\*\*Keerthi B\*\*

📧 Email: bkeerthi1528@gmail.com

🔗 LinkedIn: \[www.linkedin.com/in/siva-sai-keerthi-bondalapati-b89779248](#)