Skip to content

Commit d6bfdc2

Browse files
ravigurram8ravigurram8
authored
Merge pull request #107 from RLOpenCatalyst/topic-release
1.13.0 Release build Updates
2 parents e472dbd + 14b321b commit d6bfdc2

13 files changed

Lines changed: 270 additions & 105 deletions

‎README.md‎

Lines changed: 32 additions & 68 deletions
Original file line numberDiff line numberDiff line change
@@ -25,21 +25,36 @@ It provides a pre-built catalog of products which are ready to use out of the bo
2525

2626
| Virtual Machine Purpose | Virtual Machine Spec |
2727
| ---------------------------------- | ------------------------------------- |
28-
| Role: Portal | t2.large 2CPU, 8GB RAM, 100GB Disk |
28+
| Role: Portal | t3.large 2CPU, 8GB RAM, 100GB Disk |
2929
| Role: DB AWS DocumentDB | db.t3.large (dev) db.r5.large+ (prod) |
3030

31-
#### 2. Network Requirements
31+
#### 2. ACM or External Certificates for SSL
32+
33+
*Create a certificate for your domain*
34+
35+
To make the Research Gateway application available securely over SSL, you need a certificate issued by a public Certificate Authority (CA).
36+
If you already have a certificate for your domain issued by a third-party CA, you can import it into ACM. [See how](https://docs.aws.amazon.com/acm/latest/userguide/import-certificate.html).
37+
38+
If you already have a certificate for your domain issued by AWS in ACM, you can pass CertificateArn with launch stack ( #7 - #9) and skip the following step.
39+
40+
aws acm request-certificate --domain-name www.example.com --validation-method DNS --idempotency-token 1234 --options CertificateTransparencyLoggingPreference=DISABLED
41+
42+
You will need to validate your ownership of the domain either via DNS (recommended) or via email.
43+
44+
Fore more details on requesting a certificate, follow this [link](https://docs.aws.amazon.com/acm/latest/userguide/gs-acm-request-public.html)
45+
46+
#### 3. Network Requirements
3247

3348
1. VPC
3449
2. 3 Public Subnets
3550
3. 3 Private Subnets
3651
4. IGW
3752
5. NAT Instance / NAT Gateway
38-
6. Bastion Hosts (optional)
53+
6. Bastion Hosts (Optional)
3954
7. Application Load Balancer
4055
8. Listener
41-
9. ACM or External Certificates for SSL
42-
10. Target Group
56+
9. Target Group
57+
10. Route 53
4358

4459
<!-- trunk-ignore(markdownlint/MD036) -->
4560
*Create #1 - #5 above using the following quick-start*
@@ -49,68 +64,36 @@ It provides a pre-built catalog of products which are ready to use out of the bo
4964

5065

5166
<!-- trunk-ignore(markdownlint/MD036) -->
52-
*Create an Application Load Balancer*
53-
54-
Deploying an Application Load Balancer as part of Research Gateway deployment helps in two ways:
55-
1. Isolates your portal from being directly exposed over the internet. The ALB allows only https(s) traffic through.
56-
2. Helps to serve the application on a secure port using SSL certificates stored in AWS ACM.
57-
58-
create an ALB security Group with the following inbound and outbound rules.
59-
- Inbound rules - HTTP 80 ,HTTPS 443,SSH 22 .
60-
- outbound Rules - All Traffic.
61-
62-
Use the AWS CLI to create an Application Load Balancer choosing all three public subnets created by the quickstart above.
63-
64-
aws elbv2 create-load-balancer --name research-gw-alb --subnets subnet-abcd1234 subnet-abcd5678 subnet-abcd9876 --security-groups sg-abcd1234 --region us-east-1
65-
The output of the command above will include the ARN of the loadbalancer. e.g. arn:aws:elasticloadbalancing:region:aws_account_id:loadbalancer/app/research-gw-alb/e5ba62739c16e642
66-
67-
*Create a target group*
68-
69-
aws elbv2 create-target-group --name tg-research-gw --protocol HTTP --port 80 --target-type instance --vpc-id vpc-abcd1234 --region us-east-1
70-
The output of the command will include the ARN of the target group. e.g. arn:aws:elasticloadbalancing:region:aws_account_id:targetgroup/tg-research-gw/209a844cd01825a4
7167

68+
<!-- trunk-ignore(markdownlint/MD036) -->
69+
*Create #7 - #9 above using the following quick-start*
7270

73-
*Create a certificate for your domain*
74-
75-
To make the Research Gateway application available securely over SSL, you need a certificate issued by a public Certificate Authority (CA).
76-
If you already have a certificate for your domain issued by a third-party CA, you can import it into ACM. [See how](https://docs.aws.amazon.com/acm/latest/userguide/import-certificate.html).
77-
If you already have a certificate for your domain issued by AWS in ACM, you can skip the following step.
78-
79-
aws acm request-certificate --domain-name www.example.com --validation-method DNS --idempotency-token 1234 --options CertificateTransparencyLoggingPreference=DISABLED
80-
81-
You will need to validate your ownership of the domain either via DNS (recommended) or via email.
82-
83-
Fore more details on requesting a certificate, follow this [link](https://docs.aws.amazon.com/acm/latest/userguide/gs-acm-request-public.html)
84-
85-
*Create a listener on port 443 *
8671

87-
aws elbv2 create-listener --load-balancer-arn arn:aws:elasticloadbalancing:region:aws_account_id:loadbalancer/app/research-gw-alb/e5ba62739c16e642 --protocol HTTPS --port 443 --certificates CertificateArn=arn:aws:acm:us-east-1:aws_account_id:certificate/480cdfa8-bac6-4b99-977f-5f18441de49e --ssl-policy ELBSecurityPolicy-2016-08 --default-actions Type=forward,TargetGroupArn=arn:aws:elasticloadbalancing:region:aws_account_id:targetgroup/tg-research-gw/209a844cd01825a4 --region us-east-1
72+
[![Launch Stack](https://cdn.rawgit.com/buildkite/cloudformation-launch-stack-button-svg/master/launch-stack.svg)](https://us-west-2.console.aws.amazon.com/cloudformation/home?region=us-west-2#/stacks/create/review?templateURL=https://rglaunch-stack.s3.us-east-2.amazonaws.com/rg-alb-route53.yml)
8873

89-
*Create a listener on port 80 *
9074

91-
Note: This is only recommended for test setups and some of the features like secure links for resources will not work.
75+
<!-- trunk-ignore(markdownlint/MD036) -->
9276

93-
aws elbv2 create-listener --load-balancer-arn arn:aws:elasticloadbalancing:region:aws_account_id:loadbalancer/app/research-gw-alb/e5ba62739c16e642 --protocol HTTP --port 80 --default-actions Type=forward,TargetGroupArn=arn:aws:elasticloadbalancing:region:aws_account_id:targetgroup/tg-research-gw/209a844cd01825a4
77+
*Note* - Create #7 - #9 above using AWS CLI commands also .For that you refer to the rg_alb-tg-creation.md file from repo
9478

95-
<!-- trunk-ignore(markdownlint/MD036) -->
96-
*Set up Route 53 for your domain/sub-domain*
79+
*10.Set up Route 53 for your domain/sub-domain*
9780

9881
From the AWS Console, select Route 53 service. Create a hosted zone for your domain. If you are using an existing domain you can add a CNAME record and select the public DNS name of the ALB as the value. It is also possible to create a separate hosted zone for the sub-domain used for Research Gateway.
9982

10083
*Note* - It is possible to use Research Gateway without a domain-name but we do not recommend that for product workloads. To do so, use the public DNS name of the ALB as the URL of the Research Gateway in the setup scripts.
10184

102-
#### 3. Software Requirements
85+
#### 4. Software Requirements
10386

10487
| Virtual Machine Purpose | Software pre-requisites |
10588
| --------------------------- | ----------------------- |
10689
| Role: Portal | Python, Docker 20.04+ |
10790
| Role: DB (Option 1) MongoDB | MongoDB 3.6.23 |
108-
| | |
91+
10992

11093
The application software for Research Gateway will be made available to you as a docker image shared from Relevance Lab's Elastic Container Registry instance to your AWS account.
11194
As a part of this deployment, you will create an AMI for the portal EC2 instance which will have these softwares pre-deployed. Alternately, you can request the AMI to be shared with you by Relevance Lab and the software above will be available pre-deployed on the AMI shared with you.
11295

113-
#### 4. AWS Services required
96+
#### 5. AWS Services required
11497

11598
- AWS Cognito
11699
- Amazon S3
@@ -133,28 +116,9 @@ The following sofware needs to be installed on the Portal EC2 instance
133116
| jq | latest |
134117
| zip | latest |
135118

136-
For your convenience we have created packer scripts which allow you to create the AMI in your account.
137-
If an AMI with the pre-requisites has been shared with you, you can skip this section.
119+
For your convenience we have created packer scripts which allow you to create the AMI in your account For that you refer to the rg_AMI-creation.md file from repo
138120

139-
### Creating the AMI with pre-requisites
140-
141-
You can create the AMI with pre-requisites yourself by following these steps:
142-
143-
- [Install packer](https://learn.hashicorp.com/tutorials/packer/get-started-install-cli)
144-
- Export AWS Access Keys and Secret Keys
145-
146-
export AWS_ACCESS_KEY_ID="your_Access_Key"
147-
export AWS_SECRET_ACCESS_KEY="your_Secret_Key"
148-
export AWS_DEFAULT_REGION="Your_Region"
149-
150-
- Clone this repo on a machine.
151-
- Target Account number must be added in Admin Account-ECR and give permission to access image builds
152-
- Create a Role and attach a policy which permits ECR and EC2 actions and Replace the "iam-instance_profile" :"<your_rolename>" in builders section which is in the packer-rg.json.
153-
- Run packer build packer-rg.json
154-
155-
packer build -var 'awsRegion=your_region' -var 'vpcId=your_VPCID' -var 'subnetId=your_SubnetID' packer-rg.json
156-
- At Run time pass VPCID, SubnetID, AWSRegion as variables declared in packer-rg.json
157-
- Note that AMI id from the output
121+
If an AMI with the pre-requisites has been shared with you, you can skip this section
158122

159123
### Installing Research Gateway
160124

@@ -192,7 +156,7 @@ runid.json is created in the rgdeploy folder when you first run deploy.sh with p
192156

193157
The deployment creates EC2 Image Builder pipelines for building the RStudio and Nextflow AMIs that are used within Research Gateway. By default, these pipelines are set up to be manually triggered. You can change that in the AWS console if you wish to trigger them on a schedule.
194158

195-
steps to run pipelines: AWS console - Ec2imagebuilder – select image pipelines (Rstudio, Nextflow)-click on Actions-Run pipeline
159+
steps to run pipelines: AWS Console - EC2Imagebuilder – select image pipelines (Rstudio, Nextflow)-click on Actions- Run pipeline
196160

197161
Once a build is completed, the AMIs are automatically distributed to the regions supported by Research Gateway in your account. The AMI Ids need to be updated into your database before creating any projects.
198162

‎cft-templates/pcluster.yaml‎

Lines changed: 46 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -22,14 +22,18 @@ Metadata:
2222
default: Scheduler Configuration
2323
Parameters:
2424
- Scheduler
25-
- WorkerNodeInstanceType
26-
- WorkerNodeSubnetId
25+
- ComputeNodeInstanceType
26+
- QueueCapacityType
27+
- ComputeNodeSubnetId
2728
- ComputeEnvMinvCpus
2829
- ComputeEnvMaxvCpus
2930
- ComputeEnvDesiredvCpus
3031
- SpotBidPercentage
3132
- ResearcherName
3233
- ProjectId
34+
- DisableSimultaneousMultithreading
35+
- EnableEFA
36+
- PlacementGroup
3337
ProductName: Pcluster
3438

3539

@@ -64,7 +68,7 @@ Parameters:
6468
Scheduler:
6569
Description: Cluster scheduler
6670
Type: String
67-
Default: awsbatch
71+
Default: slurm
6872
ConstraintDescription: Must be a supported scheduler
6973
AllowedValues:
7074
- slurm
@@ -75,18 +79,18 @@ Parameters:
7579
HeadNodeInstanceType:
7680
Description: Head Node EC2 instance type
7781
Type: String
78-
Default: t3.micro
79-
AllowedValues: [t2.nano, t2.micro, t2.small, t2.medium, t3.micro, t3.small, t3.medium, t3.large, t3.xlarge]
82+
Default: t3.large
83+
AllowedValues: [t3.medium, t3.large, t3.xlarge, c5n.large]
8084
ConstraintDescription: Must be a valid EC2 instance type.
81-
WorkerNodeInstanceType:
85+
ComputeNodeInstanceType:
8286
Description: Specify the instance types to be used to carry out the computation.
8387
Type: String
8488
Default: c4.large
85-
AllowedValues: [c4.large, m4.large, r4.large, c4.4xlarge, m4.4xlarge, r4.4xlarge]
89+
AllowedValues: [c4.large, m4.large, r4.large, c4.4xlarge, m4.4xlarge, r4.4xlarge, c5n.18xlarge, c6i.32xlarge, g4dn.12xlarge]
8690
HeadNodeSubnetId:
8791
Type: AWS::EC2::Subnet::Id
8892
Description: 'Subnet you want your Head Node to launch in. You must select a public subnet.'
89-
WorkerNodeSubnetId:
93+
ComputeNodeSubnetId:
9094
Type: AWS::EC2::Subnet::Id
9195
Description: 'Subnet you want your Batch or slurm Worker Node to launch in. We recommend public subnets.'
9296
ComputeEnvDesiredvCpus:
@@ -112,7 +116,28 @@ Parameters:
112116
FileSystemId:
113117
Type: String
114118
Description: Enter the File System Id of the FSx for Lustre file system which you want to use. e.g. fs-12345678. Leave this field to default if you do not want to use FSx for Lustre
115-
Default: default
119+
Default: default
120+
QueueCapacityType:
121+
Description: The type of the compute resources used in the queue. Supported values are ONDEMAND or SPOT
122+
Type: String
123+
Default: SPOT
124+
AllowedValues: [SPOT, ONDEMAND]
125+
DisableSimultaneousMultithreading:
126+
Description: Disables hyperthreading on the compute nodes.Works with slurm Scheduler
127+
Type: String
128+
Default: "true"
129+
AllowedValues: ["true", "false"]
130+
EnableEFA:
131+
Description: Enables better network performances. Only available for certain instance types, If user select unsupported instance, EFA assigned to false.Works with slurm Scheduler
132+
Type: String
133+
Default: "false"
134+
AllowedValues: ["true", "false"]
135+
PlacementGroup:
136+
Description: Enables a PlacementGroup. Use with EnableEFA.Works with slurm Scheduler
137+
Type: String
138+
Default: "true"
139+
AllowedValues: ["true", "false"]
140+
116141

117142

118143
Resources :
@@ -222,14 +247,21 @@ Resources :
222247
- Key: cost_resource
223248
Value: !Sub ${AWS::StackName}
224249
UserData:
225-
Fn::Base64:
226-
!Sub |
250+
Fn::Base64: !Sub |
227251
#!/bin/bash
228-
set -e
252+
efaEnabled=${EnableEFA}
253+
case "${ComputeNodeInstanceType}" in
254+
c5n.18xlarge|c6i.32xlarge|g4dn.12xlarge) true
255+
;;
256+
*)
257+
efaEnabled=false
258+
;;
259+
esac
260+
set -e
229261
trap '/opt/aws/bin/cfn-signal --exit-code 1 --resource EC2Instance --region ${AWS::Region} --stack ${AWS::StackName}' ERR
230-
sudo yum update -y --security
262+
sudo yum update -y --security
231263
cd /home/ec2-user/parallel-update
232-
sudo -u ec2-user ./Provision-pcluster.sh ${Scheduler} ${AWS::Region} ${HeadNodeInstanceType} ${HeadNodeSubnetId} ${KeyPair} ${WorkerNodeInstanceType} ${ComputeEnvMinvCpus} ${ComputeEnvMaxvCpus} ${WorkerNodeSubnetId} ${ComputeEnvDesiredvCpus} ${SpotBidPercentage} ${AWS::StackName} ${CustomAMI} ${FileSystemId}
264+
sudo -u ec2-user ./Provision-pcluster.sh ${Scheduler} ${AWS::Region} ${HeadNodeInstanceType} ${HeadNodeSubnetId} ${KeyPair} ${ComputeNodeInstanceType} ${ComputeEnvMinvCpus} ${ComputeEnvMaxvCpus} ${ComputeNodeSubnetId} ${ComputeEnvDesiredvCpus} ${SpotBidPercentage} ${AWS::StackName} ${CustomAMI} ${FileSystemId} ${QueueCapacityType} ${DisableSimultaneousMultithreading} $efaEnabled ${PlacementGroup}
233265
/opt/aws/bin/cfn-signal --exit-code 0 --resource EC2Instance --region ${AWS::Region} --stack ${AWS::StackName}
234266
sleep 10
235267
aws ec2 stop-instances --instance-ids `wget -qO- http://instance-data/latest/meta-data/instance-id`

‎config/settings-config.json‎

Lines changed: 37 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -220,9 +220,14 @@
220220
"running": {
221221
"connect": [
222222
{
223-
"menu": "SSH to Pcluster HeadNode",
224-
"imageUrl": "../../assets/images/technology@2x.png",
223+
"menu": "Remote Desktop",
224+
"imageUrl": "../../assets/images/Screen-icon.png",
225225
"outputsRequired": []
226+
},
227+
{
228+
"menu": "SSH Terminal",
229+
"imageUrl": "../../assets/images/technology@2x.png",
230+
"outputsRequired": []
226231
}
227232
],
228233
"action": [
@@ -251,12 +256,22 @@
251256
"default": {
252257
"connect": [
253258
{
254-
"menu": "SSH to Pcluster HeadNode",
259+
"menu": "Remote Desktop",
260+
"imageUrl": "../../assets/images/Screen-icon.png",
261+
"outputsRequired": []
262+
},
263+
{
264+
"menu": "SSH Terminal",
255265
"imageUrl": "../../assets/images/technology@2x.png",
256266
"outputsRequired": []
257267
}
258268
],
259269
"action": [
270+
{
271+
"action":"Start",
272+
"imageUrl": "../../assets/images/play@2x.png",
273+
"outputsRequired": []
274+
},
260275
{
261276
"action": "Stop",
262277
"imageUrl": "../../assets/images/no-stopping@2x.png",
@@ -287,8 +302,8 @@
287302
"terminated"
288303
]
289304
},
290-
"rstudio": {
291-
"actions": {
305+
"rstudio": {
306+
"actions": {
292307
"running": {
293308
"connect": [
294309
{
@@ -778,6 +793,19 @@
778793
],
779794
"action": []
780795
},
796+
"external": {
797+
"connect":[
798+
{
799+
"menu": "Explore",
800+
"imageUrl": "../../assets/images/direction@2x.png"
801+
},
802+
{
803+
"menu": "Link",
804+
"imageUrl": "../../assets/images/link@2x.png"
805+
}
806+
],
807+
"action":[]
808+
},
781809
"Personal": {
782810
"connect": [
783811
{
@@ -850,6 +878,10 @@
850878
"key": "studyType",
851879
"value": "Public"
852880
},
881+
{
882+
"key": "studyType",
883+
"value": "external"
884+
},
853885
{
854886
"key": "studyType",
855887
"value": "Personal"

‎deploy.sh‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -202,6 +202,11 @@ scripts/updatessmpaths.sh "$region" "$localhome"
202202
BUCKET_TEST=$(aws s3api head-bucket --bucket "$bucketname" 2>&1)
203203
if [ -z "$BUCKET_TEST" ]; then
204204
echo "Bucket $bucketname exists, Hit Enter to continue, Ctrl-C to exit"
205+
bucket_region=$(aws s3api get-bucket-location --bucket "$bucketname" | jq -r ".LocationConstraint")
206+
if [[ "$bucket_region" != "$region" ]]; then
207+
echo "The given bucket $bucketname is not in $region. Please provide a bucket in this region."
208+
exit 1
209+
fi
205210
read -r a && echo "Copying files to bucket $bucketname"
206211
else
207212
echo "An S3 bucket with name $bucketname doesn't exist in current AWS account. Creating..."

‎docker-compose.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ services:
1818
- sp2net
1919

2020
cc-3102:
21-
image: 045938549113.dkr.ecr.us-east-2.amazonaws.com/researchportal:_fd_1.12.1_b1244
21+
image: 045938549113.dkr.ecr.us-east-2.amazonaws.com/researchportal:_fd_1.13.0_b1319
2222
secrets:
2323
- source: sp2prod-config.json
2424
target: /rlc/cc/server/app/config/config.json
@@ -42,7 +42,7 @@ services:
4242
command: /bin/bash start.sh
4343

4444
scheduler-3102:
45-
image: 045938549113.dkr.ecr.us-east-2.amazonaws.com/researchportal:_fd_1.12.1_b1244
45+
image: 045938549113.dkr.ecr.us-east-2.amazonaws.com/researchportal:_fd_1.13.0_b1319
4646
secrets:
4747
- source: sp2prod-config.json
4848
target: /rlc/cc/server/app/config/config.json

0 commit comments

Comments
 (0)