Skip to content

Commit e1e90a6

Browse files
rddefauwrddefauw
committed
First commit.
1 parent cef72e2 commit e1e90a6

141 files changed

Lines changed: 6140 additions & 5 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.gitignore‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
fargate-workshop.code-workspace
2+
.DS_Store
3+
.vscode/settings.json
4+
.env/
5+
labs/fargate-workshop-cdk/cdk.context.json
6+
labs/fargate-workshop-cdk/cdk.out/
7+
labs/fargate-workshop-cdk/fargate_workshop_cdk/__pycache__/
8+
.env
9+
cdk.out
10+
site/public/

‎README.md‎

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,12 @@
1-
## My Project
1+
# ECS Fargate Workshop
22

3-
TODO: Fill this README out!
3+
This workshop teaches you best practices for using ECS Fargate. It has three independent modules targeted at different use cases.
44

5-
Be sure to:
5+
* Development
6+
* Operations
7+
* Data engineering
68

7-
* Change the title in this README
8-
* Edit your repository description on GitHub
9+
The full documentation is available in the `site` folder, and is best viewed with [Hugo](https://learn.netlify.app/en/basics/). You can run Hugo locally by downloading Hugo for your platform, going to the `site` folder, and running `hugo serve`.
910

1011
## License Summary
1112

‎diagrams/Data Engineering with Fargate-Stream.png‎

152 KB
Loading

‎diagrams/Data Engineering with Fargate.png‎

147 KB
Loading

‎diagrams/Fargate Operations and Security.png‎

234 KB
Loading

‎diagrams/Microservices with Fargate.png‎

146 KB
Loading

‎diagrams/data-eng-lab-seq.png‎

56.8 KB
Loading

‎diagrams/data-eng-lab-seq.puml‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
@startuml
2+
3+
' Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. SPDX-License-Identifier: MIT-0
4+
5+
participant CDK #lightblue
6+
participant CodePipeline #lightblue
7+
8+
== Lab 1 ==
9+
CDK --> MSK: Deploy as prereq
10+
CDK --> DocumentDB: Deploy as prereq
11+
CDK --> S3: Deploy as prereq
12+
CDK --> DataGenerator: Deploy as prereq
13+
CDK --> SFTP: Deploy as prereq
14+
CDK --> CodePipeline: Deploy as prereq
15+
CodePipeline --> FargateCluster: Deploy
16+
== Lab 2 ==
17+
CodePipeline --> FargateService: Deploy
18+
== Lab 3 ==
19+
CodePipeline --> KafkaConnect: Deploy
20+
CodePipeline --> StreamProcessor: Deploy
21+
== Lab 4 ==
22+
CDK --> Scheduler: Deploy
23+
CodePipeline --> ScheduledTask: Deploy
24+
CodePipeline --> Batch: Deploy
25+
26+
@enduml

‎diagrams/data_engineering.puml‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
@startuml Data Engineering with Fargate
2+
3+
' Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. SPDX-License-Identifier: MIT-0
4+
5+
!define AWSPuml https://raw.githubusercontent.com/awslabs/aws-icons-for-plantuml/master/dist
6+
!include AWSPuml/AWSCommon.puml
7+
!include AWSPuml/NetworkingAndContentDelivery/all.puml
8+
!include AWSPuml/Compute/all.puml
9+
!include AWSPuml/Database/all.puml
10+
!include AWSPuml/Mobile/all.puml
11+
!include AWSPuml/Storage/all.puml
12+
!include AWSPuml/Analytics/all.puml
13+
!include AWSPuml/ApplicationIntegration/all.puml
14+
!include AWSPuml/ManagementAndGovernance/all.puml
15+
16+
skinparam frame {
17+
backgroundColor<<appmesh>> lightblue
18+
}
19+
20+
title Batch Processing with Fargate
21+
22+
package "Fargate" {
23+
ECSContainer1(schtask, "Scheduled Task", "Workers", "")
24+
ECSService(batchworker, "Batch Processors", "Workers", "")
25+
}
26+
package "Other AWS" {
27+
together {
28+
S3Bucket(batchbucket, "S3 Storage", "Object Storage", "outputs")
29+
S3Bucket(ingestbucket, "S3 Storage", "Object Storage", "inputs")
30+
}
31+
together {
32+
StepFunctions(sfn, "Step Functions", "Job Scheduler", "")
33+
CloudWatchEventTimeBased(cw, "Cloud Cron", "CloudWatch Alarm", "")
34+
}
35+
}
36+
component sftp as "SFTP Server"
37+
38+
sfn -[#blue]-> batchworker: Invoke
39+
ingestbucket -[#blue]-> batchworker: Process
40+
batchworker -[#blue]-> batchbucket: Store
41+
42+
cw -[#red]-> schtask: Invoke
43+
sftp -[#red]-> schtask: Process
44+
45+
Fargate -[hidden]> sftp
46+
cw -[hidden]> batchbucket
47+
48+
@enduml

‎diagrams/data_engineering_stream.puml‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
@startuml Data Engineering with Fargate
2+
3+
' Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. SPDX-License-Identifier: MIT-0
4+
5+
!define AWSPuml https://raw.githubusercontent.com/awslabs/aws-icons-for-plantuml/master/dist
6+
!include AWSPuml/AWSCommon.puml
7+
!include AWSPuml/NetworkingAndContentDelivery/all.puml
8+
!include AWSPuml/Compute/all.puml
9+
!include AWSPuml/Database/all.puml
10+
!include AWSPuml/Mobile/all.puml
11+
!include AWSPuml/Storage/all.puml
12+
!include AWSPuml/Analytics/all.puml
13+
!include AWSPuml/ApplicationIntegration/all.puml
14+
!include AWSPuml/ManagementAndGovernance/all.puml
15+
16+
skinparam frame {
17+
backgroundColor<<appmesh>> lightblue
18+
}
19+
20+
title Stream Processing with Fargate
21+
22+
actor "Users" as client
23+
package "DMZ" {
24+
ELBApplicationLoadBalancer(elb, "ALB", "load balancer", "Kafka Connect endpoint")
25+
}
26+
package "Fargate" {
27+
ECSService(kafkaconnect, "Kafka Connect", "Microservice", "")
28+
ECSService(kafkaconsumer, "Kafka Consumer", "Stream processor", "")
29+
}
30+
package "Other AWS" {
31+
S3Bucket(bucket, "Kafka S3 Storage", "Object Storage", "kafka data")
32+
ManagedStreamingforKafka(kafka, "Kafka Cluster", "Streaming Ingest", "")
33+
DocumentDBwithMongoDBcompatibility(docdb, "DocumentDB", "Mongo-compatible database", "")
34+
}
35+
36+
client --> elb
37+
elb --> kafkaconnect
38+
kafkaconnect --> bucket
39+
kafkaconnect <-> kafka
40+
41+
kafka --> kafkaconsumer
42+
kafkaconsumer --> docdb
43+
44+
@enduml

0 commit comments

Comments
 (0)