Skip to content

Commit 873b10e

Browse files
committed
run prettier
1 parent 0611b65 commit 873b10e

6 files changed

Lines changed: 114 additions & 102 deletions

File tree

.github/pull_request_template.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ Have any new tests been added as part of this issue? If not, try to explain why
1616

1717
- [ ] Yes
1818
- [ ] No
19-
Please write a brief description of why test coverage is not necessary here.
19+
Please write a brief description of why test coverage is not necessary here.
2020
- [ ] Not as part of this ticket. (Could be done at a later point)
2121

2222
### Documentation
@@ -26,7 +26,7 @@ as new code is added, rather than leaving it for the future.
2626

2727
- [ ] Yes
2828
- [ ] No
29-
Please write a brief description of why documentation is not necessary here.
29+
Please write a brief description of why documentation is not necessary here.
3030
- [ ] Not as part of this ticket. (Could be done at a later point)
3131

3232
### Related issues

README.md

Lines changed: 30 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -5,33 +5,33 @@ A Python utility used to archive old, unused GitHub repositories from an organis
55
## Table of Contents
66

77
- [GitHub Repository Archive Script](#github-repository-archive-script)
8-
- [Table of Contents](#table-of-contents)
9-
- [Prerequisites](#prerequisites)
10-
- [Makefile](#makefile)
11-
- [Development](#development)
12-
- [Running the Project](#running-the-project)
13-
- [Containerised (Recommended)](#containerised-recommended)
14-
- [Outside of a Container (Development only)](#outside-of-a-container-development-only)
15-
- [Deployment](#deployment)
16-
- [Overview](#overview)
17-
- [Deployment Prerequisites](#deployment-prerequisites)
18-
- [Storing the Container on AWS Elastic Container Registry (ECR)](#storing-the-container-on-aws-elastic-container-registry-ecr)
19-
- [Deploying the Lambda](#deploying-the-lambda)
20-
- [Destroying / Removing the Lambda](#destroying--removing-the-lambda)
21-
- [Linting and Testing](#linting-and-testing)
22-
- [GitHub Actions](#github-actions)
23-
- [Running Tests Locally](#running-tests-locally)
8+
- [Table of Contents](#table-of-contents)
9+
- [Prerequisites](#prerequisites)
10+
- [Makefile](#makefile)
11+
- [Development](#development)
12+
- [Running the Project](#running-the-project)
13+
- [Containerised (Recommended)](#containerised-recommended)
14+
- [Outside of a Container (Development only)](#outside-of-a-container-development-only)
15+
- [Deployment](#deployment)
16+
- [Overview](#overview)
17+
- [Deployment Prerequisites](#deployment-prerequisites)
18+
- [Storing the Container on AWS Elastic Container Registry (ECR)](#storing-the-container-on-aws-elastic-container-registry-ecr)
19+
- [Deploying the Lambda](#deploying-the-lambda)
20+
- [Destroying / Removing the Lambda](#destroying--removing-the-lambda)
21+
- [Linting and Testing](#linting-and-testing)
22+
- [GitHub Actions](#github-actions)
23+
- [Running Tests Locally](#running-tests-locally)
2424

2525
## Prerequisites
2626

2727
- A Docker Daemon (Colima is recommended)
28-
- [Colima](https://github.com/abiosoft/colima)
28+
- [Colima](https://github.com/abiosoft/colima)
2929
- Terraform (For deployment)
30-
- [Terraform](https://www.terraform.io/)
30+
- [Terraform](https://www.terraform.io/)
3131
- Python >3.12
32-
- [Python](https://www.python.org/)
32+
- [Python](https://www.python.org/)
3333
- Make
34-
- [GNU make](https://www.gnu.org/software/make/manual/make.html#Overview)
34+
- [GNU make](https://www.gnu.org/software/make/manual/make.html#Overview)
3535

3636
## Makefile
3737

@@ -119,7 +119,7 @@ Before the doing the following, make sure your Daemon is running. If using Colim
119119
When running the container, you are required to pass some environment variable.
120120

121121
| Variable | Description |
122-
|-----------------------------|-------------------------------------------------------------------------------------------|
122+
| --------------------------- | ----------------------------------------------------------------------------------------- |
123123
| GITHUB_ORG | The organisation you would like to run the tool in. |
124124
| GITHUB_APP_CLIENT_ID | The Client ID for the GitHub App which the tool uses to authenticate with the GitHub API. |
125125
| AWS_DEFAULT_REGION | The AWS Region which the Secret Manager Secret is in. |
@@ -168,7 +168,7 @@ To run the Lambda function outside of a container, we need to execute the `handl
168168
...
169169
```
170170

171-
**Please Note:** If uncommenting the above in `main.py`, make sure you re-comment the code *before* pushing back to GitHub.
171+
**Please Note:** If uncommenting the above in `main.py`, make sure you re-comment the code _before_ pushing back to GitHub.
172172

173173
2. Export the required environment variables:
174174

@@ -268,6 +268,7 @@ Within the terraform directory, there is a [service](./terraform/service/) subdi
268268
```
269269

270270
2. Fill out the appropriate environment variables file
271+
271272
- `env/dev/dev.tfvars` for sdp-dev.
272273
- `env/prod/prod.tfvars` for sdp-prod.
273274

@@ -328,34 +329,41 @@ terraform destroy -var-file=env/dev/dev.tfvars
328329
### Deployments with Concourse
329330

330331
#### Allowlisting your IP
332+
331333
To setup the deployment pipeline with concourse, you must first allowlist your IP address on the Concourse
332334
server. IP addresses are flushed everyday at 00:00 so this must be done at the beginning of every working day
333335
whenever the deployment pipeline needs to be used. Follow the instructions on the Confluence page (SDP Homepage > SDP Concourse > Concourse Login) to
334336
login. All our pipelines run on sdp-pipeline-prod, whereas sdp-pipeline-dev is the account used for
335337
changes to Concourse instance itself. Make sure to export all necessary environment variables from sdp-pipeline-prod (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN).
336338

337339
#### Setting up a pipeline
340+
338341
When setting up our pipelines, we use ecs-infra-user on sdp-dev to be able to interact with our infrastructure on AWS. The credentials for this are stored on
339342
AWS Secrets Manager so you do not need to set up anything yourself.
340343

341344
To set the pipeline, run the following script:
345+
342346
```bash
343347
chmod u+x ./concourse/scripts/set_pipeline.sh
344348
./concourse/scripts/set_pipeline.sh github-repo-archive-script
345349
```
350+
346351
Note that you only have to run chmod the first time running the script in order to give permissions.
347352
This script will set the branch and pipeline name to whatever branch you are currently on. It will also set the image tag on ECR to the current commit hash at the time of setting the pipeline.
348353

349354
The pipeline name itself will usually follow a pattern as follows: `<repo-name>-<branch-name>`
350355
If you wish to set a pipeline for another branch without checking out, you can run the following:
356+
351357
```bash
352358
./concourse/scripts/set_pipeline.sh github-repo-archive-script <branch_name>
353359
```
354360

355361
If the branch you are deploying is "main" or "master", it will trigger a deployment to the sdp-prod environment. To set the ECR image tag, you must draft a Github release pointing to the latest release of the main/master branch that has a tag in the form of vX.Y.Z. Drafting up a release will automatically deploy the latest version of the main/master branch with the associated release tag, but you can also manually trigger a build through the Concourse UI or the terminal prompt.
356362

357363
#### Triggering a pipeline
364+
358365
Once the pipeline has been set, you can manually trigger a build on the Concourse UI, or run the following command:
366+
359367
```bash
360368
fly -t aws-sdp trigger-job -j github-repo-archive-script-<branch-name>/build-and-push
361369
```

concourse/ci.yml

Lines changed: 73 additions & 73 deletions
Original file line numberDiff line numberDiff line change
@@ -1,78 +1,78 @@
11
resources:
2-
- name: resource-repo
3-
type: git
4-
icon: github
5-
source:
6-
uri: https://github.com/ONS-Innovation/github-repository-archive-script
7-
username: ((github_access_token))
8-
password: x-oauth-basic
9-
branch: ((branch))
2+
- name: resource-repo
3+
type: git
4+
icon: github
5+
source:
6+
uri: https://github.com/ONS-Innovation/github-repository-archive-script
7+
username: ((github_access_token))
8+
password: x-oauth-basic
9+
branch: ((branch))
1010

1111
jobs:
12-
- name: build-and-push
13-
public: true
14-
plan:
15-
- get: resource-repo
16-
timeout: 5m
17-
- task: build-image
18-
privileged: true
19-
config:
20-
platform: linux
21-
image_resource:
22-
type: docker-image
23-
source:
24-
repository: hashicorp/terraform
25-
inputs:
26-
- name: resource-repo
27-
params:
28-
aws_account_id: ((aws_account_sdp_((env))))
29-
aws_role_arn: arn:aws:iam::((aws_account_sdp_((env)))):role/sdp-concourse-((env))
30-
secrets: ((sdp_((env))_github_repo_archive_secrets))
31-
run: # binary used to build the image
32-
path: sh
33-
args:
34-
- -cx
35-
- |
36-
apk add --no-cache aws-cli podman jq iptables curl
12+
- name: build-and-push
13+
public: true
14+
plan:
15+
- get: resource-repo
16+
timeout: 5m
17+
- task: build-image
18+
privileged: true
19+
config:
20+
platform: linux
21+
image_resource:
22+
type: docker-image
23+
source:
24+
repository: hashicorp/terraform
25+
inputs:
26+
- name: resource-repo
27+
params:
28+
aws_account_id: ((aws_account_sdp_((env))))
29+
aws_role_arn: arn:aws:iam::((aws_account_sdp_((env)))):role/sdp-concourse-((env))
30+
secrets: ((sdp_((env))_github_repo_archive_secrets))
31+
run: # binary used to build the image
32+
path: sh
33+
args:
34+
- -cx
35+
- |
36+
apk add --no-cache aws-cli podman jq iptables curl
3737
38-
if [[ "((env))" == "prod" ]]; then
39-
tag=$(curl "https://api.github.com/repos/ONS-Innovation/github-repository-archive-script/releases" | jq -r '.[0].tag_name')
40-
export tag
41-
else
42-
export tag=((tag))
43-
fi
44-
git rev-parse --abbrev-ref HEAD
45-
chmod u+x ./resource-repo/concourse/scripts/assume_role.sh
46-
chmod u+x ./resource-repo/concourse/scripts/build_image.sh
47-
source ./resource-repo/concourse/scripts/assume_role.sh
48-
./resource-repo/concourse/scripts/build_image.sh
49-
timeout: 10m
50-
- task: terraform
51-
privileged: true
52-
config:
53-
platform: linux
54-
image_resource:
55-
type: docker-image
56-
source: {repository: hashicorp/terraform}
57-
inputs:
58-
- name: resource-repo
59-
params:
60-
secrets: ((sdp_((env))_github_repo_archive_secrets))
61-
github_access_token: ((github_access_token))
62-
run:
63-
path: sh
64-
args:
65-
- -cx
66-
- |
67-
apk add --no-cache jq curl
38+
if [[ "((env))" == "prod" ]]; then
39+
tag=$(curl "https://api.github.com/repos/ONS-Innovation/github-repository-archive-script/releases" | jq -r '.[0].tag_name')
40+
export tag
41+
else
42+
export tag=((tag))
43+
fi
44+
git rev-parse --abbrev-ref HEAD
45+
chmod u+x ./resource-repo/concourse/scripts/assume_role.sh
46+
chmod u+x ./resource-repo/concourse/scripts/build_image.sh
47+
source ./resource-repo/concourse/scripts/assume_role.sh
48+
./resource-repo/concourse/scripts/build_image.sh
49+
timeout: 10m
50+
- task: terraform
51+
privileged: true
52+
config:
53+
platform: linux
54+
image_resource:
55+
type: docker-image
56+
source: { repository: hashicorp/terraform }
57+
inputs:
58+
- name: resource-repo
59+
params:
60+
secrets: ((sdp_((env))_github_repo_archive_secrets))
61+
github_access_token: ((github_access_token))
62+
run:
63+
path: sh
64+
args:
65+
- -cx
66+
- |
67+
apk add --no-cache jq curl
6868
69-
if [[ "((env))" == "prod" ]]; then
70-
tag=$(curl "https://api.github.com/repos/ONS-Innovation/github-repository-archive-script/releases" | jq -r '.[0].tag_name')
71-
export tag
72-
else
73-
export tag=((tag))
74-
fi
75-
chmod u+x ./resource-repo/concourse/scripts/terraform_infra.sh
76-
export env=((env))
77-
./resource-repo/concourse/scripts/terraform_infra.sh
78-
timeout: 30m
69+
if [[ "((env))" == "prod" ]]; then
70+
tag=$(curl "https://api.github.com/repos/ONS-Innovation/github-repository-archive-script/releases" | jq -r '.[0].tag_name')
71+
export tag
72+
else
73+
export tag=((tag))
74+
fi
75+
chmod u+x ./resource-repo/concourse/scripts/terraform_infra.sh
76+
export env=((env))
77+
./resource-repo/concourse/scripts/terraform_infra.sh
78+
timeout: 30m

concourse/scripts/build_image.sh

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,8 +10,11 @@ container_image=$(echo "$secrets" | jq -r .container_image)
1010
# shellcheck disable=SC2154
1111
aws ecr get-login-password --region eu-west-2 | podman --storage-driver=vfs login --username AWS --password-stdin "${aws_account_id}".dkr.ecr.eu-west-2.amazonaws.com
1212

13+
# shellcheck disable=SC2154
1314
podman build -t "${container_image}":"${tag}" resource-repo
1415

16+
# shellcheck disable=SC2154
1517
podman tag "${container_image}":"${tag}" "${aws_account_id}".dkr.ecr.eu-west-2.amazonaws.com/"${container_image}":"${tag}"
18+
19+
# shellcheck disable=SC2154
1620
podman push "${aws_account_id}".dkr.ecr.eu-west-2.amazonaws.com/"${container_image}":"${tag}"
17-
# shellcheck disable=SC2154

concourse/scripts/set_pipeline.sh

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,8 @@ repo_name=${1}
55
if [[ $# -gt 1 ]]; then
66
branch=${2}
77
git rev-parse --verify "${branch}"
8-
if [[ $? -ne 0 ]]; then # shellcheck disable=SC2181
8+
# shellcheck disable=SC2181
9+
if [[ $? -ne 0 ]]; then
910
echo "Branch \"${branch}\" does not exist"
1011
exit 1
1112
fi
@@ -26,4 +27,3 @@ else
2627
fi
2728

2829
fly -t aws-sdp set-pipeline -c concourse/ci.yml -p "${repo_name}"-"${branch}" -v branch="${branch}" -v tag="${tag}" -v env="${env}"
29-

concourse/scripts/terraform_infra.sh

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,8 @@ AWS_SECRET_ACCESS_KEY=$aws_secret_access_key
2929
export AWS_ACCESS_KEY_ID
3030
export AWS_SECRET_ACCESS_KEY
3131

32-
git config --global url."https://x-access-token:$github_access_token@github.com/".insteadOf "https://github.com/" # shellcheck disable=
32+
# shellcheck disable=SC2086
33+
git config --global url."https://x-access-token:$github_access_token@github.com/".insteadOf "https://github.com/"
3334

3435
if [[ ${env} != "prod" ]]; then
3536
env="dev"
@@ -39,6 +40,7 @@ cd resource-repo/terraform/service
3940

4041
terraform init -backend-config=env/"${env}"/backend-"${env}".tfbackend -reconfigure
4142

43+
# shellcheck disable=SC2154
4244
terraform apply \
4345
-var "aws_account_id=$aws_account_id" \
4446
-var "aws_access_key_id=$aws_access_key_id" \
@@ -54,4 +56,3 @@ terraform apply \
5456
-var "lambda_memory=$lambda_memory" \
5557
-var "schedule=$schedule" \
5658
-auto-approve
57-
# shellcheck disable=SC2154

0 commit comments

Comments
 (0)