Skip to content

Commit be4f75b

Browse files
twtaylortwtaylor
committed
initial commit
0 parents  commit be4f75b

632 files changed

Lines changed: 106004 additions & 0 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.commitlintrc‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
{
2+
"extends": [
3+
"@commitlint/config-conventional"
4+
]
5+
}

‎.dockerignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
node_modules
2+
.git
3+
.gitignore

‎.editorconfig‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
[*]
2+
end_of_line = lf
3+
insert_final_newline = true
4+
charset = utf-8
5+
indent_style = space
6+
indent_size = 2

‎.github/CONTRIBUTING.md‎

Lines changed: 165 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,165 @@
1+
This project is organized as follows:
2+
3+
- `apps` This directory is the home of all applications.
4+
- `packages` This directory is the home of all shared code, used by more than one app.
5+
6+
We use `pnpm` to manage this monorepo.
7+
8+
# apps/api
9+
10+
The following describes the `apps/api` project.
11+
12+
- `src/global` This directory is for functionality common across all API versions. If you need to share a utility, constant, type etc across multiple versions of the Sort API, put that here. Top level / unversioned API routes (e.g. `/robots.txt`) belong here too.
13+
- `src/v2` Version two of the Sort API.
14+
15+
Each of the above directories follow the following subdirectory structure:
16+
17+
- `/constants` Version specific values which are not intended to change.
18+
- `/controllers` Version specific business logic.
19+
- `/mocks` Version specific mocks used in tests.
20+
- `/routes` Version specific API route definitions which map public API routes to controllers.
21+
- `/services` Version specific 3rd party service abstractions. This is where your API version specific Database/ChatGPT/Discord etc abstractions belong.
22+
- `/types` Version specific TypeScript types.
23+
- `/utils` Version specific shared helpers.
24+
25+
All mocks, services, types and utils should be added to `packages/shared` unless
26+
intended to _only_ be used in the API.
27+
28+
## File names
29+
30+
File naming convention is: `lower-case.topic.ts`.
31+
32+
- Words are lowercase and separated with `-`.
33+
34+
### Topics
35+
36+
File names use the `topic` naming convention, where the singular version of the
37+
top-level version directory name (`global`, `v2`, etc) is placed at the end of the filename before the
38+
file extention. This makes it easier to see the purpose of each file when you
39+
have multiple files open at the same time.
40+
41+
```
42+
v2/
43+
├─ services/
44+
│ ├─ user.service.ts
45+
├─ controllers/
46+
│ ├─ user.controller.ts
47+
│ ├─ organizations/
48+
│ │ ├─ invite.controller.ts # Note "controller" not "organization"
49+
├─ routes/
50+
│ ├─ user.route.ts
51+
```
52+
53+
### Test file names
54+
55+
Tests are colocated in the same directory as the functionality. Test file names
56+
are identical to the file they are testing except they end with `.test.ts`.
57+
Example:
58+
59+
```
60+
services/
61+
├─ user.service.ts
62+
├─ user.service.test.ts # <-- Contains the tests for the functionality in user.service.ts
63+
```
64+
65+
## Snake case
66+
67+
API input and response fields must use `snake_case`.
68+
69+
## Test users
70+
71+
If you are testing the [UI][] and the API together locally, you'll first need
72+
to seed your local database with our test user. Run the following command from
73+
the `apps/api` directory:
74+
75+
```sh
76+
pnpm db:reset
77+
```
78+
79+
This will run the latest version of the database locally and add our test user account to it. Now you can
80+
[log in through your local UI][local-login] without trouble.
81+
82+
## OpenAPI
83+
84+
We generate an [OpenAPI spec](https://swagger.io/specification/) from our route/controller schemas. The spec is part
85+
of our public product offering so care should be taken to ensure it's accuracy.
86+
Follow these guidelines when added or editing API endpoints.
87+
88+
### 1. OpenAPI descriptions
89+
90+
The values for all OpenAPI endpoint descriptions are found in the `src/docs`
91+
directory. Each endpoint has a file in this directory named after it's
92+
`operationId` and suffixed with `.md`. Using dedicated markdown files allows us
93+
to get full markdown syntax highlighting in our editor and no need for escaping
94+
strings etc.
95+
96+
For example, if the `operationId` is `list_changes`, then the documentation file
97+
is found in `apps/api/src/docs/list_changes.md`.
98+
99+
The values of these files are included in the generated OpenAPI spec automatically.
100+
101+
If you are adding a new endpoint, make sure to add a description file.
102+
103+
Guidelines for writing descriptions: A full sentence or paragraph(s) describing
104+
to developers the purpose of the endpoint and anything helpful. Use markdown in
105+
this field for things like links to our docs.sort.xyz site. End each sentence
106+
with a period. Take a look at some other files for examples.
107+
108+
### 2. Ensure the endpoint schema includes the following fields
109+
- `operationId` A unique name for the endpoint
110+
- `summary` A sentence fragment about what the endpoint does. Do not end with a period.
111+
112+
For `summary` and `description` fields, capitalize our product entities like `Organization`,
113+
`Issue` and `Change Request`.
114+
115+
Example
116+
117+
```ts
118+
export const getOrganizationBySlugSchema = {
119+
headers: AuthHeadersSchema,
120+
params: ParamsSchema,
121+
summary: 'Get a Sort Organization',
122+
// description -> This comes from src/docs/get_organization.md
123+
operationId: 'get_organization',
124+
response: {
125+
200: createMessageSchema(
126+
'get_organization',
127+
Type.Object({
128+
organization: OrganizationSchema
129+
})
130+
),
131+
400: ValidationErrorSchema,
132+
401: GeneralErrorSchema,
133+
500: GeneralErrorSchema
134+
}
135+
} satisfies FastifySchema
136+
```
137+
138+
You can preview what this will look like by running the API locally and visiting: http://localhost:8080/docs/static/index.html
139+
140+
### 2. Hide endpoints which are not ready or not intended for public use
141+
142+
You can prevent endpoints from being included in our public OpenAPI spec by
143+
setting `hide: true` in the endpoint schema.
144+
145+
```ts
146+
export const getMyHiddenThingsSchema = {
147+
headers: AuthHeadersSchema,
148+
params: ParamsSchema,
149+
summary: 'Get something hidden from the public',
150+
description: 'Gets hidden stuff.',
151+
operationId: 'getMyHiddenThings',
152+
hide: true, // <===
153+
...
154+
}
155+
```
156+
157+
[UI]: https://github.com/sortxyz/sorthub
158+
[local-login]: https://github.com/sortxyz/sorthub#common-issues
159+
160+
# apps/worker
161+
162+
The following describes the `apps/worker` project.
163+
164+
All mocks, services, types and utils should be added to `packages/shared` unless
165+
intended to _only_ be used in the worker.

‎.github/dependabot.yml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
version: 2
2+
updates:
3+
- package-ecosystem: 'npm'
4+
directory: '/'
5+
schedule:
6+
interval: 'daily'
7+
- package-ecosystem: 'github-actions'
8+
directory: '/'
9+
schedule:
10+
interval: 'daily'

‎.github/workflows/ci.yml‎

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
name: Continuous Integration
2+
3+
on:
4+
push:
5+
branches:
6+
- main
7+
pull_request:
8+
9+
jobs:
10+
test-workspace:
11+
name: Build, lint and test workspace
12+
runs-on: ubuntu-latest-8-core
13+
env:
14+
TEST_SNOWFLAKE_HYBRID_CONNECTION_STRING: ${{ secrets.TEST_SNOWFLAKE_HYBRID_CONNECTION_STRING }}
15+
TEST_SNOWFLAKE_HYBRID_USER: ${{ secrets.TEST_SNOWFLAKE_HYBRID_USER }}
16+
TEST_SNOWFLAKE_UNLOCK_CONNECTION_STRING: ${{ secrets.TEST_SNOWFLAKE_UNLOCK_CONNECTION_STRING }}
17+
TEST_SNOWFLAKE_V4A_CONNECTION_STRING: ${{ secrets.TEST_SNOWFLAKE_V4A_CONNECTION_STRING }}
18+
TEST_POSTGRES_AIR_QUALITY_CONNECTION_STRING: ${{ secrets.TEST_POSTGRES_AIR_QUALITY_CONNECTION_STRING }}
19+
steps:
20+
- uses: actions/checkout@v4
21+
with:
22+
token: ${{ secrets.SORT_DB_PAT }}
23+
- uses: pnpm/action-setup@v3
24+
- uses: actions/setup-node@v4
25+
with:
26+
node-version-file: ".nvmrc"
27+
cache: 'pnpm'
28+
- name: Get pnpm store directory
29+
shell: bash
30+
run: |
31+
echo "STORE_PATH=$(pnpm store path --silent)" >> $GITHUB_ENV
32+
- name: install dependencies
33+
run: |
34+
pnpm install --frozen-lockfile
35+
- name: lint project
36+
run: |
37+
pnpm run -r lint
38+
- name: start infra
39+
run: |
40+
pnpm run --filter '@sort/shared' infra:up
41+
- name: run packages/shared tests
42+
run: |
43+
pnpm run --filter '@sort/shared' test:all
44+
- name: run packages/config tests
45+
run: |
46+
pnpm run --filter '@sort/config' test
47+
- name: run packages/logger tests
48+
run: |
49+
pnpm run --filter '@sort/logger' test
50+
- name: run apps/worker tests
51+
run: |
52+
pnpm run --filter '@sort/change-request-worker' test
53+
- name: run apps/api tests
54+
run: |
55+
pnpm run --filter 'sort-api-v2' test:all
56+
- name: stop infra
57+
run: |
58+
pnpm run --filter '@sort/shared' infra:down
59+
build-docker-images:
60+
if: github.event_name == 'pull_request'
61+
name: Build the Docker images
62+
runs-on: ubuntu-latest
63+
steps:
64+
- name: Download Trivy
65+
uses: aquasecurity/setup-trivy@v0.2.2 # B/c this download fails often, run first to avoid wasting time building
66+
with:
67+
cache: true
68+
version: v0.56.1
69+
- uses: actions/checkout@v4
70+
with:
71+
token: ${{ secrets.SORT_DB_PAT }}
72+
- name: docker build api
73+
run: |
74+
docker build --target api . -t api
75+
- name: Run Trivy vulnerability scanner on Docker API image
76+
uses: aquasecurity/trivy-action@d2a392a13760cb64cb6bbd31d4bed2a7d9a5298d
77+
env:
78+
TRIVY_DB_REPOSITORY: public.ecr.aws/aquasecurity/trivy-db
79+
TRIVY_JAVA_DB_REPOSITORY: public.ecr.aws/aquasecurity/trivy-java-db
80+
with:
81+
image-ref: api
82+
format: 'table'
83+
exit-code: '1'
84+
ignore-unfixed: true
85+
vuln-type: 'os,library'
86+
severity: 'CRITICAL'
87+
skip-setup-trivy: true
88+
- name: docker build change request worker
89+
run: |
90+
docker build --target worker . -t worker
91+
- name: Run Trivy vulnerability scanner on Docker Worker image
92+
uses: aquasecurity/trivy-action@d2a392a13760cb64cb6bbd31d4bed2a7d9a5298d
93+
env:
94+
TRIVY_DB_REPOSITORY: public.ecr.aws/aquasecurity/trivy-db
95+
TRIVY_JAVA_DB_REPOSITORY: public.ecr.aws/aquasecurity/trivy-java-db
96+
with:
97+
image-ref: worker
98+
format: 'table'
99+
exit-code: '1'
100+
ignore-unfixed: true
101+
vuln-type: 'os,library'
102+
severity: 'CRITICAL'
103+
skip-setup-trivy: true
104+
automerge:
105+
name: Merge dependabot's PRs
106+
needs: test-workspace
107+
runs-on: ubuntu-latest
108+
permissions:
109+
pull-requests: write
110+
contents: write
111+
steps:
112+
- uses: fastify/github-action-merge-dependabot@v3

‎.github/workflows/helm_lint.yml‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
name: Helm Lint
2+
3+
on:
4+
pull_request:
5+
branches:
6+
- main
7+
- develop
8+
paths:
9+
- "helm/**"
10+
- "kubernetes/**"
11+
12+
jobs:
13+
api-lint:
14+
name: Running Helm lint against the chart and values for the API repository
15+
runs-on: ubuntu-latest
16+
permissions:
17+
id-token: write
18+
contents: read
19+
steps:
20+
- name: Check out code
21+
uses: actions/checkout@v4
22+
- name: Helm Lint
23+
run: |
24+
ls kubernetes/api/ | grep -v "application.yml" | xargs -I % /bin/bash -c "echo ; echo 'Testing Values and Charts for % Environment'; helm lint helm/api -f kubernetes/api/%/values.yml || exit 255; sleep 1"
25+
26+
worker-lint:
27+
name: Running Helm lint against the chart and values for the worker repository
28+
runs-on: ubuntu-latest
29+
permissions:
30+
id-token: write
31+
contents: read
32+
steps:
33+
- name: Check out code
34+
uses: actions/checkout@v4
35+
- name: Helm Lint
36+
run: |
37+
ls kubernetes/worker/ | grep -v "application.yml" | xargs -I % /bin/bash -c "echo ; echo 'Testing Values and Charts for % Environment'; helm lint helm/worker -f kubernetes/worker/%/values.yml || exit 255; sleep 1"

0 commit comments

Comments
 (0)