Skip to content

Commit 3c6c1b9

Browse files
themaroltmariobalcaJoan Reyero
authored
Kubernetes architecture (#53)
Co-authored-by: Mário Balça <mario.balca@gmail.com> Co-authored-by: Joan Reyero <joan@crowd.dev>
1 parent 5a8b768 commit 3c6c1b9

216 files changed

Lines changed: 20191 additions & 1021 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/CI-node.yaml‎

Lines changed: 19 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ name: Node.js CI
33
on:
44
pull_request:
55
paths:
6-
- "backend/**"
6+
- 'backend/**'
77
jobs:
88
lint-format:
99
runs-on: ubuntu-latest
@@ -37,8 +37,6 @@ jobs:
3737
- name: Check formatting
3838
run: npx prettier --check .
3939

40-
41-
4240
tests-main:
4341
needs: lint-format
4442
runs-on: ubuntu-latest
@@ -47,16 +45,16 @@ jobs:
4745
shell: bash
4846
working-directory: ./backend
4947

50-
steps:
51-
- name: Check out repository code
52-
uses: actions/checkout@v2
48+
steps:
49+
- name: Check out repository code
50+
uses: actions/checkout@v2
51+
52+
- name: Install root dependencies
53+
run: npm ci
5354

54-
- name: Install root dependencies
55-
run: npm ci
56-
57-
- name: Run tests
58-
working-directory: ./backend
59-
run: npm test -- --testPathIgnorePatterns=serverless
55+
- name: Run tests
56+
working-directory: ./backend
57+
run: npm test -- --testPathIgnorePatterns=serverless
6058

6159
tests-serverless:
6260
needs: lint-format
@@ -67,18 +65,12 @@ jobs:
6765
working-directory: ./backend
6866

6967
steps:
70-
- name: Check out repository code
71-
uses: actions/checkout@v2
72-
73-
- name: Install dependencies
74-
run: npm ci && cd ./src/serverless/integrations && npm ci && cd ./../dbOperations && npm ci && cd ./../microservices/nodejs && npm ci
75-
76-
- name: change db port in env
77-
working-directory: ./backend
78-
run: sed -i -e 's/5433/'5434'/g' ./.env.test && sed -i -e 's/7701/'7702'/g' ./.env.test
79-
- name: change db port in docker compose
80-
working-directory: ./backend
81-
run: sed -i -e 's/5433/'5434'/g' ../docker/docker-compose.test.yaml && sed -i -e 's/7701/'7702'/g' ../docker/docker-compose.test.yaml
82-
- name: Run tests
83-
working-directory: ./backend
84-
run: npm test -- --testPathPattern="serverless\/"
68+
- name: Check out repository code
69+
uses: actions/checkout@v2
70+
71+
- name: Install dependencies
72+
run: npm ci && cd ./src/serverless/integrations && npm ci && cd ./../dbOperations && npm ci && cd ./../microservices/nodejs && npm ci
73+
74+
- name: Run tests
75+
working-directory: ./backend
76+
run: npm test -- --testPathPattern="serverless\/"

‎README.md‎

Lines changed: 33 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
<!-- PROJECT LOGO -->
2+
23
> **_IMPORTANT:_** This project is still under active development. Be aware that future releases can lead to breaking changes.
3-
<br>
4+
> <br>
45
56
<p align="center">
67
<a href="https://github.com/CrowdDotDev/crowd.dev">
@@ -30,23 +31,27 @@
3031

3132
## About crowd.dev
3233

33-
Crowd.dev is a suite of tools to analyze, grow, and leverage online communities in order to drive business results.
34+
Crowd.dev is a suite of tools to analyze, grow, and leverage online communities in order to drive business results.
3435

3536
In recent years, community has moved to the forefront of business. But building community is hard. In the beginning, organizations have to continuously find community members and maintain a high engagement among them. Once a community is thriving, organizations are stuck managing endless tools, working with incomplete data, and failing to bring the return of community back to their business.
3637

3738
Crowd.dev is here to change this. Self-hosted or hosted by us, with developers in mind, open to extensions, and with full control over your community's data.
3839

3940
## ✨ Features
40-
* Integrate with platforms like GitHub, Discord, Slack, Twitter or DEV to establish a single source of truth for your community
41-
* Get background information about your community members and manage them with tags and automated segmentation
42-
* Detect relevant conversations and publish them in a community help center to reduce duplicate questions and get your community's content listed on search engines <a href="https://open.crowd.dev/crowd">[example]</a>
43-
* Analyze your community, create custom metrics, organize them in reports and share them publicly with your community, your investors, or your team
41+
42+
- Integrate with platforms like GitHub, Discord, Slack, Twitter or DEV to establish a single source of truth for your community
43+
- Get background information about your community members and manage them with tags and automated segmentation
44+
- Detect relevant conversations and publish them in a community help center to reduce duplicate questions and get your community's content listed on search engines <a href="https://open.crowd.dev/crowd">[example]</a>
45+
- Analyze your community, create custom metrics, organize them in reports and share them publicly with your community, your investors, or your team
4446

4547
## 🔔 Stay up-to-date
48+
4649
Crowd.dev is still in beta and we ship new features every month. To stay in the loop, leave us a star and subscribe to our <a href="https://crowd.dev/newsletter">monthly newsletter</a>. Thanks a lot! ❤️
4750

4851
## 🚀 Getting started
52+
4953
### Cloud version
54+
5055
Our <a href="https://crowd.dev/#waitlist">cloud version</a> is a fast, easy and free way to get started with crowd.dev. We're currently still in closed beta but onboard new communities every week.
5156

5257
### Self-hosted version
@@ -62,27 +67,33 @@ We currently support four integrations for self-hosting: GitHub, Twitter, Discor
6267
### Development environment
6368

6469
#### <a name="requirements">Requirements</a>
65-
- Node v16.16.0
70+
71+
- Node v16.16.0
6672
- Docker and docker-compose
6773

6874
#### <a name="getting_started">Getting started</a>
6975

7076
1. Get the mono repo from GitHub
7177

72-
```
78+
```shell
7379
git clone git@github.com:CrowdDotDev/crowd.dev.git
7480
```
7581

7682
2. Run the start script
83+
84+
```shell
85+
cd scripts
86+
./cli start
7787
```
78-
bash start.sh
79-
```
88+
8089
App will be available at https://app.localhost
8190

8291
For more information on development, you can <a href="https://docs.crowd.dev/docs/local-development">check our docs</a>.
8392

8493
## 🗺️ Roadmap
94+
8595
Our goal is to build the most powerful platform for building developer communities out there. In the upcoming months we'll focus on the following features:
96+
8697
- [ ] Completely refurbished user interface, e.g. with more background information on members
8798
- [ ] More integrations (LinkedIn, Reddit, Stack Overflow, etc.)
8899
- [ ] Organization module to find out more about the companies behind your members
@@ -92,21 +103,25 @@ Our goal is to build the most powerful platform for building developer communiti
92103
You can find more features on our [public roadmap](https://crowd.dev/roadmap). Feel free to also [open an issue](https://crowd.dev/open-an-issue) for anything you're missing.
93104

94105
## ✍️ Contribution
106+
95107
There are many ways you can contribute to crowd.dev! Here are a few options:
96108

97-
* Star this repo and follow us on <a href="https://crowd.dev/twitter">Twitter</a>.
98-
* Create issues every time you feel something is missing or goes wrong.
99-
* Upvote issues with 👍 reaction so we know what's the demand for particular issue to prioritize it within roadmap.
109+
- Star this repo and follow us on <a href="https://crowd.dev/twitter">Twitter</a>.
110+
- Create issues every time you feel something is missing or goes wrong.
111+
- Upvote issues with 👍 reaction so we know what's the demand for particular issue to prioritize it within roadmap.
100112

101113
If you would like to contribute to the development of the project please reach out first (e.g. via Discord). All contributions are highly appreciated. 🙏
102114

103115
## ⚖️ License
116+
104117
Distributed under the AGPLv3 License. See `LICENSE` for more information.
105118

106119
## 💌 Acknowledgements
120+
107121
Crowd.dev is powered by these awesome projects:
108-
* <a href="https://github.com/vuejs/vue">Vue.js</a>
109-
* <a href="https://github.com/tailwindlabs/tailwindcss">Tailwind CSS</a>
110-
* <a href="https://github.com/cube-js/cube.js">Cube.js</a>
111-
* <a href="https://github.com/superfaceai/one-sdk-js">Superface</a>
112-
* <a href="https://github.com/meilisearch/meilisearch">Meilisearch</a>
122+
123+
- <a href="https://github.com/vuejs/vue">Vue.js</a>
124+
- <a href="https://github.com/tailwindlabs/tailwindcss">Tailwind CSS</a>
125+
- <a href="https://github.com/cube-js/cube.js">Cube.js</a>
126+
- <a href="https://github.com/superfaceai/one-sdk-js">Superface</a>
127+
- <a href="https://github.com/meilisearch/meilisearch">Meilisearch</a>

‎backend/.dockerignore‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,4 +3,6 @@
33
**/venv*
44
**/.webpack
55
**/.serverless
6-
**/.cubestore
6+
**/.cubestore
7+
**/.idea
8+
**/.vscode

‎backend/.env.dist.compose‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
# Api settings
2+
CROWD_PREMIUM_API_URL=http://premium-api:5000
3+
4+
# SQS settings
5+
CROWD_SQS_HOST="sqs"
6+
CROWD_SQS_NODEJS_WORKER_QUEUE="http://sqs:9324/000000000000/nodejs-worker.fifo"
7+
CROWD_SQS_PYTHON_WORKER_QUEUE="http://sqs:9324/000000000000/python-worker.fifo"
8+
CROWD_SQS_PREMIUM_PYTHON_WORKER_QUEUE="http://sqs:9324/000000000000/premium-python-worker.fifo"
9+
10+
# S3 settings
11+
CROWD_S3_HOST="s3"
12+
13+
# Db settings
14+
CROWD_DB_READ_HOST="db"
15+
CROWD_DB_WRITE_HOST="db"
16+
17+
# Search engine settings
18+
CROWD_SEARCH_ENGINE_HOST="http://search-engine:7700"
19+
20+
# CubeJS settings
21+
CROWD_CUBEJS_URL="http://cubejs:4000"

‎backend/.env.dist.local‎

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
# Global settings
2+
KUBE_MODE=1
3+
CROWD_EDITION=community
4+
TENANT_MODE=multi
5+
6+
# API settings
7+
CROWD_API_URL=https://anton.localhost/api
8+
CROWD_API_FRONTEND_URL=https://app.localhost
9+
CROWD_API_FRONTEND_URL_WITH_SUBDOMAINS=
10+
CROWD_API_JWT_SECRET=your-secret
11+
CROWD_API_JWT_EXPIRES_IN='100 years'
12+
CROWD_PREMIUM_API_URL=http://localhost:5000
13+
14+
# SQS settings
15+
CROWD_SQS_HOST=localhost
16+
CROWD_SQS_PORT=9324
17+
CROWD_SQS_NODEJS_WORKER_QUEUE=http://localhost:9324/000000000000/nodejs-worker.fifo
18+
CROWD_SQS_PYTHON_WORKER_QUEUE=http://localhost:9324/000000000000/python-worker.fifo
19+
CROWD_SQS_PREMIUM_PYTHON_WORKER_QUEUE=http://localhost:9324/000000000000/premium-python-worker.fifo
20+
CROWD_SQS_AWS_ACCOUNT_ID=000000000000
21+
CROWD_SQS_AWS_ACCESS_KEY_ID=x
22+
CROWD_SQS_AWS_SECRET_ACCESS_KEY=x
23+
CROWD_SQS_AWS_REGION=elasticmq
24+
25+
# S3 settings
26+
CROWD_S3_HOST=localhost
27+
CROWD_S3_PORT=9000
28+
CROWD_S3_INTEGRATION_ASSETS_BUCKET=crowd-integrations-assets
29+
CROWD_S3_MICROSERVICES_ASSETS_BUCKET=crowd-microservices-assets
30+
CROWD_S3_AWS_ACCOUNT_ID=000000000000
31+
CROWD_S3_AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
32+
CROWD_S3_AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
33+
CROWD_S3_AWS_REGION=eu-central-1
34+
35+
# DB settings
36+
CROWD_DB_READ_HOST=localhost
37+
CROWD_DB_WRITE_HOST=localhost
38+
CROWD_DB_PORT=5432
39+
CROWD_DB_USERNAME=postgres
40+
CROWD_DB_PASSWORD=example
41+
CROWD_DB_DATABASE=crowd-web
42+
43+
# CubeJS settings
44+
CROWD_CUBEJS_URL=http://localhost:4000
45+
CROWD_CUBEJS_JWT_SECRET=137ea167812145c6d77452a58d7dd29b
46+
CROWD_CUBEJS_JWT_EXPIRY=2h
47+
48+
# Search engine settings
49+
CROWD_SEARCH_ENGINE_HOST=http://localhost:7700
50+
CROWD_SEARCH_ENGINE_API_KEY=dev
51+
CROWD_SEARCH_ENGINE_CONVERSATIONS_INDEX=conversations
52+
CROWD_SEARCH_ENGINE_SETTINGS_INDEX=settings
53+
54+
# Segment settings
55+
CROWD_SEGMENT_WRITE_KEY=TdX3BLaZuHpHyzN2lcDiNiRHDSH9Piyl
56+
57+
# Netlify settings
58+
CROWD_NETLIFY_API_KEY=
59+
CROWD_NETLIFY_SITE_DOMAIN=open.localhost
60+
61+
# Sendgrid settings
62+
CROWD_SENDGRID_KEY=
63+
CROWD_SENDGRID_EMAIL_FROM=
64+
CROWD_SENDGRID_NAME_FROM=
65+
CROWD_SENDGRID_TEMPLATE_EMAIL_ADDRESS_VERIFICATION=
66+
CROWD_SENDGRID_TEMPLATE_INVITATION=
67+
CROWD_SENDGRID_TEMPLATE_PASSWORD_RESET=
68+
CROWD_SENDGRID_TEMPLATE_WEEKLY_ANALYTICS=
69+
CROWD_SENDGRID_WEEKLY_ANALYTICS_UNSUBSCRIBE_GROUP_ID=
70+
71+
# Stripe settings
72+
CROWD_STRIPE_PRICE_PREMIUM=
73+
CROWD_STRIPE_PRICE_ENTERPRISE=
74+
CROWD_STRIPE_SECRET_KEY=
75+
CROWD_STRIPE_WEBHOOK_SIGNING_SECRET=
76+
77+
# Twitter settings
78+
CROWD_TWITTER_CLIENT_ID=
79+
CROWD_TWITTER_CLIENT_SECRET=
80+
81+
# Slack settings
82+
CROWD_SLACK_CLIENT_ID=
83+
CROWD_SLACK_CLIENT_SECRET=
84+
85+
# Google settings
86+
CROWD_GOOGLE_CLIENT_ID=
87+
CROWD_GOOGLE_CLIENT_SECRET=
88+
CROWD_GOOGLE_CALLBACK_URL=
89+
90+
# Facebook settings
91+
CROWD_FACEBOOK_CLIENT_ID=
92+
CROWD_FACEBOOK_CLIENT_SECRET=
93+
CROWD_FACEBOOK_CALLBACK_URL=
94+
95+
# Discord settings
96+
CROWD_DISCORD_TOKEN=
97+
98+
# Github settings
99+
CROWD_GITHUB_APP_ID=
100+
CROWD_GITHUB_CLIENT_ID=
101+
CROWD_GITHUB_CLIENT_SECRET=
102+
CROWD_GITHUB_PRIVATE_KEY=
103+
CROWD_GITHUB_WEBHOOK_SECRET=
104+
105+
# Cohere settings
106+
CROWD_COHERE_API_KEY=
107+
108+
# Vector settings
109+
CROWD_VECTOR_API_KEY=
110+
CROWD_VECTOR_INDEX=

‎backend/.env.test‎

100644100755
Lines changed: 6 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,6 @@
1-
NODE_ENV='test'
2-
DATABASE_USERNAME='postgres'
3-
DATABASE_DIALECT='postgres'
4-
DATABASE_PASSWORD='example'
5-
DATABASE_DATABASE='crowd-web'
6-
DATABASE_HOST_READ='localhost'
7-
DATABASE_HOST_WRITE='localhost'
8-
DATABASE_LOGGING='false'
9-
DATABASE_TRANSACTIONS='false'
10-
DATABASE_PORT='5433'
11-
AUTH_JWT_SECRET='jwt_secret'
12-
AUTH_JWT_EXPIRES_IN='1d'
13-
SERVICE='default'
14-
INTEGRATION_SERVICE='integrations'
15-
SEARCH_ENGINE_HOST='http://127.0.0.1:7701'
16-
SEARCH_ENGINE_API_KEY='masterKey'
1+
# DB settings
2+
CROWD_DB_PORT=5433
3+
4+
# Search engine settings
5+
CROWD_SEARCH_ENGINE_HOST=http://localhost:7701
6+
CROWD_SEARCH_ENGINE_API_KEY=masterKey

‎backend/Dockerfile.kube‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# syntax = docker/dockerfile:experimental
2+
FROM node:16-alpine
3+
4+
WORKDIR /usr/crowd/backend
5+
COPY package-lock.json package.json ./
6+
7+
RUN npm install
8+
9+
COPY . .

‎backend/Dockerfile.kube.dockerignore‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
**/.git
2+
**/node_modules
3+
**/venv*
4+
**/.webpack
5+
**/.serverless
6+
**/.cubestore
7+
**/.env
8+
**/.env.*
9+
**/.idea
10+
**/.vscode
11+
**/dist
12+
server-config/
13+
util/
14+
src/serverless/microservices/python

0 commit comments

Comments
 (0)