Skip to content

Commit 8414799

Browse files
tryshanktryshank
authored
Merge branch 'master' into fix/password-reset-null-check
2 parents 8577847 + fcc0d41 commit 8414799

119 files changed

Lines changed: 6880 additions & 5822 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/AI_PR_REVIEW_GUIDELINES.md‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# AI Pull Request Review Guidelines for Countly Server
2+
3+
These guidelines are for AI-based review bots (e.g., GitHub Copilot) to ensure all pull requests (PRs) to Countly Server meet project standards for quality, security, and maintainability.
4+
5+
---
6+
7+
## 1. General Code Quality
8+
- **Linting:** All code must pass ESLint and shellcheck validation. Reject PRs with lint errors.
9+
- **Comments:** Public functions must use JSDoc. Complex logic should be commented, but avoid redundant comments.
10+
- **Error Handling:** All errors must be handled and surfaced to the frontend or logs.
11+
12+
## 2. Backend (Node.js)
13+
- **API Security:** All endpoints must use validation from `api/utils/rights.js` (e.g., `validateRead`, `validateCreate`).
14+
- **Parameter Validation:** All input parameters must be validated and type-checked. Reject PRs with missing or unsafe validation.
15+
- **Cross-App Security:** All write/delete operations must check `app_id` to prevent cross-app access.
16+
- **MongoDB:** Use projections to limit returned fields. Use batchers for frequent/bulk operations. Create indexes for new collections.
17+
- **Audit Logging:** All create/update/delete actions must dispatch `/systemlogs` events.
18+
19+
## 3. Frontend (Vue.js)
20+
- **Component Naming:** Use PascalCase for JS, kebab-case for templates.
21+
- **Props/Events:** Use props down, events up. Do not modify parent state directly.
22+
- **Security:** Never use `v-html` with user input. Use `countlyCommon.encodeHtml()` for manual sanitization.
23+
- **Testing:** All interactive elements must have `data-test-id` attributes for UI testing.
24+
- **Computed Properties:** Prefer computed properties over watchers for derived state.
25+
26+
## 4. CSS & Styling
27+
- **SASS:** Use SCSS syntax and `@use` for imports. Do not use `@import`.
28+
- **BEM:** All new classes must use BEM naming with `cly-vue-` prefix.
29+
- **Bulma:** Use Bulma classes with `bu-` prefix for layout.
30+
- **No Deprecated Paths:** Do not add CSS to deprecated directories.
31+
32+
## 5. Security
33+
- **XSS:** All API output must be escaped. Frontend must treat API data as text. Never use `v-html` with unsanitized input.
34+
- **MongoDB Injection:** Always cast credentials to strings. Validate objects for MongoDB operators.
35+
- **File Uploads:** Only allow whitelisted file types. Sanitize all filenames.
36+
- **Command Line:** Use `spawn` with argument arrays, never `exec` with user input.
37+
38+
## 6. Testing & Documentation
39+
- **Tests:** PRs must include tests for new features, edge cases, and cleanup. Plugins must test app lifecycle hooks.
40+
- **Docs:** Update or add documentation for new/changed features, including JSDoc for public functions.
41+
42+
## 7. Plugin Development
43+
- **Structure:** Plugins must follow the standard structure (`api/api.js`, `frontend/app.js`, etc.).
44+
- **Lifecycle:** Plugins with collections must handle app create/delete and user data deletion events.
45+
46+
## 8. Pull Request Hygiene
47+
- **No Secrets:** PRs must not include credentials, secrets, or sensitive data.
48+
- **Changelogs:** Update `CHANGELOG.md` for user-facing changes.
49+
- **License:** All code must comply with AGPL-3.0 and Countly's Section 7 modifications.
50+
51+
---
52+
53+
**References:**
54+
- [CODING_GUIDELINES.md](../CODING_GUIDELINES.md)
55+
- [SECURITY.md](../docs/SECURITY.md)
56+
- [VUEJS_GUIDELINES.md](../docs/VUEJS_GUIDELINES.md)
57+
- [CSS_STYLE_GUIDE.md](../docs/CSS_STYLE_GUIDE.md)
58+
- [UI_TESTING.md](../docs/UI_TESTING.md)
59+
60+
---
61+
62+
**If a PR does not meet these requirements, request changes with specific feedback.**

‎.github/copilot-instructions.md‎

Lines changed: 250 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,250 @@
1+
# Countly Server - AI Coding Agent Instructions
2+
3+
## Project Overview
4+
Countly is a product analytics platform built with **Node.js 22+**, **MongoDB**, and **Vue 2** (with Element UI). The architecture is plugin-based: core functionality lives in `api/` and `frontend/`, while features are implemented as plugins in `plugins/`.
5+
6+
## Architecture
7+
8+
### Multi-Process Architecture
9+
Countly runs as multiple services (start via `npm run start:all:dev`):
10+
- **API Server** (`api/api.js`) - SDK data ingestion on port 3001
11+
- **Frontend** (`frontend/express/app.js`) - Dashboard on port 6001
12+
- **Job Server** (`jobServer/index.js`) - Background job processing
13+
- **Aggregator** (`api/aggregator.js`) - Data aggregation
14+
- **Ingestor** (`api/ingestor.js`) - High-volume data ingestion
15+
16+
### Plugin System
17+
Plugins extend Countly via event hooks. Each plugin has this structure:
18+
```
19+
plugins/<name>/
20+
├── api/api.js # Backend API endpoints (required)
21+
├── frontend/app.js # Express middleware/routes
22+
├── frontend/public/ # Static assets (JS, CSS, templates)
23+
├── package.json # Plugin metadata
24+
├── install.js # Installation hook
25+
└── tests.js # Plugin tests
26+
```
27+
28+
## Backend Development Checklist
29+
30+
### API Endpoint Security (REQUIRED)
31+
**Every endpoint must use validation** from `api/utils/rights.js`:
32+
```javascript
33+
const { validateRead, validateCreate, validateUpdate, validateDelete } = require('../../../api/utils/rights.js');
34+
35+
// Read operations
36+
validateRead(params, FEATURE_NAME, () => { /* handler */ });
37+
38+
// Write operations - always include app_id in queries!
39+
validateDelete(params, FEATURE_NAME, () => {
40+
// CORRECT: Include app_id to prevent cross-app access
41+
db.collection("items").deleteOne({_id: params.qstring.id, app_id: params.app_id + ""});
42+
});
43+
```
44+
45+
### Parameter Validation
46+
Always validate and type-check input parameters:
47+
```javascript
48+
var argProps = {
49+
'name': { 'required': true, 'type': 'String' },
50+
'count': { 'required': false, 'type': 'Number' }
51+
};
52+
var validation = common.validateArgs(params.qstring.args, argProps, true);
53+
if (!validation.obj) {
54+
common.returnMessage(params, 400, 'Error: ' + validation.errors);
55+
return false;
56+
}
57+
58+
// Parse JSON safely
59+
if (typeof params.qstring.data === "string") {
60+
try {
61+
params.qstring.data = JSON.parse(params.qstring.data);
62+
} catch (ex) {
63+
params.qstring.data = {};
64+
}
65+
}
66+
```
67+
68+
### MongoDB Performance
69+
```javascript
70+
// Use read batcher for frequently accessed documents
71+
common.readBatcher.getOne("events", {'_id': params.app_id}, (err, event) => {});
72+
73+
// Use write batcher for multiple updates to same document
74+
common.writeBatcher.add("users", id, {'$inc': updateData});
75+
76+
// Always use projection to limit returned fields
77+
db.collection('plugins').findOne({_id: 'plugins'}, {projection: {'myfield': 1}});
78+
```
79+
80+
### App Lifecycle Events (Required for plugins with collections)
81+
```javascript
82+
// Create indexes when new app is created
83+
plugins.register("/i/apps/create", function(ob) {
84+
common.db.collection('app_mydata' + ob.appId).ensureIndex({"field": 1}, {background: true});
85+
});
86+
87+
// Clean up when app is deleted
88+
plugins.register("/i/apps/delete", function(ob) {
89+
common.db.collection('app_mydata' + ob.appId).drop();
90+
});
91+
92+
// Handle user data deletion (GDPR)
93+
plugins.register("/i/app_users/delete", function(ob) {
94+
common.db.collection("app_mydata" + ob.app_id).remove({uid: {$in: ob.uids}});
95+
});
96+
```
97+
98+
### Audit Logging
99+
Log all create/update/delete actions:
100+
```javascript
101+
plugins.dispatch("/systemlogs", {params: params, action: "item_created", data: newItem});
102+
plugins.dispatch("/systemlogs", {params: params, action: "item_edited", data: {before: oldItem, update: changes}});
103+
```
104+
105+
## Frontend Development (Vue 2)
106+
107+
### Component Conventions
108+
```javascript
109+
// Use PascalCase for component names
110+
var MyComponent = countlyVue.views.create({
111+
template: countlyVue.T("/myplugin/templates/mytemplate.html"),
112+
mixins: [countlyVue.mixins.auth(FEATURE_NAME)],
113+
data: function() { return { /* state */ }; },
114+
computed: { /* prefer computed over watchers */ },
115+
methods: { /* handlers */ }
116+
});
117+
118+
// Register route with kebab-case component names in templates
119+
app.route('/dashboard/myfeature', 'myfeature', function() {
120+
new countlyVue.views.BackboneWrapper({ component: MyComponent }).render();
121+
});
122+
```
123+
124+
### Vue Best Practices
125+
- **DO**: Use `@event` instead of `v-on:event`, `:prop` instead of `v-bind:prop`
126+
- **DO**: Prefer computed properties over data + watchers
127+
- **DO**: Add `data-test-id` attributes for testable elements
128+
- **DON'T**: Use `v-html` with user input (XSS risk)
129+
- **DON'T**: Modify parent state directly - use props down, events up
130+
- **DON'T**: Use global component registration unless truly global
131+
132+
### Data Test IDs for UI Testing
133+
```html
134+
<!-- Add data-test-id for Cypress tests -->
135+
<button data-test-id="submit-form-button">Submit</button>
136+
<input data-test-id="username-input" type="text">
137+
138+
<!-- Dynamic test IDs in Vue -->
139+
<el-tab :data-test-id="'tab-' + tab.name + '-link'">
140+
```
141+
142+
## Security Requirements
143+
144+
### XSS Prevention
145+
- API output is auto-escaped via `common.returnOutput()` and `common.returnMessage()`
146+
- Frontend: Treat API data as HTML, use `{{ msg }}` for unescaped user input
147+
- Use `countlyCommon.encodeHtml()` for manual sanitization
148+
149+
### MongoDB Injection Prevention
150+
```javascript
151+
// Always cast credentials to strings
152+
params.username = params.username + "";
153+
params.password = params.password + "";
154+
```
155+
156+
### File Upload Security
157+
```javascript
158+
// Validate file types
159+
if (type !== "image/png" && type !== "image/gif" && type !== "image/jpeg") {
160+
fs.unlink(tmp_path, function() {});
161+
return;
162+
}
163+
// Sanitize filenames
164+
var safeFileName = common.sanitizeFilename(params.qstring.filename);
165+
```
166+
167+
### Command Line Security
168+
```javascript
169+
// Use spawn with array args, NOT exec with string concatenation
170+
var cp = require('child_process');
171+
cp.spawn("command", [userInput]); // Safe
172+
// exec("command " + userInput); // UNSAFE - allows injection
173+
```
174+
175+
## Testing
176+
177+
```bash
178+
npm run test:unit # Unit tests (no Docker)
179+
npm run test:api-core # Core API tests
180+
npm run test:lite-plugins # CE plugin tests
181+
npm run test:plugin -- <name> # Single plugin tests
182+
183+
# Linting
184+
countly plugin lint <pluginname>
185+
countly plugin lintfix <pluginname>
186+
187+
# Shell script validation
188+
countly shellcheck
189+
```
190+
191+
### Plugin Test Requirements
192+
- Test empty state, various inputs, and cleanup
193+
- Verify app lifecycle handlers work correctly
194+
- Include tests in `plugins/<name>/tests.js`
195+
196+
## CSS & Styling
197+
198+
- Use **SASS** (SCSS syntax) for stylesheets
199+
- Use **BEM naming** with `cly-vue-` prefix for all new classes
200+
- Use **Bulma** classes prefixed with `bu-` for grid/layout
201+
- Don't use `@import`, use `@use` in SASS files
202+
- Compile with `npx grunt sass` or `npx grunt dist-all`
203+
204+
## JSDoc Comments
205+
206+
Document all public functions:
207+
```javascript
208+
/**
209+
* Calculates percent change between periods.
210+
* @param {number} previous - data for previous period
211+
* @param {number} current - data for current period
212+
* @returns {object} {"percent": "20%", "trend": "u"}
213+
*/
214+
```
215+
216+
## Custom Scripts
217+
218+
Scripts in `bin/scripts/` must include:
219+
- Header comment with description, server type, path, command
220+
- All configurable variables with comments
221+
- Dry run option for destructive operations
222+
- Idempotent behavior (safe to run multiple times)
223+
224+
## Development Commands
225+
226+
```bash
227+
npm run start:all:dev # All services with hot reload
228+
npx grunt dist-all # Build all static assets (required after JS changes)
229+
npx grunt locales # Build locale files
230+
231+
# Plugin management
232+
node bin/commands/scripts/plugin.js enable <name>
233+
node bin/commands/scripts/plugin.js disable <name>
234+
```
235+
236+
## Key Files Reference
237+
238+
| Purpose | Location |
239+
|---------|----------|
240+
| Plugin manager | `plugins/pluginManager.js` |
241+
| Common utilities | `api/utils/common.js` |
242+
| Authorization | `api/utils/rights.js` |
243+
| Vue core | `frontend/express/public/javascripts/countly/vue/core.js` |
244+
| Sample plugin | `plugins/empty/` |
245+
| TypeScript types | `types/` |
246+
| Coding guidelines | `CODING_GUIDELINES.md` |
247+
| Vue.js guidelines | `docs/VUEJS_GUIDELINES.md` |
248+
| CSS style guide | `docs/CSS_STYLE_GUIDE.md` |
249+
| Security guidelines | `docs/SECURITY.md` |
250+
| UI testing guide | `docs/UI_TESTING.md` |

‎.github/workflows/codeql-analysis.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@ jobs:
3737

3838
steps:
3939
- name: Checkout repository
40-
uses: actions/checkout@v5
40+
uses: actions/checkout@v6
4141

4242
# Initializes the CodeQL tools for scanning.
4343
- name: Initialize CodeQL

‎.github/workflows/deploy.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ jobs:
1919

2020
steps:
2121
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
22-
- uses: actions/checkout@v5
22+
- uses: actions/checkout@v6
2323

2424
- name: Enable command line
2525
shell: bash
@@ -45,7 +45,7 @@ jobs:
4545
runs-on: ubuntu-latest
4646
steps:
4747
- name: Check out the repo
48-
uses: actions/checkout@v5
48+
uses: actions/checkout@v6
4949

5050
- name: Log in to Docker Hub
5151
uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef

‎.github/workflows/docker-image.yml‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ jobs:
1212
runs-on: ubuntu-latest
1313
steps:
1414
- name: Check out the repo
15-
uses: actions/checkout@v5
15+
uses: actions/checkout@v6
1616

1717
- name: Set output
1818
id: vars
@@ -43,7 +43,7 @@ jobs:
4343
runs-on: ubuntu-latest
4444
steps:
4545
- name: Check out the repo
46-
uses: actions/checkout@v5
46+
uses: actions/checkout@v6
4747

4848
- name: Set output
4949
id: vars
@@ -74,7 +74,7 @@ jobs:
7474
runs-on: ubuntu-latest
7575
steps:
7676
- name: Check out the repo
77-
uses: actions/checkout@v5
77+
uses: actions/checkout@v6
7878

7979
- name: Set output
8080
id: vars
@@ -105,7 +105,7 @@ jobs:
105105
runs-on: ubuntu-latest
106106
steps:
107107
- name: Check out the repo
108-
uses: actions/checkout@v5
108+
uses: actions/checkout@v6
109109

110110
- name: Set output
111111
id: vars

0 commit comments

Comments
 (0)