|
| 1 | +# AGENTS.md — UTMStack Repository Guide |
| 2 | + |
| 3 | +## Overview |
| 4 | + |
| 5 | +UTMStack is a SIEM/XDR platform. Multi-language monorepo with three technology stacks: |
| 6 | + |
| 7 | +| Directory | Language | Build | Notes | |
| 8 | +|---|---|---|---| |
| 9 | +| `backend/` | Java 17 (Spring Boot 3.1, JHipster 7.3) | Maven (`./mvnw`) | WAR packaging, Liquibase migrations | |
| 10 | +| `frontend/` | Angular 7 (TypeScript 3.2, Node 14) | Angular CLI + npm | Very old Angular — no modern APIs | |
| 11 | +| `agent/` | Go 1.25.5 | `go build` | Cross-compiled for Linux/Windows/macOS | |
| 12 | +| `agent-manager/` | Go 1.25.5 | `go build` | Distributes agent binaries | |
| 13 | +| `utmstack-collector/` | Go 1.25.5 | `go build` | Log collector service | |
| 14 | +| `as400/` | Go 1.25.5 | `go build` | AS400 log collector | |
| 15 | +| `plugins/*/` | Go 1.25.5 | `go build` | 16 independent plugin modules | |
| 16 | +| `shared/` | Go 1.25.1 | — | Shared library (consumed via `replace`) | |
| 17 | +| `installer/` | Go 1.25.1 | `go build` | Self-updating installer binary | |
| 18 | +| `user-auditor/` | Java 11 | Maven | Standalone microservice | |
| 19 | +| `web-pdf/` | Java 11 | Maven | Standalone microservice | |
| 20 | + |
| 21 | +## Build Commands |
| 22 | + |
| 23 | +### Backend (Java) |
| 24 | +```bash |
| 25 | +cd backend |
| 26 | +./mvnw # Run Spring Boot dev server (port 8080) |
| 27 | +./mvnw -Pprod clean package # Production WAR build → target/utmstack.war |
| 28 | +./mvnw clean test # Run tests |
| 29 | +``` |
| 30 | +- Maven settings: `backend/settings.xml` authenticates to GitHub Packages via `$MAVEN_TK` env var (GitHub PAT). |
| 31 | +- Spring profiles: `dev` (default), `prod`, `tls`. Config in `backend/src/main/resources/config/`. |
| 32 | +- Docker dev database: `docker-compose -f backend/src/main/docker/mysql.yml up -d` |
| 33 | +- **No `src/test/` directory exists** — tests are embedded in `src/main/java/`. |
| 34 | + |
| 35 | +### Frontend (Angular 7) |
| 36 | +```bash |
| 37 | +cd frontend |
| 38 | +npm install |
| 39 | +npm start # ng serve --host 0.0.0.0 |
| 40 | +npm run build # ng build --prod --base-href / |
| 41 | +npm test # ng test (Karma + Jasmine) |
| 42 | +npm run lint # ng lint (TSLint, NOT ESLint) |
| 43 | +``` |
| 44 | +- **Node 14.16.1 required** (pinned in CI). Newer Node will break `node-sass@4`. |
| 45 | +- Linter is **TSLint** (`tslint.json`), not ESLint. Uses `codelyzer` rules. |
| 46 | +- Output: `dist/utm-stack` |
| 47 | +- Styles use SCSS (see `angular.json` schematic config). |
| 48 | +- **CI sets `NODE_OPTIONS=--max_old_space_size=8192`** for builds — you may need this locally too. |
| 49 | + |
| 50 | +### Go Components |
| 51 | +Each Go module is independent. Build from its directory: |
| 52 | +```bash |
| 53 | +cd agent |
| 54 | +go build -o utmstack_agent_service . |
| 55 | +``` |
| 56 | + |
| 57 | +**`shared/` replace directives:** Only `agent/go.mod` and `agent/updater/go.mod` have `replace github.com/utmstack/UTMStack/shared => ../shared`. You cannot build those two modules outside the repo structure. All other Go modules (plugins, agent-manager, installer, etc.) are fully independent. |
| 58 | + |
| 59 | +**Agent ldflags:** Agent binaries require `-ldflags "-X 'github.com/utmstack/UTMStack/agent/config.REPLACE_KEY=<secret>'"` to inject an auth prefix. Without it, agents won't authenticate. CI uses `$AGENT_SECRET_PREFIX` secret. |
| 60 | + |
| 61 | +**Cross-compilation:** Set `GOOS`/`GOARCH`/`CGO_ENABLED=0` before `go build`. CI builds Linux (amd64/arm64), Windows (amd64/arm64), and macOS (arm64). |
| 62 | + |
| 63 | +**Geolocation data:** The event processor needs external CSV files downloaded from `storage.googleapis.com/utmstack-updates/dependencies/geolocation/` at build time. |
| 64 | + |
| 65 | +### Plugins |
| 66 | +Each plugin under `plugins/*/` is a standalone Go module. Build output is a single binary named `com.utmstack.<name>.plugin`. The `event_processor.Dockerfile` copies all 16 plugin binaries into the event processor container. |
| 67 | + |
| 68 | +### Installer |
| 69 | +```bash |
| 70 | +cd installer |
| 71 | +bash build.sh # Uses ldflags for config injection |
| 72 | +``` |
| 73 | +`build.sh` sets `GOPRIVATE=github.com/utmstack` and injects `DEFAULT_BRANCH`, `INSTALLER_VERSION`, `REPLACE` (encryption salt), and `PUBLIC_KEY` via `-ldflags`. |
| 74 | + |
| 75 | +## CI / CD |
| 76 | + |
| 77 | +Two active deployment pipelines: |
| 78 | + |
| 79 | +- **v11** (`.github/workflows/v11-deployment-pipeline.yml`) — current active line. Triggers on push to `release/v11*` branches (dev) or prerelease events (rc). |
| 80 | +- **v10** (`.github/workflows/v10-deployment-pipeline.yml`) — legacy. Triggers on push to `release/v10*` and prerelease/release events. |
| 81 | + |
| 82 | +Reusable workflow building blocks: |
| 83 | +- `reusable-java.yml` — builds Maven projects, pushes Docker image to `ghcr.io/utmstack/utmstack/<image>:<tag>` |
| 84 | +- `reusable-golang.yml` — runs `go test ./...`, builds binary, pushes Docker image |
| 85 | +- `reusable-node.yml` — Node 14.16.1, `npm install && npm run-script build`, pushes Docker image |
| 86 | +- `reusable-sign-agent.yml` — signs Windows binaries (jsign + GCP KMS) and macOS binaries (codesign + notarytool) |
| 87 | +- `reusable-basic.yml` — Docker build-only, no compile step |
| 88 | + |
| 89 | +All Docker images push to **ghcr.io/utmstack/utmstack/**. Agent binary signing requires GCP KMS credentials (Windows) or Apple Developer certificates (macOS). |
| 90 | + |
| 91 | +## Key Architecture Notes |
| 92 | + |
| 93 | +- **Event processor** is the core log correlation engine. It's a standalone Go-based service that loads compiled plugin binaries at runtime. The `event_processor.Dockerfile` expects all plugins pre-built alongside it. |
| 94 | +- **Backend** serves the REST API and the Angular frontend. The frontend is a separate directory but gets built and deployed as its own container. |
| 95 | +- **Agent** runs on endpoints (Windows/Linux/macOS). Communicates with `agent-manager` via gRPC. The agent-manager container bundles all agent binaries for distribution. |
| 96 | +- **Filters** (`filters/`) and **rules** (`rules/`) are YAML files defining log parsing filters and detection rules. The backend Dockerfile copies these into the container at `/utmstack/filters` and `/utmstack/rules`. |
| 97 | +- **UTMStack Collector** (`utmstack-collector/`) and **AS400** (`as400/`) are log collectors — separate from the endpoint agent. |
| 98 | + |
| 99 | +## Gotchas |
| 100 | + |
| 101 | +- **No `src/test/` in backend** — Maven test phase runs but the directory doesn't exist yet. Tests are embedded in `src/main/java/`. |
| 102 | +- **Agent ldflags required** — `config.REPLACE_KEY` is injected at build time. Without it, agents won't authenticate. |
| 103 | +- **Backend uses GitHub Packages** — `settings.xml` references `maven.pkg.github.com/utmstack/**`. You need a GitHub PAT with `read:packages` scope in `$MAVEN_TK`. |
| 104 | +- **Frontend is Angular 7** — many modern Angular patterns (standalone components, signals, etc.) do not exist. CLI is v7.3.6. |
| 105 | +- **Node 14 required for frontend** — CI pins 14.16.1. `npm install` on Node 16+ will fail on `node-sass@4`. |
| 106 | +- **`npm run build` needs 8 GB heap** — CI sets `NODE_OPTIONS=--max_old_space_size=8192`. |
| 107 | +- **Installer build requires ldflags** — `installer/build.sh` shows the required `-ldflags` for `DEFAULT_BRANCH`, `INSTALLER_VERSION`, `REPLACE`, and `PUBLIC_KEY`. |
| 108 | +- **Geolocation data must be downloaded** — event processor needs CSV files from GCS at build time. |
| 109 | +- **`geolocation/` directory is gitignored** — must be populated at build time from GCS. |
| 110 | +- **`.plugin` files are gitignored** — plugin binaries are build artifacts, not committed. |
0 commit comments