Skip to content

Commit c517e36

Browse files
xeioexxeioex
committed
Added agent developer guide.
The new top-level AGENTS.md (with CLAUDE.md symlinked to it) is the canonical index for agent and contributor instructions. Detailed guides live under docs/agent/: docs/agent/engine-dev.md building, testing, debugging the engine and the nginx modules. docs/agent/js-dev.md writing JavaScript for either engine, with the common nginx API surface and engine differences. docs/agent/js-dev-njs.md specifics of the deprecated njs engine and migration to QuickJS.
1 parent c67d183 commit c517e36

5 files changed

Lines changed: 829 additions & 0 deletions

File tree

‎AGENTS.md‎

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
# njs — agent instructions
2+
3+
njs is a JavaScript engine integrated with NGINX. It ships as:
4+
5+
- a standalone CLI (`build/njs`) for testing and scripting,
6+
- two NGINX modules: `ngx_http_js_module` and `ngx_stream_js_module`,
7+
- two interchangeable JS engines selectable per location/server via the
8+
`js_engine` directive:
9+
- **njs** — the built-in engine, deprecated since 1.0.0.
10+
- **QuickJS** — recommended. Set `js_engine qjs;` in nginx.conf.
11+
12+
This file is the index. Detailed instructions live under [`docs/agent/`](docs/agent/).
13+
14+
## Pick your task
15+
16+
| If you are doing... | Read |
17+
|---|---|
18+
| Editing C in `src/`, `external/`, `nginx/` — engine, modules, build system | [docs/agent/engine-dev.md](docs/agent/engine-dev.md) |
19+
| Writing JavaScript that runs in njs (CLI or NGINX), targeting either engine | [docs/agent/js-dev.md](docs/agent/js-dev.md) |
20+
| Writing JavaScript that must run on the deprecated njs engine | [docs/agent/js-dev-njs.md](docs/agent/js-dev-njs.md) |
21+
22+
## 1. Engine and module development (C)
23+
24+
You are extending or fixing the engine, the QuickJS integration, or the
25+
nginx modules.
26+
27+
Quick facts:
28+
29+
- **Build (CLI):** `./configure && make njs` → `build/njs`. Rebuild is fast.
30+
- **Build (NGINX):** configure NGINX with `--add-module=<njs>/nginx` (static)
31+
or `--add-dynamic-module=<njs>/nginx` (dynamic) in a separate NGINX tree.
32+
- **Dual engine = dual code.** Most external modules ship both an `njs_*.c`
33+
and a `qjs_*.c` implementation. If you change behavior on one side, change
34+
it on the other.
35+
- **Tests:** `make unit_test`, `make lib_test`, `make test262`. NGINX
36+
integration tests under `nginx/t/` run with
37+
`prove -I <tests-lib> nginx/t/`.
38+
- **Code style:** NGINX conventions — 4 spaces (no tabs), 80-column limit,
39+
no trailing whitespace, newline after closing brace, `-Werror` build.
40+
- **Commit subjects:** past tense, prefixed
41+
(`HTTP:`, `Stream:`, `Core:`, `QuickJS:`, `Tests:`, …), ≤67 characters.
42+
43+
Full details, sanitizer builds, VM architecture, and object model:
44+
[docs/agent/engine-dev.md](docs/agent/engine-dev.md).
45+
46+
## 2. Writing JavaScript for njs (CLI or NGINX)
47+
48+
You are writing `.js` modules that run inside `js_content` / `js_filter` /
49+
`js_set` / `js_access` / `js_preread` handlers, or under the standalone CLI.
50+
51+
Orientation:
52+
53+
- **Default to the QuickJS engine** (`js_engine qjs;`). The built-in njs
54+
engine is deprecated since 1.0.0; write new code for QuickJS.
55+
- **Language baseline.** QuickJS is ES2023; the njs engine is ES5.1 strict
56+
with a curated ES6+ subset. See the
57+
[compatibility page](https://nginx.org/en/docs/njs/compatibility.html).
58+
- **Nginx drives the engine, not the JS.** Code only runs from
59+
directive-bound entry points (HTTP: `js_content`, `js_access`,
60+
`js_header_filter`, `js_body_filter`, `js_set`, `js_periodic`).
61+
- **Quick test (CLI):** `./build/njs -c '<code>'` or `./build/njs file.js`.
62+
- **Test inside NGINX:** `prove -I <tests-lib> nginx/t/<your>.t` with
63+
`TEST_NGINX_GLOBALS_HTTP='js_engine qjs;'` (and the same for `_STREAM`).
64+
65+
Everything else — full integration-point semantics, `nginx.conf` wiring
66+
(`js_shared_dict_zone`, `resolver` + `js_fetch_*`, `js_import` /
67+
`js_path` / `js_engine`), bindings (`r`, `s`, `ngx.fetch`, `ngx.shared`,
68+
`crypto`, …), engine-only features, do/don't recipes:
69+
[docs/agent/js-dev.md](docs/agent/js-dev.md). For code that must run on
70+
the deprecated njs engine, also see
71+
[docs/agent/js-dev-njs.md](docs/agent/js-dev-njs.md).
72+
73+
## Resources
74+
75+
- [njs official documentation](https://nginx.org/en/docs/njs/)
76+
- [Reference (API surface)](https://nginx.org/en/docs/njs/reference.html)
77+
- [Compatibility](https://nginx.org/en/docs/njs/compatibility.html)
78+
- [Engine selection](https://nginx.org/en/docs/njs/engine.html)
79+
- [njs-examples repo](https://github.com/nginx/njs-examples/)

‎CLAUDE.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
AGENTS.md

‎docs/agent/engine-dev.md‎

Lines changed: 304 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,304 @@
1+
# Engine and module development (C)
2+
3+
This document covers C development inside the njs repository: the engine
4+
core (`src/`), the external module wrappers (`external/`), the NGINX
5+
modules (`nginx/`), and the build system (`auto/`, `configure`).
6+
7+
For per-task orientation see the top-level [AGENTS.md](../../AGENTS.md).
8+
9+
## Building
10+
11+
njs has no autotools/cmake. The shell-based `configure` script generates
12+
`build/Makefile`. Always run `make clean` before reconfiguring with
13+
different options — the Makefile is not regenerated in place.
14+
15+
### Standalone CLI
16+
17+
```bash
18+
./configure
19+
make -j$(nproc) njs # build/njs
20+
```
21+
22+
### njs with QuickJS backend
23+
24+
QuickJS is built separately and linked into njs.
25+
26+
```bash
27+
# Build libquickjs.a in the QuickJS source tree
28+
( cd <QUICKJS_SRC> && CFLAGS=-fPIC make libquickjs.a )
29+
30+
# Configure njs to use it
31+
make clean
32+
./configure \
33+
--cc-opt='-I<QUICKJS_SRC>' \
34+
--ld-opt='-L<QUICKJS_SRC>'
35+
make -j$(nproc) njs
36+
```
37+
38+
### NGINX module (static / dynamic)
39+
40+
njs builds as an NGINX module from a separate NGINX source tree.
41+
42+
```bash
43+
# Static
44+
cd <NGINX_SRC>
45+
./auto/configure --add-module=<NJS_SRC>/nginx --with-stream --with-debug
46+
make -j$(nproc)
47+
48+
# Dynamic
49+
./auto/configure --add-dynamic-module=<NJS_SRC>/nginx --with-stream
50+
make -j$(nproc) modules
51+
```
52+
53+
Adding `--with-cc-opt='-I<QUICKJS_SRC>'` and
54+
`--with-ld-opt='-L<QUICKJS_SRC>'` enables QuickJS in the NGINX module.
55+
56+
### AddressSanitizer build
57+
58+
```bash
59+
./auto/configure --add-module=<NJS_SRC>/nginx --with-stream --with-debug \
60+
--with-cc=clang \
61+
--with-cc-opt='-O0 -fsanitize=address' \
62+
--with-ld-opt='-fsanitize=address'
63+
make -j$(nproc)
64+
```
65+
66+
The njs `configure` exposes `--address-sanitizer=YES` directly when
67+
building the CLI; prefer `clang` on arm64 (gcc ASan is slow there).
68+
69+
### Configure options (njs)
70+
71+
| Option | Purpose |
72+
|---|---|
73+
| `--cc=FILE` | C compiler (default: gcc) |
74+
| `--cc-opt=OPTIONS` | Additional CFLAGS |
75+
| `--ld-opt=OPTIONS` | Additional LDFLAGS |
76+
| `--debug=YES` | Runtime checks |
77+
| `--debug-memory=YES` | Memory allocation tracing |
78+
| `--debug-opcode=YES` | Per-instruction execution trace |
79+
| `--debug-generator=YES` | Bytecode generator trace |
80+
| `--address-sanitizer=YES` | AddressSanitizer (use with `clang`) |
81+
| `--with-quickjs` | Require QuickJS to be present |
82+
| `--no-openssl` / `--no-libxml2` / `--no-zlib` | Drop optional deps |
83+
84+
Run `./configure --help` for the complete list.
85+
86+
## Testing
87+
88+
```bash
89+
make unit_test # 5800+ language and API tests
90+
make lib_test # internal data structures (hash, rbtree, unicode)
91+
make test262 # ECMAScript test262 compliance suite
92+
make test # shell tests + unit_test + test262
93+
```
94+
95+
NGINX integration tests live under `nginx/t/` and use Perl's `prove`
96+
harness against `Test::Nginx`.
97+
98+
```bash
99+
TMPDIR=$(mktemp -d) \
100+
TEST_NGINX_BINARY=<NGINX_BIN> \
101+
prove -I <TESTS_LIB> nginx/t/
102+
```
103+
104+
Useful environment variables:
105+
106+
| Variable | Effect |
107+
|---|---|
108+
| `TEST_NGINX_BINARY` | Path to the nginx binary (required) |
109+
| `TEST_NGINX_VERBOSE=1` | Verbose harness output |
110+
| `TEST_NGINX_LEAVE=1` | Keep test artifacts in `$TMPDIR/nginx-test-*` |
111+
| `TEST_NGINX_CATLOG=1` | Dump `error.log` after the run |
112+
| `TEST_NGINX_GLOBALS=<conf>` | Inject global-scope config (e.g. `load_module ...`) |
113+
| `TEST_NGINX_GLOBALS_HTTP='js_engine qjs;'` | Run http tests under QuickJS |
114+
| `TEST_NGINX_GLOBALS_STREAM='js_engine qjs;'` | Same, for stream tests |
115+
116+
Use a per-run `TMPDIR=$(mktemp -d)` to isolate artifacts across concurrent
117+
runs and avoid destructive `rm -fr /tmp/nginx-test*`.
118+
119+
For more on the harness see `<TESTS_LIB>/Test/Nginx.pm`.
120+
121+
## Validation checklist
122+
123+
Before submitting a change:
124+
125+
1. `./configure && make -j$(nproc)` compiles without warnings (`-Werror`).
126+
2. `make unit_test` and `make lib_test` pass.
127+
3. If you touched `src/`, also run `make test262`.
128+
4. If you touched `nginx/`, run `prove -I <TESTS_LIB> nginx/t/`,
129+
once with the default engine and once with
130+
`TEST_NGINX_GLOBALS_HTTP='js_engine qjs;'`.
131+
5. New source files: update `auto/sources` (njs core),
132+
`auto/modules` (njs external modules), or
133+
`auto/qjs_modules` (QuickJS external modules).
134+
6. Dual-engine: if you added/changed behavior in an `njs_*.c` module,
135+
mirror it in the corresponding `qjs_*.c` (and vice versa).
136+
137+
## Code style and commits
138+
139+
NGINX coding style:
140+
141+
- 4 spaces, no tabs.
142+
- 80-column line limit.
143+
- No trailing whitespace.
144+
- Newline after closing brace.
145+
- Comments explain *why*, not *what*; avoid em-dashes.
146+
- `-Werror` is on by default — fix all warnings.
147+
148+
Commit messages:
149+
150+
- Past tense subject (`Added X`, `Fixed Y`).
151+
- Subject ≤67 chars, body wrapped to ~80 chars.
152+
- Subject prefix: `HTTP:`, `Stream:`, `Core:`, `QuickJS:`, `Tests:`,
153+
`Modules:`, etc.
154+
- One logical change per commit; rebase/squash before submitting.
155+
156+
## Project layout
157+
158+
```
159+
njs/
160+
├── configure # build entry point
161+
├── auto/ # shell-based build system
162+
│ ├── sources # njs core source list
163+
│ ├── modules # njs external module list
164+
│ ├── qjs_modules # QuickJS external module list
165+
│ └── cc, options, ... # compiler/option detection
166+
├── src/ # engine core (C)
167+
│ ├── njs_vm.c / njs_vmcode.c # virtual machine
168+
│ ├── njs_lexer.c # tokenizer
169+
│ ├── njs_parser.c # parser
170+
│ ├── njs_generator.c # bytecode generator
171+
│ ├── njs_object.c / njs_array.c # built-in types
172+
│ ├── njs_promise.c / njs_async.c
173+
│ ├── njs_value.h # value representation
174+
│ ├── njs.h # public C API
175+
│ ├── qjs.c # QuickJS engine wrapper
176+
│ └── test/ # C unit tests
177+
├── external/ # extension modules
178+
│ ├── njs_shell.c # CLI entry point (main())
179+
│ ├── njs_*_module.c # njs-engine modules (crypto, fs, ...)
180+
│ └── qjs_*_module.c # QuickJS-engine counterparts
181+
├── nginx/ # NGINX module integration
182+
│ ├── ngx_http_js_module.c
183+
│ ├── ngx_stream_js_module.c
184+
│ ├── ngx_js.c # core nginx-JS bindings
185+
│ ├── config # NGINX build glue
186+
│ └── t/ # Perl integration tests
187+
├── test/ # functional test suite
188+
│ ├── js/ # JS language feature tests
189+
│ ├── harness/ # test framework utilities
190+
│ └── shell_test.exp # interactive shell tests (Expect)
191+
└── ts/ # TypeScript type definitions
192+
```
193+
194+
Public C API: `src/njs.h`. VM internals: `src/njs_vm.h`,
195+
`src/njs_value.h`. CLI entry point: `external/njs_shell.c`.
196+
197+
## VM architecture (njs engine)
198+
199+
The QuickJS backend uses upstream QuickJS internals (see
200+
[bellard.org/quickjs](https://bellard.org/quickjs/)). What follows is the
201+
**njs engine** internals only.
202+
203+
### Register-based VM
204+
205+
Each instruction has operands that are immediate values or **indexes**.
206+
An index is encoded as:
207+
208+
```
209+
index | level_type (4 bits) | var_type (4 bits)
210+
```
211+
212+
### Level types (storage location)
213+
214+
```
215+
NJS_LEVEL_LOCAL = 0 // local variable in current frame
216+
NJS_LEVEL_CLOSURE = 1 // closure variable from parent frame
217+
NJS_LEVEL_GLOBAL = 2 // global variable
218+
NJS_LEVEL_STATIC = 3 // static / absolute scope
219+
```
220+
221+
Values are addressed as `vm->levels[NJS_LEVEL_*][index]`.
222+
223+
### Variable types
224+
225+
```
226+
NJS_VARIABLE_CONST = 0
227+
NJS_VARIABLE_LET = 1
228+
NJS_VARIABLE_CATCH = 2
229+
NJS_VARIABLE_VAR = 3
230+
NJS_VARIABLE_FUNCTION = 4
231+
```
232+
233+
### Bytecode example
234+
235+
```
236+
$ ./build/njs -d
237+
>> var a = 42; function f(v) { return v + 1 }
238+
239+
shell:main
240+
1 | 00000 MOVE 0123 0133
241+
1 | 00024 STOP 0033
242+
243+
shell:f
244+
1 | 00000 ADD 0203 0103 0233
245+
1 | 00032 RETURN 0203
246+
```
247+
248+
`MOVE 0123 0133` copies the value at index `0x0133` to `0x0123`.
249+
`ADD a b c` computes `a = b + c`. Indexes are printed in hex and encode
250+
level and variable type.
251+
252+
## Object model (njs engine)
253+
254+
For performance and footprint, a JS object is split into a **local mutable
255+
hash** for the current object and a **shared hash** holding inherited
256+
properties. Built-ins are lazily materialized: the shared definitions stay
257+
shared until first mutation. For functions, the first access copies the
258+
function from the shared hash into the local mutable hash so the
259+
per-object copy can be modified.
260+
261+
Key entry points:
262+
263+
- `njs_value_property()` — top-level property lookup.
264+
- `njs_property_query()` — lookup with descriptor result.
265+
- `njs_object_property_query()` — object-level walk including prototype.
266+
- `njs_prop_private_copy()` — promotion from shared to local on write.
267+
268+
## Debugging
269+
270+
### CLI
271+
272+
```bash
273+
./build/njs -c '<code>' # one-shot
274+
./build/njs -d # interactive, with disassembly
275+
./build/njs -d script.js # dump bytecode for a script
276+
./build/njs -o script.js # opcode trace
277+
./build/njs -h # full option list
278+
```
279+
280+
Select the JavaScript engine with `-n <engine>` (case-insensitive; default
281+
is `njs`):
282+
283+
```bash
284+
./build/njs -n njs -c 'console.log(typeof Map)' # built-in engine
285+
./build/njs -n QuickJS -c 'console.log(typeof Map)' # QuickJS backend
286+
```
287+
288+
`-n QuickJS` requires the binary to be built with QuickJS linked in (see
289+
[njs with QuickJS backend](#njs-with-quickjs-backend) above); otherwise
290+
the CLI reports `unknown engine "QuickJS"`.
291+
292+
### Opcode trace
293+
294+
Built with `--debug-opcode=YES`, `./build/njs -o script.js` prints each
295+
instruction as it executes — `ENTER`/`EXIT` for function boundaries,
296+
opcode mnemonics for everything else. Useful for confirming control flow
297+
through bytecode without a debugger.
298+
299+
### Test failures (NGINX)
300+
301+
With `TEST_NGINX_LEAVE=1`, each test leaves
302+
`$TMPDIR/nginx-test-<random>/` containing the generated `nginx.conf`,
303+
`error.log`, and any artifacts. `TEST_NGINX_CATLOG=1` dumps the log to
304+
stdout automatically.

0 commit comments

Comments
 (0)