|
| 1 | +# Qtil — CodeQL Utility Library |
| 2 | + |
| 3 | +## Architecture |
| 4 | + |
| 5 | +Qtil is a **multi-language CodeQL utility library** published as `advanced-security/qtil`. It has a layered structure: |
| 6 | + |
| 7 | +- **Core pack** (`src/`): Language-agnostic utilities — tuples, lists, strings, graphs, inheritance helpers, parameterization aids. Entry point: `src/qtil/Qtil.qll` re-exports everything via one `module Qtil { ... }`. |
| 8 | +- **Language packs** (`cpp/`, `java/`, `python/`, etc.): Each wraps the core pack and adds language-specific features (e.g., `TwoOperands`, `QlFormat`, `CustomPathProblem`). Entry point pattern: `cpp/src/qtil/Cpp.qll` imports `Common::Qtil` then adds cpp-specific modules. |
| 9 | +- **Test packs** (`test/`, `<lang>/test/`): Mirror the src structure. Core tests use the `Qnit` unit testing framework; language tests analyze real source files. |
| 10 | + |
| 11 | +Language-specific features like `TwoOperands`, `QlFormat`, and `CustomPathProblem` require language types and are intentionally **not** imported in the core `Qtil.qll` — they live in language packs only. |
| 12 | + |
| 13 | +## Developer Workflow |
| 14 | + |
| 15 | +- **CodeQL CLI version**: Pinned in `.codeqlversion` (currently `2.20.1`). CI reads this file. |
| 16 | +- **Install packs**: `codeql pack install src` (core), `codeql pack install <lang>/src` (language-specific). |
| 17 | +- **Run tests**: `codeql test run test/` (core) or `codeql test run <lang>/test/` (language-specific). |
| 18 | +- **Format code**: `codeql query format --in-place <file.ql|file.qll>`. CI enforces formatting on all `.ql`/`.qll` files. |
| 19 | +- **Code generation**: `python scripts/generate_fnqll.py` generates arity-based `.qll` files in `src/qtil/fn/generated/` from Jinja2 templates in `scripts/templates/`. Never hand-edit generated files. |
| 20 | +- **Version bumps**: All packs share the same base version. When updating, bump `version` in `src/qlpack.yml` and all `<lang>/src/qlpack.yml` files, and update each language pack's dependency on `advanced-security/qtil` to match. Test packs don't need version bumps. See `.github/skills/update-pack-versions/SKLL.md`. |
| 21 | + |
| 22 | +## CodeQL Conventions |
| 23 | + |
| 24 | +- **QLDoc**: Every public module, class, and predicate gets a `/** ... */` doc comment with usage examples. |
| 25 | +- **Imports**: Use `private import` for internal dependencies. Only the entry-point module (`Qtil.qll` / `Cpp.qll`) uses public imports. |
| 26 | +- **Naming**: Modules are `PascalCase`, predicates are `camelCase`, type parameters are single uppercase letters (`T`, `R`, `A`, `B`). |
| 27 | +- **Parameterized modules**: Heavy use with nested sub-modules. Example: `Ordered<T>::GroupBy<Division>::Type`. Use `Final<T>::Type` instead of `final class FinalFoo = Foo;` aliases. |
| 28 | +- **Signature types**: Use the hierarchy from `SignatureTypes.qll` — `FiniteType`, `FiniteStringableType`, `InfiniteType`, `InfiniteStringableType`. Use `Signature<T>::Type` to create ad-hoc signature types from existing types. |
| 29 | +- **`bindingset`**: Required on infinite types and predicates operating on infinite domains. |
| 30 | + |
| 31 | +## Testing with Qnit |
| 32 | + |
| 33 | +Core library tests use Qnit (`import qtil.testing.Qnit`), not standard CodeQL test assertions: |
| 34 | + |
| 35 | +```ql |
| 36 | +class MyTest extends Test, Case { |
| 37 | + override predicate run(Qnit test) { |
| 38 | + if someCondition() |
| 39 | + then test.pass("unique description") |
| 40 | + else test.fail("failure description") |
| 41 | + } |
| 42 | +} |
| 43 | +``` |
| 44 | + |
| 45 | +- Each test class should pass or fail with a **unique** string. |
| 46 | +- Expected file for passing tests: `| 1 test passed. |` or `| All N tests passed. |` |
| 47 | +- Language-specific tests (e.g., `cpp/test/`) use standard CodeQL test format with `test.cpp` + `.ql` + `.expected` pipe-delimited tables. |
| 48 | +- Test helper libraries (`.qll`) go in the test directory alongside test files. |
| 49 | + |
| 50 | +## Key Files |
| 51 | + |
| 52 | +| Path | Purpose | |
| 53 | +|------|---------| |
| 54 | +| `src/qtil/Qtil.qll` | Core module entry point — all language-agnostic exports | |
| 55 | +| `<lang>/src/qtil/<Lang>.qll` | Language pack entry point (e.g., `Cpp.qll`) | |
| 56 | +| `src/qtil/parameterization/SignatureTypes.qll` | Reusable signature type hierarchy | |
| 57 | +| `src/qtil/parameterization/SignaturePredicates.qll` | Signature predicates by arity (Nullary–Senary) | |
| 58 | +| `scripts/generate_fnqll.py` | Code generator for arity-based modules | |
| 59 | +| `.codeqlversion` | Pinned CodeQL CLI version | |
| 60 | +| `.codeqlmanifest.json` | Workspace manifest listing all qlpacks | |
0 commit comments