|
2 | 2 |
|
3 | 3 | ## Development Guidelines |
4 | 4 |
|
5 | | -* project is a monorepo Rust workspace, java bindings in `/java`, python bindings in `/vortex-python` |
6 | | -* run `cargo build -p` to build a specific crate |
7 | | -* use `cargo clippy --all-targets --all-features` to make sure a project is free of lint issues. Please do this every |
8 | | - time you reach a stopping point or think you've finished work. |
9 | | -* run `cargo +nightly fmt --all` to format Rust source files. Please do this every time you reach a stopping point or |
10 | | - think you've finished work. |
11 | | -* run `cargo xtask public-api` to re-generate the public API lock files. Please do this every time you reach a stopping |
12 | | - point or think you've finished work. |
13 | | -* you can try running |
14 | | - `cargo fix --lib --allow-dirty --allow-staged && cargo clippy --fix --lib --allow-dirty --allow-staged` to |
15 | | - automatically many fix minor errors. |
16 | | -* when iterating on CI failures, fetch only failed job logs first (`gh run view <run-id> --job <job-id> --log-failed`) |
17 | | - and run narrow local repro commands for the affected crate/tests before running workspace-wide checks. |
18 | | -* if `gh` commands fail with `error connecting to api.github.com` in sandbox, immediately rerun with escalated network |
19 | | - permissions instead of retrying in sandbox. |
20 | | -* if cargo fails with `sccache: error: Operation not permitted`, rerun the command with `RUSTC_WRAPPER=` so rustc runs |
21 | | - directly. |
22 | | -* run docs doctests from the docs directory (`make -C docs doctest`) so the correct Sphinx Makefile target is used. |
| 5 | +- project is a monorepo Rust workspace, java bindings in `/java`, python bindings in |
| 6 | + `/vortex-python` |
| 7 | +- run `cargo build -p` to build a specific crate |
| 8 | +- use `cargo clippy --all-targets --all-features` to make sure a project is free of lint issues. |
| 9 | + Please do this every time you reach a stopping point or think you've finished work. |
| 10 | +- run `cargo +nightly fmt --all` to format Rust source files. Please do this every time you reach a |
| 11 | + stopping point or think you've finished work. |
| 12 | +- run `./scripts/public-api.sh` to re-generate the public API lock files. Please do this every time |
| 13 | + you reach a stopping point or think you've finished work. |
| 14 | +- you can try running |
| 15 | + `cargo fix --lib --allow-dirty --allow-staged && cargo clippy --fix --lib --allow-dirty --allow-staged` |
| 16 | + to automatically many fix minor errors. |
| 17 | +- when iterating on CI failures, fetch only failed job logs first |
| 18 | + (`gh run view <run-id> --job <job-id> --log-failed`) and run narrow local repro commands for the |
| 19 | + affected crate/tests before running workspace-wide checks. |
| 20 | +- if `gh` commands fail with `error connecting to api.github.com` in sandbox, immediately rerun with |
| 21 | + escalated network permissions instead of retrying in sandbox. |
| 22 | +- if cargo fails with `sccache: error: Operation not permitted`, rerun the command with |
| 23 | + `RUSTC_WRAPPER=` so rustc runs directly. |
| 24 | +- run docs doctests from the docs directory (`make -C docs doctest`) so the correct Sphinx Makefile |
| 25 | + target is used. |
23 | 26 |
|
24 | 27 | ## Architecture |
25 | 28 |
|
26 | | -* `vortex-buffer` defines zero-copy aligned `Buffer<T>` and `BufferMut<T>` that are guaranteed |
27 | | - to be aligned to `T` (or whatever requested runtime alignment). |
28 | | -* `vortex-array/src/dtype` contains the basic `DType` logical type enum that is the basis of the Vortex |
29 | | - type system |
30 | | -* `vortex-array` contains the basic `Array` trait, as well as several encodings which impl |
31 | | - that trait for each encoding. It includes all of most of the Apache Arrow encodings. |
32 | | -* More exotic compressed encodings live in the crates inside of `/encodings/*` |
33 | | -* File IO is defined in `vortex-file`. It uses the concept of a `LayoutReader` defined |
34 | | - in `vortex-layout` crate. |
35 | | -* `/vortex-python` contains the python bindings. rst flavored docs for the project are in `/docs` |
| 29 | +- `vortex-buffer` defines zero-copy aligned `Buffer<T>` and `BufferMut<T>` that are guaranteed to be |
| 30 | + aligned to `T` (or whatever requested runtime alignment). |
| 31 | +- `vortex-array/src/dtype` contains the basic `DType` logical type enum that is the basis of the |
| 32 | + Vortex type system |
| 33 | +- `vortex-array` contains the basic `Array` trait, as well as several encodings which impl that |
| 34 | + trait for each encoding. It includes all of most of the Apache Arrow encodings. |
| 35 | +- More exotic compressed encodings live in the crates inside of `/encodings/*` |
| 36 | +- File IO is defined in `vortex-file`. It uses the concept of a `LayoutReader` defined in |
| 37 | + `vortex-layout` crate. |
| 38 | +- `/vortex-python` contains the python bindings. rst flavored docs for the project are in `/docs` |
36 | 39 |
|
37 | 40 | ## Code Style |
38 | 41 |
|
39 | | -* Prefer `impl AsRef<T>` to `&T` for public interfaces where possible, e.g. `impl AsRef<Path>` |
40 | | -* Avoid usage of unsafe where not necessary, use zero-cost safe abstractions wherever possible, |
41 | | - or cheap non-zero-cost abstractions. |
42 | | -* Every new public API definition must have a doc comment. Examples are nice to have but not |
| 42 | +- Prefer `impl AsRef<T>` to `&T` for public interfaces where possible, e.g. `impl AsRef<Path>` |
| 43 | +- Avoid usage of unsafe where not necessary, use zero-cost safe abstractions wherever possible, or |
| 44 | + cheap non-zero-cost abstractions. |
| 45 | +- Every new public API definition must have a doc comment. Examples are nice to have but not |
43 | 46 | strictly required. |
44 | | -* Use `vortex_err!` to create a `VortexError` with a format string and `vortex_bail!` to do the same but immediately |
45 | | - return it as a `VortexResult<T>` to the surrounding context. |
46 | | -* When writing tests, strongly consider using `rstest` cases to parameterize repetitive test logic. |
47 | | -* If you want to create a large number of tests to an existing file module called `foo.rs`, and if you think doing so |
48 | | - would |
49 | | - be too many to inline in a `tests` submodule within `foo.rs`, then first promote `foo` to a directory module. You can |
50 | | - do |
51 | | - this by running `mkdir foo && mv foo.rs foo/mod.rs`. Then, you can create a test file `foo/tests.rs` that you include |
52 | | - in `foo/mod.rs` with the appropriate test config flag. |
53 | | -* If you encounter clippy errors in tests that should only pertain to production code (e.g., prohibiting panic/unwrap, |
54 | | - possible numerical truncation, etc.), then consider allowing those lints at the test module level. |
55 | | -* Prefer naming test modules `tests`, not `test`. |
56 | | -* Prefer having test return VortexResult<()> and use ? over unwrap. |
57 | | -* All imports must be at the top of the module, never inside functions. The only exception is `#[cfg(test)]` blocks, |
58 | | - where imports should be at the top of the test module. Function-scoped imports are only acceptable when (a) required, |
59 | | - or (b) it would be exceptionally verbose otherwise, such as a match statement where left and right sides have similar |
60 | | - names. |
61 | | -* Imports should be preferred over qualified identifiers. |
62 | | -* Only write comments that explain non-obvious logic or important context. Avoid commenting simple or self-explanatory |
63 | | - code. |
64 | | -* Use `assert_arrays_eq!` macro for comparing arrays in tests instead of element-by-element comparison. |
65 | | -* Keep tests concise and to the point - avoid unnecessary setup or verbose assertions. |
66 | | -* Run tests for a specific crate with `cargo test -p <crate-name>` (e.g., `cargo test -p vortex-array`). |
| 47 | +- Use `vortex_err!` to create a `VortexError` with a format string and `vortex_bail!` to do the same |
| 48 | + but immediately return it as a `VortexResult<T>` to the surrounding context. |
| 49 | +- When writing tests, strongly consider using `rstest` cases to parameterize repetitive test logic. |
| 50 | +- If you want to create a large number of tests to an existing file module called `foo.rs`, and if |
| 51 | + you think doing so would be too many to inline in a `tests` submodule within `foo.rs`, then first |
| 52 | + promote `foo` to a directory module. You can do this by running |
| 53 | + `mkdir foo && mv foo.rs foo/mod.rs`. Then, you can create a test file `foo/tests.rs` that you |
| 54 | + include in `foo/mod.rs` with the appropriate test config flag. |
| 55 | +- If you encounter clippy errors in tests that should only pertain to production code (e.g., |
| 56 | + prohibiting panic/unwrap, possible numerical truncation, etc.), then consider allowing those lints |
| 57 | + at the test module level. |
| 58 | +- Prefer naming test modules `tests`, not `test`. |
| 59 | +- Prefer having test return VortexResult<()> and use ? over unwrap. |
| 60 | +- All imports must be at the top of the module, never inside functions. The only exception is |
| 61 | + `#[cfg(test)]` blocks, where imports should be at the top of the test module. Function-scoped |
| 62 | + imports are only acceptable when (a) required, or (b) it would be exceptionally verbose otherwise, |
| 63 | + such as a match statement where left and right sides have similar names. |
| 64 | +- Imports should be preferred over qualified identifiers. |
| 65 | +- Only write comments that explain non-obvious logic or important context. Avoid commenting simple |
| 66 | + or self-explanatory code. |
| 67 | +- Use `assert_arrays_eq!` macro for comparing arrays in tests instead of element-by-element |
| 68 | + comparison. |
| 69 | +- Keep tests concise and to the point - avoid unnecessary setup or verbose assertions. |
| 70 | +- Run tests for a specific crate with `cargo test -p <crate-name>` (e.g., |
| 71 | + `cargo test -p vortex-array`). |
67 | 72 |
|
68 | 73 | ## Other |
69 | 74 |
|
70 | | -* When summarizing your work, please produce summaries in valid Markdown that can be easily copied/pasted to Github. |
| 75 | +- When summarizing your work, please produce summaries in valid Markdown that can be easily |
| 76 | + copied/pasted to Github. |
71 | 77 |
|
72 | 78 | ## Commits |
73 | 79 |
|
74 | | -* All commits must be signed of by the committers in the form `Signed-off-by: "COMMITTER" <COMMITTER_EMAIL>`. |
| 80 | +- All commits must be signed of by the committers in the form |
| 81 | + `Signed-off-by: "COMMITTER" <COMMITTER_EMAIL>`. |
0 commit comments