|
| 1 | +# Vortex |
| 2 | + |
| 3 | +## Development Guidelines |
| 4 | + |
| 5 | +* project is a monorepo Rust workspace, java bindings in `/java`, python bindings in `/vortex-python` |
| 6 | +* run `cargo build -p` to build a specific crate |
| 7 | +* use `cargo clippy --all-targets --all-features` to make sure a project is free of lint issues. Please do this every |
| 8 | + time you reach a stopping point or think you've finished work. |
| 9 | +* run `cargo +nightly fmt --all` to format Rust source files. Please do this every time you reach a stopping point or |
| 10 | + think you've finished work. |
| 11 | +* run `cargo xtask public-api` to re-generate the public API lock files. Please do this every time you reach a stopping |
| 12 | + point or think you've finished work. |
| 13 | +* you can try running |
| 14 | + `cargo fix --lib --allow-dirty --allow-staged && cargo clippy --fix --lib --allow-dirty --allow-staged` to |
| 15 | + automatically many fix minor errors. |
| 16 | + |
| 17 | +## Architecture |
| 18 | + |
| 19 | +* `vortex-buffer` defines zero-copy aligned `Buffer<T>` and `BufferMut<T>` that are guaranteed |
| 20 | + to be aligned to `T` (or whatever requested runtime alignment). |
| 21 | +* `vortex-array/src/dtype` contains the basic `DType` logical type enum that is the basis of the Vortex |
| 22 | + type system |
| 23 | +* `vortex-array` contains the basic `Array` trait, as well as several encodings which impl |
| 24 | + that trait for each encoding. It includes all of most of the Apache Arrow encodings. |
| 25 | +* More exotic compressed encodings live in the crates inside of `/encodings/*` |
| 26 | +* File IO is defined in `vortex-file`. It uses the concept of a `LayoutReader` defined |
| 27 | + in `vortex-layout` crate. |
| 28 | +* `/vortex-python` contains the python bindings. rst flavored docs for the project are in `/docs` |
| 29 | + |
| 30 | +## Code Style |
| 31 | + |
| 32 | +* Prefer `impl AsRef<T>` to `&T` for public interfaces where possible, e.g. `impl AsRef<Path>` |
| 33 | +* Avoid usage of unsafe where not necessary, use zero-cost safe abstractions wherever possible, |
| 34 | + or cheap non-zero-cost abstractions. |
| 35 | +* Every new public API definition must have a doc comment. Examples are nice to have but not |
| 36 | + strictly required. |
| 37 | +* Use `vortex_err!` to create a `VortexError` with a format string and `vortex_bail!` to do the same but immediately |
| 38 | + return it as a `VortexResult<T>` to the surrounding context. |
| 39 | +* When writing tests, strongly consider using `rstest` cases to parameterize repetitive test logic. |
| 40 | +* If you want to create a large number of tests to an existing file module called `foo.rs`, and if you think doing so |
| 41 | + would |
| 42 | + be too many to inline in a `tests` submodule within `foo.rs`, then first promote `foo` to a directory module. You can |
| 43 | + do |
| 44 | + this by running `mkdir foo && mv foo.rs foo/mod.rs`. Then, you can create a test file `foo/tests.rs` that you include |
| 45 | + in `foo/mod.rs` with the appropriate test config flag. |
| 46 | +* If you encounter clippy errors in tests that should only pertain to production code (e.g., prohibiting panic/unwrap, |
| 47 | + possible numerical truncation, etc.), then consider allowing those lints at the test module level. |
| 48 | +* Prefer naming test modules `tests`, not `test`. |
| 49 | +* Prefer having test return VortexResult<()> and use ? over unwrap. |
| 50 | +* All imports must be at the top of the module, never inside functions. The only exception is `#[cfg(test)]` blocks, |
| 51 | + where imports should be at the top of the test module. Function-scoped imports are only acceptable when (a) required, |
| 52 | + or (b) it would be exceptionally verbose otherwise, such as a match statement where left and right sides have similar |
| 53 | + names. |
| 54 | +* Imports should be preferred over qualified identifiers. |
| 55 | +* Only write comments that explain non-obvious logic or important context. Avoid commenting simple or self-explanatory |
| 56 | + code. |
| 57 | +* Use `assert_arrays_eq!` macro for comparing arrays in tests instead of element-by-element comparison. |
| 58 | +* Keep tests concise and to the point - avoid unnecessary setup or verbose assertions. |
| 59 | +* Run tests for a specific crate with `cargo test -p <crate-name>` (e.g., `cargo test -p vortex-array`). |
| 60 | + |
| 61 | +## Other |
| 62 | + |
| 63 | +* When summarizing your work, please produce summaries in valid Markdown that can be easily copied/pasted to Github. |
| 64 | + |
| 65 | +## Commits |
| 66 | + |
| 67 | +* All commits must be signed of by the committers in the form `Signed-off-by: "COMMITTER" <COMMITTER_EMAIL>`. |
0 commit comments