Skip to content

CONTRIBUTING.md

Latest commit

 

History

History
157 lines (100 loc) · 4.79 KB

CONTRIBUTING.md

File metadata and controls

157 lines (100 loc) · 4.79 KB
Raw

Contributing

Setup

Prereqs:

  • mise (recommended for tool version management)
  • rust toolchain (installed via mise)
  • pre-commit (optional but recommended)

Clone and install tools:

mise install

Install JS dependencies (for tsc):

pnpm install --ignore-scripts

Enable the mise shell hook (if you haven't already), then re-open your shell or cd back into the repo. This repo adds ./target/debug to your PATH via mise.toml, so binaries built in debug mode are on your PATH while you're in this directory.

Install git hooks:

pre-commit install

If you want to run this via mise:

mise run pre-commit-install

Local SDK Development

The CLI depends on braintrust-sdk-rust. The default dependency is pinned to a git rev in Cargo.toml.

To override with a local checkout:

cp .cargo/config.toml.example .cargo/config.toml

Then ensure the path in .cargo/config.toml points to your local SDK checkout (default: ../braintrust-sdk-rust). This file is ignored by git.

Note: when the local override is enabled, Cargo will treat the SDK as a path dependency and update Cargo.lock accordingly. The committed lockfile should reflect the git dependency (for CI). If you need to update Cargo.lock, temporarily move .cargo/config.toml out of the way, run cargo generate-lockfile, then restore it.

Running

Build:

cargo build

Run the CLI:

cargo run -- sql "SELECT 1"

Run evals (JS default runner):

bt eval path/to/foo.eval.ts

Run evals locally (no logs sent):

bt eval --local path/to/foo.eval.ts

Eval fixture tests:

mise run eval-tests

Note: eval fixtures use Node + pnpm (or npm) to install dependencies in tests/evals/js/*.

Required env vars:

  • BRAINTRUST_API_KEY: API key used for login

Optional env vars:

  • BRAINTRUST_API_URL: override API endpoint (default https://api.braintrust.dev)
  • BRAINTRUST_DEFAULT_PROJECT: default project name

Formatting and Linting

Pre-commit runs:

  • cargo fmt --all
  • cargo clippy --all-targets --all-features -- -D warnings

Stable Releases

Stable releases are prepared with a release bump PR and published after GitHub Environment approval.

  1. Open Actions in braintrustdata/bt.
  2. Select release-bump-pr.
  3. Click Run workflow and enter the target version without a leading v (for example 0.9.2).
  4. Review and merge the generated PR. The PR updates Cargo.toml and Cargo.lock.
  5. After the PR merges to main, the release workflow reads the version from Cargo.toml, creates and pushes the matching tag (for example v0.9.2), builds release artifacts, then waits for the release GitHub Environment approval before publishing.
  6. Approve the release deployment to upload artifacts, create the GitHub Release, and publish installers.
  7. Wait for the post-release smoke install jobs to pass.

You can also manually run release with an existing tag input such as v0.9.2. Tags must match v<major>.<minor>.<patch>.

Release artifacts and installers are generated by cargo-dist from dist-workspace.toml. The release workflow publishes the shell and PowerShell installers plus target archives/checksums for the configured platforms.

Publishing PR Canary Releases

Use the release-canary workflow when you need a one-off installer build from a PR branch.

GitHub UI

  1. Open Actions in braintrustdata/bt.
  2. Select release-canary.
  3. Click Run workflow.
  4. Choose the PR branch in the branch/ref selector.
  5. Run the workflow and wait for the announce job to complete.
  6. Open the run summary and copy the install command from Canary Install Commands.

GitHub CLI

gh workflow run release-canary.yml --repo braintrustdata/bt --ref <pr-branch>

Then open the run in Actions and copy the install command from the summary.

Notes:

  • The workflow publishes an immutable tag: canary-<branch-slug>-<short-sha>.
  • It also updates a moving branch tag: canary-<branch-slug> when the run is for the latest commit on that branch.

Windows Release Signing

Release workflows Authenticode-sign Windows artifacts via Azure Trusted Signing (the sign-windows-artifacts composite action). Signing runs when these GitHub Actions repository secrets are configured:

  • AZURE_CLIENT_ID
  • AZURE_TENANT_ID
  • AZURE_SUBSCRIPTION_ID

The signing account is selected by these repository variables:

  • AZURE_SIGNING_ENDPOINT
  • AZURE_SIGNING_ACCOUNT_NAME
  • AZURE_SIGNING_CERT_PROFILE

If the secrets are absent (for example, on forks or PRs without access), the signing step is skipped and published bt.exe remains unsigned. The verify-windows-signature action surfaces unsigned builds via a neutral check run rather than failing the job.