Skip to content

small metal benchmark runners #7456

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
joseph-isaacs merged 3 commits into from
Apr 27, 2026
+11 −6
Merged

small metal benchmark runners #7456

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
ede2ac5
wip
joseph-isaacs Apr 15, 2026
824e3c4
wip
joseph-isaacs Apr 15, 2026
1872230
Merge branch 'develop' into ji/metal-small
joseph-isaacs Apr 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/nightly-bench.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,4 +72,4 @@ jobs:
matrix:
machine_type:
- id: x86
instance_name: c6id.8xlarge
Copy link
Copy Markdown
Contributor

@0ax1 0ax1 Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

consider turning off freq scaling always running at max:

# Pin CPU frequency governor to performance to prevent dynamic frequency scaling.
for gov in /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor; do
  [[ -w "$gov" ]] && echo performance > "$gov" || true
done

instance_name: i7i.metal-24xl
4 changes: 2 additions & 2 deletions .github/workflows/sql-benchmarks.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ on:
machine_type:
required: false
type: string
default: c6id.8xlarge
default: i7i.metal-24xl
benchmark_matrix:
required: false
type: string
Expand Down Expand Up @@ -270,7 +270,7 @@ jobs:

runs-on: >-
${{ github.repository == 'vortex-data/vortex'
&& format('runs-on={0}/runner=bench-dedicated/tag={1}{2}', github.run_id, matrix.id, (inputs.mode != 'pr' || github.event.pull_request.head.repo.fork == false) && '/extras=s3-cache' || '')
&& format('runs-on={0}/runner=bench-dedicated/instance-type={1}/tag={2}{3}', github.run_id, inputs.machine_type, matrix.id, (inputs.mode != 'pr' || github.event.pull_request.head.repo.fork == false) && '/extras=s3-cache' || '')
|| 'ubuntu-latest' }}
steps:
- uses: runs-on/action@v2
Expand Down
11 changes: 8 additions & 3 deletions scripts/bench-taskset.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,17 @@ fi


if [[ -z "${BENCH_CPUS:-}" ]]; then
cpu_count="$(nproc)"
BENCH_CPUS="2-$((cpu_count - 1))"
if command -v numactl >/dev/null 2>&1; then
Copy link
Copy Markdown
Contributor

@AdamGS AdamGS Apr 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

worth setting TOKIO_WORKER_THREADS here to make sure tokio only spawns one thread per CPU core we use

# All CPUs on NUMA node 0, skipping CPUs 0-1 to avoid OS interference
BENCH_CPUS=$(numactl --hardware | awk '/^node 0 cpus:/{sep=""; for(i=4;i<=NF;i++){if($i+0>1){printf "%s%s",sep,$i; sep=","}}}')
else
cpu_count="$(nproc)"
BENCH_CPUS="2-$((cpu_count - 1))"
fi
fi

Copy link
Copy Markdown
Contributor Author

@joseph-isaacs joseph-isaacs Apr 16, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# Pin CPU frequency governor to performance to prevent dynamic frequency scaling.
for gov in /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor; do
[[ -w "$gov" ]] && echo performance > "$gov" || true
done

Like this?

if command -v numactl >/dev/null 2>&1; then
exec numactl --physcpubind="$BENCH_CPUS" --localalloc "$@"
exec numactl --physcpubind="$BENCH_CPUS" --membind=0 "$@"
fi
Comment thread
AdamGS marked this conversation as resolved.

exec taskset -c "$BENCH_CPUS" "$@"
Loading