-
Notifications
You must be signed in to change notification settings - Fork 2
Add Db2/MSSQL/Oracle engines and TPC-H, Wikipedia, YCSB, CH-benCHmark benchmarks #21
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
ribalba
wants to merge
3
commits into
main
Choose a base branch
from
new-dbs-and-benchmarks
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,85 @@ | ||
| # BenchBase load-driver image for the Wikipedia and YCSB scenarios. | ||
| # | ||
| # BenchBase builds one self-contained distribution per database profile (each | ||
| # bundles that engine's JDBC driver), so we build every profile we need and copy | ||
| # them all into one runtime image. A scenario then runs: | ||
| # cd /benchbase/benchbase-<profile> && java -jar benchbase.jar -b <bench> -c <cfg> ... | ||
| # | ||
| # Build via ./benchmarks/benchbase/build-image.sh (sets the build args below). | ||
| # syntax=docker/dockerfile:1 | ||
|
|
||
| # --- build stage: compile each requested profile from source --------------- | ||
| # BenchBase's pom.xml pins Java 23 (maven.compiler.release=23). | ||
| FROM eclipse-temurin:23-jdk AS build | ||
|
|
||
| # Moving default; pin a commit SHA via build-image.sh for a reproducible build. | ||
| ARG BENCHBASE_REF=main | ||
| # Space-separated BenchBase Maven profiles to bake in. | ||
| ARG BENCHBASE_PROFILES="postgres mysql mariadb sqlserver oracle" | ||
|
|
||
| RUN apt-get update \ | ||
| && apt-get install -y --no-install-recommends git ca-certificates \ | ||
| && rm -rf /var/lib/apt/lists/* | ||
|
|
||
| WORKDIR /src | ||
| RUN git clone https://github.com/cmu-db/benchbase.git . \ | ||
| && git checkout "${BENCHBASE_REF}" | ||
|
|
||
| # Patch an upstream Wikipedia data-generator bug: UpdatePage truncates the | ||
| # generated revComment to 254 chars before writing revision.rev_comment, but | ||
| # writes the *untruncated* value into recentchanges.rc_comment (also | ||
| # varchar(255)). On any strict engine (PostgreSQL, and MySQL/MariaDB/Oracle/ | ||
| # SQL Server in their default strict modes) the occasional >255-char comment | ||
| # makes the rc_comment insert fail with "value too long" (SQLSTATE 22001), | ||
| # silently dropping that UpdatePage txn. We apply the same substring truncation | ||
| # already used for rev_comment. Anchored on the `// rc_comment` trailing comment | ||
| # so it survives line-number drift; the grep guard fails the build loudly if | ||
| # upstream changes shape and the patch silently no-ops. | ||
| # Upstream: https://github.com/cmu-db/benchbase (procedures/UpdatePage.java) | ||
| RUN set -eux; \ | ||
| f=src/main/java/com/oltpbenchmark/benchmarks/wikipedia/procedures/UpdatePage.java; \ | ||
| sed -i 's|revComment); // rc_comment|revComment.substring(0, Math.min(revComment.length(), 255 - 1))); // rc_comment|' "$f"; \ | ||
| grep -q 'revComment.substring(0, Math.min(revComment.length(), 255 - 1))); // rc_comment' "$f"; \ | ||
| ! grep -q 'param++, revComment); // rc_comment' "$f" | ||
|
|
||
| # Patch the upstream Oracle Wikipedia DDL: five varchar2 columns (ipb_timestamp, | ||
| # page_touched, rev_timestamp, user_touched, user_token) carry a MySQL-style | ||
| # binary default of N literal NUL bytes written as '\0\0...'. Oracle does not | ||
| # process backslash escapes in string literals, so it reads each '\0' as the two | ||
| # characters backslash+zero -- e.g. varchar2(14) DEFAULT '\0\0\0\0\0\0\0\0\0\0\0\0\0\0' | ||
| # is a 28-char default into a 14-char column, which fails CREATE TABLE with | ||
| # ORA-01401 (inserted value too large for column). The sibling timestamp columns | ||
| # in the same file use DEFAULT '', so we normalise every varchar2 NUL-byte | ||
| # default to ''. The grep guard fails the build loudly if upstream changes shape | ||
| # and the patch silently no-ops. | ||
| # Upstream: https://github.com/cmu-db/benchbase (benchmarks/wikipedia/ddl-oracle.sql) | ||
| RUN set -eux; \ | ||
| f=src/main/resources/benchmarks/wikipedia/ddl-oracle.sql; \ | ||
| sed -i "s/\(varchar2([0-9]*)\) DEFAULT '[\\0]*'/\1 DEFAULT ''/g" "$f"; \ | ||
| ! grep -qF "\0" "$f" | ||
|
|
||
| # Build each profile and unpack its tarball into /dist/benchbase-<profile>/. | ||
| # The Maven dependency cache (~/.m2) persists across profiles within this layer, | ||
| # so only the first profile pays the full download cost. Tests are skipped. | ||
| RUN set -eux; \ | ||
| mkdir -p /dist; \ | ||
| for p in ${BENCHBASE_PROFILES}; do \ | ||
| echo ">> building BenchBase profile: $p"; \ | ||
| ./mvnw -q -B clean package -P "$p" -DskipTests; \ | ||
| tar -xzf "target/benchbase-$p.tgz" -C /dist; \ | ||
| done; \ | ||
| ls -d /dist/benchbase-* | ||
|
|
||
| # --- runtime stage: JRE + the unpacked distributions ----------------------- | ||
| FROM eclipse-temurin:23-jre | ||
| LABEL org.opencontainers.image.source="https://github.com/cmu-db/benchbase" \ | ||
| org.opencontainers.image.description="CMU BenchBase, multi-profile, load driver for DBMS-bench" | ||
|
|
||
| WORKDIR /benchbase | ||
| COPY --from=build /dist/ /benchbase/ | ||
| # Fail the build early if no profile made it in. | ||
| RUN ls -d /benchbase/benchbase-* >/dev/null | ||
|
|
||
| # Stay alive so GMT can exec each flow step (create+load, then execute) into the | ||
| # running container, the same way it drives the hammerdb container. | ||
| CMD ["sleep", "infinity"] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| #!/usr/bin/env bash | ||
| # Builds the `benchbase` image used to drive the Wikipedia and YCSB scenarios. | ||
| # | ||
| # BenchBase (https://github.com/cmu-db/benchbase) builds one self-contained | ||
| # distribution per database profile (each bundles that engine's JDBC driver). | ||
| # This image bakes in a distribution for every engine we benchmark so a SINGLE | ||
| # image/container drives all databases of a benchmark — keeping the load | ||
| # generator identical across engines. Each profile lands in | ||
| # /benchbase/benchbase-<profile>/ inside the image; the usage scenarios cd into | ||
| # the right one and run `java -jar benchbase.jar ...`. | ||
| # | ||
| # Run this ONCE before running the Wikipedia/YCSB scenarios (after `docker login`): | ||
| # ./benchmarks/benchbase/build-image.sh | ||
| # It builds and pushes the image to Docker Hub as ribalba/benchbase:latest, which | ||
| # the usage scenarios pull. Set NO_PUSH=1 to build only. | ||
| # | ||
| # For a reproducible paper build, pin BENCHBASE_REF to a commit SHA: | ||
| # BENCHBASE_REF=<sha> ./benchmarks/benchbase/build-image.sh | ||
| set -euo pipefail | ||
|
|
||
| OUT_IMAGE="${OUT_IMAGE:-ribalba/benchbase:latest}" | ||
| # BenchBase has no GitHub releases; `main` is the moving default. Pin a commit | ||
| # SHA via BENCHBASE_REF for a reproducible build. | ||
| BENCHBASE_REF="${BENCHBASE_REF:-main}" | ||
| # Engines we benchmark. BenchBase profile names (note: postgres->pg, mariadb->maria, | ||
| # sqlserver->mssql in this repo's directory naming). No Db2 profile exists. | ||
| BENCHBASE_PROFILES="${BENCHBASE_PROFILES:-postgres mysql mariadb sqlserver oracle}" | ||
| HERE="$(cd "$(dirname "$0")" && pwd)" | ||
|
|
||
| echo ">> building $OUT_IMAGE" | ||
| echo " ref: $BENCHBASE_REF" | ||
| echo " profiles: $BENCHBASE_PROFILES" | ||
| docker build \ | ||
| --build-arg BENCHBASE_REF="$BENCHBASE_REF" \ | ||
| --build-arg BENCHBASE_PROFILES="$BENCHBASE_PROFILES" \ | ||
| -t "$OUT_IMAGE" \ | ||
| "$HERE" | ||
|
|
||
| if [ "${NO_PUSH:-0}" = 1 ]; then | ||
| echo ">> built $OUT_IMAGE (push skipped via NO_PUSH=1)" | ||
| else | ||
| echo ">> pushing $OUT_IMAGE to Docker Hub (requires prior 'docker login')" | ||
| docker push "$OUT_IMAGE" | ||
| echo ">> done: pushed $OUT_IMAGE" | ||
| fi |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
Repository: green-coding-solutions/DBMS-bench
Length of output: 1401
🏁 Script executed:
Repository: green-coding-solutions/DBMS-bench
Length of output: 13513
🏁 Script executed:
Repository: green-coding-solutions/DBMS-bench
Length of output: 13362
🏁 Script executed:
Repository: green-coding-solutions/DBMS-bench
Length of output: 311
🏁 Script executed:
Repository: green-coding-solutions/DBMS-bench
Length of output: 243
Replace
privileged: truewith the actual Db2 capability requirementREADME.md:69-72— the runtime compose setup only addsCAP_IPC_OWNERviacap_add; it doesn’t require fullprivileged: true.