Skip to content

Commit 6ced439

Browse files
committed
feat: Add sql-runner wrapper script to simplify flink run (#321)
1 parent d9aad9c commit 6ced439

4 files changed

Lines changed: 49 additions & 7 deletions

File tree

README.md

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -78,14 +78,17 @@ docker run -d --rm -it \
7878
In a separate terminal, run:
7979

8080
```bash
81-
docker exec -it runner flink run flink-sql-runner.jar --sqlfile /flink/sql/flink.sql
81+
docker exec -it runner sql-runner --sqlfile /flink/sql/flink.sql
8282
```
8383

8484
The job will be submitted to the embedded JobManager and executed using the local TaskManager.
8585

8686
> [!NOTE]
87-
> The `flink-sql-runner.jar` is a symlink placed in the Flink root directory (`/opt/flink`) for easier access, but the actual file resides in its own plugin directory: `/opt/flink/plugins/flink-sql-runner`.
88-
> It is possible to add any Flink arguments or run any accessible JAR, just like with a vanilla `flink run` command.
87+
> The [`sql-runner`](https://github.com/DataSQRL/flink-sql-runner/blob/main/flink-sql-runner/src/main/docker/sql-runner) wrapper submits the runner through `flink run` so you do not need to specify the main class or placeholder JAR manually.
88+
> The placeholder JAR is necessary because `sql-runner.jar` is shipped under Flink's `lib/` fodler, while the Flink CLI still requires a job JAR argument for `flink run`.
89+
> By default, all arguments are passed to the SQL runner, for example `sql-runner --sqlfile /flink/sql/flink.sql`.
90+
> To pass options to `flink run`, put them before `--`; arguments after `--` are passed to the SQL runner, for example `sql-runner -s /path/to/savepoint -- --planfile /flink/plan.json`.
91+
> You can also omit using `sql-runner` and directly use `flink run`, just make sure to pass the main class and use the `noop.jar`.
8992
9093
4\. Inspect output
9194
If your SQL uses the print connector as a sink, you can check logs via:

flink-sql-runner/src/main/docker/Dockerfile

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ COPY iceberg-flink-runtime-*.jar /opt/flink/lib
2525
COPY iceberg-aws-bundle-*.jar /opt/flink/lib
2626
COPY stdlib-utils-*.jar /opt/flink/lib
2727
COPY flink-sql-runner.uber.jar /opt/flink/lib/sql-runner.uber.jar
28+
COPY --chmod=755 sql-runner /opt/flink/bin/sql-runner
2829
COPY --chmod=755 entrypoint.sh /entrypoint.sh
2930

3031
RUN rm -rf /opt/flink/lib/flink-table-planner-loader-*.jar \
Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
#!/usr/bin/env bash
2+
#
3+
# Copyright © 2026 DataSQRL (contact@datasqrl.com)
4+
#
5+
# Licensed under the Apache License, Version 2.0 (the "License");
6+
# you may not use this file except in compliance with the License.
7+
# You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS,
13+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
# See the License for the specific language governing permissions and
15+
# limitations under the License.
16+
#
17+
18+
flink_args=()
19+
runner_args=()
20+
split_seen=false
21+
22+
for arg in "$@"; do
23+
if [[ "$split_seen" == false && "$arg" == "--" ]]; then
24+
split_seen=true
25+
continue
26+
fi
27+
28+
if [[ "$split_seen" == true ]]; then
29+
runner_args+=("$arg")
30+
else
31+
flink_args+=("$arg")
32+
fi
33+
done
34+
35+
if [[ "$split_seen" == false ]]; then
36+
runner_args=("${flink_args[@]}")
37+
flink_args=()
38+
fi
39+
40+
exec flink run "${flink_args[@]}" -c com.datasqrl.flinkrunner.CliRunner /opt/flink/noop.jar "${runner_args[@]}"

flink-sql-runner/src/test/java/com/datasqrl/flinkrunner/AbstractITSupport.java

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -158,16 +158,14 @@ protected String flinkRun(List<String> sqlRunnerArgs) throws Exception {
158158

159159
protected String flinkRun(List<String> sqlRunnerArgs, @Nullable String savepointPath)
160160
throws Exception {
161-
var execCmd = new ArrayList<>(List.of("flink", "run"));
161+
var execCmd = new ArrayList<>(List.of("sql-runner"));
162162

163163
if (savepointPath != null) {
164164
execCmd.add("-s");
165165
execCmd.add(savepointPath);
166+
execCmd.add("--");
166167
}
167168

168-
execCmd.add("-c");
169-
execCmd.add("com.datasqrl.flinkrunner.CliRunner");
170-
execCmd.add("noop.jar");
171169
execCmd.addAll(sqlRunnerArgs);
172170

173171
var execRes = flinkContainer.execInContainer(execCmd.toArray(new String[0]));

0 commit comments

Comments
 (0)