Skip to content

Commit f1d1dc1

Browse files
committed
[SPARK-45706][PYTHON][DOCS] Makes entire Binder build fails fast during setting up
### What changes were proposed in this pull request? This PR proposes to make entire Binder build fails fast during setting up to prevent the Binder image to be successfully built which it cannot be rebuilt later on the same commit. ### Why are the changes needed? Binder build is currently broken for Spark 3.5.0: https://mybinder.org/v2/gh/apache/spark/ce5ddad9903?filepath=python%2Fdocs%2Fsource%2Fgetting_started%2Fquickstart_df.ipynb Seems like we uploaded PySpark late into PyPI, and the installation steps just slightly ignored the failure (a user triggered the first docker image for Binder, and that's being reused at that time PySpark wasn't uploaded to PyPI). ![Screenshot 2023-10-27 at 5 42 26 PM](https://github.com/apache/spark/assets/6477701/9030e4a1-2afa-43a2-aee0-dda01abb46ce) ### Does this PR introduce _any_ user-facing change? Yes, it fixes the user-facing live notebooks (at https://spark.apache.org/docs/latest/api/python/index.html). ### How was this patch tested? Manually tested in my fork: - https://mybinder.org/v2/gh/HyukjinKwon/spark/ce5ddad9903?filepath=python%2Fdocs%2Fsource%2Fgetting_started%2Fquickstart_df.ipynb - https://mybinder.org/v2/gh/HyukjinKwon/spark/ce5ddad9903?filepath=python%2Fdocs%2Fsource%2Fgetting_started%2Fquickstart_connect.ipynb - https://mybinder.org/v2/gh/HyukjinKwon/spark/ce5ddad9903?filepath=python%2Fdocs%2Fsource%2Fgetting_started%2Fquickstart_ps.ipynb ### Was this patch authored or co-authored using generative AI tooling? No. Closes apache#43553 from HyukjinKwon/SPARK-45706. Authored-by: Hyukjin Kwon <gurwls223@apache.org> Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
1 parent efa891c commit f1d1dc1

1 file changed

Lines changed: 5 additions & 0 deletions

File tree

binder/postBuild

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,11 @@
2020
# This file is used for Binder integration to install PySpark available in
2121
# Jupyter notebook.
2222

23+
# SPARK-45706: Should fail fast. Otherwise, the Binder image is successfully
24+
# built, and it cannot be rebuilt.
25+
set -o pipefail
26+
set -e
27+
2328
VERSION=$(python -c "exec(open('python/pyspark/version.py').read()); print(__version__)")
2429
TAG=$(git describe --tags --exact-match 2>/dev/null)
2530

0 commit comments

Comments
 (0)