Extend progressive cache to bazel-managed Go and pip (#47007)

### What does this PR do?
Wire up `bazel`-managed Go and `pip` caches to `XDG_CACHE_HOME` in the `tools/bazel*` scripts, and add the corresponding paths to the progressive GitLab runner cache keyed on `.go-version` and `.python-version`.

### Motivation
Extend #43274's XDG-as-single-cache-root design to Go and `pip`, whose XDG support has been steadlessly growing from "Partial" to "Supported": https://wiki.archlinux.org/title/XDG_Base_Directory.

By landing in `$XDG_CACHE_HOME`, they inherit the progressive-cache policy from #46151: only `main` pushes to them, keeping growth bounded.
Keying on language version files further contains growth by resetting the cache at version upgrade boundaries rather than accumulating superseded artifacts.

### Additional Notes
No worry: the new cache paths are already excluded from omnibus source trees via `**/.cache/**/*` source filters.

Coming soon: we might want to leverage upcoming `--strict_repo_env` (with `bazel` 8.6.0, #47011), for which we'll anyway have to list propagated environment variables.

Near future: as the omnibus-bazel transition progresses, other caches (`cache_omnibus_ruby_deps`, `go_deps`, `go_tools_deps`, `go_tools_deps_arm64`, etc.) are expected to shrink until no longer applicable.

Co-authored-by: regis.desgroppes <regis.desgroppes@datadoghq.com>

Author: rdesgroppes

.gitlab/build/bazel/defs.yml

@@ -16,9 +16,13 @@
       policy: pull$BAZEL_CACHE_POLICY_SUFFIX
       when: on_success
     - &bazel_cache
-      key: bazel-$CI_RUNNER_DESCRIPTION
+      key:
+        prefix: bazel-$CI_RUNNER_DESCRIPTION
+        files: [ .go-version, .python-version ]
       paths:
         - .cache/bazel/*/cache
+        - .cache/go-build
+        - .cache/go/mod
         - .cache/pip
       policy: pull$BAZEL_CACHE_POLICY_SUFFIX
       when: on_success

tools/bazel

@@ -18,8 +18,19 @@ if [ ! -d "${XDG_CACHE_HOME:-}" ]; then
     # shellcheck disable=SC2016
     >&2 echo '    docker run --env=XDG_CACHE_HOME=/cache --volume="$HOME/.cache:/cache" ...'
   fi
-elif [ -n "${CI:-}" ]; then
-  exec "$BAZEL_REAL" ${1:+"$1" --config=ci} "${@:2}"
+fi
+
+# Ensure `bazel` & managed toolchains honor `XDG_CACHE_HOME` as per https://wiki.archlinux.org/title/XDG_Base_Directory
+if [ -n "${XDG_CACHE_HOME:-}" ]; then
+  if [[ "$XDG_CACHE_HOME" != /* ]]; then
+    >&2 echo "🔴 XDG_CACHE_HOME ($XDG_CACHE_HOME) must denote an absolute path!"
+    exit 2
+  fi
+  unset GOCACHE                              # https://pkg.go.dev/os#UserCacheDir
+  export GOMODCACHE="$XDG_CACHE_HOME"/go/mod # https://wiki.archlinux.org/title/XDG_Base_Directory#Partial
+  unset PIP_CACHE_DIR                        # https://pip.pypa.io/en/stable/topics/caching/#default-paths
+
+  [ -n "${CI:-}" ] && exec "$BAZEL_REAL" ${1:+"$1" --config=ci} "${@:2}"
 fi
 
 exec "$BAZEL_REAL" "$@"

tools/bazel.bat

@@ -21,8 +21,19 @@ if not exist "%XDG_CACHE_HOME%" (
   )
 )
 
-:: Make `bazel` honor $XDG_CACHE_HOME if set as it does on POSIX OSes: https://github.com/bazelbuild/bazel/issues/27808
+:: Ensure `bazel` & managed toolchains honor `XDG_CACHE_HOME` if set: https://github.com/bazelbuild/bazel/issues/27808
 if defined XDG_CACHE_HOME (
+  set "XDG_CACHE_HOME=!XDG_CACHE_HOME:/=\!"
+  if "!XDG_CACHE_HOME:~1,2!" neq ":\" if "!XDG_CACHE_HOME:~0,2!" neq "\\" (
+    >&2 echo 🔴 XDG_CACHE_HOME ^(!XDG_CACHE_HOME!^) must denote an absolute path!
+    exit /b 2
+  )
+  :: https://pkg.go.dev/os#UserCacheDir
+  set "GOCACHE=%XDG_CACHE_HOME%\go-build"
+  :: https://wiki.archlinux.org/title/XDG_Base_Directory#Partial
+  set "GOMODCACHE=%XDG_CACHE_HOME%\go\mod"
+  :: https://pip.pypa.io/en/stable/topics/caching/#default-paths
+  set "PIP_CACHE_DIR=%XDG_CACHE_HOME%\pip"
   set "bazel_home=%XDG_CACHE_HOME%\bazel"
   set bazel_home_startup_option="--output_user_root=!bazel_home!"
 ) else (