Share the ~200MB VTK/Qt/TBB runtime cache across Pkg.updates; add update!()

A plain `Pkg.add`-installed package lives in a content-hashed folder
that gets a BRAND NEW hash on every single Pkg.update, even for a
one-line .jl change. deps/build.jl was extracting the full runtime zip
into that ephemeral folder, so every update silently re-downloaded and
re-extracted the whole ~200MB VTK/Qt/TBB bundle again. Fixed: the
runtime now lives at ~/.julia/gmtvtk_runtime/, keyed off the shared
Julia depot rather than the per-version package folder, so it's fetched
once ever. src/libgmtvtk.jl looks there too (falling back to it only
when there's no local deps/build/gmtvtk.dll, so an active dev build via
deps/build.bat always wins first). Verified both files resolve to the
identical path and the extraction layout is correct.

Add src/selfupdate.jl (InteractiveGMT.update!()): for a `] dev`-installed
checkout (fixed directory, never moves across updates -- unlike
Pkg.add), pulls the latest source in place via Julia's BUNDLED LibGit2
(no system git.exe required) and rebuilds. Pkg.update() deliberately
skips dev'd packages, so this fills that gap. Verified fetch+merge!
against this actual repo.

Adds LibGit2 + Pkg (both stdlibs) to Project.toml deps -- needed by
selfupdate.jl, same reasoning as the earlier Downloads fix.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Author: joa-quim

Project.toml

@@ -7,11 +7,15 @@ version = "0.1.0"
 Distributed = "8ba89e20-285c-5b6f-9357-94700520ee1b"
 Downloads = "f43a241f-c20a-4ad4-852c-f6b1247861c6"
 GMT = "5752ebe1-31b9-557e-87aa-f909b540aa54"
+LibGit2 = "76f85450-5226-5b5a-8eaa-529ad045b433"
+Pkg = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"
 PrecompileTools = "aea7be01-6a6a-4083-8856-8a6e6704d82a"
 
 [compat]
 Downloads = "1"
 GMT = "1.40"
+LibGit2 = "1"
+Pkg = "1"
 julia = "1.10"
 
 [extras]

deps/build.jl

@@ -4,21 +4,32 @@
 #
 #   * FULL runtime zip (gmtvtk.dll + bundled VTK/Qt/TBB + Qt plugins) — changes rarely,
 #     only when the VTK/Qt/TBB module set changes. Pinned by deps/RUNTIME_VERSION (a git
-#     tag, e.g. "runtime-0.1"). Downloaded ONCE, on first install (no marker in deps/build/).
+#     tag, e.g. "runtime-0.1"). Downloaded ONCE EVER (see SHARED_ROOT below).
 #
 #   * DLL-ONLY zip (just gmtvtk.dll) — can change daily as the C++ side is edited. Lives at
 #     a FIXED, reused release tag (DLL_TAG below); its one asset gets overwritten in place
 #     (`gh release upload dll-latest gmtvtk-win64.zip --clobber`) — no new tag per day.
 #     Re-downloaded on every `Pkg.build("InteractiveGMT")`.
-
+#
+# A regular `Pkg.add`-installed (non-dev) package lives in a content-hashed folder
+# (~/.julia/packages/InteractiveGMT/<hash>/) that gets a BRAND NEW <hash> on every single
+# Pkg.update, even for a one-line .jl change unrelated to the C++ side. If the ~200 MB
+# VTK/Qt/TBB runtime were extracted INTO that folder (as an earlier version of this file did),
+# every update would silently re-download and re-extract the entire runtime again --
+# unacceptable. Fix: extract the runtime into SHARED_ROOT, a location keyed off the Julia
+# DEPOT itself (~/.julia), not off this ephemeral package folder -- the same physical spot
+# survives every Pkg.update, `Pkg.add` or `Pkg.develop` alike, so the runtime is fetched once,
+# ever, no matter how many times the package updates. src/libgmtvtk.jl looks in this same
+# SHARED_ROOT (falling back to it only when there's no LOCAL deps/build/gmtvtk.dll -- i.e. a
+# developer's own `deps/build.bat` build always wins first).
 using Downloads
 
 const REPO    = "GenericMappingTools/InteractiveGMT"
 const DLL_TAG = "dll-latest"   # fixed tag; its one asset is re-uploaded in place, never retagged
 
-const DEPS_DIR = @__DIR__
-const PKG_ROOT = normpath(joinpath(DEPS_DIR, ".."))   # zip paths are relative to here (deps/build/..., src/..., data/...)
-const MARKER   = joinpath(DEPS_DIR, "build", ".full_runtime_installed")
+const DEPS_DIR     = @__DIR__
+const SHARED_ROOT  = joinpath(first(Base.DEPOT_PATH), "gmtvtk_runtime")   # survives every Pkg.update; zip paths (deps/build/...) are relative to here
+const MARKER       = joinpath(SHARED_ROOT, "deps", "build", ".full_runtime_installed")
 
 function runtime_tag()
     f = joinpath(DEPS_DIR, "RUNTIME_VERSION")
@@ -35,13 +46,10 @@ release_url(tag::String, asset::String) =
 # "does not look like a tar archive" / "Error exit delayed from previous errors".
 const TAR = joinpath(get(ENV, "SystemRoot", "C:\\Windows"), "System32", "tar.exe")
 
-# Both zips also contain Project.toml/data/src (the "full" one, for the standalone
-# zip/NSIS user who isn't going through Julia Pkg at all) — but under Pkg those files
-# already exist on disk from the git checkout, and Pkg marks them READ-ONLY on Windows.
-# Extracting the whole archive then fails with "Can't stat existing object: Permission
-# denied" trying to overwrite them. Only deps/build/ (the binaries) is actually needed
-# here, so restrict extraction to that subtree — sidesteps the permission error AND
-# avoids redundantly re-writing files git already gave us.
+# The full zip also contains Project.toml/data/src (for the standalone zip/NSIS user who isn't
+# going through Julia Pkg at all) — irrelevant here since Pkg already gave us those via git, and
+# SHARED_ROOT only ever needs the binaries. Restrict extraction to deps/build/ so SHARED_ROOT
+# doesn't waste disk space on a redundant copy of data/ and src/.
 function fetch_and_extract(url::String, dest::String)
     isfile(TAR) || error("$TAR not found — need Windows 10 1803+ (bsdtar) to unzip gmtvtk binaries")
     zip = joinpath(tempdir(), basename(url))
@@ -58,14 +66,15 @@ end
 
 function main()
     if !isfile(MARKER)
-        # First install: full runtime bundle, pinned to a coarse, rarely-bumped tag.
-        fetch_and_extract(release_url(runtime_tag(), "iGMT-win64-full.zip"), PKG_ROOT)
+        # First install EVER on this machine: full runtime bundle, pinned to a coarse,
+        # rarely-bumped tag. Never repeated after this, even across many future updates.
+        fetch_and_extract(release_url(runtime_tag(), "iGMT-win64-full.zip"), SHARED_ROOT)
         touch(MARKER)
     else
-        # Update: DLL only, always the same rolling tag/asset.
-        fetch_and_extract(release_url(DLL_TAG, "gmtvtk-win64.zip"), PKG_ROOT)
+        # Every subsequent build: DLL only (~1 MB), always the same rolling tag/asset.
+        fetch_and_extract(release_url(DLL_TAG, "gmtvtk-win64.zip"), SHARED_ROOT)
     end
-    @info "InteractiveGMT: gmtvtk binaries installed"
+    @info "InteractiveGMT: gmtvtk binaries installed" SHARED_ROOT
 end
 
 main()

src/InteractiveGMT.jl

@@ -20,6 +20,7 @@ using PrecompileTools: @setup_workload, @compile_workload
 
 # --- C-API DLL loader (resolved at runtime in __init__; see libgmtvtk.jl) ----------------
 include("libgmtvtk.jl")
+include("selfupdate.jl") # update!() -- pull + rebuild in place, for a `] dev`-installed checkout
 
 # --- handles, event loop, in-window Julia console ----------------------------------------
 include("types.jl")

src/libgmtvtk.jl

@@ -14,16 +14,30 @@ const _PKGROOT = normpath(joinpath(@__DIR__, ".."))
 # Libdl without adding it to [deps]: the stdlib module is re-exposed inside Base.
 const Libdl = Base.Libc.Libdl
 
+# Where's gmtvtk.dll? Checked in order:
+#   1. A LOCAL dev build (deps/build/gmtvtk.dll next to this checkout, from deps/build.bat) --
+#      always wins first, so a developer actively rebuilding the DLL never picks up a stale
+#      cached copy.
+#   2. SHARED_LIB: the depot-wide runtime cache deps/build.jl fetches into
+#      (~/.julia/gmtvtk_runtime/deps/build/gmtvtk.dll) -- this is the Pkg.add/Pkg.develop path.
+#      It's keyed off the Julia DEPOT itself, not off this package's own (possibly
+#      content-hashed, possibly-different-every-update) folder, so the SAME ~200 MB VTK/Qt/TBB
+#      runtime is reused across every future update instead of being re-fetched into a new
+#      folder each time.
+const _LOCAL_LIB  = joinpath(_PKGROOT, "deps", "build", "gmtvtk.dll")
+const _SHARED_LIB = joinpath(first(Base.DEPOT_PATH), "gmtvtk_runtime", "deps", "build", "gmtvtk.dll")
+const _LIB      = isfile(_LOCAL_LIB) ? _LOCAL_LIB : _SHARED_LIB
+const _BIN_DIR  = dirname(_LIB)
+
 # Toolchain runtime DLL dirs (this machine). Dependent DLLs (Qt6*, vtk*) resolve from PATH at
 # load time. Override any of these via the matching ENV var BEFORE `using InteractiveGMT`.
 #
 # A GMTVTK_PACKAGE=ON build (see deps/CMakeLists.txt) drops every VTK/Qt/TBB runtime DLL plus the
 # Qt platform plugins (platforms/qwindows.dll, via windeployqt) into deps/build/ next to
-# gmtvtk.dll itself — that's the NSIS-installed layout, with no VTK/Qt toolchain on the
-# destination machine at all. Detect that bundle and point straight at it; otherwise fall back
-# to this dev machine's hard-coded toolchain paths (ENV overrides always win either way).
-const _LIB      = joinpath(_PKGROOT, "deps", "build", "gmtvtk.dll")
-const _BIN_DIR  = dirname(_LIB)
+# gmtvtk.dll itself — that's the NSIS-installed / shared-runtime-cache layout, with no VTK/Qt
+# toolchain on the destination machine at all. Detect that bundle and point straight at it;
+# otherwise fall back to this dev machine's hard-coded toolchain paths (ENV overrides always
+# win either way).
 const _BUNDLED  = isdir(joinpath(_BIN_DIR, "platforms"))
 const _VTK_BIN = get(ENV, "INTERACTIVEGMT_VTK_BIN", _BUNDLED ? _BIN_DIR : raw"C:\programs\compa_libs\VTK-9.6.2\compileds\bin")
 const _QT_BIN  = get(ENV, "INTERACTIVEGMT_QT_BIN",  _BUNDLED ? _BIN_DIR : raw"C:\programs\Qt6\6.11.1\msvc2022_64\bin")

src/selfupdate.jl

@@ -0,0 +1,50 @@
+# selfupdate.jl — update!() pulls the latest InteractiveGMT source in place and rebuilds the
+# binaries, for a `] dev`-installed checkout ONLY.
+#
+# Why this exists: `] dev https://github.com/GenericMappingTools/InteractiveGMT` clones ONCE to
+# a fixed, permanent directory (~/.julia/dev/InteractiveGMT by default) that never moves again --
+# unlike a plain `Pkg.add`, which re-checks-out into a brand NEW content-hashed folder on every
+# single `Pkg.update`. A fixed directory means a Desktop shortcut never goes stale. The one thing
+# `dev` doesn't give you for free is an update mechanism: Pkg.update() deliberately skips dev'd
+# packages (you're expected to manage their git state yourself). update!() is that missing piece
+# — using Julia's BUNDLED LibGit2, not a system `git.exe`, so end users never need git installed.
+
+using LibGit2
+import Pkg
+
+"""
+    InteractiveGMT.update!()
+
+Pull the latest InteractiveGMT source in place (fast-forward only) and rebuild the binaries.
+Only works for a `] dev`-installed checkout — a plain `Pkg.add` install should use
+`Pkg.update("InteractiveGMT")` instead.
+"""
+function update!()
+	isdir(joinpath(_PKGROOT, ".git")) || error(
+		"InteractiveGMT at $_PKGROOT isn't a git checkout -- update! only works for a " *
+		"`] dev`-installed copy. For a plain `Pkg.add` install, use " *
+		"Pkg.update(\"InteractiveGMT\") instead.")
+
+	repo = LibGit2.GitRepo(_PKGROOT)
+	try
+		@info "InteractiveGMT: fetching latest changes..." path=_PKGROOT
+		LibGit2.fetch(repo)
+		before = LibGit2.head_oid(repo)
+		ok = LibGit2.merge!(repo; fastforward=true)
+		ok || error("InteractiveGMT: local changes or diverged history -- couldn't fast-forward. " *
+		            "This checkout is a normal git repo at $_PKGROOT; resolve manually (e.g. `git status`).")
+		after = LibGit2.head_oid(repo)
+		if before == after
+			@info "InteractiveGMT: already up to date."
+			return nothing
+		end
+		@info "InteractiveGMT: updated." from = string(before)[1:8] to = string(after)[1:8]
+	finally
+		close(repo)
+	end
+
+	@info "InteractiveGMT: rebuilding binaries..."
+	Pkg.build("InteractiveGMT")
+	@info "InteractiveGMT: update complete. Restart Julia to use the new version."
+	return nothing
+end