import parquet-wasm from local bundle rather than CDN

[cornhundred]

Hi, is it possible to import parquet-wasm using esbuild rather than a CDN? I see that parquet-wasm is a dependency in the package.json

lonboard/package.json

Lines 5 to 20 in c8cf2e1

	"dependencies": {
	"@anywidget/react": "^0.0.4",
	"@babel/runtime": "^7.24.1",
	"@deck.gl/core": "^8.9.35",
	"@deck.gl/extensions": "^8.9.35",
	"@deck.gl/layers": "^8.9.35",
	"@deck.gl/react": "^8.9.35",
	"@geoarrow/deck.gl-layers": "^0.3.0-beta.16",
	"apache-arrow": "^15.0.2",
	"maplibre-gl": "^3.6.2",
	"parquet-wasm": "0.5.0",
	"react": "^18.2.0",
	"react-dom": "^18.2.0",
	"react-map-gl": "^7.1.7",
	"uuid": "^9.0.1"
	},

but lonboard is also referencing a CDN here:

lonboard/src/parquet.ts

Lines 2 to 19 in c8cf2e1

	import _initParquetWasm, { readParquet } from "parquet-wasm/esm/arrow2";
	import * as arrow from "apache-arrow";
	// NOTE: this version must be synced exactly with the parquet-wasm version in
	// use.
	const PARQUET_WASM_VERSION = "0.5.0";
	const PARQUET_WASM_CDN_URL = `https://cdn.jsdelivr.net/npm/parquet-wasm@${PARQUET_WASM_VERSION}/esm/arrow2_bg.wasm`;
	let WASM_READY: boolean = false;
	export async function initParquetWasm() {
	if (WASM_READY) {
	return;
	}
	await _initParquetWasm(PARQUET_WASM_CDN_URL);
	WASM_READY = true;
	}

I ask because I'm trying to test out lonboard on a service that prevents me from making front-end requests to CDNs (I would have to request the URL be added to an allowlist, but I want to see if I can avoid this by bundling the code with esbuild but I was running into errors when I tried to use the bundled version - kylebarron/parquet-wasm#488).

[kylebarron]

I agree, this change would be nice. But it's unclear to me how to get a reference to the local URL within Jupyter to the current bundle. I asked in the anywidget discord

[cornhundred]

Thanks @kylebarron, I was able to find a workaround that might be relevant here kylebarron/parquet-wasm#488 (comment)

[kylebarron]

Trevor responded on discord that serializing the wasm on the model is probably the best approach

[prusswan]

Until #461 gets merged, the easiest way (imho) is to let user specify a different cdn url (e.g. http://localhost:8888/files/). For now, we need to resort to modifying the js bundle.

[kylebarron]

What's your use case where you can't use the default CDN?

[prusswan]

Offline or standalone usage

…

On Fri, Oct 11, 2024, 4:47 AM Kyle Barron ***@***.***> wrote: What's your use case where you can't use the default CDN? — Reply to this email directly, view it on GitHub <#457 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFEBYWBHLNJEADOMVJ4X33Z23RWFAVCNFSM6AAAAABPXMRYFGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMBWGAYDSOJSGM> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[kylebarron]

But offline you won't be able to see the base map; does that not matter for you?

[prusswan]

But offline you won't be able to see the base map; does that not matter for you?

Usually I have a copy of the tiles for performance/practical reasons - it is faster than getting over the internet, while in other cases I am generating them anyway. If BitmapTileLayer (which already works with locally hosted Jupyter file urls) can be expanded to support/include pmtiles it would be perfect (sounds like it is in the works with #494)