[FEAT] `hatchet.isolate` to integrate with various sandbox or isolation providers

[abelanger5]

Is your feature request related to a problem? Please describe.

(Experimental/WIP)

A common request is to run one part of a Hatchet workflow within a sandbox or isolate, like Modal, E2B, Daytona, etc.

We'd like to build in experimental support for sandbox providers via hatchet.isolate to wrap a task. The isolate class will need to accept a factory to support configuration across multiple providers. This isolate class will need to expose a common set of methods, such as resume, snapshot, start, etc.

We are also interested in providing proxy support for routing OpenTelemetry traces from the isolate to Hatchet; this will need some investigation.

[gregfurman]

FYI Docker has released new sandbox functionality that is already available in newer versions via a sandboxd.sock unix socket/daemon (see https://docs.docker.com/reference/cli/docker/sandbox/) There's no first-class support for it via the SDK but I've done some tinkering and we could easily just make direct http calls to the sandbox socket i.e

curl --unix-socket ~/.docker/sandboxes/sandboxd.sock http://localhost/vm

While this functionality has been deprecated in favour of a dedicated sbx CLI that requires auth (see https://docs.docker.com/ai/sandboxes/usage/), the direct socket approach is definitely still relevant 👀