feat: library / embedded mode by mnafees · Pull Request #4355 · hatchet-dev/hatchet

mnafees · 2026-07-05T22:12:21Z

Description

Adds the ability to spawn an embedded Hatchet engine from a Go SDK client simply by providing a URL to a PostgreSQL DB.

import (
    hatchet "github.com/hatchet-dev/hatchet/sdks/go"

    _ "github.com/hatchet-dev/hatchet/embed"
)

client, err := hatchet.NewClient(hatchet.WithEmbeddedPostgres(databaseURL))
if err != nil {
    return err
}
defer func() { _ = client.Close(context.Background()) }()

This runs a full Hatchet engine (no UI) in-process with the Go binary it is embedded in.

Partially resolves #3680

Type of change

New feature (non-breaking change which adds functionality)

Checklist

Changes have been:

Tested (unit, integration, or manually with steps specified)
Linted and formatted

🤖 AI Disclosure

I acknowledge that an LLM was used in the creation of this Pull Request, in accordance with Hatchet's AI_POLICY.md.

Details: [e.g. generating tests, writing docs]

vercel · 2026-07-05T22:12:26Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
hatchet-docs	Ready	Preview, Comment	Jul 17, 2026 4:08pm

github-actions · 2026-07-12T21:22:03Z

⚠️ Optional test failure: The load-deadlock job failed on this PR (optimistic-scheduling=false). This check is non-mandatory and does not block merging, but may be worth investigating. View logs

Copilot

Pull request overview

Adds a new in-process “embedded engine” mode via github.com/hatchet-dev/hatchet/embed, enabling Go applications to start a Hatchet engine (API + engine, no UI) inside the host process and obtain a configured Go SDK client. This advances the “library-only mode” effort from #3680 by providing a runnable embed package plus example binaries/scripts.

Changes:

Introduces the new embed package (startup, config options, keyset persistence, version resolution, fleet sizing) and an embed/example module with engine/worker/trigger/basic examples.
Refactors advisory-lock hashing into pkg/repository/sqlchelpers.AdvisoryLockKey and reuses it from pkg/repository/task.go.
Adds a local dev runner script for embedded mode and minor workflow/task tweaks.

Reviewed changes

Copilot reviewed 16 out of 17 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
Taskfile.yaml	Makes version bump tasks fail-fast via `set -e`.
pkg/repository/task.go	Switches advisory-lock key hashing to shared helper.
pkg/repository/sqlchelpers/advisory.go	Adds shared advisory-lock key hashing helper.
hack/dev/embed/run.sh	Adds a local script to spin up multiple embedded engines/workers and trigger runs.
embed/version.go	Resolves Hatchet module version from build info for embedded startup.
embed/options.go	Adds embed configuration options and validation.
embed/keyset.go	Persists/loads local encryption keysets with an advisory lock.
embed/fleet.go	Checks active fleet size from DB to report “joining a fleet” status.
embed/example/worker/main.go	Example worker process for embedded engines.
embed/example/trigger/main.go	Example trigger tool to run workflows and summarize distribution.
embed/example/go.sum	Dependency lockfile for the standalone example module.
embed/example/go.mod	Standalone example module (with `replace` to local repo).
embed/example/engine/main.go	Example embedded engine launcher that writes connection info to a file.
embed/example/basic/main.go	Minimal single-process embedded engine + worker example.
embed/embed.go	Core embedded instance lifecycle (migrations, seeding, token minting, goroutine startup, SDK client).
cmd/hatchet-engine/main.go	Removes an outdated FIXME comment on Version.
.github/workflows/build.yml	Normalizes quoting for python-version values.

mnafees · 2026-07-13T11:09:29Z

+	if err := os.Setenv("DATABASE_URL", cfg.postgresURL); err != nil {
+		return nil, fmt.Errorf("could not set DATABASE_URL: %w", err)
+	}
+	if cfg.adminEmail != "" {
+		_ = os.Setenv("ADMIN_EMAIL", cfg.adminEmail)
+	}
+	if cfg.adminPassword != "" {
+		_ = os.Setenv("ADMIN_PASSWORD", cfg.adminPassword)
+	}


I don't like this since I think it makes sense to simply override these env vars from within the embed engine since we have config level options. Open to what you think @abelanger5

+	fleetStatus := "starting a new fleet"
+	if fleetSize > 0 {
+		fleetStatus = fmt.Sprintf("joining a fleet of %d engine(s)", fleetSize)
+	}
+	fmt.Fprintf(os.Stderr, "embed engine ready: api=%s grpc=%s | %s\n", apiURL, grpcBroadcast, fleetStatus)
+


+	fmt.Fprintf(os.Stderr, "embed engine ready: api=%s grpc=%s | %s\n", apiURL, grpcBroadcast, fleetStatus)
+


mnafees · 2026-07-13T11:04:50Z

The idea here is to have a shared hatchet_embed.keyset DB table which keeps the autogenerated keyset information for such embedded instances and the user does not need to generate a keyset themselves.

When WithKeysets is provided we always use that instead of this path.

Can convert to required sqlc constructs once this approach is approved.

mnafees · 2026-07-13T11:07:26Z

+
+const hatchetModulePath = "github.com/hatchet-dev/hatchet"
+
+func resolveVersion() (string, error) {


Just resolving the library tagged version from the respective go.mod file

github-actions · 2026-07-13T11:34:48Z

⚠️ Optional test failure: The load-deadlock job failed on this PR (optimistic-scheduling=true). This check is non-mandatory and does not block merging, but may be worth investigating. View logs

github-actions · 2026-07-13T11:35:50Z

⚠️ Optional test failure: The load-deadlock job failed on this PR (optimistic-scheduling=true). This check is non-mandatory and does not block merging, but may be worth investigating. View logs

github-actions · 2026-07-13T11:36:07Z

⚠️ Optional test failure: The load-deadlock job failed on this PR (optimistic-scheduling=false). This check is non-mandatory and does not block merging, but may be worth investigating. View logs

github-actions · 2026-07-16T15:03:04Z

⚠️ Optional test failure: The load-deadlock job failed on this PR (optimistic-scheduling=false). This check is non-mandatory and does not block merging, but may be worth investigating. View logs

mnafees added 26 commits July 1, 2026 12:06

initial commit for no-auth local mode

c1ce69f

no-auth authz

e0eb1c0

CLI and docs

31afe32

fix build error

62f1aaf

error boundary

afeb5fb

documentation changes

9e16f52

local keyset

ad36f41

Merge branch 'main' into nafees/local-no-auth

b398a8d

no auth keyset in hatchet lite + docs

3160acb

rm stale file

1b90b11

edit doc

7b91740

no auth user default to seed admin

5ee5b2e

fmt docs

4a3f9cb

simple test for noauth tokens

8190d2c

Merge branch 'main' into nafees/local-no-auth

1239b41

auth disabled go build tag

b574633

dev image

a2621cf

rename strings

f7b85f6

rename files

ee09f00

update docs

186511d

prose rewording

2801cf2

upload authmode disabled keyset

7e70b77

format

6f35c6a

remove code

955ceca

fix precommitlint

e84bee1

embed mode: initial commit

aa4700c

mnafees self-assigned this Jul 5, 2026

github-actions Bot added github_actions Pull requests that update GitHub Actions code sdk-go Related to the Go SDK labels Jul 5, 2026

vercel Bot deployed to Preview July 12, 2026 21:13 View deployment

runnable example

ceb615f

vercel Bot deployed to Preview July 12, 2026 21:53 View deployment

mnafees marked this pull request as ready for review July 13, 2026 10:53

mnafees requested review from abelanger5 and Copilot July 13, 2026 10:53

Copilot started reviewing on behalf of mnafees July 13, 2026 10:53 View session

Copilot AI reviewed Jul 13, 2026

View reviewed changes

mnafees commented Jul 13, 2026

View reviewed changes

Merge branch 'main' into nafees/library-mode

ae90179

mnafees commented Jul 13, 2026

View reviewed changes

mnafees added 2 commits July 13, 2026 13:21

migrate returns an error

78536f7

Merge branch 'main' into nafees/library-mode

ac66915

vercel Bot deployed to Preview July 13, 2026 11:27 View deployment

abelanger5 reviewed Jul 15, 2026

View reviewed changes

Comment thread cmd/hatchet-engine/main.go Outdated

Comment thread embed/example/trigger/main.go Outdated

Comment thread pkg/repository/task.go Outdated

mnafees added 2 commits July 16, 2026 15:46

Merge branch 'main' into nafees/library-mode

6ce2849

in Go SDK

398025a

github-actions Bot added the sdk-go Related to the Go SDK label Jul 16, 2026

vercel Bot deployed to Preview July 16, 2026 14:35 View deployment

fix api off

97d3ffa

vercel Bot deployed to Preview July 16, 2026 14:53 View deployment

abelanger5 reviewed Jul 17, 2026

View reviewed changes

Comment thread embed/example/trigger/main.go Outdated

mnafees added 2 commits July 17, 2026 17:54

Merge branch 'main' into nafees/library-mode

1e5e83c

auto resolve ports

9af0b82

vercel Bot deployed to Preview July 17, 2026 16:08 View deployment

		fmt.Fprintf(os.Stderr, "embed engine ready: api=%s grpc=%s \| %s\n", apiURL, grpcBroadcast, fleetStatus)


		const hatchetModulePath = "github.com/hatchet-dev/hatchet"

		func resolveVersion() (string, error) {

Uh oh!

Conversation

mnafees commented Jul 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

Checklist

Uh oh!

vercel Bot commented Jul 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 12, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

mnafees Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mnafees Jul 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mnafees Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Jul 13, 2026

Uh oh!

github-actions Bot commented Jul 13, 2026

Uh oh!

github-actions Bot commented Jul 13, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Jul 16, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mnafees commented Jul 5, 2026 •

edited

Loading

vercel Bot commented Jul 5, 2026 •

edited

Loading

mnafees Jul 13, 2026 •

edited

Loading