feat: in-process embedded permissions and per-schema-version caching by josephschorr · Pull Request #3166 · authzed/spicedb

josephschorr · 2026-06-09T02:22:43Z

Summary

Adds the ability to embed SpiceDB's permission engine in-process (no gRPC server) and makes per-schema-version caching of derived artifacts possible, eliminating repeated work on the check hot path.

Changes

datalayer: cache schema-derived artifacts on the stored-schema cache. The stored-schema cache already shares one instance per schema version (keyed by schema hash). Bundle lazily-built, schema-derived caches (compiled caveats today; type systems / reachability later) into that instance via a new CachedSchema type, so they share the schema's lifetime and are discarded, together, when the schema changes. Register a kind with RegisterDerivedCache; retrieve the per-schema instance with GetDerivedCache. The stored-schema cache now holds *datalayer.CachedSchema; pkg/datastore is unchanged.
datastore: honor schema mode when writing bootstrap data. NewDatastore's bootstrap always wrote schema in legacy per-definition form regardless of the configured schema mode, so a server reading from the unified schema would find none. Adds a BootstrapSchemaMode option; the zero value preserves prior behavior.
caveats: cache compiled caveats per schema version. ComputeCheck constructs a fresh CaveatRunner per call, so its per-runner deserialized-caveat cache never persists across checks. Deserializing a caveat rebuilds its CEL environment, which is expensive and — with a rich caveat type set — dominated check cost. This caches compiled caveats on the schema-derived cache, keyed by caveat name, so the CEL environment is built once per schema version rather than once per check. Falls back to per-runner caching when the reader is not backed by a unified stored schema.
embedded: add in-process permissions library. Adds pkg/embedded, a focused library for running permission checks in-process against a datastore via the dispatch engine — no gRPC, caveat context passed as native Go values. Includes a README and tests.
datastore: cost the stored-schema cache by estimated byte size. The cache previously admitted every entry at a fixed cost of 1, so a byte-denominated MaxCost (the server's 32MiB default, or the embedded SchemaCacheMaxCostBytes) actually bounded the number of cached schema versions rather than memory. Each entry is now costed by ReadOnlyStoredSchema.EstimatedSize(): a rough schema byte size plus, for each registered schema-derived cache kind, an estimate of the bytes it adds once populated. Datastore readers pass the exact serialized size they just read; other callers fall back to a cheap schema-text proxy, avoiding a full proto.Size walk (~1.5ms on a 1k-definition schema). RegisterDerivedCache now takes a size estimator, and the compiled-caveat cache estimates per-caveat CEL overhead.

Testing

mage lint:all passes; unit suite passes.
New tests: pkg/datalayer (derived-cache registry: lazy build, per-schema isolation, duplicate/unregistered handling), internal/caveats (compiled-caveat cache), pkg/embedded (checks across both legacy and unified schema modes, incl. caveated/conditional results), and stored-schema cost estimation (EstimatedSize reflects schema bytes plus registered derived-cache estimators).

codecov · 2026-06-09T02:26:21Z

Codecov Report

❌ Patch coverage is 78.28283% with 43 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
pkg/embedded/permissions.go	85.72%	8 Missing and 4 partials ⚠️
pkg/datastore/derivedcache.go	75.61%	7 Missing and 3 partials ⚠️
internal/caveats/run.go	75.00%	4 Missing and 2 partials ⚠️
internal/caveats/schemacache.go	76.93%	4 Missing and 2 partials ⚠️
pkg/datalayer/hashcache.go	62.50%	2 Missing and 1 partial ⚠️
pkg/datastore/datastore.go	50.00%	2 Missing and 1 partial ⚠️
pkg/validationfile/loader.go	25.00%	2 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

tstirrat15

See comments

tstirrat15 · 2026-06-15T16:20:13Z

+	if err := datalayer.RegisterDerivedCache(compiledCaveatCacheKey, func() any { return &CompiledCaveatCache{} }); err != nil {
+		spiceerrors.MustPanicf("failed to register compiled caveat cache: %v", err)
+	}


Are we expecting to have many of these? Like do we need a registry, or can this be singleton?

Registry, to avoid an import cycle: a typed field per cache kind would make pkg/datastore depend on the packages that own those caches (caveats now, type systems/reachability later) — which already depend on it. Registration inverts that. We only expect a handful.

tstirrat15 · 2026-06-15T16:21:24Z

+)
+
+// CachedSchema bundles a read-only stored schema with lazily-built, schema-derived caches
+// (for example compiled caveats, type systems, or reachability). Instances are produced and


These being the other examples of things that we would register?

Yes — compiled caveats today; type systems and reachability later.

tstirrat15 · 2026-06-15T16:22:06Z

+
+// DerivedCacheKey identifies a kind of schema-derived cache (e.g. compiled caveats). Create
+// one per kind, typically as a package-level var, via NewDerivedCacheKey.
+type DerivedCacheKey struct{ name string }


For my own edification: why is this a struct with a single member, rather than a subtype of string directly?

So the key is opaque — it can only come from NewDerivedCacheKey, not a bare string literal at some call site. Can make it a string subtype if you'd rather.

tstirrat15 · 2026-06-15T16:23:03Z

+// GetDerivedCache returns the schema-derived cache of type T registered under key for the
+// given cached schema, building it lazily on first access.
+func GetDerivedCache[T any](c *CachedSchema, key DerivedCacheKey) T {
+	return c.derivedCache(key).(T)


MustBugF here? Or are we not worried about a panic here?

Yeah — it now returns an error instead of relying on the bare type assertion.

tstirrat15 · 2026-06-15T16:28:44Z

+	// SchemaCacheMaxCostBytes, when > 0, enables an in-memory stored-schema cache of the
+	// given size in bytes. Caching the stored schema across checks is what allows
+	// schema-derived caches (e.g. compiled caveats) to persist; strongly recommended when
+	// SchemaMode reads from the unified schema.
+	SchemaCacheMaxCostBytes int64


curious - why do we need a max cost here but not on the other schemas?

It's the memory bound for the one cache the embedded library stands up. It's a real byte budget now — each entry is costed by the schema size plus the derived caches hanging off it.

tstirrat15 · 2026-06-15T16:29:20Z

+}
+
+// Permissions issues in-process permission checks against a datastore.
+type Permissions struct {


PermissionsChecker? Or is the idea that this is PermissionsService minus the Service?

Holding the name until the API shape settles (your thread with miparnisari).

tstirrat15 · 2026-06-15T16:30:48Z

+// CheckRequest is a single, fully-consistent permission check. The caveat context is passed
+// as native Go values (no structpb / base64 round-trip).
+type CheckRequest struct {


Maybe mark that this should track the protobuf API shape? Is there a way to get a derived golang shape from the protobuf shape? Like I'm assuming this is because we don't want to pay the serde cost on Structs?

Exactly — passing native Go values avoids building a structpb.Struct just for the server to convert it straight back. Added a note that it mirrors v1.CheckPermissionRequest and should be kept in sync.

tstirrat15 · 2026-06-15T16:31:48Z

+	}
+}
+
+// Close releases resources held by the checker. It does not close the datastore.


With the datastore, is that because we don't want to have to do dependency checking on the datastore?

The caller owns the datastore — it may be shared and outlive the checker, so closing it here would be wrong. Close only releases what this object created.

miparnisari

I'm not a fan of the approach taken in this PR. You're constructing things that we already have (e.g you wrote a new CheckRequest object and embedded.NewPermissions) and we don't offer the entire API surface. We are also losing request validation I think?

Can we do something like this instead? Reuse our existing objects:

type server struct {
    dl  datalayer
    v1svc.NewPermissionsServer
    v1svc.NewSchemaServer
    v1svc.NewWatchServer
}

// server inherits the entire API surface, plus:

func  New(dl datalayer, options ...) *server {
    // configure everything
    // including eg the protovalidate object
}

func (s *server) Close() {
}

It's probably more work, but i think it's better for the long run since we don't duplicate code.

If you look at our existing devcontext code, it looks very similar to what I suggested 😉 we could replace all of it with this server object.

spicedb/pkg/development/devcontext.go

Lines 169 to 217 in 9552207

    
           func (dc *DevContext) RunV1InMemoryService() (*grpc.ClientConn, func(), error) { 
        
           	listener := bufconn.Listen(defaultConnBufferSize) 
        
           	s := grpc.NewServer( 
        
           		grpc.ChainUnaryInterceptor( 
        
           			datalayer.UnaryServerInterceptor(dc.DataLayer), 
        
           			consistency.UnaryServerInterceptor("development", consistency.TreatMismatchingTokensAsError), 
        
           		), 
        
           		grpc.ChainStreamInterceptor( 
        
           			datalayer.StreamServerInterceptor(dc.DataLayer), 
        
           			consistency.StreamServerInterceptor("development", consistency.TreatMismatchingTokensAsError), 
        
           		), 
        
           	) 
        
           	validator, err := genutil.NewProtoValidator() 
        
           	if err != nil { 
        
           		return nil, nil, err 
        
           	} 
        
           	ps := v1svc.NewPermissionsServer(dc.Dispatcher, v1svc.PermissionsServerConfig{ 
        
           		MaxUpdatesPerWrite:               50, 
        
           		MaxPreconditionsCount:            50, 
        
           		MaximumAPIDepth:                  50, 
        
           		MaxCaveatContextSize:             0, 
        
           		ExpiringRelationshipsEnabled:     true, 
        
           		CaveatTypeSet:                    caveattypes.Default.TypeSet, 
        
           		PerformanceInsightMetricsEnabled: false, 
        
           	}, validator) 
        
           	ss := v1svc.NewSchemaServer(v1svc.SchemaServerConfig{ 
        
           		CaveatTypeSet:                    caveattypes.Default.TypeSet, 
        
           		AdditiveOnly:                     false, 
        
           		ExpiringRelsEnabled:              true, 
        
           		PerformanceInsightMetricsEnabled: false, 
        
           	}, validator) 
        
           	v1.RegisterPermissionsServiceServer(s, ps) 
        
           	v1.RegisterSchemaServiceServer(s, ss) 
        
           	go func() { 
        
           		if err := s.Serve(listener); err != nil { 
        
           			log.Err(err).Msg("error when serving in-memory server") 
        
           		} 
        
           	}() 
        
           	conn, err := grpchelpers.NewBufferedClient(listener) 
        
           	return conn, func() { 
        
           		conn.Close() 
        
           		listener.Close() 
        
           		s.Stop() 
        
           	}, err 
        
           }

josephschorr · 2026-06-24T20:37:33Z

I'm not a fan of the approach taken in this PR. You're constructing things that we already have (e.g you wrote a new CheckRequest object and embedded.NewPermissions) and we don't offer the entire API surface. We are also losing request validation I think?

Can we do something like this instead? Reuse our existing objects:

Not without re-introducing a requirement to encode/decode obejcts as structpb's, which is a loss of performance we should not need to pay, especially for this simplified use case

miparnisari · 2026-06-24T23:44:37Z

which is a loss of performance we should not need to pay

The penalty comes only in caveated checks. IMO it doesn't justify the maintenance burden that comes with the duplication and the poor DX that comes from only offering one API (CheckPermission).

josephschorr · 2026-06-25T17:38:33Z

which is a loss of performance we should not need to pay

The penalty comes only in caveated checks. IMO it doesn't justify the maintenance burden that comes with the duplication and the poor DX that comes from only offering one API (CheckPermission).

It is a non-trivial amount of overhead and many "in memory" systems do use caveats for dynamic or real-time behavior, so I do think it is worth it

Add a lazily-built, schema-derived cache layer to ReadOnlyStoredSchema. Because a single ReadOnlyStoredSchema instance is shared across callers for a given schema version via the stored-schema cache, derived caches (e.g. compiled caveats) hung off it live exactly as long as that schema version and are discarded, together, when the schema changes. Cache kinds register a factory at init via RegisterDerivedCache and are retrieved with the generic GetDerivedCache. The schema-reader adapter exposes the shared stored schema so consumers can reach it.

NewDatastore's bootstrap always wrote schema in legacy per-definition form regardless of the configured schema mode, so a server reading from the unified schema would find none. Add a BootstrapSchemaMode option so the bootstrap datalayer writes schema in the requested mode. The zero value preserves the prior (legacy) behavior. When bootstrapping the first schema under the unified schema, the populate path reads the (absent) existing schema to resolve references; tolerate ErrSchemaNotFound there and resolve against the definitions being written, the same way the schema-write service does.

Deserializing a caveat rebuilds its CEL environment, which is expensive. When the reader is backed by a unified stored schema, cache compiled caveats on that schema version (via the schema-derived cache on ReadOnlyStoredSchema) so the cost is paid once per schema rather than once per check. Falls back to per-runner caching when no stored schema is available.

Add pkg/embedded, a thin in-process wrapper over the dispatch engine (computed.ComputeCheck + a local-only dispatcher) that issues fully-consistent permission checks against a datastore without standing up a gRPC server. Caveat context is passed as native Go values, avoiding the structpb/base64 round-trip the gRPC path pays. The schema is optionally cached across checks, which lets the schema-derived compiled-caveat cache persist.

The stored-schema cache previously admitted every entry at a fixed cost of 1, so a MaxCost configured in bytes (e.g. the server's 32MiB default, or the embedded SchemaCacheMaxCostBytes) actually bounded the number of cached schema versions rather than memory. Cost each entry by ReadOnlyStoredSchema.EstimatedSize() instead: a rough schema byte size plus, for each registered schema-derived cache kind, an estimate of the bytes it adds once populated. Datastore readers pass the exact serialized size they just read; other callers fall back to a cheap schema-text proxy, avoiding a full proto walk (proto.Size is ~1.5ms on a 1k-definition schema). RegisterDerivedCache now takes a size estimator, and the compiled-caveat cache estimates per-caveat CEL overhead.

josephschorr requested a review from a team as a code owner June 9, 2026 02:22

github-actions Bot added area/cli Affects the command line area/api v1 Affects the v1 API area/tooling Affects the dev or user toolchain (e.g. tests, ci, build tools) labels Jun 9, 2026

josephschorr force-pushed the embedded-permissions branch 2 times, most recently from 96c22be to 7fc3209 Compare June 9, 2026 19:33

tstirrat15 reviewed Jun 15, 2026

View reviewed changes

miparnisari requested changes Jun 16, 2026

View reviewed changes

josephschorr force-pushed the embedded-permissions branch from 7fc3209 to 44a772b Compare June 25, 2026 17:37

github-actions Bot added area/datastore Affects the storage system and removed area/api v1 Affects the v1 API labels Jun 25, 2026

josephschorr added 5 commits June 25, 2026 12:12

josephschorr force-pushed the embedded-permissions branch from 44a772b to a6f9516 Compare June 25, 2026 19:12

	func (dc DevContext) RunV1InMemoryService() (grpc.ClientConn, func(), error) {
	listener := bufconn.Listen(defaultConnBufferSize)

	s := grpc.NewServer(
	grpc.ChainUnaryInterceptor(
	datalayer.UnaryServerInterceptor(dc.DataLayer),
	consistency.UnaryServerInterceptor("development", consistency.TreatMismatchingTokensAsError),
	),
	grpc.ChainStreamInterceptor(
	datalayer.StreamServerInterceptor(dc.DataLayer),
	consistency.StreamServerInterceptor("development", consistency.TreatMismatchingTokensAsError),
	),
	)
	validator, err := genutil.NewProtoValidator()
	if err != nil {
	return nil, nil, err
	}
	ps := v1svc.NewPermissionsServer(dc.Dispatcher, v1svc.PermissionsServerConfig{
	MaxUpdatesPerWrite: 50,
	MaxPreconditionsCount: 50,
	MaximumAPIDepth: 50,
	MaxCaveatContextSize: 0,
	ExpiringRelationshipsEnabled: true,
	CaveatTypeSet: caveattypes.Default.TypeSet,
	PerformanceInsightMetricsEnabled: false,
	}, validator)
	ss := v1svc.NewSchemaServer(v1svc.SchemaServerConfig{
	CaveatTypeSet: caveattypes.Default.TypeSet,
	AdditiveOnly: false,
	ExpiringRelsEnabled: true,
	PerformanceInsightMetricsEnabled: false,
	}, validator)

	v1.RegisterPermissionsServiceServer(s, ps)
	v1.RegisterSchemaServiceServer(s, ss)

	go func() {
	if err := s.Serve(listener); err != nil {
	log.Err(err).Msg("error when serving in-memory server")
	}
	}()

	conn, err := grpchelpers.NewBufferedClient(listener)
	return conn, func() {
	conn.Close()
	listener.Close()
	s.Stop()
	}, err
	}

Uh oh!

Conversation

josephschorr commented Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Testing

Uh oh!

codecov Bot commented Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

tstirrat15 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

miparnisari left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

josephschorr commented Jun 24, 2026

Uh oh!

miparnisari commented Jun 24, 2026

Uh oh!

josephschorr commented Jun 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

josephschorr commented Jun 9, 2026 •

edited

Loading

codecov Bot commented Jun 9, 2026 •

edited

Loading

miparnisari left a comment •

edited

Loading