Skip to content

Firestore emulator rules get() intermittently fails as PERMISSION_DENIED after transaction lock timeout #10518

@EthanSK

Description

@EthanSK

Environment info

firebase-tools: 15.13.0
Firestore emulator: cloud-firestore-emulator-v1.20.4
Platform: macOS 26.4.1 arm64
Node: v22.22.2
Java: OpenJDK 25.0.1

Test case

A Firestore rules helper validates project subcollection writes by calling get() on the parent project doc, then another get() on the channel collaborator doc. Under local emulator load, multiple concurrent client writes under the same project subcollections intermittently fail during rule evaluation.

Rules shape, simplified:

match /projects/{id} {
  match /{document=**} {
    allow create: if hasProjectPerm(id, 'project.update');
    allow update: if hasProjectPerm(id, 'project.update');
  }
}

function hasProjectPerm(projectId, perm) {
  let project = get(/databases/$(database)/documents/projects/$(projectId)).data;
  return project != null && hasChannelPerm(project.ownedByChannelId, perm);
}

function hasChannelPerm(channelId, perm) {
  return isAuthed() && channelRoleHasPerm(roleForChannelUser(channelId), perm);
}

function roleForChannelUser(channelId) {
  let collaboratorDoc = get(/databases/$(database)/documents/channels-private/$(channelId)/channel-collaborators/$(request.auth.uid)).data;
  return collaboratorDoc != null && collaboratorDoc.collaborationStatus == 'active' ? collaboratorDoc.role : null;
}

The client sends several setDoc(..., { merge: true }) writes concurrently to paths like:

  • projects/<projectId>/timeline-clip-timings/<timelineId>
  • projects/<projectId>/timeline-clip-timings/<timelineId>/clip-script-word-timings/<clipId>
  • projects/<projectId>/clips/<clipId>

The referenced project and collaborator docs exist, and the same authenticated user can perform controlled writes successfully when the emulator is not under this contention.

Steps to reproduce

  1. Start emulators with Firestore + Functions + Storage, with imported data and Firestore rules enabled:
firebase emulators:start --config=firebase.emulator.generated.json --only functions,firestore,storage --project staging --import=./emulator-export-data --export-on-exit=./emulator-export-data
  1. Run a UI flow that performs multiple concurrent writes under the same projects/<projectId>/... subtree. Each write hits the rules helper above.
  2. Watch the browser SDK receive FirebaseError: PERMISSION_DENIED even though permissions/data are valid.
  3. Inspect firestore-debug.log.

Expected behavior

Rules-side get() calls should either succeed using the existing documents or, if the emulator is overloaded, surface an emulator/internal/retryable failure. They should not be reported to the SDK as a security-rule PERMISSION_DENIED for valid permissions.

Actual behavior

The emulator intermittently reports a rules service call failure for get(/projects/<projectId>), wrapped as PERMISSION_DENIED:

evaluation error at L472:26 for 'create' @ L472, evaluation error at L473:26 for 'update' @ L473, Service call error. Function: [get], Argument: [path_value {
  segments { simple: "databases" }
  segments { simple: "(default)" }
  segments { simple: "documents" }
  segments { simple: "projects" }
  segments { simple: "<projectId>" }
}]. for 'update' @ L473

The same log contains repeated transaction lock timeouts from the emulator internals:

WARNING: Operation failed: Transaction lock timeout.
com.google.cloud.datastore.core.exception.DatastoreException: Transaction lock timeout.
    at com.google.cloud.datastore.emulator.impl.transactions.ReactiveLockManager.acquireLocks(ReactiveLockManager.java:70)
    at com.google.cloud.datastore.emulator.impl.storage.FlatLocalEntityStore$StronglyConsistentReadWriteView.<init>(FlatLocalEntityStore.java:147)
    at com.google.cloud.datastore.emulator.impl.storage.FlatLocalEntityStore.readWrite(FlatLocalEntityStore.java:67)
    at com.google.cloud.datastore.emulator.impl.firestore.FirestoreEmulatorHelper.commitHelper(FirestoreEmulatorHelper.java:363)

And suppressed state errors around the failed rules evaluation:

Suppressed: com.google.cloud.datastore.core.exception.DatastoreException: Can't swap from CLOSED to CLOSED.
    at com.google.cloud.datastore.emulator.impl.transactions.EmulatorTransactionManager.finish(EmulatorTransactionManager.java:589)
    at com.google.cloud.datastore.emulator.impl.storage.FlatLocalEntityStore$StronglyConsistentReadWriteView.close(FlatLocalEntityStore.java:167)
    at com.google.cloud.datastore.emulator.impl.firestore.FirestoreEmulatorHelper.commitHelper(FirestoreEmulatorHelper.java:362)

Later in the same emulator session, there are also many WebChannel backpressure warnings:

WARNING: Failed to send a new message due to too many pending messagings in the back channel (10001). May need enable flow control.
WARNING: [ERROR] NETWORK_ERROR ()

In one local run, firestore-debug.log had 9 instances of the rules Service call error. Function: [get], 320 Transaction lock timeout entries, and 830 back-channel warnings.

Notes

This appears to be an emulator contention/rules-engine issue rather than a rules logic issue:

  • The project document exists.
  • The collaborator document exists and grants the authenticated user the expected permission.
  • A direct authenticated write against the same emulator succeeds outside the high-contention flow.
  • The failure is intermittent and improves after restarting the emulator.

I noticed newer firebase-tools releases bundle Firestore emulator v1.21.0, but I do not see release notes specifically mentioning this lock-timeout/rules-get failure mode. I can retest on v1.21.0 if helpful.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions