Describe the bug
When using API policy source type, if a policy file is deleted in bundle.tar.gz, the deletion is not propagated to client.
To Reproduce
For example, this is a setup where bundle.tar.gz contains policy1.cedar and policy2.cedar, then after 40 seconds, the policy2.cedar file is deleted:
services:
policies:
image: python:3.12-alpine
ports:
- "5555:5555"
command: |
sh -c '
set -e
POLICY="permit (principal, action, resource) when { false };"
echo "$$POLICY" > policy1.cedar
echo "$$POLICY" > policy2.cedar
tar -czf bundle.tar.gz *.cedar
# Start a background timer to remove the file after 40 seconds
sh -c "sleep 40 && rm -v policy2.cedar && tar -czf bundle.tar.gz *.cedar" &
echo "Serving on port 5555..."
python3 -m http.server --bind 0.0.0.0 5555
'
opal_server:
image: permitio/opal-server:0.9.4-alpine
environment:
- OPAL_BROADCAST_URI=postgres://opal:opal@opal_broadcast:5432/opal
- UVICORN_NUM_WORKERS=4
- OPAL_POLICY_BUNDLE_URL=http://policies:5555
- OPAL_POLICY_SOURCE_TYPE=API
- OPAL_POLICY_REPO_POLLING_INTERVAL=10
- OPAL_DATA_CONFIG_SOURCES={"config":{"entries":[{"url":"http://opal_server:7002/policy-data","topics":["policy_data"],"dst_path":"/static"}]}}
- OPAL_FILTER_FILE_EXTENSIONS=.cedar
- OPAL_POLICY_REPO_POLICY_EXTENSIONS=.cedar
ports:
- "7002:7002"
depends_on:
- opal_broadcast
- policies
opal_cedar_agent:
image: permitio/opal-client-cedar:0.9.4
ports:
- "7766:7766"
- "8180:8180"
- "8181:8181"
- "7000:7000"
environment:
- OPAL_SERVER_URL=http://opal_server:7002
- OPAL_DATA_UPDATER_ENABLED=false
depends_on:
- opal_server
opal_broadcast:
image: postgres:16-alpine
environment:
- POSTGRES_DB=opal
- POSTGRES_USER=opal
- POSTGRES_PASSWORD=opalWhen I run this, at startup, the client received 2 policies:
$ curl -X GET http://localhost:8180/v1/policies
[{"id":"policy1.cedar","content":"permit(\n principal,\n action,\n resource\n) when {\n false\n};"},{"id":"policy2.cedar","content":"permit(\n principal,\n action,\n resource\n) when {\n false\n};"}]
After 40 seconds, server detected changes as suggested by the logs:
2026-04-21T17:43:16.223118+0000 | opal_common.sources.api_policy_source | INFO | Fetching changes from remote: 'http://policies:5555'
2026-04-21T17:43:16.227587+0000 | opal_common.sources.api_policy_source | INFO | Etag is turned off, you may want to turn it on at your bundle server
2026-04-21T17:43:16.227821+0000 | opal_common.sources.api_policy_source | INFO | Bundle hash is 6d7e8c5476ae97462b271f43239304d4b19796295f2f81c3ff7816b75fd592fb
2026-04-21T17:43:16.227916+0000 | opal_common.sources.api_policy_source | INFO | New bundle found, hash is: 6d7e8c5476ae97462b271f43239304d4b19796295f2f81c3ff7816b75fd592fb
2026-04-21T17:43:16.240610+0000 | opal_common.sources.api_policy_source | INFO | Found new version: old version hash was 'd3897a494ef6cfb23d943c08b6b057b3de5e8826a4d348f4ab7d8a714d665f80', new version hash is '6d7e8c5476ae97462b271f43239304d4b19796295f2f81c3ff7816b75fd592fb'
2026-04-21T17:43:16.244165+0000 | opal_server.policy.watcher.callbacks |WARNING | new commits detected but no tracked files were affected: 'a85eb5b3484d7abb361bbf7b24d001a3a002964e' -> '26f4ea8424f869dbcb61745de4b58630557b9a57'
But running the curl request on client still returned 2 policies even after one of the policy has been deleted in the source:
$ curl -X GET http://localhost:8180/v1/policies
[{"id":"policy1.cedar","content":"permit(\n principal,\n action,\n resource\n) when {\n false\n};"},{"id":"policy2.cedar","content":"permit(\n principal,\n action,\n resource\n) when {\n false\n};"}]
It maybe caused by TarFileToLocalGitExtractor.commit_local_git() in opal-common/opal_common/git_utils/tar_file_to_local_git_extractor.py in:
local_git.index.add(self.policy_bundle_git_add_pattern)
which can be replaced with git.add(..., all=True) to include deleted files:
local_git.git.add(self.policy_bundle_git_add_pattern, all=True)
Expected behavior
After the second policy is deleted from the source, it should also be removed from the client so that curl -X GET http://localhost:8180/v1/policies should only return 1 policy like:
[{"id":"policy1.cedar","content":"permit(\n principal,\n action,\n resource\n) when {\n false\n};"}]
OPAL version
Describe the bug
When using API policy source type, if a policy file is deleted in
bundle.tar.gz, the deletion is not propagated to client.To Reproduce
For example, this is a setup where
bundle.tar.gzcontainspolicy1.cedarandpolicy2.cedar, then after 40 seconds, thepolicy2.cedarfile is deleted:When I run this, at startup, the client received 2 policies:
After 40 seconds, server detected changes as suggested by the logs:
But running the curl request on client still returned 2 policies even after one of the policy has been deleted in the source:
It maybe caused by
TarFileToLocalGitExtractor.commit_local_git()inopal-common/opal_common/git_utils/tar_file_to_local_git_extractor.pyin:which can be replaced with
git.add(..., all=True)to include deleted files:Expected behavior
After the second policy is deleted from the source, it should also be removed from the client so that
curl -X GET http://localhost:8180/v1/policiesshould only return 1 policy like:OPAL version