Skip to content

StorageFactory::getDevice() omits DSN bucket prefix for S3-compatible endpoints (url= / custom host) #239

Description

@ibaraki-douji

When using OPR_EXECUTOR_CONNECTION_STORAGE with an S3-compatible backend (Garage, MinIO, etc.) via the url= DSN parameter, StorageFactory creates an Utopia\Storage\Device\S3 instance without prepending the bucket from the DSN path to the storage root.

Appwrite’s own getDevice() in app/init/resources.php does prepend the bucket:

$bucketRoot = (!empty($bucket) ? "{$bucket}/" : '') . ltrim($root, '/');
return new S3($bucketRoot, $accessKey, $accessSecret, $url, $region, $acl);

Executor StorageFactory (0.25.1) does not:

if ($url !== '' && $url !== '0') {
    return new S3($root, $accessKey, $accessSecret, $url, $dsnRegion, $acl);
}

Impact

Builds/deployments that upload artifacts to storage fail at the final step:

Failed to move built code to storage

Executor logs / S3 errors show requests against the wrong key prefix, e.g.:

  • NoSuchBucket: Bucket not found: storage (path-style treats storage as the bucket)
  • or uploads to /storage/builds/... instead of /<bucket>/storage/builds/...

Environment

  • openruntimes/executor:0.25.1
  • S3-compatible storage (Garage) with path-style access
  • OPR_EXECUTOR_CONNECTION_STORAGE using url= for custom endpoint

DSN format (related)

utopia-php DSN also requires a host in the connection string. This is invalid:

s3://accessKey:secret@/bucket?region=garage&url=http%3A%2F%2Fhost%3A3900

Error: Unable to parse DSN: host is required

This works (host is a placeholder when url= is used):

s3://accessKey:secret@localhost/bucket?region=garage&url=http%3A%2F%2Fhost%3A3900

Docs currently show @/bucket in some examples; worth clarifying that a host is required.

Reproduction

  1. Set:
OPR_EXECUTOR_CONNECTION_STORAGE=s3://KEY:SECRET@localhost/mybucket?region=garage&url=http%3A%2F%2F127.0.0.1%3A3900
  1. In PHP inside the executor container:
use OpenRuntimes\Executor\StorageFactory;

$dsn = getenv('OPR_EXECUTOR_CONNECTION_STORAGE');
$dev = StorageFactory::getDevice('/storage/builds/app-test', $dsn);

echo $dev->getRoot(); 
// Actual:   storage/builds/app-test
// Expected: mybucket/storage/builds/app-test

$path = $dev->getPath('artifact.tar.gz');
$dev->upload('/tmp/artifact.tar.gz', $path);
// Fails against Garage/MinIO (wrong bucket/key prefix)
  1. Trigger a function/site build via Appwrite — build runs, upload at end fails.

Expected behavior

For S3 + custom endpoint (url= or host= + insecure), StorageFactory should match Appwrite:

$bucketRoot = ($bucket !== '' && $bucket !== '0')
    ? ltrim($bucket, '/') . '/' . ltrim($root, '/')
    : ltrim($root, '/');

if ($url !== '' && $url !== '0') {
    return new S3($bucketRoot, $accessKey, $accessSecret, $url, $dsnRegion, $acl);
}

if ($host !== '' && $host !== '0') {
    $host = $insecure ? 'http://' . $host : $host;
    return new S3(root: $bucketRoot, accessKey: $accessKey, secretKey: $accessSecret, host: $host, region: $dsnRegion, acl: $acl);
}

AWS branch (new AWS(...)) can stay as-is (bucket passed separately).

Suggested fix

Patch src/Executor/StorageFactory.php as above, and add a test covering:

  • DSN with url= + bucket path
  • getDevice('/storage/builds/app-xyz', $dsn) → root mybucket/storage/builds/app-xyz
  • upload to a path-style S3-compatible endpoint succeeds

Workaround

Mount a patched StorageFactory.php into the executor container until upstream fixes it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions