I've stumbled upon weird error messages with TileDB (2.22) on Windows, that I eventually figured out to be related to too long filenames.
Let's consider the following test.py (example given with tiledb-py for ease, but the original issue comes from the use of the TileDB C++ API from the GDAL TileDB driver):
import tiledb
import numpy as np
import shutil
dim = tiledb.Dim(name="dim", domain=(1, 4), tile=2, dtype=np.int32)
dom = tiledb.Domain(dim)
attr = tiledb.Attr(name="attr", dtype=np.int32)
schema = tiledb.ArraySchema(domain=dom, sparse=False, attrs=[attr])
filename = "z:/" + ('X' * 190)
try:
tiledb.Array.create(filename, schema)
finally:
shutil.rmtree(filename)When run, it throws:
Traceback (most recent call last):
File "C:\dev\gdal space\build_conda\test.py", line 11, in <module>
tiledb.Array.create(filename, schema)
File "tiledb\libtiledb.pyx", line 976, in tiledb.libtiledb.Array.create
File "tiledb\libtiledb.pyx", line 342, in tiledb.libtiledb._raise_ctx_err
File "tiledb\libtiledb.pyx", line 327, in tiledb.libtiledb._raise_tiledb_error
tiledb.cc.TileDBError: [TileDB::IO] Error: Cannot write to file 'C:\dev\gdal space\build_conda'; File opening error CreateFile GetLastError 5 (0x00000005): Access denied
Note that it tries to write to the current directory, which has nothing to do with the array I'm creating ("z:/XXXXXXXXXXXXXXX....XXXX")
I didn't try to create too long filenames to be annoying. The original issue came actually from using pytest with the tmp_path fixture for GDAL regression test suite. With a test like (can't be run by you as requires development version of the GDAL TileDB driver, but just to give the nominal context where the issue was triggered):
def test_tiledb_write_overviews(tmp_path, use_group):
dsname = str(tmp_path / "test_tiledb_write_overviews.tiledb")
src_ds = gdal.Open("data/rgbsmall.tif")
ds = gdal.GetDriverByName("TileDB").CreateCopy(dsname, src_ds)
ds.BuildOverviews("NEAR", [2])
Behind the scenes this creates a "test_tiledb_write_overviews_1" TileDB array as a subdirectory of str(tmp_path / "test_tiledb_write_overviews.tiledb" + ".ovr"). And it turns out that tmp_path generates quite long temporary directory names (here tmp_path evaluates to C:\Users\evenr\AppData\Local\Temp\pytest-of-evenr\pytest-95\test_tiledb_write_overviews_Fa0\)
The error message I got here was slightly different and scarier, as it looked like an attempt at deleting my current working directory:
E RuntimeError: TileDB: TileDBRasterBand::IRasterIO() failed: TileDB internal: [OrderedWriter::dowork] ([TileDB::IO] Error: Failed to delete file 'C:\dev\gdal space\autotest\gdrivers' DeleteFile GetLastError 5 (0x00000005): Access denied
E )
This exception is thrown when running tiledb::Query::submit()
It would be good if TileDB could be anticipate, at array creation, that some of its internal filenames are going to go over the Windows filename size limit , and throw an explicit exception when that happens ((or use \\?\ prefixing an appropriate Windows API so that long filenames work)
I've stumbled upon weird error messages with TileDB (2.22) on Windows, that I eventually figured out to be related to too long filenames.
Let's consider the following test.py (example given with tiledb-py for ease, but the original issue comes from the use of the TileDB C++ API from the GDAL TileDB driver):
When run, it throws:
Note that it tries to write to the current directory, which has nothing to do with the array I'm creating ("z:/XXXXXXXXXXXXXXX....XXXX")
I didn't try to create too long filenames to be annoying. The original issue came actually from using pytest with the tmp_path fixture for GDAL regression test suite. With a test like (can't be run by you as requires development version of the GDAL TileDB driver, but just to give the nominal context where the issue was triggered):
Behind the scenes this creates a "test_tiledb_write_overviews_1" TileDB array as a subdirectory of str(tmp_path / "test_tiledb_write_overviews.tiledb" + ".ovr"). And it turns out that tmp_path generates quite long temporary directory names (here tmp_path evaluates to
C:\Users\evenr\AppData\Local\Temp\pytest-of-evenr\pytest-95\test_tiledb_write_overviews_Fa0\)The error message I got here was slightly different and scarier, as it looked like an attempt at deleting my current working directory:
This exception is thrown when running
tiledb::Query::submit()It would be good if TileDB could be anticipate, at array creation, that some of its internal filenames are going to go over the Windows filename size limit , and throw an explicit exception when that happens ((or use
\\?\prefixing an appropriate Windows API so that long filenames work)