WHy doesn't fsync apply to standalone deletes (if we don't fsync isn't there some chance that a deleted file will get "undeleted" on crash)?
it should! but it doesnt right now, the documented mentions only write-like operations and not deletes
|
/// When enabled, [`LocalFileSystem`] calls [`File::sync_all`] on written files and fsyncs |
|
/// the affected parent directories before a write operation |
|
/// ([`put_opts`](ObjectStore::put_opts), [`copy_opts`](ObjectStore::copy_opts), |
|
/// [`rename_opts`](ObjectStore::rename_opts), and multipart upload completion) returns |
|
/// success. This guarantees that both the file contents and the directory entries pointing |
|
/// to them are durable on stable storage, matching the implicit durability contract of |
|
/// remote object stores such as S3 or GCS. |
Let's do a follow up to make standalone delete / delete_stream crash-durable
Originally posted by @kevinjqliu in #758 (comment)
it should! but it doesnt right now, the documented mentions only write-like operations and not deletes
arrow-rs-object-store/src/local.rs
Lines 310 to 316 in c9db3e2
Let's do a follow up to make standalone
delete/delete_streamcrash-durableOriginally posted by @kevinjqliu in #758 (comment)