Docs gets corrupted state when app terminated after doc.set_bytes() success

[audkar]

Problem:

Doc state is not fully finalized after doc.set_bytes(<..>) completes with success. Seems that data to FsStore is not written completely.

Example case:

1. Call doc.set_bytes(<..>) on Docs with FsStore.
2. await for completion success
3. Terminate app immediately without waiting for clean shutdown.
4. Start application again, and try to get the content for all Entries doc.get_many(query) and store.get_bytes(hash)
5. Observe that the entry in Docs exists, but the data in the store not.

This is problematic because docs then try to sync with other endpoints, but noone has that data in a store.

Expectation:

• doc.set_bytes(<..>) should complete only when everything is written.

---

I think this test showcases the issue, where state is not finalized after doc.set_bytes(<..>) return.

Click to expand full code block

On Linux the store file is locked for the second FsStore::load(store_path).await.unwrap(); indefinitely if the original Docs dereferenced and the test never finishes.

    #[tokio::test]
    async fn test_doc_blob() {
        let store_path = "asdf.blobs";
        {
            let secret_key = { SecretKey::generate(&mut rand::rng()) };
            let endpoint = Endpoint::empty_builder(RelayMode::Disabled)
                .secret_key(secret_key)
                .bind()
                .await
                .unwrap();
            let mdns = MdnsDiscovery::builder().build(endpoint.id()).unwrap();
            endpoint.discovery().add(mdns.clone());

            let pinger = iroh_ping::Ping::new();
            // let blobs_store = MemStore::new();
            let blobs_store = FsStore::load(store_path).await.unwrap();
            let gossip = Gossip::builder().spawn(endpoint.clone());
            let docs = Docs::memory()
                .spawn(endpoint.clone(), (*blobs_store).clone(), gossip.clone())
                .await
                .unwrap();

            let _router = Router::builder(endpoint.clone())
                .accept(iroh_ping::ALPN, pinger.clone())
                .accept(iroh_blobs::ALPN, BlobsProtocol::new(&blobs_store, None))
                .accept(iroh_gossip::ALPN, gossip.clone())
                .accept(iroh_docs::ALPN, docs.clone())
                .spawn();

            let doc = docs.create().await.unwrap();
            let _hash = doc
                .set_bytes(docs.author_default().await.unwrap(), "my_key", "my_value")
                .await
                .unwrap();
        }

        let _blobs_store = FsStore::load(store_path).await.unwrap();  // Does not have data
    }