Perf Test Footprint Serialization

[raymondwjang]

>>> stack = COO.from_numpy(xr.DataArray(np.clip(np.random.random((500, 500, 1000)) -0.9995, min=0)))  # assume each comp has ~130 pixels
>>> %timeit pickle.dumps(stack)
346 μs ± 1.32 μs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

[raymondwjang]

>>> pkl = pickle.dumps(stack)
>>> %timeit pickle.loads(pkl)
60 μs ± 161 ns per loop (mean ± std. dev. of 7 runs, 10,000 loops each)

[sneakers-the-rat]

hell ya

[raymondwjang]

Upper bound test

Still surprisingly fast? This is assuming 50 pixel diameter per component (absolutely gigantic considering the entire frame is normally like 500 x 500)

Serialization

stack = COO.from_numpy(xr.DataArray(np.clip(np.random.random((500, 500, 1000)) -0.99, min=0)))  # assuming ~2500 pixels per footprint
%timeit pkl = pickle.dumps(stack)
7.28 ms ± 83.5 μs per loop (mean ± std. dev. of 7 runs, 100 loops each)

Deserialization

%timeit pickle.loads(pkl)
1.79 ms ± 44.2 μs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

[sneakers-the-rat]

Ok good to know, thats technically not possible right? There cant be 1000 analyzable cells in a 500x500 frame if each of them is 50x50 anyway. 10ms roundtrip would be prohibitive at imaging speeds, but seems like its not really possible to get there under normal circumstances

[sneakers-the-rat]

I guess realistic upper bound test would be consulting largest imaging dataset with most cells we can find

[raymondwjang]

yeah the video would look absolutely ridiculous lol very little chance a calcium imaging video could even look like this

[sneakers-the-rat]

In any case, fast enough vs. Serial processing to do concurrent