1.5.x cloud benchmarks#3615
Conversation
kill_all() is called after cloud_preflight_warmup completes. ProvidedShardManager wraps an already-running process we don't own, so neither kill nor restart should crash the binary. Both are now silent no-ops, matching UnavailableShardManager.
…ot own its memory environment
…e continuing the test
kmatasfp
force-pushed
the
1.5.x-cloud-benchmarks
branch
7 times, most recently
from
c5df097 to
2556fa3
Compare
|
|
||
| assert_eq!(result1.len(), 2, "G1002"); // TODO: this is temporarily not working because of using the dynamic invoke API and not having structured information in the oplog | ||
| assert_eq!(result2.len(), 2, "imported-function"); | ||
| assert_eq!(result2.len(), 1, "imported-function"); |
There was a problem hiding this comment.
this and other similar test changes are because we do not call host function to get agent metadata anymore
| if matches!(get_asyncness(&constructor_method.sig), Asyncness::Future) { | ||
| return syn::Error::new_spanned( | ||
| &constructor_method.sig.ident, | ||
| "Agent constructors must be synchronous. Async constructors can call host APIs during snapshot restore and make recovery fall back to full replay.", |
There was a problem hiding this comment.
prob need a better message here, regardless Agent constructor should not be async imho, adds more complexity than value, unless I am missing something
There was a problem hiding this comment.
I disagree, I think that would be too limiting (no awaitable rpc, no http calls etc) - and whether it does side effects or not (it has to be able to!) does not actually depend on the "asyncness", in p2 all host functions are sync and in p3 we get async ones beside them.
There was a problem hiding this comment.
But I see how this interferes with load-snapshot initializing the agent. I need to think more about it :)
There was a problem hiding this comment.
One real-world example: it has been a useful pattern to spawn child agents and trigger some background operation in the constructor. During recovery, of course this does not get triggered again because it's persisted in the oplog. But if load-snapshot calls the same constructor, it's going to be re-trigger the same background operation with a fresh idempotency key (or crash if we detect it as an unallowed side-effect). Both are wrong. The assumption was so far that "it's the user's responsibility" but it's probably too easy to do it wrong.
| pub mod websocket; | ||
|
|
||
| #[cfg(any(test, feature = "test-support"))] | ||
| pub mod snapshot_recovery_test_support { |
There was a problem hiding this comment.
could not think of a better way how to do this, I need some hook inside the snapshot recovery code that actually records that we recovered from snapshot or not. Rust makes it easier than other languages to use test Spy with conditional compilation
kmatasfp
force-pushed
the
1.5.x-cloud-benchmarks
branch
2 times, most recently
from
54ce165 to
63aff3d
Compare
kmatasfp
force-pushed
the
1.5.x-cloud-benchmarks
branch
from
63aff3d to
13df001
Compare
3 participants
Also contains replacement for memory semaphore plus fixes eviction deadlock.
for memory semaphore replacement see: