feat: add workflow benchmarks and cloud orchestration support for: aws, azure, gcp#303
Open
userlaurin wants to merge 238 commits into
Open
feat: add workflow benchmarks and cloud orchestration support for: aws, azure, gcp#303userlaurin wants to merge 238 commits into
userlaurin wants to merge 238 commits into
Conversation
… sebs/cloudflare folder for comprehensive cloudflare doc and next steps
… if all configs are set up manually
…ge is needed and the wrappers have to work
…, at this point the other benchmarks may be translated to nodejs
…and 000 to nodejs. fixed support for nosql in wrapper and sebs. fixed durable objects implementation for sebs
When running the handler as a thread and then awaiting said thread (asyncio.to_thread), we can run asyncio.run() in the subsequent call stack.
…ironment variables
…er files and package dependencies
…new download method
…d Parallel states
…esult and enhance async_invoke to parse execution result
Add an Azure workflow HTTP trigger that exposes Durable Functions as SeBS library-style workflow invocations by starting the orchestration, polling the status URL, and returning the completed workflow output. Fix Azure workflow packaging/runtime behavior: - preserve benchmark-relative imports by importing activity modules as packages - add wrapper-level User-Agent handling for outbound requests - query the correct workflow HTTP entrypoint after publish - avoid duplicate HTTP trigger handling - quote Azure app settings and restart apps after env updates - support cached workflow trigger deserialization Fix workflow result handling for nested Map states inside Parallel branches so genome workflow outputs preserve the expected branch payload shape. Run high-CPU genome workflows on an EP3 Linux Premium plan with distinct app names, and limit genome activity concurrency in host.json to avoid shared /tmp collisions in benchmark code. Fix Azure blob range reads to use inclusive end-byte semantics, matching the benchmark expectations, and improve regression failure diagnostics so invocation exceptions are written to result JSON instead of being masked by harness errors.
…chmarks-data to the appropriate branch "workflows"
|
Important Review skippedToo many files! This PR contains 203 files, which is 53 over the limit of 150. To get a review, narrow the scope: Upgrade to a paid plan to raise the limit. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (203)
You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
added 8 commits
June 20, 2026 21:23
- move Redis measurement config to provider runtime env/config - propagate workflow request ids into collected measurements - collect workflow measurements during regression runs - avoid masking workflow invocation failures in regression output - refresh cached workflow runtime configuration for measurement envs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add the workflow benchmark suite, including FSM-based workflow definitions,
input generation, output validation, documentation, and regression test
coverage for workflow executions.
Implement workflow deployment and invocation support for AWS Step Functions,
GCP Workflows, and Azure Durable Functions, including platform-specific
generators, triggers, packaging, wrappers, and storage helpers needed by
workflow tasks.
Add workflow benchmark definitions for gen, function invocation, trip booking,
parallel sleep/download, selfish detour, video, map-reduce, auth, ExCamera,
ML, and 1000 Genomes workloads.
Also update configs, dependencies, and benchmark data references for workflow
regression runs.
Regression: workflow suites pass across AWS, GCP, Azure, and Cloudflare.