Add sensitivity analysis example workflow#866
Draft
cvolkcvolk wants to merge 2 commits into
Draft
Conversation
Signed-off-by: Clemens Volk <cvolk@nvidia.com>
alexmillane
reviewed
Jul 6, 2026
alexmillane
left a comment
alexmillane
left a comment
Collaborator
There was a problem hiding this comment.
Looks great!
One small suggestion!
Comment on lines
+123
to
+147
| .. todo:: | ||
|
|
||
| Replace this dry-run figure with the final plot from an evaluation that contains enough | ||
| successful and failed episodes to support an interpretation. | ||
|
|
||
| .. figure:: ../../../images/droid_camera_sensitivity_dry_run.png | ||
| :width: 100% | ||
| :alt: Dry-run sensitivity report for three wrist-camera translation components | ||
| :align: center | ||
|
|
||
| Temporary report from the five-episode dry run. Horizontal axes show offsets in metres. The | ||
| blue curve is the estimated distribution for successful episodes, its shading marks the 5% | ||
| to 95% range, and the grey dashed line is the uniform sampling distribution. A blue curve | ||
| close to the dashed line suggests no clear relationship; concentration in one region | ||
| suggests a stronger association with success. | ||
|
|
||
| As an illustration of how to read the shapes, imagine that the same pattern came from a larger | ||
| evaluation containing both successes and failures. The takeaways could then be written as: | ||
|
|
||
| * **Horizontal displacement:** Pi0.5 appears more successful when the wrist camera moves to the | ||
| right of its nominal position. | ||
| * **Vertical displacement:** Pi0.5 appears relatively robust to vertical movement because the | ||
| posterior remains close to the uniform reference. | ||
| * **Depth displacement:** Pi0.5 appears more successful when the wrist camera moves about 3 mm | ||
| forward from its nominal position. |
Collaborator
There was a problem hiding this comment.
What do you think about including in the repo a jsonl file from a larger evaluation.
That way we could show some meaningful results and a user could run the same analysis?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Document variation and sensitivity workflows
Detailed description