Skip to content

Differential testing#119

Merged
smiasojed merged 9 commits into
mainfrom
sm/layout
Jul 10, 2026
Merged

Differential testing#119
smiasojed merged 9 commits into
mainfrom
sm/layout

Conversation

@smiasojed

@smiasojed smiasojed commented Jul 1, 2026

Copy link
Copy Markdown
Collaborator

Adds a new differential-testing crate that validates the SDK’s on-chain storage representation against Solidity compiler output by compiling small Solidity fixtures with solc, executing the deployed bytecode on revm, and comparing the resulting {slot -> 32 bytes} storage maps to what pvm-storage writes into MockHost.

@socket-security

socket-security Bot commented Jul 1, 2026

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addedrevm@​40.0.310010093100100

View full report

@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown

Benchmark Size Comparison (vs main)

Artifact Baseline Current Delta Status
allowlist_no-alloc.debug.polkavm 63984 63991 +0.01% OK
allowlist_no-alloc.release.polkavm 3168 3168 +0.00% OK
allowlist_with-alloc.debug.polkavm 75597 75607 +0.01% OK
allowlist_with-alloc.release.polkavm 3567 3567 +0.00% OK
amm_reserves_no-alloc.debug.polkavm 42894 42902 +0.02% OK
amm_reserves_no-alloc.release.polkavm 1949 1949 +0.00% OK
amm_reserves_with-alloc.debug.polkavm 54614 54614 +0.00% OK
amm_reserves_with-alloc.release.polkavm 2288 2288 +0.00% OK
fibonacci_builder-dsl.debug.polkavm 37239 37239 +0.00% OK
fibonacci_builder-dsl.release.polkavm 638 638 +0.00% OK
fibonacci_no-alloc.debug.polkavm 35852 35852 +0.00% OK
fibonacci_no-alloc.release.polkavm 595 595 +0.00% OK
fibonacci_with-alloc.debug.polkavm 46306 46306 +0.00% OK
fibonacci_with-alloc.release.polkavm 600 600 +0.00% OK
multi_builder-dsl.debug.polkavm 59390 59398 +0.01% OK
multi_builder-dsl.release.polkavm 3070 3070 +0.00% OK
multi_no-alloc.debug.polkavm 62095 62099 +0.01% OK
multi_no-alloc.release.polkavm 2968 2968 +0.00% OK
multi_with-alloc.debug.polkavm 73140 73140 +0.00% OK
multi_with-alloc.release.polkavm 3096 3096 +0.00% OK
mytoken_builder-dsl.debug.polkavm 56536 56536 +0.00% OK
mytoken_builder-dsl.release.polkavm 3715 3715 +0.00% OK
mytoken_no-alloc.debug.polkavm 56121 60330 +7.50% FAIL
mytoken_no-alloc.release.polkavm 2955 2955 +0.00% OK
mytoken_storage_no-alloc.debug.polkavm 62508 66578 +6.51% FAIL
mytoken_storage_no-alloc.release.polkavm 3087 3087 +0.00% OK
mytoken_storage_with-alloc.debug.polkavm 75133 79208 +5.42% FAIL
mytoken_storage_with-alloc.release.polkavm 4130 4130 +0.00% OK
mytoken_with-alloc.debug.polkavm 70209 74414 +5.99% FAIL
mytoken_with-alloc.release.polkavm 3980 3980 +0.00% OK
---------------------------------------------------- ------------ ------------ ---------- --------
Total 911424 928020 +1.82%

REGRESSION: 4 artifact(s) exceed 5% threshold

  • mytoken_no-alloc.debug.polkavm: 56121 -> 60330 (+7.50%)
  • mytoken_storage_no-alloc.debug.polkavm: 62508 -> 66578 (+6.51%)
  • mytoken_storage_with-alloc.debug.polkavm: 75133 -> 79208 (+5.42%)
  • mytoken_with-alloc.debug.polkavm: 70209 -> 74414 (+5.99%)

Run 29021013263 | e66df89 | 2026-07-09 13:43 UTC

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new differential-testing crate that validates the SDK’s on-chain storage representation against Solidity compiler output by compiling small Solidity fixtures with solc, executing the deployed bytecode on revm, and comparing the resulting {slot -> 32 bytes} storage maps to what pvm-storage writes into MockHost.

Changes:

  • Introduces pvm-solc-differential with a comprehensive suite of storage-representation fixtures (packing, mappings, dynamics, vectors/arrays, clearing, signed ints).
  • Adds a MockHost::storage_dump() helper to enumerate all written storage slots for full-map comparisons.
  • Wires the new crate into the workspace and CI, running it behind the solc-tests feature.

Reviewed changes

Copilot reviewed 7 out of 8 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
crates/pvm-solc-differential/src/storage.rs Implements the solc+revm ground-truth harness and a wide set of differential storage representation tests.
crates/pvm-solc-differential/src/lib.rs Adds crate-level documentation and gates the test modules behind test + solc-tests.
crates/pvm-solc-differential/src/common.rs Adds a shared solc --standard-json runner and JSON parsing for test modules.
crates/pvm-solc-differential/Cargo.toml Defines the new crate and gates heavy dependencies (revm, alloy-core, serde_json) behind solc-tests.
crates/pvm-contract-types/src/mock_host.rs Adds storage_dump() to snapshot all stored key/value pairs for differential assertions.
Cargo.toml Adds pvm-solc-differential to the workspace members list.
Cargo.lock Updates the lockfile for the newly introduced optional dependencies.
.github/workflows/ci.yml Adds a CI step to run the new solc differential storage-representation tests.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread crates/pvm-contract-types/src/mock_host.rs
smiasojed and others added 2 commits July 2, 2026 11:34
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@smiasojed smiasojed marked this pull request as ready for review July 2, 2026 14:17
Comment thread crates/pvm-solc-differential/src/storage.rs
Comment thread crates/pvm-solc-differential/src/common.rs Outdated
Comment thread crates/pvm-solc-differential/src/storage.rs
Comment thread crates/pvm-solc-differential/Cargo.toml Outdated

@re-gius re-gius left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@smiasojed smiasojed requested a review from pkhry July 9, 2026 13:18

@pkhry pkhry left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, but maybe we can also add a-la quickcheck testing for storage value equivalences too?

@smiasojed

Copy link
Copy Markdown
Collaborator Author

lgtm, but maybe we can also add a-la quickcheck testing for storage value equivalences too?

I agree, but it is a longer story I will handle it in a separate PR

@smiasojed smiasojed merged commit 32054eb into main Jul 10, 2026
5 of 6 checks passed
@smiasojed smiasojed deleted the sm/layout branch July 10, 2026 10:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants