Skip to content

Commit 1344086

Browse files
committed
Merge remote-tracking branch 'origin/main' into wat
2 parents b805201 + 03ac8ab commit 1344086

1,998 files changed

Lines changed: 22612 additions & 18402 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/main.yml

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -237,6 +237,7 @@ jobs:
237237
- run: cargo check --no-default-features --features addr2line
238238
- run: cargo check --no-default-features --features json-from-wast
239239
- run: cargo check --no-default-features --features completion
240+
- run: cargo check --no-default-features --features wast
240241
- run: cargo check --no-default-features -p wit-parser
241242
- run: cargo check --no-default-features -p wit-parser --features wat
242243
- run: cargo check --no-default-features -p wit-parser --features serde
@@ -274,12 +275,31 @@ jobs:
274275
- run: cargo check --no-default-features -p wasm-smith --features component-model
275276
- run: cargo check --no-default-features -p wasm-smith --features wasmparser
276277
- run: cargo check --no-default-features -p wasm-smith --features wasmparser,component-model
278+
- run: cargo check --no-default-features -p wasm-metadata
279+
- run: cargo check --no-default-features -p wasm-metadata --features serde
280+
- run: cargo check --no-default-features -p wasm-metadata --features oci
277281
- run: |
278282
if cargo tree -p wasm-smith --no-default-features -e no-dev | grep wasmparser; then
279283
echo wasm-smith without default features should not depend on wasmparser
280284
exit 1
281285
fi
282286
287+
# Double-check that files and such related to the `tests/cli` test suite are
288+
# up-to-date.
289+
generated_files_up_to_date:
290+
name: Check generated files are up-to-date
291+
runs-on: ubuntu-latest
292+
steps:
293+
- uses: actions/checkout@v4
294+
with:
295+
submodules: true
296+
- uses: ./.github/actions/install-rust
297+
- run: rm -rf tests/snapshots
298+
- run: rustc ci/generate-spec-tests.rs && ./generate-spec-tests
299+
- run: find tests/cli -name '*.stderr' | xargs rm
300+
- run: BLESS=1 cargo test --test cli
301+
- run: git diff --exit-code
302+
283303
doc:
284304
runs-on: ubuntu-latest
285305
steps:
@@ -326,6 +346,7 @@ jobs:
326346
- test_extra_features
327347
- test-prefer-btree-collections
328348
- clippy
349+
- generated_files_up_to_date
329350
if: always()
330351

331352
steps:

CONTRIBUTING.md

Lines changed: 29 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -47,49 +47,48 @@ and then using Cargo to execute all tests:
4747
$ cargo test --workspace
4848
```
4949

50-
Running the spec test suite can be done with:
50+
The majority of all tests is located in `tests/cli/*` and can be run with:
5151

5252
```
53-
$ cargo test --test roundtrip
53+
$ cargo test --test cli
5454
```
5555

56-
and running a single spec test can be done with an argument to this command as a
57-
string filter on the filename.
56+
Each individual file is a standalone test and documents at the top how to run
57+
it. Running individual tests can be done through:
58+
59+
```
60+
$ cargo test --test cli -- test_name_filter
61+
```
62+
63+
Or you can use `cargo run` plus the test's arguments to run a single test.
5864

5965
```
60-
$ cargo test --test roundtrip binary-leb128.wast
66+
$ cargo run wast tests/cli/empty.wast
6167
```
6268

63-
Many tests are also located in the top-level `tests/*` folder. This is organized
64-
into a few suites:
69+
Running just the spec test suite can be done with:
6570

66-
* `tests/cli/*` - these files are run by the `tests/cli.rs` test file and are
67-
intended to be tests for the CLI itself. They start with `;; RUN: ...` headers
68-
to indicate what commands should run and adjacent files indicate the expected
69-
output.
71+
```
72+
$ cargo test --test cli -- spec
73+
```
7074

71-
* `tests/local/*` - these are handwritten `*.wat` and `*.wast` tests. The
72-
`*.wat` files must all validate as valid modules and `*.wast` files run their
73-
directives in the same manner as the spec test suite. This folder additional
74-
subfolders for specific classes of tests, for example `missing-features` has
75-
all optional wasm features disabled to test what happens when a feature is
76-
implemented but disabled at runtime. The `component-model` folder contains all
77-
tests related to enabling the component model feature.
75+
and running a single spec test can be done with an argument to this command as a
76+
string filter on the filename.
7877

79-
* `tests/testsuite` - this is a git submodule pointing to the [upstream test
80-
suite repository](https://github.com/WebAssembly/testsuite/) and is where spec
81-
tests come from.
78+
```
79+
$ cargo test --test cli binary-leb128.wast
80+
$ cargo run wast tests/testsuite/binary-leb128.wast --ignore-error-messages
81+
```
8282

83-
* `tests/roundtrip.rs` - this is the main driver for the `local` and `testsuite`
84-
folders. This will crawl over all files in those folders and execute what
85-
tests it can. This means running `*.wast` directives such as `assert_invalid`.
86-
Additionally all valid wasm modules are printed with `wasmprinter` and then
87-
parsed again with `wat` to ensure that they can be round-tripped through the
88-
crates.
83+
The `tests/cli` tests suite is documented as a self-describing test at
84+
`tests/cli/readme.wat`. Each test describes what subcommand of `wasm-tools` is
85+
run and the test is itself the input.
8986

90-
* `tests/snapshots` - this contains golden output files which correspond to the
91-
`wasmprinter`-printed version of binaries of all tests. These files are used
92-
to view the impact of changes to `wasmprinter`.
87+
The `tests/testsuite` folder is a git submodule pointing to the [upstream test
88+
suite repository](https://github.com/WebAssembly/testsuite/) and is where spec
89+
tests come from. The `tests/snapshots` folder contains golden output files
90+
which correspond to the `wasmprinter`-printed version of binaries of all tests.
91+
These files are used to view the impact of changes to `wasmprinter`.
9392

9493
Many tests throughout the repository have automatically generated files
9594
associated with them which reflect the expected output of an operation. This is

Cargo.toml

Lines changed: 16 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -105,7 +105,7 @@ wasmtime = { version = "29.0.1", default-features = false, features = ['cranelif
105105

106106
wasm-compose = { version = "0.227.1", path = "crates/wasm-compose" }
107107
wasm-encoder = { version = "0.227.1", path = "crates/wasm-encoder", default-features = false }
108-
wasm-metadata = { version = "0.227.1", path = "crates/wasm-metadata" }
108+
wasm-metadata = { version = "0.227.1", path = "crates/wasm-metadata", default-features = false }
109109
wasm-mutate = { version = "0.227.1", path = "crates/wasm-mutate" }
110110
wasm-shrink = { version = "0.227.1", path = "crates/wasm-shrink" }
111111
wasm-smith = { version = "0.227.1", path = "crates/wasm-smith" }
@@ -168,7 +168,7 @@ wit-encoder = { workspace = true, optional = true }
168168
wit-parser = { workspace = true, optional = true, features = ['decoding', 'wat', 'serde'] }
169169

170170
# Dependencies of `metadata`
171-
wasm-metadata = { workspace = true, features = ["clap"], optional = true }
171+
wasm-metadata = { workspace = true, features = ["clap", "default"], optional = true }
172172
bytesize = { workspace = true }
173173

174174
# Dependencies of `wit-smith`
@@ -178,6 +178,9 @@ wit-smith = { workspace = true, features = ["clap"], optional = true }
178178
addr2line = { version = "0.24.0", optional = true }
179179
gimli = { workspace = true, optional = true }
180180

181+
# Dependencies of `wast`
182+
pretty_assertions = { workspace = true, optional = true }
183+
181184
[target.'cfg(not(target_family = "wasm"))'.dependencies]
182185
is_executable = { version = "1.0.1", optional = true }
183186

@@ -193,10 +196,6 @@ wast = { workspace = true }
193196
name = "cli"
194197
harness = false
195198

196-
[[test]]
197-
name = "roundtrip"
198-
harness = false
199-
200199
[features]
201200
# By default, all subcommands are built
202201
default = [
@@ -217,6 +216,7 @@ default = [
217216
'addr2line',
218217
'completion',
219218
'json-from-wast',
219+
'wast',
220220
]
221221

222222
# Each subcommand is gated behind a feature and lists the dependencies it needs
@@ -253,3 +253,13 @@ wit-smith = ['dep:wit-smith', 'arbitrary']
253253
addr2line = ['dep:addr2line', 'dep:gimli', 'dep:wasmparser']
254254
completion = ['dep:clap_complete']
255255
json-from-wast = ['dep:serde_derive', 'dep:serde_json', 'dep:wast', 'dep:serde']
256+
wast = [
257+
'dep:wast',
258+
'wasm-encoder/wasmparser',
259+
'dep:pretty_assertions',
260+
'validate',
261+
# These subcommands are executed from `wasm-tools wast` so make sure they're
262+
# built-in if this is enabled.
263+
'dump',
264+
'json-from-wast',
265+
]

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -155,6 +155,7 @@ that can be use programmatically as well:
155155
| `wasm-tools addr2line` | | | Translate wasm offsets to filename/line numbers with DWARF |
156156
| `wasm-tools completion` | | | Generate shell completion scripts for `wasm-tools` |
157157
| `wasm-tools json-from-wast` | | | Convert a `*.wast` file into JSON commands |
158+
| `wasm-tools wast` | | | Validate the structure of a `*.wast` file |
158159

159160
[wasmparser]: https://crates.io/crates/wasmparser
160161
[wat]: https://crates.io/crates/wat

ci/generate-spec-tests.rs

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
//! Helper program to generate files in `tests/cli/spec/*` which correspond to
2+
//! running spec tests in `tests/testsuite/*`.
3+
4+
use std::fs;
5+
use std::path::Path;
6+
7+
fn main() {
8+
let _ = fs::remove_dir_all("./tests/cli/spec");
9+
copy_tests("tests/testsuite".as_ref(), "tests/cli/spec".as_ref());
10+
}
11+
12+
/// Recursively visit `src` and, for all test files, create a file in `dst` to
13+
/// run the test.
14+
fn copy_tests(src: &Path, dst: &Path) {
15+
fs::create_dir(&dst).unwrap();
16+
for entry in src.read_dir().unwrap() {
17+
let entry = entry.unwrap();
18+
19+
let src = entry.path();
20+
let dst = dst.join(entry.file_name());
21+
if entry.file_type().unwrap().is_dir() {
22+
copy_tests(&src, &dst);
23+
continue;
24+
}
25+
26+
if src.extension().and_then(|s| s.to_str()) != Some("wast") {
27+
continue;
28+
}
29+
30+
copy_test(&src, &dst);
31+
}
32+
}
33+
34+
/// Creates `dst` as a file to run `src` as a test.
35+
fn copy_test(src: &Path, dst: &Path) {
36+
// The legacy exception-handling proposal is not currently supported because
37+
// it uses the folded form of s-expressions which are not implemented here.
38+
// Regardless just skip these spec tests.
39+
if src.iter().any(|p| p == "legacy") {
40+
return;
41+
}
42+
43+
let mut contents = format!(";; RUN: wast \\\n");
44+
contents.push_str(";; --assert default \\\n");
45+
contents.push_str(";; --snapshot tests/snapshots \\\n");
46+
47+
// This test specifically tests various forms of unicode which are
48+
// default-disallowed, so explicitly allow it here.
49+
if src.ends_with("names.wast") {
50+
contents.push_str(";; --allow-confusing-unicode \\\n");
51+
}
52+
53+
// Historically wasm-tools tried to match the upstream error message. This
54+
// generally led to a large sequence of matches here which is not easy to
55+
// maintain and is particularly difficult when test suites and proposals
56+
// conflict with each other (e.g. one asserts one error message and another
57+
// asserts a different error message). Overall we didn't benefit a whole lot
58+
// from trying to match errors so just assume the error is roughly the same
59+
// and otherwise don't try to match it.
60+
contents.push_str(";; --ignore-error-messages \\\n");
61+
62+
// Push a `--features=..` flag for the spec tests. Spec tests often need a
63+
// precise set of features different from the defaults of `wasm-tools` so
64+
// it's always overridden here.
65+
let features = match find_proposal(src) {
66+
None => "wasm2",
67+
Some("annotations") => "wasm2",
68+
Some("threads") => "wasm1,threads",
69+
Some("function-references") => "wasm2,function-references,tail-call",
70+
Some("wasm-3.0") => "wasm3",
71+
Some("gc") => "wasm2,function-references,gc,tail-call",
72+
Some("multi-memory") => "wasm2,multi-memory",
73+
Some("extended-const") => "wasm2,extended-const",
74+
Some("exception-handling") => "wasm2,exceptions,tail-call",
75+
Some("custom-page-sizes") => "wasm3,custom-page-sizes",
76+
Some("wide-arithmetic") => "wasm2,wide-arithmetic",
77+
Some("tail-call") => "wasm2,tail-call",
78+
Some("relaxed-simd") => "wasm2,relaxed-simd",
79+
Some("memory64") => "wasm3",
80+
Some(proposal) => panic!("unsupported proposal: {}", proposal),
81+
};
82+
contents.push_str(&format!(";; --features={features} \\\n"));
83+
84+
// And finally push a path to the test itself.
85+
contents.push_str(&format!(";; {}\n", src.display()));
86+
87+
fs::write(dst, contents).unwrap();
88+
}
89+
90+
/// Finds the wasm proposal, if present, within `src`.
91+
fn find_proposal(src: &Path) -> Option<&str> {
92+
// Look for `foo` in `.../proposals/foo/...`
93+
let mut parts = src.iter();
94+
while let Some(next) = parts.next() {
95+
if next.to_str() == Some("proposals") {
96+
return parts.next()?.to_str();
97+
}
98+
}
99+
None
100+
}

crates/wasm-metadata/Cargo.toml

Lines changed: 21 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -14,16 +14,25 @@ workspace = true
1414
anyhow = { workspace = true }
1515
clap = { workspace = true, optional = true }
1616
indexmap = { workspace = true, features = ["serde"] }
17-
serde = { workspace = true }
18-
serde_derive = { workspace = true }
19-
serde_json = { workspace = true }
20-
spdx = { workspace = true }
21-
url = { workspace = true }
17+
serde = { workspace = true, optional = true }
18+
serde_derive = { workspace = true, optional = true }
19+
serde_json = { workspace = true, optional = true }
20+
spdx = { workspace = true, optional = true }
21+
url = { workspace = true, optional = true }
2222
wasm-encoder = { workspace = true, features = ['std', 'component-model'] }
23-
wasmparser = { workspace = true, features = [
24-
'std',
25-
'component-model',
26-
'hash-collections',
27-
] }
28-
auditable-serde = "0.8.0"
29-
flate2 = "1.1.0"
23+
wasmparser = { workspace = true, features = ['std', 'component-model', 'hash-collections'] }
24+
auditable-serde = { version = "0.8.0", optional = true }
25+
flate2 = { version = "1.1.0", optional = true }
26+
27+
[features]
28+
default = ['oci', 'serde']
29+
oci = [
30+
'dep:auditable-serde',
31+
'dep:flate2',
32+
'dep:url',
33+
'dep:spdx',
34+
'dep:serde_json',
35+
'serde',
36+
]
37+
38+
serde = ['dep:serde_derive', 'dep:serde']

0 commit comments

Comments
 (0)