|
| 1 | +# OpenROAD Testing Guide |
| 2 | + |
| 3 | +OpenROAD has two types of tests: **integration tests** (Tcl) and **unit tests** (C++). |
| 4 | + |
| 5 | +## CAUTION -- Dual Registration Required |
| 6 | + |
| 7 | +**Every new test MUST be registered in BOTH build systems. Missing either one is a build break.** |
| 8 | + |
| 9 | +| Build System | Registration File | Block/Macro | |
| 10 | +|-------------|-------------------|-------------| |
| 11 | +| **CMake** | `src/<module>/test/CMakeLists.txt` | `or_integration_tests(...)` | |
| 12 | +| **Bazel** | `src/<module>/test/BUILD` | `regression_test(...)` | |
| 13 | + |
| 14 | +Forgetting the Bazel `BUILD` file is the most common mistake -- CMake-only registration |
| 15 | +silently passes local `make test` but the **test will be missing from Bazel CI**. |
| 16 | + |
| 17 | +## Integration Tests |
| 18 | + |
| 19 | +Integration tests are Tcl scripts located at `src/<module>/test` or `src/<module>/test/tcl`. |
| 20 | + |
| 21 | +### Running Integration Tests |
| 22 | + |
| 23 | +`./regression` is a ctest wrapper that filters by module label. |
| 24 | + |
| 25 | +```bash |
| 26 | +cd src/rsz/test |
| 27 | +./regression -R buffer_ports1 # run one test (regex match) |
| 28 | +./regression -R "repair.*hier" # run tests matching pattern |
| 29 | +./regression -j4 -V # all rsz tests, parallel, verbose |
| 30 | + |
| 31 | +# Or equivalently from the build directory: |
| 32 | +cd build |
| 33 | +ctest -R "rsz\.buffer_ports1\.tcl" |
| 34 | +``` |
| 35 | + |
| 36 | +### Creating a New Integration Test |
| 37 | + |
| 38 | +Best practice reference: `src/rsz/test/repair_tie12_hier.tcl` |
| 39 | + |
| 40 | +1. **Header comment**: First line must explain the test purpose |
| 41 | + ```tcl |
| 42 | + # [Brief description of the test] |
| 43 | + ``` |
| 44 | + |
| 45 | +2. **Set test name**: Used for output file generation |
| 46 | + ```tcl |
| 47 | + set test_name <integration_test_name> |
| 48 | + ``` |
| 49 | + |
| 50 | +3. **Output naming with `make_result_file`**: Use for temporary output files |
| 51 | + ```tcl |
| 52 | + set verilog_filename "${test_name}.v" |
| 53 | + set out_verilog [make_result_file $verilog_filename] |
| 54 | + write_verilog $out_verilog |
| 55 | + ``` |
| 56 | + |
| 57 | +4. **Verification with `diff_file`**: Compare against golden files |
| 58 | + ```tcl |
| 59 | + diff_file ${test_name}.vok $out_verilog |
| 60 | + ``` |
| 61 | + |
| 62 | +5. **Generate `<test_name>.ok` file**: Create golden test log by executing the script, redirecting stdout, and removing the openroad banner at the top. |
| 63 | + |
| 64 | +6. **Registration (BOTH files for CMake and Bazel -- do NOT skip Bazel)** |
| 65 | + |
| 66 | +### Test Framework Flags |
| 67 | +- `-no_splash -no_init -exit` |
| 68 | + |
| 69 | +### Golden File Notes |
| 70 | +- `.ok` = log golden file (full stdout), `.vok` = verilog golden file |
| 71 | +- **`diff_file` reports only the first difference** -- when regenerating golden files, regenerate the entire output file rather than relying on incremental diffs. |
| 72 | + |
| 73 | +## Unit Tests |
| 74 | + |
| 75 | +Unit tests are C++ files located at `src/<module>/test/cpp`. |
| 76 | + |
| 77 | +### Test Registration |
| 78 | +- CMake: `src/<module>/test/CMakeLists.txt` |
| 79 | +- Bazel: `src/<module>/test/BUILD` or `src/<module>/test/cpp/BUILD` |
0 commit comments