Skip to content

Commit 3004e97

Browse files
Brennan1994Brennan1994
committed
add examples.
1 parent 1578789 commit 3004e97

3 files changed

Lines changed: 282 additions & 1 deletion

File tree

‎HEC.FDA.TestingUtility/Configuration/TestConfiguration.cs‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ public class GlobalSettings
2828
public string LocalTempDirectory { get; set; } = Path.Combine(Path.GetTempPath(), "FDATests");
2929

3030
[JsonPropertyName("timeoutMinutes")]
31-
public int TimeoutMinutes { get; set; } = 30;
31+
public int TimeoutMinutes { get; set; } = 120;
3232
}
3333

3434
public class StudyConfiguration

‎HEC.FDA.TestingUtility/README.md‎

Lines changed: 262 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,262 @@
1+
# HEC.FDA.TestingUtility
2+
3+
A command-line regression testing tool for HEC-FDA studies. It loads FDA study databases, runs computations (stage damage, scenarios, alternatives, and alternative comparisons), and produces a comprehensive CSV report of the results.
4+
5+
## Purpose
6+
7+
This utility runs FDA computations outside the GUI so that results can be validated programmatically. Typical use cases include:
8+
9+
- **Regression testing** - Run computations on known studies and compare the CSV output against baseline results to detect unintended changes.
10+
- **Batch computation** - Execute all computations in one or more studies without manual interaction.
11+
- **CI/CD integration** - Automate computation verification as part of a build pipeline.
12+
13+
## Quick Start
14+
15+
### 1. Build the project
16+
17+
```bash
18+
dotnet build HEC.FDA.TestingUtility/HEC.FDA.TestingUtility.csproj
19+
```
20+
21+
### 2. Create a configuration file
22+
23+
Copy `example-config.json` from this project directory and update `networkSourcePath` to point to your FDA study folder (the folder containing the `.sqlite` or `.db` file):
24+
25+
```json
26+
{
27+
"testSuiteId": "quick-start",
28+
"globalSettings": {
29+
"localTempDirectory": "C:/temp/FDATests",
30+
"timeoutMinutes": 60
31+
},
32+
"studies": [
33+
{
34+
"studyId": "my-study",
35+
"studyName": "My Study",
36+
"networkSourcePath": "C:/path/to/my/study/folder",
37+
"runAllStageDamage": true,
38+
"runAllScenarios": true,
39+
"runAllAlternatives": true,
40+
"runAllAlternativeComparisons": true,
41+
"computations": []
42+
}
43+
]
44+
}
45+
```
46+
47+
Setting the `runAll*` flags to `true` auto-discovers every element of that type in the study, so you don't need to list them individually. If you only want to run specific elements, set the flags to `false` and use `computations` instead:
48+
49+
```json
50+
{
51+
"testSuiteId": "targeted-run",
52+
"globalSettings": {},
53+
"studies": [
54+
{
55+
"studyId": "my-study",
56+
"studyName": "My Study",
57+
"networkSourcePath": "C:/path/to/my/study/folder",
58+
"runAllStageDamage": false,
59+
"runAllScenarios": false,
60+
"runAllAlternatives": false,
61+
"runAllAlternativeComparisons": false,
62+
"computations": [
63+
{ "type": "stagedamage", "elementName": "My Stage Damage" },
64+
{ "type": "scenario", "elementName": "Existing Conditions" },
65+
{ "type": "alternative", "elementName": "Proposed Levee" }
66+
]
67+
}
68+
]
69+
}
70+
```
71+
72+
You can list computations in any order -- the utility automatically sorts them by dependency (stage damage first, then scenarios, then alternatives, then alternative comparisons).
73+
74+
### 3. Run the utility
75+
76+
```bash
77+
# Run all studies in the config, write results to a "results" folder
78+
dotnet run --project HEC.FDA.TestingUtility -- compute -c my-test.json -o results
79+
80+
# Run only a single study by its studyId
81+
dotnet run --project HEC.FDA.TestingUtility -- compute -c my-test.json -o results -s "my-study"
82+
83+
# Filter to multiple studies
84+
dotnet run --project HEC.FDA.TestingUtility -- compute -c my-test.json -s "study-a" -s "study-b"
85+
```
86+
87+
### 4. Check the output
88+
89+
After the run completes, look for `results_report.csv` in the output directory. It contains separate sections for scenario EAD, damage by category, performance metrics, alternative EqAD, stage damage summaries, and alternative comparison results.
90+
91+
The console output will also show a summary:
92+
93+
```
94+
=== Summary ===
95+
Completed: 5
96+
Errors: 0
97+
Duration: 2m 34.1s
98+
99+
CSV report saved to: results/results_report.csv
100+
```
101+
102+
An exit code of `0` means all computations passed. An exit code of `1` means at least one failed -- scroll up in the console output to find lines marked `ERROR`.
103+
104+
## Prerequisites
105+
106+
- .NET 9.0 SDK (Windows, targets `net9.0-windows`)
107+
- Access to FDA study databases (`.sqlite` or `.db` files) located on a network share or local directory
108+
- NuGet sources configured as described in the root `CLAUDE.md`
109+
110+
## Building
111+
112+
```bash
113+
dotnet build HEC.FDA.TestingUtility/HEC.FDA.TestingUtility.csproj
114+
```
115+
116+
## Usage
117+
118+
The utility uses the `System.CommandLine` library and exposes a `compute` subcommand:
119+
120+
```bash
121+
dotnet run --project HEC.FDA.TestingUtility -- compute --config <path-to-config.json> [options]
122+
```
123+
124+
### Options
125+
126+
| Option | Alias | Required | Description |
127+
|---|---|---|---|
128+
| `--config` | `-c` | Yes | Path to a JSON configuration file defining the studies and computations to run. |
129+
| `--output` | `-o` | No | Output directory for generated files. Defaults to the current working directory. |
130+
| `--study` | `-s` | No | Filter to one or more specific study IDs. Can be specified multiple times. |
131+
132+
### Examples
133+
134+
Run all studies defined in a config file:
135+
136+
```bash
137+
dotnet run --project HEC.FDA.TestingUtility -- compute -c tests/regression.json -o results/
138+
```
139+
140+
Run only a specific study:
141+
142+
```bash
143+
dotnet run --project HEC.FDA.TestingUtility -- compute -c tests/regression.json -s "muncie"
144+
```
145+
146+
## Configuration File
147+
148+
The configuration is a JSON file with the following structure:
149+
150+
```json
151+
{
152+
"testSuiteId": "regression-v1",
153+
"globalSettings": {
154+
"localTempDirectory": "C:/temp/FDATests",
155+
"timeoutMinutes": 30
156+
},
157+
"studies": [
158+
{
159+
"studyId": "muncie",
160+
"studyName": "Muncie Indiana",
161+
"networkSourcePath": "\\\\server\\share\\studies\\Muncie",
162+
"runAllStageDamage": true,
163+
"runAllScenarios": true,
164+
"runAllAlternatives": true,
165+
"runAllAlternativeComparisons": true,
166+
"computations": []
167+
}
168+
]
169+
}
170+
```
171+
172+
### Global Settings
173+
174+
| Field | Default | Description |
175+
|---|---|---|
176+
| `localTempDirectory` | System temp + `FDATests` | Directory where studies are copied locally before computation. |
177+
| `timeoutMinutes` | `30` | Maximum wall-clock time for all computations before cancellation. |
178+
179+
### Study Configuration
180+
181+
| Field | Description |
182+
|---|---|
183+
| `studyId` | Short identifier used in the CSV report and for the `--study` filter. |
184+
| `studyName` | Human-readable name printed during execution. |
185+
| `networkSourcePath` | Path to the study folder containing the `.sqlite` or `.db` database file. |
186+
| `runAllStageDamage` | Auto-discover and run all stage damage elements in the study. |
187+
| `runAllScenarios` | Auto-discover and run all scenario elements. |
188+
| `runAllAlternatives` | Auto-discover and run all alternative elements. |
189+
| `runAllAlternativeComparisons` | Auto-discover and run all alternative comparison report elements. |
190+
| `computations` | Explicit list of computations (see below). Combined with auto-discovered elements. |
191+
192+
### Computation Entry
193+
194+
Each entry in the `computations` array targets a specific element:
195+
196+
```json
197+
{
198+
"type": "scenario",
199+
"elementName": "Existing Conditions"
200+
}
201+
```
202+
203+
Valid `type` values (case-insensitive):
204+
205+
| Type | Description |
206+
|---|---|
207+
| `stagedamage` | Computes stage-damage curves from hydraulics and inventory data. |
208+
| `scenario` | Runs a Monte Carlo scenario simulation using convergence criteria from study properties. |
209+
| `alternative` | Computes annualized damages for an alternative using its base and future scenario results. |
210+
| `alternativecomparison` | Compares with-project alternatives against a without-project alternative. |
211+
212+
Computations are automatically sorted in dependency order: stage damage -> scenario -> alternative -> alternative comparison. This means you can list them in any order and the utility will execute them correctly.
213+
214+
## How It Works
215+
216+
1. **Study Loading** - The study folder is copied from `networkSourcePath` to a local temp directory to avoid locking network files. The SQLite database is opened and all element types are loaded into the `StudyCache` in dependency order (terrains, impact areas, hydraulics, frequencies, inventories, stage damage, scenarios, alternatives, etc.).
217+
218+
2. **Computation** - Each computation is dispatched to the appropriate runner:
219+
- `StageDamageRunner` - Builds stage-damage configuration from hydraulics and inventory, then calls `ScenarioStageDamage.Compute()`.
220+
- `ScenarioRunner` - Creates `ImpactAreaScenarioSimulation` objects and runs `Scenario.Compute()` with the study's convergence criteria.
221+
- `AlternativeRunner` - Calls `Alternative.AnnualizationCompute()` using the base/future scenario results and study discount rate / period of analysis.
222+
- `AlternativeComparisonRunner` - Computes alternatives and then runs `AlternativeComparisonReport.ComputeAlternativeComparisonReport()`.
223+
224+
3. **Result Saving** - Computed results are saved back to the temp study database so that downstream computations (e.g., alternatives depending on scenario results) can access them.
225+
226+
4. **CSV Report** - A single `results_report.csv` file is written to the output directory containing sections for:
227+
- Scenario results (mean and percentile EAD by impact area)
228+
- Scenario damage by category
229+
- Scenario performance (long-term risk, AEP, assurance)
230+
- Scenario assurance of AEP
231+
- Alternative results (mean and percentile EqAD)
232+
- Alternative damage by category
233+
- Stage damage summary (point counts, stage ranges, median integrals)
234+
- Alternative comparison summary (EqAD reduced, base/future EAD reduced)
235+
- Alternative comparison by damage category
236+
237+
5. **Cleanup** - When a study finishes, the temp copy is deleted automatically.
238+
239+
## Project Structure
240+
241+
```
242+
HEC.FDA.TestingUtility/
243+
Program.cs # CLI entry point (System.CommandLine)
244+
ComputeRunner.cs # Orchestrates study loading, computation, and reporting
245+
Configuration/
246+
TestConfiguration.cs # JSON config deserialization models
247+
Services/
248+
StudyLoader.cs # Copies study to temp, opens DB, loads all elements
249+
StageDamageRunner.cs # Stage damage computation
250+
ScenarioRunner.cs # Scenario computation + element lookup helpers
251+
AlternativeRunner.cs # Alternative annualization computation
252+
AlternativeComparisonRunner.cs # Alternative comparison report computation
253+
Reporting/
254+
CsvReportFactory.cs # Builds the multi-section CSV report
255+
```
256+
257+
## Exit Codes
258+
259+
| Code | Meaning |
260+
|---|---|
261+
| `0` | All computations completed successfully. |
262+
| `1` | One or more computations failed, or the configuration was invalid. |

‎HEC.FDA.TestingUtility/example-config.json‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
{
2+
"testSuiteId": "quick-start",
3+
"globalSettings": {
4+
"localTempDirectory": "C:/temp/FDATests",
5+
"timeoutMinutes": 60
6+
},
7+
"studies": [
8+
{
9+
"studyId": "my-study",
10+
"studyName": "My Study",
11+
"networkSourcePath": "C:/path/to/my/study/folder",
12+
"runAllStageDamage": true,
13+
"runAllScenarios": true,
14+
"runAllAlternatives": true,
15+
"runAllAlternativeComparisons": true,
16+
"computations": []
17+
}
18+
]
19+
}

0 commit comments

Comments
 (0)