Skip to content

Commit 5906fc1

Browse files
authored
Merge branch 'main' into fix/issue-1488-cpufj-incumbent
2 parents a7df065 + 52ca23a commit 5906fc1

4 files changed

Lines changed: 57 additions & 69 deletions

File tree

skills/cuopt-numerical-optimization-api-cli/BENCHMARK.md

Lines changed: 11 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -7,11 +7,11 @@ This benchmark summarizes 3-Tier Evaluation from NVSkills-Eval results for the s
77
## Evaluation Summary
88

99
- Skill: `cuopt-numerical-optimization-api-cli`
10-
- Evaluation date: 2026-05-29
10+
- Evaluation date: 2026-07-01
1111
- NVSkills-Eval profile: `external`
12-
- Environment: `local`
12+
- Environment: `astra-sandbox`
1313
- Dataset: 1 evaluation tasks
14-
- Attempts per task: 2
14+
- Attempts per task: 1
1515
- Pass threshold: 50%
1616
- Overall verdict: PASS
1717

@@ -54,34 +54,26 @@ Task composition is derived from the evaluation dataset when possible. Entries w
5454

5555
| Dimension | Num | `claude-code` | `codex` |
5656
|---|---:|---:|---:|
57-
| Security | 2 | 100% (+0%) | 100% (+0%) |
58-
| Correctness | 2 | 100% (+0%) | 97% (+5%) |
59-
| Discoverability | 2 | 100% (+0%) | 84% (+5%) |
60-
| Effectiveness | 2 | 78% (+2%) | 76% (+4%) |
61-
| Efficiency | 2 | 93% (-0%) | 78% (-0%) |
57+
| Security | 1 | 100% (+0%) | 100% (+0%) |
58+
| Correctness | 1 | 30% (+0%) | 77% (+47%) |
59+
| Discoverability | 1 | 0% (+0%) | 84% (+84%) |
60+
| Effectiveness | 1 | 39% (-4%) | 41% (+4%) |
61+
| Efficiency | 1 | 27% (-0%) | 77% (+49%) |
6262

6363
Score values show skill-assisted performance. Values in parentheses show uplift versus the no-skill baseline when baseline data is available.
6464

6565
## Tier 1: Static Validation Summary
6666

67-
Tier 1 validation passed with observations. NVSkills-Eval ran 9 checks and found 8 total findings.
67+
Tier 1 validation passed with observations. NVSkills-Eval ran 1 checks and found 2 total findings.
6868

6969
Top findings:
7070

7171
- MEDIUM SCHEMA/body_recommended_section: Missing recommended section: '## Instructions' (`skills/cuopt-numerical-optimization-api-cli/SKILL.md`)
72-
- LOW QUALITY/quality_discoverability: Broad description without negative triggers may cause over-triggering (`skills/cuopt-numerical-optimization-api-cli/SKILL.md`)
73-
- LOW QUALITY/quality_discoverability: No '## Purpose' section (`skills/cuopt-numerical-optimization-api-cli/SKILL.md`)
74-
- LOW QUALITY/quality_reliability: No prerequisites/requirements documented (`skills/cuopt-numerical-optimization-api-cli/SKILL.md`)
75-
- LOW QUALITY/quality_reliability: No limitations documented (`skills/cuopt-numerical-optimization-api-cli/SKILL.md`)
72+
- LOW SCHEMA/author_format: Author must be of the form 'Name <email@host>' (`skills/cuopt-numerical-optimization-api-cli/SKILL.md`)
7673

7774
## Tier 2: Deduplication Summary
7875

79-
Tier 2 validation passed. NVSkills-Eval ran 2 checks and found 0 total findings.
80-
81-
Notable observations:
82-
83-
- Context Deduplication: Collected 5 file(s)
84-
- Inter-Skill Deduplication: Parsed skill 'cuopt-numerical-optimization-api-cli': 141 char description
76+
This tier was not run or did not produce findings in this report.
8577

8678
## Publication Recommendation
8779

skills/cuopt-numerical-optimization-api-cli/SKILL.md

Lines changed: 25 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
name: cuopt-numerical-optimization-api-cli
33
version: "26.08.00"
4-
description: LP, MILP, and QP (beta) with cuOpt — CLI only (MPS files, cuopt_cli). Use when the user is solving LP, MILP, or QP from MPS via command line.
4+
description: LP, MILP, and QP (beta) with cuOpt — CLI only (MPS/LP/QPS files, cuopt_cli). Use when the user is solving LP, MILP, or QP from an MPS or LP file via command line.
55
license: Apache-2.0
66
metadata:
77
author: NVIDIA cuOpt Team
@@ -17,63 +17,53 @@ metadata:
1717

1818
# cuOpt Numerical Optimization — CLI
1919

20-
Solve LP, MILP, and QP problems from MPS files via `cuopt_cli`. The same command, options, and MPS workflow apply across all three; QP uses the standard MPS quadratic-objective extension.
20+
Solve LP, MILP, and QP problems from MPS or LP files via `cuopt_cli`. The same command and options apply across all three; QP is supported in both MPS (QPS) and LP files.
2121

2222
Confirm problem type and formulation (variables, objective, constraints, variable types) before coding.
2323

24-
This skill is **CLI only** (MPS input).
24+
This skill is **CLI only** (MPS or LP file input).
2525

2626
## Basic usage
2727

2828
```bash
29-
# Solve LP or MILP from MPS file
30-
cuopt_cli problem.mps
31-
32-
# With options
33-
cuopt_cli problem.mps --time-limit 120 --mip-relative-tolerance 0.01
29+
cuopt_cli <problem-file> [options]
3430
```
3531

36-
## Common options
32+
The first positional argument is the input file. The format is chosen automatically from the extension — MPS, QPS, and LP files are all accepted (including `.gz` / `.bz2` compressed variants); run `cuopt_cli --help` for the exact list of supported extensions.
3733

38-
```bash
39-
cuopt_cli --help
34+
## Options
4035

41-
# Time limit (seconds)
42-
cuopt_cli problem.mps --time-limit 120
36+
**`cuopt_cli --help` is the authoritative list — don't work from a hard-coded subset.** The CLI exposes every solver setting as a flag, generated from the parameter list at runtime: take any parameter documented in the cuOpt settings reference and replace underscores with hyphens (`time_limit``--time-limit`, `mip_relative_gap``--mip-relative-gap`). So if a parameter is documented, the flag exists; `--help` and the [solver-settings docs](https://docs.nvidia.com/cuopt/user-guide/latest/) are the sources of truth for names, meanings, and defaults.
4337

44-
# MIP gap tolerance (stop when within X% of optimal)
45-
cuopt_cli problem.mps --mip-relative-tolerance 0.001
38+
A few options are CLI-specific (not solver parameters) and worth knowing because you wouldn't derive them from a parameter name:
4639

47-
# MIP absolute tolerance
48-
cuopt_cli problem.mps --mip-absolute-tolerance 0.0001
40+
- `--params-file <file>` — supply many parameters from a `key = value` config file instead of repeating flags.
41+
- `--relaxation` — solve the continuous relaxation of a MILP (drop integrality).
42+
- `--initial-solution <file>` — warm-start from a solution file.
4943

50-
# Presolve, iteration limit, method
51-
cuopt_cli problem.mps --presolve --iteration-limit 10000 --method 1
52-
```
44+
Run `cuopt_cli --help` for the complete, current set.
45+
46+
## Authoring input files
47+
48+
MPS, QPS, and LP are precise, externally-specified file formats. Don't hand-author them from memory or from a partial recollection of the layout — column-position rules, marker conventions, the quadratic-objective encoding, and sign/scaling conventions are easy to get subtly wrong, and a malformed file either fails to parse or **silently encodes a different model than intended**.
5349

54-
## MPS format (required sections, in order)
50+
If you're building a model from data (rather than solving a file you already have), **define the problem through the cuOpt Python API or your preferred modeling interface — don't generate an MPS or LP file by hand to feed the CLI.** These interfaces take coefficients, bounds, and variable types as native data structures, so there's no text format to get wrong. The cuOpt Python API solves and returns the solution in the same program (see the `cuopt-numerical-optimization-api-python` skill); a separate modeling library (e.g. PuLP, Pyomo, JuMP) can export a valid file for the CLI or call its own solver. Reach for `cuopt_cli` when you *already* have a model file — e.g. a benchmark instance or a file exported by one of these tools.
5551

56-
1. **NAME** — problem name
57-
2. **ROWS** — N (objective), L/G/E (constraints)
58-
3. **COLUMNS** — variable names, row names, coefficients
59-
4. **RHS** — right-hand side values
60-
5. **BOUNDS** (optional) — LO, UP, FX, BV, LI, UI
61-
6. **ENDATA**
52+
When a file is genuinely the right artifact, still generate it programmatically rather than by hand:
6253

63-
Integer variables: use `'MARKER' 'INTORG'` before and `'MARKER' 'INTEND'` after the integer columns.
54+
- **From cuOpt's Python API** — build the model, then export it with `model.writeMPS("problem.mps")` (emits a valid MPS file, including the quadratic objective for QP) and solve with `cuopt_cli problem.mps`.
55+
- **From another modeling tool** — most LP/MILP modeling libraries and solvers can export standard MPS or LP files; pass the exported file straight to `cuopt_cli`.
6456

65-
## QP via CLI (beta)
57+
If you must read or write these formats by hand anyway, work from the format's full specification (and the cuOpt repo docs at `docs/cuopt/source/cuopt-cli/` for cuOpt-specific conventions such as the quadratic-objective encoding) — not from an example alone.
6658

67-
Quadratic objectives extend the standard MPS workflow — same `cuopt_cli` command, same options. Check `cuopt_cli --help` for QP-specific flags and the repo docs at `docs/cuopt/source/cuopt-cli/` for the quadratic-objective MPS format.
59+
Either way, **validate before trusting the result**: `cuopt_cli` logs `Read file ...` on a successful parse, and reports the variable/constraint counts and objective — sanity-check those against your intended model.
6860

69-
**QP rules:**
70-
- **MINIMIZE only.** For maximization, negate the objective coefficients (and Q entries) in the MPS file.
71-
- **Continuous variables only** — do not mix integer markers with quadratic objectives.
61+
**QP notes (beta):** quadratic objectives are **MINIMIZE only** (for maximization, negate the objective, including the quadratic terms) and require **continuous variables only** (no integer variables mixed with a quadratic objective). Check `cuopt_cli --help` for QP-specific flags.
7262

7363
## Troubleshooting
7464

75-
- **Failed to parse MPS**Check ENDATA, section order (NAME, ROWS, COLUMNS, RHS, [BOUNDS], ENDATA), integer markers.
76-
- **Infeasible**Check constraint directions (L/G/E) and RHS values.
65+
- **Parsing input file failed**Confirm the extension matches the format (`.lp` vs `.mps`/`.qps`); an unrecognized extension is rejected before parsing. The parser error names the offending line — fix it against the format spec, or (more reliably) regenerate the file from a modeling tool rather than patching it by hand.
66+
- **Infeasible**Re-check the model against your intended formulation: constraint directions, right-hand sides, and variable bounds.
7767

7868
## Examples
7969

skills/cuopt-numerical-optimization-api-cli/skill-card.md

Lines changed: 20 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,27 +1,33 @@
11
## Description: <br>
2-
LP, MILP, and QP (beta) with cuOpt — CLI only (MPS files, cuopt_cli). Use when the user is solving LP, MILP, or QP from MPS via command line. <br>
2+
LP, MILP, and QP (beta) with cuOpt — CLI only (MPS/LP/QPS files, cuopt_cli). Use when the user is solving LP, MILP, or QP from an MPS or LP file via command line. <br>
33

44
This skill is ready for commercial/non-commercial use. <br>
55

66
## Owner
77
NVIDIA <br>
88

99
### License/Terms of Use: <br>
10-
Apache 2.0 <br>
10+
Apache-2.0 <br>
1111
## Use Case: <br>
12-
Developers and engineers solving LP, MILP, and QP optimization problems from MPS files via the cuopt_cli command-line interface. <br>
12+
Developers and engineers solving LP, MILP, and QP optimization problems from MPS or LP files via command line using NVIDIA cuOpt's GPU-accelerated solver. <br>
1313

1414
### Deployment Geography for Use: <br>
1515
Global <br>
1616

17+
## Requirements / Dependencies: <br>
18+
**Requires API Key or External Credential:** [Not Specified] <br>
19+
**Credential Type(s):** [None identified] <br>
20+
21+
Do not include secrets in prompts/logs/output; use least-privilege credentials; rotate keys as appropriate. <br>
22+
1723
## Known Risks and Mitigations: <br>
1824
Risk: Review before execution as proposals could introduce incorrect or misleading guidance into skills. <br>
1925
Mitigation: Review and scan skill before deployment. <br>
2026

2127
## Reference(s): <br>
22-
- [cuOpt User Guide](https://docs.nvidia.com/cuopt/user-guide/latest/introduction.html) <br>
23-
- [cuopt-examples](https://github.com/NVIDIA/cuopt-examples) <br>
24-
- [Sample MPS Assets](assets/README.md) <br>
28+
- [cuOpt User Guide](https://docs.nvidia.com/cuopt/user-guide/latest/) <br>
29+
- [cuOpt Examples](https://github.com/NVIDIA/cuopt-examples) <br>
30+
- [cuOpt Releases](https://github.com/NVIDIA/cuopt/releases) <br>
2531

2632

2733
## Skill Output: <br>
@@ -31,13 +37,13 @@ Mitigation: Review and scan skill before deployment. <br>
3137
**Other Properties Related to Output:** [None] <br>
3238

3339
## Evaluation Agents Used: <br>
34-
- Claude Code (`claude-code`) <br>
35-
- Codex (`codex`) <br>
40+
- claude-code <br>
41+
- codex <br>
3642

3743

3844

3945
## Evaluation Tasks: <br>
40-
Evaluated against 1 evaluation task with 2 attempts per task via NVSkills-Eval (external profile, local environment). Pass threshold: 50%. <br>
46+
Evaluated against 1 evaluation task (positive skill-activation case) in NVSkills-Eval external profile. <br>
4147

4248
## Evaluation Metrics Used: <br>
4349
Reported benchmark dimensions: <br>
@@ -61,11 +67,11 @@ Underlying evaluation signals used in this run: <br>
6167
## Evaluation Results: <br>
6268
| Dimension | Num | `claude-code` | `codex` |
6369
|---|---:|---:|---:|
64-
| Security | 2 | 100% (+0%) | 100% (+0%) |
65-
| Correctness | 2 | 100% (+0%) | 97% (+5%) |
66-
| Discoverability | 2 | 100% (+0%) | 84% (+5%) |
67-
| Effectiveness | 2 | 78% (+2%) | 76% (+4%) |
68-
| Efficiency | 2 | 93% (-0%) | 78% (-0%) |
70+
| Security | 1 | 100% (+0%) | 100% (+0%) |
71+
| Correctness | 1 | 30% (+0%) | 77% (+47%) |
72+
| Discoverability | 1 | 0% (+0%) | 84% (+84%) |
73+
| Effectiveness | 1 | 39% (-4%) | 41% (+4%) |
74+
| Efficiency | 1 | 27% (-0%) | 77% (+49%) |
6975

7076
## Skill Version(s): <br>
7177
26.08.00 (source: frontmatter) <br>

0 commit comments

Comments
 (0)