|
2 | 2 | { |
3 | 3 | "id": "numopt-py-eval-001-lp-api-call-sequence", |
4 | 4 | "question": "I want to solve a small LP (continuous variables only, maximize a linear objective with linear constraints) using the cuOpt Python API. List the API calls in order \u2014 name each method, one line per method, no full runnable script.", |
5 | | - "expected_skill": "cuopt-numerical-optimization-api-python", |
| 5 | + "expected_skill": "cuopt-numerical-optimization-api", |
6 | 6 | "expected_script": null, |
7 | 7 | "ground_truth": "The agent produces an ordered list of API calls without a runnable script. The list, in order: (1) Import Problem, CONTINUOUS, and MAXIMIZE from cuopt.linear_programming.problem, and SolverSettings from cuopt.linear_programming.solver_settings. (2) Construct Problem('name'). (3) For each decision variable, call problem.addVariable(lb=..., vtype=CONTINUOUS, name=...). (4) For each constraint, call problem.addConstraint(<linear expression> <= or >= or == <rhs>, name=...). (5) Call problem.setObjective(<linear expression>, sense=MAXIMIZE). (6) Construct SolverSettings(); call set_parameter('time_limit', ...) for time budget. (7) Call problem.solve(settings). (8) Check problem.Status.name in ['Optimal', 'PrimalFeasible'] (PascalCase status names \u2014 case-sensitive). (9) Read problem.ObjValue for the objective, and each variable's .getValue() for its optimal value. The agent uses LP (not MILP / QP) because all variables are continuous and the objective is linear. Mentions that status names are PascalCase (Optimal, not OPTIMAL or optimal) \u2014 case sensitivity matters.", |
8 | 8 | "expected_behavior": [ |
|
19 | 19 | { |
20 | 20 | "id": "numopt-py-eval-002-status-case-sensitivity", |
21 | 21 | "question": "My cuOpt Python LP solve runs without error but the result block never executes. Here is the check I wrote: if problem.Status.name == 'OPTIMAL': print(problem.ObjValue). What is wrong and how do I fix it?", |
22 | | - "expected_skill": "cuopt-numerical-optimization-api-python", |
| 22 | + "expected_skill": "cuopt-numerical-optimization-api", |
23 | 23 | "expected_script": null, |
24 | 24 | "ground_truth": "The check silently fails because cuOpt status names use PascalCase, not ALL_CAPS. The string 'OPTIMAL' never matches. The correct LP status values to check are 'Optimal' and 'PrimalFeasible'. The fixed check is: if problem.Status.name in ['Optimal', 'PrimalFeasible']: print(problem.ObjValue). For MILP the correct values are 'Optimal' and 'FeasibleFound'. This is a common silent bug \u2014 the solve completes successfully but the code path that reads results is skipped because the string comparison always returns False.", |
25 | 25 | "expected_behavior": [ |
|
33 | 33 | { |
34 | 34 | "id": "numopt-py-eval-003-integer-vs-continuous-workers", |
35 | 35 | "question": "I am modeling a staffing problem where I need to decide how many nurses to assign to each ward. Should the nurse count variables be INTEGER or CONTINUOUS in the cuOpt Python API, and what vtype constant do I use for each?", |
36 | | - "expected_skill": "cuopt-numerical-optimization-api-python", |
| 36 | + "expected_skill": "cuopt-numerical-optimization-api", |
37 | 37 | "expected_script": null, |
38 | 38 | "ground_truth": "Nurse counts should be INTEGER because nurses are discrete countable entities \u2014 you cannot assign 2.7 nurses to a ward. The vtype constant is INTEGER (imported from cuopt.linear_programming.problem). The addVariable call would be: problem.addVariable(lb=0, vtype=INTEGER, name='ward_a_nurses'). This makes the problem a MILP, not an LP. CONTINUOUS would be wrong here because it allows fractional values, which are meaningless for headcounts. The rule is: 'how many things' (people, vehicles, machines) \u2192 INTEGER; 'how much of something' (hours, tonnes, dollars) \u2192 CONTINUOUS.", |
39 | 39 | "expected_behavior": [ |
|
48 | 48 | { |
49 | 49 | "id": "numopt-py-eval-004-qp-maximize-workaround", |
50 | 50 | "question": "I want to maximize a quadratic objective using the cuOpt Python QP API. When I pass sense=MAXIMIZE to setObjective, I get an error. What is the correct approach?", |
51 | | - "expected_skill": "cuopt-numerical-optimization-api-python", |
| 51 | + "expected_skill": "cuopt-numerical-optimization-api", |
52 | 52 | "expected_script": null, |
53 | 53 | "ground_truth": "The cuOpt QP solver only supports MINIMIZE \u2014 MAXIMIZE is rejected for quadratic objectives. The correct workaround is to negate all coefficients in the objective and minimize the negated expression. For example, to maximize -0.04*x1*x1 - 0.02*x2*x2 (a concave quadratic with NSD Q), minimize 0.04*x1*x1 + 0.02*x2*x2 with sense=MINIMIZE. The resulting problem.ObjValue will be the negated maximum; multiply by -1 to recover the true maximum. All variables must remain CONTINUOUS \u2014 integer QP is not supported. The Q matrix of the original maximization problem must be negative semi-definite (NSD) for the problem to be concave and have a finite maximum; after negation it becomes PSD, which is what the solver expects. Maximizing a convex quadratic (positive coefficients) is unbounded and not a meaningful use case.", |
54 | 54 | "expected_behavior": [ |
|
62 | 62 | { |
63 | 63 | "id": "numopt-c-eval-001-milp-api-call-sequence", |
64 | 64 | "question": "I want to solve a small MILP (some integer variables, linear objective, linear constraints) with the cuOpt C API. List the C functions and structs I need in order \u2014 names only, one line each, no full source.", |
65 | | - "expected_skill": "cuopt-numerical-optimization-api-c", |
| 65 | + "expected_skill": "cuopt-numerical-optimization-api", |
66 | 66 | "expected_script": null, |
67 | 67 | "ground_truth": "The agent produces an ordered list of C API entry points without writing a full source file: include cuopt/mathematical_optimization/cuopt_c.h, then call cuOptCreateRangedProblem with sense CUOPT_MINIMIZE or CUOPT_MAXIMIZE, then cuOptSolve(problem, settings, &solution), then cuOptGetObjectiveValue.", |
68 | 68 | "expected_behavior": [ |
|
73 | 73 | { |
74 | 74 | "id": "numopt-c-eval-002-parameter-function-wrong-name", |
75 | 75 | "question": "I am setting a time limit on my cuOpt C API solver with this call: cuOptSetIntParameter(settings, CUOPT_TIME_LIMIT, 60.0). My colleague says the function name is wrong. What is the correct function, and what other parameter-setting functions does the C API provide?", |
76 | | - "expected_skill": "cuopt-numerical-optimization-api-c", |
| 76 | + "expected_skill": "cuopt-numerical-optimization-api", |
77 | 77 | "expected_script": null, |
78 | 78 | "ground_truth": "The function name cuOptSetIntParameter does not exist in the cuOpt C API \u2014 it is a common mistake. The correct function for float parameters (including CUOPT_TIME_LIMIT, tolerances) is cuOptSetFloatParameter. The C API provides three parameter-setting functions: cuOptSetFloatParameter for float params such as time limits and tolerances, cuOptSetIntegerParameter (not cuOptSetIntParameter) for integer params such as CUOPT_LOG_TO_CONSOLE and method selection, and cuOptSetParameter for string params. CUOPT_TIME_LIMIT is a float parameter so the correct call is cuOptSetFloatParameter(settings, CUOPT_TIME_LIMIT, 60.0).", |
79 | 79 | "expected_behavior": [ |
|
86 | 86 | { |
87 | 87 | "id": "numopt-c-eval-003-csr-constraint-matrix", |
88 | 88 | "question": "I am building the constraint matrix for a cuOpt C LP. The problem has 2 constraints and 2 variables. Constraint 1: 3x1 + 4x2 <= 5.4. Constraint 2: 2.7x1 + 10.1x2 <= 4.9. Show me the row_offsets, col_indices, and values arrays for the CSR representation, and explain what each array means.", |
89 | | - "expected_skill": "cuopt-numerical-optimization-api-c", |
| 89 | + "expected_skill": "cuopt-numerical-optimization-api", |
90 | 90 | "expected_script": null, |
91 | 91 | "ground_truth": "The CSR (Compressed Sparse Row) format uses three arrays. row_offsets has length num_constraints+1 = 3: {0, 2, 4}. Element i gives the starting index in col_indices/values for row i; the last element is the total number of nonzeros (4 here). col_indices = {0, 1, 0, 1}: the column index of each nonzero, ordered by row. values = {3.0, 4.0, 2.7, 10.1}: the nonzero values in the same order. Constraint upper bounds are {5.4, 4.9} and lower bounds are {-CUOPT_INFINITY, -CUOPT_INFINITY} since both constraints are <=. These arrays are passed to cuOptCreateRangedProblem.", |
92 | 92 | "expected_behavior": [ |
|
100 | 100 | { |
101 | 101 | "id": "numopt-c-eval-004-qp-restrictions", |
102 | 102 | "question": "I want to solve a QP with integer variables using the cuOpt C API. A colleague says this is not supported. Is that correct, and what are the restrictions for QP in the cuOpt C API?", |
103 | | - "expected_skill": "cuopt-numerical-optimization-api-c", |
| 103 | + "expected_skill": "cuopt-numerical-optimization-api", |
104 | 104 | "expected_script": null, |
105 | 105 | "ground_truth": "The colleague is correct \u2014 integer QP is not supported in the cuOpt C API. The QP restrictions are: (1) minimization only \u2014 CUOPT_MINIMIZE is required; to maximize a quadratic objective, negate all objective coefficients and Q matrix entries; (2) continuous variables only \u2014 all variables must use CUOPT_CONTINUOUS, integer variables are not supported for QP; (3) the Q matrix should be positive semi-definite (PSD) for a convex, well-posed problem. The same library, include paths, and build pattern as LP/MILP are used; only the problem-creation call differs for QP.", |
106 | 106 | "expected_behavior": [ |
|
114 | 114 | { |
115 | 115 | "id": "numopt-cli-eval-001-mps-sections-and-cli-command", |
116 | 116 | "question": "I have an LP problem I want to solve with cuopt_cli from an MPS file, with a 60-second time limit and 1% MIP gap (in case I add integers later). List the MPS sections in required order, and the cuopt_cli command line.", |
117 | | - "expected_skill": "cuopt-numerical-optimization-api-cli", |
| 117 | + "expected_skill": "cuopt-numerical-optimization-api", |
118 | 118 | "expected_script": null, |
119 | 119 | "ground_truth": "The agent lists the MPS sections in the required order: NAME, ROWS (N row for the objective, L/G/E rows for constraints), COLUMNS (variable-name, row-name, coefficient triples), RHS (right-hand-side values), BOUNDS (optional \u2014 LO/UP/FX/BV/LI/UI), ENDATA. For integer variables, integer markers are 'MARKER' 'INTORG' before and 'MARKER' 'INTEND' after the integer columns. The cuopt_cli invocation is: cuopt_cli problem.mps --time-limit 60 --mip-relative-tolerance 0.01. The agent mentions cuopt_cli --help as the canonical source for all flags. Does not invent flags like --max-time or --gap that are not in the skill. Notes that cuopt_cli ships with the cuopt Python package (install via pip or conda first if not present).", |
120 | 120 | "expected_behavior": [ |
|
0 commit comments