Move doc from comments to meta (JuliaSmoothOptimizers#407)

* Move doc from comments to meta (aircrfta)

* up

* do not move license

* add new fields in Meta

* Move comments to META

* fix

* add refs

* move things around hs

* add doc

* Add a bib creator

* fix

* fix conflict

* allow multiple URL

* fill-in meta

* up some refs

* clean COPS test problems

* clean HS2

* Fix split :origin and :origin_notes

* Add missing bibtex and links

* Add lib ref

* small fixes

* 59 files with http://dx.doi.org/ → https://doi.org/

* Update SKILL

* uniform bib

Author: tmigot

.claude/skills/find-bib/SKILL.md

@@ -0,0 +1,162 @@
+---
+name: find-bib
+description: Find or construct a BibTeX reference for an OptimizationProblems.jl meta file. Given a DOI, problem name, or free-form reference text, searches for the DOI online, fetches BibTeX from doi2bib.org, and formats the result as a Julia raw string ready to paste into src/Meta/<name>.jl.
+argument-hint: <problem-name | DOI | "free-form reference text">
+allowed-tools: [Read, Glob, Grep, WebSearch, WebFetch]
+---
+
+# find-bib
+
+Find or construct a BibTeX reference and format it as a Julia raw string for `src/Meta/<name>.jl`.
+
+## Arguments
+
+The user invoked this skill with: $ARGUMENTS
+
+This can be:
+- A **problem name** from `src/Meta/` (e.g. `zangwil3`, `hs1`) — skill will read the file and extract context
+- A **DOI** (e.g. `10.1145/355934.355936` or `https://doi.org/10.1145/355934.355356`)
+- **Free-form reference text** (title, authors, journal, year — quoted or unquoted)
+- **Empty** — if no argument is given, ask the user to specify a problem name or reference
+
+---
+
+## Instructions
+
+Follow these steps in order.
+
+### Step 1 — Gather source material
+
+If `$ARGUMENTS` looks like a problem name (no spaces, no slashes, exists as `src/Meta/<name>.jl`):
+- Read `src/Meta/$ARGUMENTS.jl`
+- Extract `:url`, `:notes`, `:origin_notes`, and the existing `:reference` field (may be empty)
+- Use this text as the source for Steps 2–4
+
+Otherwise, treat `$ARGUMENTS` directly as a DOI or free-form reference text.
+
+### Step 2 — Extract or find a DOI
+
+1. Regex-scan all gathered text for a DOI pattern: `10\.\d{4,}/\S+`
+   - DOIs appear in `:url`, `:notes`, `:origin_notes`, or in the argument itself
+2. If no DOI is found, run a **WebSearch** to locate one:
+   - Try: `"<title or key terms>" "<first author>" DOI`
+   - Try: `site:doi.org "<title or key terms>"`
+   - Try CrossRef: `site:search.crossref.org "<title or key terms>"`
+3. Extract the DOI string from any search result that contains one.
+
+### Step 3 — Fetch BibTeX
+
+If a DOI was found (say `10.1145/355934.355936`), try these endpoints in order:
+
+1. **CrossRef** (primary): `https://api.crossref.org/works/10.1145/355934.355936/transform/application/x-bibtex`
+2. **doi2bib.org** (fallback): `https://www.doi2bib.org/bib/10.1145/355934.355936`
+
+The response is plain-text BibTeX — clean up whitespace if needed, then apply these normalizations:
+
+- **Citation key**: CrossRef often returns auto-generated keys like `More_1981` or `more1981testingunconstrained`. Always rename the key to the `Author1Author2YYYY` format described in Step 4 (e.g. `MoreGarbowHillstrom1981`).
+- **`pages` field**: normalize to BibTeX double-hyphen. Replace Unicode en-dash `–` or a single hyphen `-` between page numbers with `--` (e.g. `175--184`).
+
+If both endpoints fail or return an error, fall through to Step 4.
+
+### Step 4 — Construct BibTeX manually (fallback)
+
+If no DOI was found or doi2bib.org failed, construct the best possible BibTeX from all available information.
+
+Choose the entry type:
+
+| Type | Use when |
+|---|---|
+| `@article` | journal or conference paper |
+| `@book` | book or edited volume |
+| `@techreport` | institutional or technical report |
+| `@misc` | dataset, software, website, or unclear |
+
+**Citation key format:** `Author1Author2YYYY` using last names only (e.g. `MoreGarbowHillstrom1981`, `HockSchittkowski1981`). For a single author: `AuthorYYYY`. For institutional authors use a compact CamelCase form (e.g. `NISTStRD`).
+
+**`pages` field:** always use double-hyphen: `175--184`. Convert Unicode en-dash `–`, em-dash `—`, or single hyphen `-` between page numbers to `--`.
+
+**LaTeX encoding for special characters:** `Mor{\'e}`, `{\'E}`, etc.
+
+**Flag uncertain fields** with a trailing `% UNVERIFIED` comment on that line.
+
+### Step 5 — Present the result
+
+Show three things:
+
+**1. Status line** — one sentence: whether the DOI was found, and from where.
+
+**2. The BibTeX entry** in a code block:
+```bibtex
+@article{AuthorYear,
+  author  = {Last, First and Last2, First2},
+  title   = {Title of the Article},
+  journal = {Journal Name},
+  year    = {YYYY},
+  volume  = {V},
+  number  = {N},
+  pages   = {P1--P2},
+  doi     = {10.xxxx/xxxxx}
+}
+```
+
+**3. The Julia snippet** ready to paste into `src/Meta/<name>.jl`:
+```julia
+:reference => raw"""
+@article{AuthorYear,
+  author  = {Last, First and Last2, First2},
+  title   = {Title of the Article},
+  journal = {Journal Name},
+  year    = {YYYY},
+  volume  = {V},
+  number  = {N},
+  pages   = {P1--P2},
+  doi     = {10.xxxx/xxxxx}
+}
+""",
+```
+
+### Step 6 — Suggest next steps
+
+- List any fields marked `% UNVERIFIED` that the user should check manually.
+- If a DOI was found and the file's `:url` field does not already contain it, suggest adding `https://doi.org/<DOI>` to `:url`. The `:url` field supports multiple URLs as a **comma-separated string**. If a URL is already present, append the new one: `"https://existing.url, https://doi.org/<DOI>"`.
+- If the problem name was given, name the exact file to edit: `src/Meta/<name>.jl`.
+
+---
+
+## BibTeX type templates (reference)
+
+```bibtex
+@article{AuthorYear,
+  author  = {Last, First},
+  title   = {Title},
+  journal = {Journal},
+  year    = {YYYY},
+  volume  = {V},
+  number  = {N},
+  pages   = {P1--P2},
+  doi     = {10.xxxx/xxxxx}
+}
+
+@book{AuthorYear,
+  author    = {Last, First},
+  title     = {Book Title},
+  publisher = {Publisher},
+  address   = {City},
+  year      = {YYYY}
+}
+
+@techreport{AuthorYear,
+  author      = {Last, First},
+  title       = {Report Title},
+  institution = {Institution},
+  number      = {Report Number},
+  year        = {YYYY}
+}
+
+@misc{AuthorYear,
+  author       = {Last, First},
+  title        = {Title},
+  year         = {YYYY},
+  howpublished = {\url{https://...}}
+}
+```

.gitignore

@@ -2,4 +2,6 @@
 *.jl.*.cov
 *.jl.mem
 Manifest.toml
+*.bib
+*/settings.local.json
 .DS_Store

docs/src/contributing.md

@@ -22,8 +22,8 @@ Here is a to-do list, to help you add new problems:
 In both cases, the function must have the same name `problem_name` as the file.
 The function should be only exported from `src/ADNLPProblems/problem_name.jl` and `src/PureJuMP/problem_name.jl`.
 * When submitting a problem, please pay particular attention to the documentation. We would like to gather as much information as possible on the provenance of problems, other problem sets where the problems are present, and general information on the problem. 
-The documentation should be added to the file in the `PureJuMP` folder.
-* New problems can be scalable, see [ADNLPProblems/arglina.jl](https://github.com/JuliaSmoothOptimizers/OptimizationProblems.jl/blob/main/src/ADNLPProblems/arglina.jl) and [PureJuMP/arglina.jl](https://github.com/JuliaSmoothOptimizers/OptimizationProblems.jl/blob/main/src/PureJuMP/arglina.jl) for examples. In that case, the first keyword parameter should be the number of variables `n::Int` and have the default value `default_nvar` (constant predefined in the module). If your problem has restrictions on the number of variables, e.g., `n` should be odd, or `n` should have the form `4k + 3`, then, instead of throwing errors when the restrictions are not satisfied, you should instead use the number of variables to be as close to `n` as possible. For example, if you want `n` odd and `n = 100` is passed, you can internally convert to `n = 99`. If you want `n = 4k + 3`, and `n = 100` is passed, then compute `k = round(Int, (n - 3) / 4)` and update `n`. When such an internal adjustment is made, emit a warning indicating the requested `n` and the effective value used.
+The documentation should be added to the corresponding fields in the `Meta` folder.
+* New problems can be scalable, see [ADNLPProblems/arglina.jl](https://github.com/JuliaSmoothOptimizers/OptimizationProblems.jl/blob/main/src/ADNLPProblems/arglina.jl) and [PureJuMP/arglina.jl](https://github.com/JuliaSmoothOptimizers/OptimizationProblems.jl/blob/main/src/PureJuMP/arglina.jl) for examples. In that case, the first keyword parameter should be the number of variables `n::Int` and have the default value `default_nvar` (constant predefined in the module). If your problem has restrictions on the number of variables, e.g., `n` should be odd, or `n` should have the form `4k + 3`, then, instead of throwing errors when the restrictions are not satisfied, you should instead use the number of variables to be as close to `n` as possible. For example, if you want `n` odd and `n = 100` is passed, you can internally convert to `n = 99`. If you want `n = 4k + 3`, and `n = 100` is passed, then compute `k = round(Int, (n - 3) / 4)` and update `n`.
 * A first version of the `meta` can be generated using `generate_meta`. A `String` is returned that can be copy-pasted into the `Meta` folder, and then edited.
 
 ```julia

docs/src/meta.md

@@ -59,4 +59,52 @@ OptimizationProblems.get_lanczos1_nls_nequ()
 To filter all NLS problems in the metadata DataFrame:
 ```@example 1
 nls_problems = OptimizationProblems.meta[OptimizationProblems.meta.objtype .== :least_squares, :name]
-```
+```
+
+### Test-set membership (`:lib`)
+
+The `:lib` column records membership in named optimization test-set collections.
+Each entry is a comma-separated list of `"Collection:ID"` pairs; an empty string
+means the problem has no known test-set membership.
+
+```@example 1
+OptimizationProblems.hs1_meta[:lib]  # "CUTEst:HS1, HS:1"
+```
+
+Known collections (keys of `OptimizationProblems.LIB_REFERENCES`):
+
+| Key | Description |
+|-----|-------------|
+| `AMPGO` | Gavana's Global Optimization benchmark suite |
+| `COPS` | COPS 3.0 collection (Dolan & Moré, 2004) |
+| `CUTEst` | CUTEst testing environment (Gould et al., 2015) |
+| `HS` | Hock & Schittkowski (1981), Vol. 187 |
+| `HS2` | Schittkowski (1987), Vol. 282 |
+| `Luksan` | Luksan, Matonoha & Vlček (2003) — Modified CUTE problems |
+| `LuksanSparse` | Luksan, Matonoha & Vlček (2010) — Sparse test problems |
+| `MGH` | Moré, Garbow & Hillstrom (1981) |
+| `NIST` | NIST/ITL Statistical Reference Datasets |
+
+To filter by collection, use Julia's `contains` function:
+
+```@example 1
+meta = OptimizationProblems.meta
+cops_problems = meta[contains.(meta.lib, "COPS"), [:name, :lib]]
+```
+
+```@example 1
+cutest_problems = meta[contains.(meta.lib, "CUTEst"), [:name, :lib]]
+```
+
+`OptimizationProblems.LIB_REFERENCES` provides the canonical BibTeX entry for
+each collection, and [`export_bibtex`](@ref) automatically appends them when
+`include_lib_refs = true` (the default).
+
+## Problem'source information
+
+The following code will create a .bib file regrouping all the BibTex citations.
+```julia
+using OptimizationProblems
+export_bibtex()
+```
+Fields documenting the origin of the problem may be incomplete and any help is welcome.

src/Meta/AMPGO02.jl

@@ -17,6 +17,26 @@ AMPGO02_meta = Dict(
   :defined_everywhere => missing,
   :origin => :unknown,
   :implementation => :both,
+  :url => "http://infinity77.net/global_optimization/test_functions_1d.html#d-test-functions",
+  :notes => raw"""
+A one dimensional optimization problem
+""",
+  :origin_notes => raw"""
+Problem 2 in
+http://infinity77.net/global_optimization/test_functions_1d.html#d-test-functions
+Andrea Gavana
+S. Goyette, Sherbrooke 2016/2017
+""",
+  :reference => raw"""
+@misc{GavanaGOTestSuite,
+  author       = {Gavana, Andrea},
+  title        = {Global Optimization Benchmarks},
+  year         = {2013},
+  howpublished = {\url{http://infinity77.net/global_optimization/}},
+  note         = {Collection of benchmark functions and algorithm comparisons for global optimization (including AMPGO)}
+}
+""",
+  :lib => "AMPGO:2",
 )
 get_AMPGO02_nvar(; n::Integer = default_nvar, kwargs...) = 1
 get_AMPGO02_ncon(; n::Integer = default_nvar, kwargs...) = 0

src/Meta/AMPGO03.jl

@@ -17,6 +17,26 @@ AMPGO03_meta = Dict(
   :defined_everywhere => missing,
   :origin => :unknown,
   :implementation => :both,
+  :url => "http://infinity77.net/global_optimization/test_functions_1d.html#d-test-functions",
+  :notes => raw"""
+A one dimensional optimization problem
+""",
+  :origin_notes => raw"""
+Problem 3 in
+http://infinity77.net/global_optimization/test_functions_1d.html#d-test-functions
+Andrea Gavana
+S. Goyette, Sherbrooke 2016/2017
+""",
+  :reference => raw"""
+@misc{GavanaGOTestSuite,
+  author       = {Gavana, Andrea},
+  title        = {Global Optimization Benchmarks},
+  year         = {2013},
+  howpublished = {\url{http://infinity77.net/global_optimization/}},
+  note         = {Collection of benchmark functions and algorithm comparisons for global optimization (including AMPGO)}
+}
+""",
+  :lib => "AMPGO:3",
 )
 get_AMPGO03_nvar(; n::Integer = default_nvar, kwargs...) = 1
 get_AMPGO03_ncon(; n::Integer = default_nvar, kwargs...) = 0

src/Meta/AMPGO04.jl

@@ -17,6 +17,26 @@ AMPGO04_meta = Dict(
   :defined_everywhere => missing,
   :origin => :unknown,
   :implementation => :both,
+  :url => "http://infinity77.net/global_optimization/test_functions_1d.html#d-test-functions",
+  :notes => raw"""
+A one dimensional optimization problem
+""",
+  :origin_notes => raw"""
+Problem 4 in
+http://infinity77.net/global_optimization/test_functions_1d.html#d-test-functions
+Andrea Gavana
+S. Goyette, Sherbrooke 2016/2017
+""",
+  :reference => raw"""
+@misc{GavanaGOTestSuite,
+  author       = {Gavana, Andrea},
+  title        = {Global Optimization Benchmarks},
+  year         = {2013},
+  howpublished = {\url{http://infinity77.net/global_optimization/}},
+  note         = {Collection of benchmark functions and algorithm comparisons for global optimization (including AMPGO)}
+}
+""",
+  :lib => "AMPGO:4",
 )
 get_AMPGO04_nvar(; n::Integer = default_nvar, kwargs...) = 1
 get_AMPGO04_ncon(; n::Integer = default_nvar, kwargs...) = 0

src/Meta/AMPGO05.jl

@@ -17,6 +17,26 @@ AMPGO05_meta = Dict(
   :defined_everywhere => missing,
   :origin => :unknown,
   :implementation => :both,
+  :url => "http://infinity77.net/global_optimization/test_functions_1d.html#d-test-functions",
+  :notes => raw"""
+A one dimensional optimization problem
+""",
+  :origin_notes => raw"""
+Problem 5 in
+http://infinity77.net/global_optimization/test_functions_1d.html#d-test-functions
+Andrea Gavana
+S. Goyette, Sherbrooke 2016/2017
+""",
+  :reference => raw"""
+@misc{GavanaGOTestSuite,
+  author       = {Gavana, Andrea},
+  title        = {Global Optimization Benchmarks},
+  year         = {2013},
+  howpublished = {\url{http://infinity77.net/global_optimization/}},
+  note         = {Collection of benchmark functions and algorithm comparisons for global optimization (including AMPGO)}
+}
+""",
+  :lib => "AMPGO:5",
 )
 get_AMPGO05_nvar(; n::Integer = default_nvar, kwargs...) = 1
 get_AMPGO05_ncon(; n::Integer = default_nvar, kwargs...) = 0

src/Meta/AMPGO06.jl

@@ -17,6 +17,26 @@ AMPGO06_meta = Dict(
   :defined_everywhere => missing,
   :origin => :unknown,
   :implementation => :both,
+  :url => "http://infinity77.net/global_optimization/test_functions_1d.html#d-test-functions",
+  :notes => raw"""
+A one dimensional optimization problem
+""",
+  :origin_notes => raw"""
+Problem 6 in
+http://infinity77.net/global_optimization/test_functions_1d.html#d-test-functions
+Andrea Gavana
+S. Goyette, Sherbrooke 2016/2017
+""",
+  :reference => raw"""
+@misc{GavanaGOTestSuite,
+  author       = {Gavana, Andrea},
+  title        = {Global Optimization Benchmarks},
+  year         = {2013},
+  howpublished = {\url{http://infinity77.net/global_optimization/}},
+  note         = {Collection of benchmark functions and algorithm comparisons for global optimization (including AMPGO)}
+}
+""",
+  :lib => "AMPGO:6",
 )
 get_AMPGO06_nvar(; n::Integer = default_nvar, kwargs...) = 1
 get_AMPGO06_ncon(; n::Integer = default_nvar, kwargs...) = 0

src/Meta/AMPGO07.jl

@@ -17,6 +17,26 @@ AMPGO07_meta = Dict(
   :defined_everywhere => missing,
   :origin => :unknown,
   :implementation => :both,
+  :url => "http://infinity77.net/global_optimization/test_functions_1d.html#d-test-functions",
+  :notes => raw"""
+A one dimensional optimization problem
+""",
+  :origin_notes => raw"""
+Problem 7 in
+http://infinity77.net/global_optimization/test_functions_1d.html#d-test-functions
+Andrea Gavana
+S. Goyette, Sherbrooke 2016/2017
+""",
+  :reference => raw"""
+@misc{GavanaGOTestSuite,
+  author       = {Gavana, Andrea},
+  title        = {Global Optimization Benchmarks},
+  year         = {2013},
+  howpublished = {\url{http://infinity77.net/global_optimization/}},
+  note         = {Collection of benchmark functions and algorithm comparisons for global optimization (including AMPGO)}
+}
+""",
+  :lib => "AMPGO:7",
 )
 get_AMPGO07_nvar(; n::Integer = default_nvar, kwargs...) = 1
 get_AMPGO07_ncon(; n::Integer = default_nvar, kwargs...) = 0