During the port to JupyterBook. we semi-deliberately dropped the various doctests.
This was because:
a) JupyterBook does not have Sphinx's test mode, so the tests can't be run in the same way within a JupyterBook project and
b) As we all know, doctests can be quite ugly because of the workarounds one has to use to test against output text, rather than variable values.
c) It was already a lot of work to do the page port. Adding the tests after porting, as necessary, seemed like a reasonable option.
However, we do like tests, so I have added a some test machinery, that you can see described in CONTRIBUTING.md - in the form of cells marked remove-cell with asserts in them. These don't appear in the built output or the JupyterLite notebooks, but do get run as part of the book build process.
If I wanted to add those tests back, and I enjoyed using AI, I think I'd use AI, with prompts something like:
"""
The current book is in the form JupyterBook Markdown. We ported to this form from Sphinx ReST. The last commit in this form is 9e2eefc . Commit 8854996 is the Sphinx / ReST to JupyterBook / Markdown port.
Notice that, as part of the port, we dropped the many doctest blocks. Here's one example of a doctest block, from advanced/advanced_numpy/index.rst
Block of memory
----------------
>>> x = np.array([1, 2, 3], dtype=np.int32)
>>> x.data
<... at ...>
>>> bytes(x.data)
b'\x01\x00\x00\x00\x02\x00\x00\x00\x03\x00\x00\x00'
Notice the >>> for code markers, and the ... in <... at ...>, to avoid testing output that is either too variable to be sensibly tested, or content not important to test.
In the Markdown, we simply removed the doctest blocks, as these are not supported.
However, we now want to restore tests for not-trivial operations in the notebooks.
In order to do that, we have test machinery that, instead of using doctests, introduces a specific test cell, that uses asserts. See CONTRIBUTING.md in the current code for an example.
Your job is to:
- Carefully analyze the relationship between the ReST and the ported Markdown to detect places where we no longer have doctests - i.e. where the doctests have been lost.
- Determine whether the doctests are useful as tests - i.e. whether the doctests are merely placeholders for trivial output (that is true for the example above), or whether they assert some result that may plausibly change over Python or Numpy or Scipy or other package versions.
- Where the doctests are useful as tests, write a test cell of the form given in
CONTRIBUTING.md corresponding to the useful test in the doctest block, and insert it into the Markdown notebook after the corresponding code.
Here is the example test cell from CONTRIBUTING.md:
```{code-cell}
log_a = sp.special.gammaln(500)
log_b = sp.special.gammaln(499)
log_res = log_a - log_b
res = np.exp(log_res)
res
```
```{code-cell}
:tags: [remove-cell, test]
assert np.allclose(res, 499)
```
I suspect some iteration of those prompts would give you useful tests for a substantial proportion of the original doctest blocks, and the combination of AI + iteration + edits would take about 5 hours.
Another approach to restoring the tests would be to start again from scratch with the port, but I suspect that would take many times longer.
During the port to JupyterBook. we semi-deliberately dropped the various doctests.
This was because:
a) JupyterBook does not have Sphinx's test mode, so the tests can't be run in the same way within a JupyterBook project and
b) As we all know, doctests can be quite ugly because of the workarounds one has to use to test against output text, rather than variable values.
c) It was already a lot of work to do the page port. Adding the tests after porting, as necessary, seemed like a reasonable option.
However, we do like tests, so I have added a some test machinery, that you can see described in
CONTRIBUTING.md- in the form of cells markedremove-cellwith asserts in them. These don't appear in the built output or the JupyterLite notebooks, but do get run as part of the book build process.If I wanted to add those tests back, and I enjoyed using AI, I think I'd use AI, with prompts something like:
"""
The current book is in the form JupyterBook Markdown. We ported to this form from Sphinx ReST. The last commit in this form is 9e2eefc . Commit 8854996 is the Sphinx / ReST to JupyterBook / Markdown port.
Notice that, as part of the port, we dropped the many doctest blocks. Here's one example of a doctest block, from
advanced/advanced_numpy/index.rstNotice the
>>>for code markers, and the...in<... at ...>, to avoid testing output that is either too variable to be sensibly tested, or content not important to test.In the Markdown, we simply removed the doctest blocks, as these are not supported.
However, we now want to restore tests for not-trivial operations in the notebooks.
In order to do that, we have test machinery that, instead of using doctests, introduces a specific test cell, that uses asserts. See
CONTRIBUTING.mdin the current code for an example.Your job is to:
CONTRIBUTING.mdcorresponding to the useful test in the doctest block, and insert it into the Markdown notebook after the corresponding code.Here is the example test cell from
CONTRIBUTING.md:I suspect some iteration of those prompts would give you useful tests for a substantial proportion of the original doctest blocks, and the combination of AI + iteration + edits would take about 5 hours.
Another approach to restoring the tests would be to start again from scratch with the port, but I suspect that would take many times longer.