doctests are not tested as part of CI

[ugognw]

The doctests are not run as part of the CI process despite a dedicated tox environment (doctests). Currently, all but 3 doctests fail on main (bdfb17a):

Doctest summary
===============
  107 tests
  104 failures in tests
    0 failures in setup code
    0 failures in cleanup code
build finished with problems, 18 warnings.

Both running the doctests as part of the CI (e.g., as a job step under the docs job in testing.yml):

...
   docs:
...
      - name: Test doctests
        run: tox -e doctests
...

and resolving the errors will help identify bad examples and keep examples up to date as the library changes.

PS: One can run the doctests in any given file using:

python -m doctest -v <FILE>

doctest directives can be specified using the doctest option-o (e.g., -o ELLIPSIS).

[rkingsbury]

Thanks for flagging this. To be honest, the doctests are left over from a time before I had a proper unit test suite; I don't think they are necessary from the standpoint of code quality. However, they might be helpful to users as documentation (basically as examples), in which case we should update them to pass. If we go that route, we should also add doctests to the CI which can be run whenever the docs are built.

How do you think is best to proceed?

[ugognw]

Fair, this is what I would suggest:

1. Format all code blocks in documentation such that they are recognized by the doctest sphinx extension (Some doctest code blocks in markdown files are not formatted correctly)
2. Update conf.py with doctest configuration (e.g., doctest setup, default directives, etc.)
3. Fix doctests to pass
4. Add job step for tox -e doctests to testing GH action

I've started work at fix-doctests.