Adds summary statistics to coverage report

[jorgecarleitao]

Subject: Adds a table with a summary of the coverage report:

+-----------------------+----------+--------------+
| Module                | Coverage | Undocumented |
+=======================+==========+==============+
| schemaflow.exceptions | 100.00%  | 0            |
+-----------------------+----------+--------------+
| schemaflow.pipe       | 83.33%   | 1            |
+-----------------------+----------+--------------+
| schemaflow.pipeline   | 100.00%  | 0            |
+-----------------------+----------+--------------+
| schemaflow.types      | 70.00%   | 3            |
+-----------------------+----------+--------------+
| TOTAL                 | 82.61%   | 4            |
+-----------------------+----------+--------------+

Feature or Bugfix

• Feature

Purpose

The current implementation of ext.coverage outputs which methods/classes/functions are documented. This PR adds a short summary of this report in terms of documented objects / total number of objects, both per module and total.

The purpose of this is to support a currently not mainstream but relevant use-case: a coverage report on the number of objects that are documented.

By having the statistics on the report or on the stdout, a regex expression can capture the coverage percentage (re.search(r'TOTAL.*?([0-9.]{4,6}\%)', d).group(1) in Python regex) and use it e.g. as another CI report, in a Badge, etc.

The current version of this PR does not do anything by default to not break backward compatibility. Two options were added to the configuration to allow a table to be outputted to the report and/or to stdout.

There are still steps missing:

1. simplify code
2. unit-test
3. documentation

however, I have the following questions before proceeding:

1. is this something that you would like to have?
2. should we make the coverage report to contain the statistics by default?
3. should we make the coverage to output the table to stdout by default?

My hypothesis is that the answer is affirmative for all 3 cases: the most common usage of coverage is to know how the coverage of docs is and a nice table as output seems a good output for the command. However, I am not sure about backward compatibility.

[codecov]

Codecov Report

Merging #5474 into master will decrease coverage by 0.11%.
The diff coverage is 25.92%.

@@            Coverage Diff             @@
##           master    #5474      +/-   ##
==========================================
- Coverage   83.07%   82.95%   -0.12%     
==========================================
  Files         289      291       +2     
  Lines       38820    38886      +66     
  Branches     5840     5854      +14     
==========================================
+ Hits        32250    32259       +9     
- Misses       5207     5255      +48     
- Partials     1363     1372       +9

Impacted Files	Coverage Δ
tests/test_ext_coverage.py	100% <100%> (ø)	⬆️
sphinx/ext/coverage.py	65.98% <24.52%> (-10.86%)	⬇️
sphinx/environment/__init__.py	76.71% <0%> (-1.57%)	⬇️
sphinx/environment/collectors/__init__.py	77.77% <0%> (-1.17%)	⬇️
sphinx/locale/__init__.py	76.76% <0%> (-0.33%)	⬇️
sphinx/environment/collectors/asset.py	91.46% <0%> (-0.11%)	⬇️
sphinx/cmd/quickstart.py	66.86% <0%> (-0.1%)	⬇️
sphinx/registry.py	74.46% <0%> (-0.1%)	⬇️
sphinx/util/__init__.py	67.97% <0%> (-0.08%)	⬇️
sphinx/application.py	77.74% <0%> (-0.07%)	⬇️
... and 29 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 408219c...453aed3. Read the comment docs.

[AA-Turner]

Probably need to bump the env version here so that old pickle files are ignored.

A

[AA-Turner]

Thank you @jorgecarleitao!

A