New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
FEA Add examples recommender system #1125
Changes from all commits
28f4461
d27c9fe
0e027a7
d74b825
7b9959c
81f742e
c9c4589
cbc55cd
fc5b8a4
8624f68
d042591
8a04e69
29fe98b
4a23db1
5b1b757
fa2389a
3c8e957
9b4f6f0
2d217e3
1d611dc
7d1f621
aeb5489
108f523
ed7c6fc
a52b5c1
9b5ef86
a2cc203
8dd83d0
8672914
94e5a40
bbe8a20
e7be84c
9b2ec7c
6231bca
9c56f62
c6ea293
12fd707
581c7f4
cd01604
904be25
72526be
ec7ed38
40bb513
8706c81
8d11a7a
b1083a7
083c541
30dc03d
00e78c8
0714ef0
1c8351d
c29c621
69966a2
c0dee6f
a8daadc
a06f026
92aeff5
3feac7b
8dea184
5289260
0dafc32
a214c05
829ee56
76c0a81
4c7dafb
f5d849f
2a21ce6
a3e98eb
d1ec903
6464c4f
67dccf0
8092e14
eae2a4a
3ba4d26
494d177
1189948
96c557a
1859608
fbea244
a56e6ca
38404d4
063df21
44d2885
878772a
dd762b1
767c6cb
a16867b
d5398f3
43b97ed
c69f075
8e11051
dc32406
e5e0045
2d3cb5d
488f64d
f591624
53e2740
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -30,6 +30,8 @@ | |
with open("requirements.txt") as fid: | ||
install_requires = [line.strip() for line in fid if line.strip()] | ||
|
||
extras_require = {"recommender": ["numpy"]} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @larsoner , if we add this, I wonder if we should add other optional extras like There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Probably at some point, yeah |
||
|
||
setup( | ||
name="sphinx-gallery", | ||
description=description, # noqa: E501, analysis:ignore | ||
|
@@ -55,6 +57,7 @@ | |
author="Óscar Nájera", | ||
author_email="najera.oscar@gmail.com", | ||
install_requires=install_requires, | ||
extras_require=extras_require, | ||
python_requires=">=3.8", | ||
license="3-clause BSD", | ||
classifiers=[ | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -18,6 +18,7 @@ | |
import os | ||
import pathlib | ||
from xml.sax.saxutils import quoteattr, escape | ||
from itertools import chain | ||
|
||
from sphinx.errors import ConfigError, ExtensionError | ||
import sphinx.util | ||
|
@@ -39,6 +40,7 @@ | |
from .interactive_example import post_configure_jupyterlite_sphinx | ||
from .interactive_example import create_jupyterlite_contents | ||
from .directives import MiniGallery, ImageSg, imagesg_addnode | ||
from .recommender import ExampleRecommender, _write_recommendations | ||
|
||
_KNOWN_CSS = ( | ||
"sg_gallery", | ||
|
@@ -85,6 +87,7 @@ def __call__(self, gallery_conf, script_vars): | |
"download_all_examples": True, | ||
"abort_on_example_error": False, | ||
"only_warn_on_example_error": False, | ||
"recommender": {"enable": False}, | ||
"failing_examples": {}, | ||
"passing_examples": [], | ||
"stale_examples": [], # ones that did not need to be run due to md5sum | ||
|
@@ -663,6 +666,42 @@ def generate_gallery_rst(app): | |
costs += subsection_costs | ||
write_computation_times(gallery_conf, target_dir, subsection_costs) | ||
|
||
# Build recommendation system | ||
if gallery_conf["recommender"]["enable"]: | ||
try: | ||
import numpy as np # noqa: F401 | ||
except ImportError: | ||
raise ConfigError("gallery_conf['recommender'] requires numpy") | ||
|
||
recommender_params = copy.deepcopy(gallery_conf["recommender"]) | ||
recommender_params.pop("enable") | ||
recommender_params.pop("rubric_header", None) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Good catch! Would we be able to get a test to check config processing? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is something similar as done in c69f075 what you had in mind? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes thanks |
||
recommender = ExampleRecommender(**recommender_params) | ||
|
||
gallery_py_files = [] | ||
# root and subsection directories containing python examples | ||
gallery_directories = [gallery_dir_abs_path] + subsecs | ||
for current_dir in gallery_directories: | ||
src_dir = os.path.join(gallery_dir_abs_path, current_dir) | ||
# sort python files to have a deterministic input across call | ||
py_files = sorted( | ||
[ | ||
fname | ||
for fname in Path(src_dir).iterdir() | ||
if fname.suffix == ".py" | ||
], | ||
key=gallery_conf["within_subsection_order"](src_dir), | ||
) | ||
gallery_py_files.append( | ||
[os.path.join(src_dir, fname) for fname in py_files] | ||
) | ||
# flatten the list of list | ||
gallery_py_files = list(chain.from_iterable(gallery_py_files)) | ||
|
||
recommender.fit(gallery_py_files) | ||
for fname in gallery_py_files: | ||
_write_recommendations(recommender, fname, gallery_conf) | ||
|
||
# generate toctree with subsections | ||
if gallery_conf["nested_sections"] is True: | ||
subsections_toctree = _format_toctree( | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I know this is already implied above but could we explicitly say that only the examples within a single gallery (and it's sub galleries) are used for computing closest examples.
Also this is probably obvious but can we add that only recommendations for
.py
files will be generated.