Minimum dependency test job #2816
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## master #2816 +/- ##
==========================================
+ Coverage 74.11% 74.56% +0.45%
==========================================
Files 115 115
Lines 12685 12713 +28
==========================================
+ Hits 9402 9480 +78
+ Misses 3283 3233 -50
|
Test failures on 141eb6a252 failed, 650 passed, 59 skipped, 5 xfailed, 1038 warnings, 128 errors in 451.20s (0:07:31) Noting some of the big causes: Expected warnings not thrownA few times Older versions of pandas do not support
|
|
We were getting a lot of errors from dask tests because they were relying on test helpers from anndata 0.10. It's a small number of functions, but it depends on the types in the compat module so is difficult to copy out. To work around this I've temporarily bumped the minimum required version of anndata up to 0.10, but we definitely shouldn't actually do that. |
Test failures as of d36b977Significantly improved! 56 failed, 1236 passed, 96 skipped, 19 xfailed, 9 xpassed, 763 warnings in 595.02s (0:09:55) Remaining errors include:
FAILED scanpy/get/get.py::scanpy.get.get.obs_df
FAILED scanpy/plotting/_baseplot_class.py::scanpy.plotting._baseplot_class.BasePlot.add_totals
FAILED scanpy/tests/test_metrics.py::test_consistency[morans_i-allclose] - AssertionError:
FAILED scanpy/tests/test_paga.py::test_paga_plots[-paga] - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_paga.py::test_paga_plots[continuous-func1] - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_paga.py::test_paga_plots[continuous_obs-func2] - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_paga.py::test_paga_plots[continuous_multiple-func3] - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_paga.py::test_paga_pie - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_neighbors.py::test_distances_euclidean[umap] - ModuleNotFoundError: No module named 'pynndescent'
FAILED scanpy/tests/test_neighbors.py::test_distances_euclidean[gauss] - ModuleNotFoundError: No module named 'pynndescent'
FAILED scanpy/tests/test_neighbors.py::test_distances_all[pynndescent] - ModuleNotFoundError: No module named 'pynndescent'
FAILED scanpy/tests/test_plotting.py::test_tracksplot - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_plotting.py::test_heatmap - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_plotting.py::test_dotplot_matrixplot_stacked_violin[dotplot-fn0] - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_plotting.py::test_multiple_plots - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_plotting.py::test_dotplot_matrixplot_stacked_violin[dotplot3-fn2] - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_plotting.py::test_dotplot_matrixplot_stacked_violin[dotplot_std_scale_group-fn3] - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_plotting.py::test_dotplot_matrixplot_stacked_violin[dotplot_dict-fn4] - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_plotting.py::test_dotplot_matrixplot_stacked_violin[matrixplot-fn5] - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_plotting.py::test_dotplot_matrixplot_stacked_violin[matrixplot_std_scale_var_dict-fn6] - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_plotting.py::test_dotplot_matrixplot_stacked_violin[matrixplot_std_scale_group-fn7] - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_plotting.py::test_correlation - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_plotting.py::test_rank_genes_groups[ranked_genes_dotplot-fn6] - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_plotting.py::test_dotplot_matrixplot_stacked_violin[stacked_violin-fn9] - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_plotting.py::test_rank_genes_groups[ranked_genes_dotplot_gene_names-fn7] - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_plotting.py::test_dotplot_matrixplot_stacked_violin[stacked_violin_std_scale_var_dict-fn10] - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_plotting.py::test_rank_genes_groups[ranked_genes_dotplot_logfoldchange-fn8] - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_plotting.py::test_rank_genes_groups[ranked_genes_dotplot_logfoldchange_vcenter-fn9] - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_plotting.py::test_rank_genes_groups[ranked_genes_heatmap-fn2] - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_plotting.py::test_dotplot_obj - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_plotting.py::test_rank_genes_groups[ranked_genes_matrixplot-fn10] - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_plotting.py::test_rank_genes_groups[ranked_genes_heatmap_swap_axes-fn3] - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_plotting.py::test_rank_genes_groups[ranked_genes_heatmap_swap_axes_vcenter-fn4] - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_plotting.py::test_rank_genes_groups[ranked_genes_matrixplot_gene_names_symbol-fn11] - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_plotting.py::test_stacked_violin_obj - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_plotting.py::test_rank_genes_groups[ranked_genes_matrixplot_n_genes_negative-fn12] - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_plotting_utils.py::test_validate_palette_no_mod[asarray-viridis] - ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
FAILED scanpy/tests/test_plotting_utils.py::test_validate_palette_no_mod[asarray-rgba] - ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
FAILED scanpy/tests/test_plotting.py::test_rank_genes_groups[ranked_genes_matrixplot_swap_axes-fn13] - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_plotting.py::test_rank_genes_groups[ranked_genes_stacked_violin-fn5] - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_plotting.py::test_rank_genes_groups[ranked_genes_matrixplot_swap_axes_vcenter-fn14] - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_plotting.py::test_rank_genes_groups[ranked_genes_tracksplot-fn15] - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_preprocessing.py::test_regress_out_ordinal - ImportError: cannot import name '_centered' from 'scipy.signal.signaltools' (/mnt/workspace/mambaforge/envs/scanpy-min-deps-test/lib/python3.9/site-p...
FAILED scanpy/tests/test_preprocessing.py::test_regress_out_layer - ImportError: cannot import name '_centered' from 'scipy.signal.signaltools' (/mnt/workspace/mambaforge/envs/scanpy-min-deps-test/lib/python3.9/site-p...
FAILED scanpy/tests/test_preprocessing.py::test_regress_out_view - ImportError: cannot import name '_centered' from 'scipy.signal.signaltools' (/mnt/workspace/mambaforge/envs/scanpy-min-deps-test/lib/python3.9/site-p...
FAILED scanpy/tests/test_preprocessing.py::test_regress_out_categorical - ImportError: cannot import name '_centered' from 'scipy.signal.signaltools' (/mnt/workspace/mambaforge/envs/scanpy-min-deps-test/lib/python3.9/site-p...
FAILED scanpy/tests/test_preprocessing.py::test_regress_out_constants - ImportError: cannot import name '_centered' from 'scipy.signal.signaltools' (/mnt/workspace/mambaforge/envs/scanpy-min-deps-test/lib/python3.9/site-p...
FAILED scanpy/tests/test_preprocessing.py::test_regress_out_constants_equivalent - ImportError: cannot import name '_centered' from 'scipy.signal.signaltools' (/mnt/workspace/mambaforge/envs/scanpy-min-deps-test/lib/python3.9/site-p...
FAILED scanpy/tests/test_plotting.py::test_genes_symbols[heatmap-heatmap] - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_plotting.py::test_genes_symbols[dotplot-dotplot] - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_plotting.py::test_genes_symbols[matrixplot-matrixplot] - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_plotting.py::test_genes_symbols[stacked_violin-stacked_violin] - AssertionError: Error: Image files did not match.
FAILED scanpy/tests/test_plotting.py::test_genes_symbols[tracksplot-tracksplot] - AssertionError: Error: Image files did not match.
FAILED scanpy/tools/_dendrogram.py::scanpy.tools._dendrogram.dendrogram
FAILED scanpy/tests/notebooks/test_pbmc3k.py::test_pbmc3k - ImportError: cannot import name '_centered' from 'scipy.signal.signaltools' (/mnt/workspace/mambaforge/envs/scanpy-min-deps-test/lib/python3.9/site-p...
FAILED scanpy/tests/test_plotting.py::test_scatterplots[umap_with_edges-fn17] - AssertionError: Error: Image files did not match.
============================== 56 failed, 1236 passed, 96 skipped, 19 xfailed, 9 xpassed, 763 warnings in 595.02s (0:09:55) ============================== |
|
@ivirshup Re: |
|
The tolerances need to be tight enough that the tests do anything though … I’ve seen and fixed quite some tests where the tolerances meant that completely broken output was accepted. |
flying-sheep
requested changes
Jan 18, 2024
This is exactly what I'm seeing in my PR. |
|
So, bumping pandas to above 2.0 fixes most of the plotting tests. Almost all the differences were in ordering, and frequently where the ordering would change, but the dendrogram being displayed would not. It's not immediately obvious to me which piece of code is the problem, so I am going to temporarily bump the required pandas version and come back to it after resolving the other remaining issues. |
|
@ivirshup If I remember correctly, you said that there was a specific anndata operation within the scanpy If you could point me to the test/line of code causing the order to change, I'd have a better sense. |
|
The array ordering thing ends up getting more complicated... But this is something we should decide on a policy for in |
flying-sheep
requested changes
Feb 13, 2024
There was a problem hiding this comment.
I’m happy with it, except for
-
the obscure semantics of version specifications: Minimum dependency test job #2816 (comment)
I get that it’s a predictable hack (always add
==.*to the end), but it’s not how people think about versions. And how a given project’s maintainers think about versions is essential to know if we want to tinker with that version number in an automated way. -
.loc[self.categories_order if self.categories_order is not None else self.categories]Minimum dependency test job #2816 (comment)Seems like if we want to support pandas <2, we need to use that pattern or abstract it away somehow. Please do that.
|
My priority are intuitive semantics so people can add or bump dependencies without 100% understanding the algorithm of the minimum dependency script. So I can think of options:
In both versions, there’s no hidden semantics in What does your experience while implementing this so far say to these? Any other ideas? |
| "umap-learn>=0.3.10", | ||
| "packaging", | ||
| "pynndescent>=0.5", | ||
| "packaging>=21.3", |
There was a problem hiding this comment.
This is calver, but they have never issued more than one release per month so it doesn’t matter.
flying-sheep
approved these changes
Feb 15, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
TODO:
MinVerSome thoughts