Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve stateful documentation #3881

Merged
merged 10 commits into from
Feb 15, 2024
Merged

Conversation

td-anne
Copy link
Contributor

@td-anne td-anne commented Feb 13, 2024

In #3511, a number of issues with the stateful documentation are raised. Some are simple confusion, some are outright errors in the documentation. This PR makes no code changes but attempts to clarify and correct the documentation to better explain how hypothesis works.

Rendered docs: https://hypothesis--3881.org.readthedocs.build/en/3881/stateful.html#rule-based-state-machines

@td-anne
Copy link
Contributor Author

td-anne commented Feb 13, 2024

Is there a CI tag I can add so that it doesn't waste time running CI apart from the doc build?

I'm not sure how to run the doc build locally either, as it doesn't appear to follow the usual Sphinx layout I'm familiar with.

@td-anne
Copy link
Contributor Author

td-anne commented Feb 13, 2024

The pyodide run failed in a way I can only describe as "flaky":

=========================== short test summary info ============================
FAILED hypothesis-python/tests/cover/test_filter_rewriting.py::test_rewrite_filter_chains_with_some_unhandled[s0] - hypothesis.errors.FailedHealthCheck: Data generation is extremely slow: Only produced 6 valid examples in 1.02 seconds (21 invalid ones and 0 exceeded maximum size). Try decreasing size of the data you're generating (with e.g. max_size or max_leaves parameters).
               count | fraction |    slowest draws (seconds)
    Draw 1   |   27  |     47%  |   0.020,  0.020,  0.020,  0.020,  0.020
     data    |   27  |     26%  |   0.010,  0.010,  0.010,  0.010,  0.010
  predicates |   27  |     26%  |   0.010,  0.010,  0.010,  0.010,  0.010
See https://hypothesis.readthedocs.io/en/latest/healthchecks.html for more information about this. If you want to disable just this health check, add HealthCheck.too_slow to the suppress_health_check settings for this test.
====== 1 failed, 3088 passed, 10 skipped, 1 xfailed in 957.97s (0:15:57) =======
Error: Process completed with exit code 1.

Is this worth doing something about? (Like, filing a bug report, I mean.)

@Zac-HD
Copy link
Member

Zac-HD commented Feb 13, 2024

Including [skip ci] in the commit message will do what you'd guess. You can run sphinx locally with make documentation, or by manually running sphinx-build [...] docs from the hypothesis-python/ directory.

The Pyodide build is a bit slower than the others, so I'm not particularly surprised to see such issues... I'd just ignore for now, it seems rare and the build is not required so it shouldn't block us. I'll track in our "make CI faster" issue #3524 if it keeps happening.

Copy link
Member

@Zac-HD Zac-HD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Anne! I think this will be a great improvement for people getting started with stateful testing ❤️

@Zac-HD
Copy link
Member

Zac-HD commented Feb 14, 2024

@tybug https://github.com/HypothesisWorks/hypothesis/actions/runs/7906413070/job/21581186710?pr=3881#step:7:202

@tybug
Copy link
Member

tybug commented Feb 14, 2024

thanks, looking into it!

@Zac-HD Zac-HD merged commit fad79fc into HypothesisWorks:master Feb 15, 2024
48 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants