Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize _chunked_even_finite() #699

Merged
merged 4 commits into from
Mar 19, 2023

Conversation

elliotwutingfeng
Copy link
Contributor

@elliotwutingfeng elliotwutingfeng commented Mar 17, 2023

Issue reference

#698

Changes

  • Optimize _chunked_even_finite by yielding the chunk as a slice of the iterable, without using a for-loop to build up the chunk.

Implementation Details

yield list(islice(...)) was used to ensure that chunks for non-list iterables such as strings are yielded as lists, and to handle non-sliceable iterables like sets and dicts.

I used the following in IPython as a rudimentary benchmark.

OS: Linux x64 CPU: AMD Ryzen 7 5800X

%timeit _ = [list(chunked_even(range(x), y)) for x in range(500) for y in range(1,500)]
# Before: 2.9 s ± 24.6 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
# After: 1.39 s ± 12.3 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

%timeit _ = [list(chunked_even([z for z in range (x)], y)) for x in range(500) for y in range(1,500)]
# Before: 3.61 s ± 11.1 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
# After: 1.91 s ± 19.4 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

%timeit _ = [list(chunked_even({z:z for z in range(x)}, y)) for x in range(500) for y in range(1,500)]
# Before: 4.33 s ± 34.1 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
# After: 2.84 s ± 24.4 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

%timeit _ = [list(chunked_even(set(z for z in range(x)), y)) for x in range(500) for y in range(1,500)]
# Before: 4.31 s ± 26.1 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
# After: 2.72 s ± 7.32 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

@elliotwutingfeng elliotwutingfeng marked this pull request as draft March 17, 2023 10:15
@elliotwutingfeng elliotwutingfeng marked this pull request as ready for review March 17, 2023 10:45
@elliotwutingfeng elliotwutingfeng marked this pull request as draft March 17, 2023 13:54
@elliotwutingfeng elliotwutingfeng marked this pull request as ready for review March 17, 2023 14:09
@bbayles
Copy link
Collaborator

bbayles commented Mar 19, 2023

This looks good - thanks.

@bbayles bbayles linked an issue Mar 19, 2023 that may be closed by this pull request
@bbayles bbayles merged commit 3e705a4 into more-itertools:master Mar 19, 2023
@elliotwutingfeng elliotwutingfeng deleted the chunked_even branch March 20, 2023 02:17
lengau pushed a commit to canonical/charmcraft that referenced this pull request Jul 25, 2023
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [more-itertools](https://togithub.com/more-itertools/more-itertools) |
`==9.1.0` -> `==10.0.0` |
[![age](https://developer.mend.io/api/mc/badges/age/pypi/more-itertools/10.0.0?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/more-itertools/10.0.0?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/more-itertools/9.1.0/10.0.0?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/more-itertools/9.1.0/10.0.0?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|

---

### Release Notes

<details>
<summary>more-itertools/more-itertools (more-itertools)</summary>

###
[`v10.0.0`](https://togithub.com/more-itertools/more-itertools/releases/tag/v10.0.0):
Version 10.0.0

[Compare
Source](https://togithub.com/more-itertools/more-itertools/compare/v9.1.0...v10.0.0)

##### What's Changed

- Update recipes.iter_index to match CPython PR 102360 by
[@&#8203;bbayles](https://togithub.com/bbayles) in
[more-itertools/more-itertools#690
- fixup - add missing commas to the readme function table by
[@&#8203;lonnen](https://togithub.com/lonnen) in
[more-itertools/more-itertools#692
- fixup remove 3.6 from tox by
[@&#8203;lonnen](https://togithub.com/lonnen) in
[more-itertools/more-itertools#693
- seekable: Add relative_seek by
[@&#8203;karlb](https://togithub.com/karlb) in
[more-itertools/more-itertools#694
- Optimize \_chunked_even_finite() by
[@&#8203;elliotwutingfeng](https://togithub.com/elliotwutingfeng) in
[more-itertools/more-itertools#699
- Indexing of combinations with replacement by
[@&#8203;Schoyen](https://togithub.com/Schoyen) in
[more-itertools/more-itertools#689
- Add notes for transposing empty inputs by
[@&#8203;XuehaiPan](https://togithub.com/XuehaiPan) in
[more-itertools/more-itertools#700
- Add the polynomial_eval recipe by
[@&#8203;bbayles](https://togithub.com/bbayles) in
[more-itertools/more-itertools#703
- Add nth_combination_with_replacement by
[@&#8203;Schoyen](https://togithub.com/Schoyen) in
[more-itertools/more-itertools#704
- Add sum_of_squares, sync with itertools by
[@&#8203;bbayles](https://togithub.com/bbayles) in
[more-itertools/more-itertools#706
- Issue
[#&#8203;707](https://togithub.com/more-itertools/more-itertools/issues/707):
fix `iterate()` to enable `func` to raise StopIteration + 3 unittests by
[@&#8203;jrebiffe](https://togithub.com/jrebiffe) in
[more-itertools/more-itertools#708
- Update polynomial_from roots and convolve by
[@&#8203;bbayles](https://togithub.com/bbayles) in
[more-itertools/more-itertools#709
- Issue
[#&#8203;677](https://togithub.com/more-itertools/more-itertools/issues/677):
Improve `partition` by [@&#8203;pochmann](https://togithub.com/pochmann)
in
[more-itertools/more-itertools#710
- Issue
[#&#8203;713](https://togithub.com/more-itertools/more-itertools/issues/713):
Fix `partial_product` (also simplify and clean up) by
[@&#8203;pochmann](https://togithub.com/pochmann) in
[more-itertools/more-itertools#714
- Issue
[#&#8203;711](https://togithub.com/more-itertools/more-itertools/issues/711):
Optimize `pairwise` by [@&#8203;pochmann](https://togithub.com/pochmann)
in
[more-itertools/more-itertools#712
- Issue
[#&#8203;715](https://togithub.com/more-itertools/more-itertools/issues/715):
Simplify/optimize `partial_product` by
[@&#8203;pochmann](https://togithub.com/pochmann) in
[more-itertools/more-itertools#716
- Issue
[#&#8203;717](https://togithub.com/more-itertools/more-itertools/issues/717):
Improve `duplicates_justseen` by
[@&#8203;pochmann](https://togithub.com/pochmann) in
[more-itertools/more-itertools#718
- Fix unique_in_window to match described behavior by
[@&#8203;elliotwutingfeng](https://togithub.com/elliotwutingfeng) in
[more-itertools/more-itertools#720
- Add polynomial_derivative recipe by
[@&#8203;bbayles](https://togithub.com/bbayles) in
[more-itertools/more-itertools#723
- Update recipes with CPython PRs: 105403 and 106371 by
[@&#8203;bbayles](https://togithub.com/bbayles) in
[more-itertools/more-itertools#731
- Changes for version 10.0.0 by
[@&#8203;bbayles](https://togithub.com/bbayles) in
[more-itertools/more-itertools#734
- Delay computation of numeric_range len until needed by
[@&#8203;eltoder](https://togithub.com/eltoder) in
[more-itertools/more-itertools#674

##### New Contributors

- [@&#8203;karlb](https://togithub.com/karlb) made their first
contribution in
[more-itertools/more-itertools#694
- [@&#8203;elliotwutingfeng](https://togithub.com/elliotwutingfeng) made
their first contribution in
[more-itertools/more-itertools#699
- [@&#8203;Schoyen](https://togithub.com/Schoyen) made their first
contribution in
[more-itertools/more-itertools#689
- [@&#8203;XuehaiPan](https://togithub.com/XuehaiPan) made their first
contribution in
[more-itertools/more-itertools#700
- [@&#8203;jrebiffe](https://togithub.com/jrebiffe) made their first
contribution in
[more-itertools/more-itertools#708
- [@&#8203;pochmann](https://togithub.com/pochmann) made their first
contribution in
[more-itertools/more-itertools#710
- [@&#8203;eltoder](https://togithub.com/eltoder) made their first
contribution in
[more-itertools/more-itertools#674

**Full Changelog**:
more-itertools/more-itertools@v9.1.1...v10.0.0

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "every weekend" in timezone Etc/UTC,
Automerge - "before 07:00" in timezone Etc/UTC.

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://developer.mend.io/github/canonical/charmcraft).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNi4xMS4wIiwidXBkYXRlZEluVmVyIjoiMzYuMTEuMCIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Optimization: _chunked_even_finite()
2 participants