Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BUG: mode not sorting values for arrow backed strings #55621

Merged
merged 9 commits into from Oct 25, 2023
Merged

BUG: mode not sorting values for arrow backed strings #55621

merged 9 commits into from Oct 25, 2023

Conversation

phofl
Copy link
Member

@phofl phofl commented Oct 21, 2023

  • closes #xxxx (Replace xxxx with the GitHub issue number)
  • Tests added and passed if fixing a bug or adding a new feature
  • All code checks passed.
  • Added type annotations to new arguments/methods/functions.
  • Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

@phofl phofl added Strings String extension data type and string data Arrow pyarrow functionality labels Oct 21, 2023
@phofl phofl added this to the 2.1.2 milestone Oct 21, 2023
mroeschke
mroeschke reviewed Oct 22, 2023
View reviewed changes
pandas/tests/groupby/test_groupby.py Outdated
object,
pytest.param(
"string[pyarrow_numpy]",
marks=pytest.mark.skipif(pa_version_under7p0, reason="arrow not installed"),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you use an importorskip in the test when dtype is pyarrow_numpy? (avoids having to change pa_version_under7p0 every time we bump pyarrow)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thoughts about creating a pyarrow_installed variable?

I like this pattern a bit better than the if inside the test

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thoughts about creating a pyarrow_installed variable?

Sure this would be good too

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

lithomas1
lithomas1 reviewed Oct 22, 2023
View reviewed changes
pandas/tests/groupby/test_groupby.py Outdated
object,
pytest.param(
"string[pyarrow_numpy]",
marks=pytest.mark.skipif(pa_installed, reason="arrow not installed"),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This condition is backwards?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh thx, oversight

mroeschke
mroeschke reviewed Oct 23, 2023
View reviewed changes
pandas/tests/groupby/test_groupby.py Outdated
True,
pytest.param(
True,
marks=pytest.mark.skipif(not pa_installed, reason="arrow not installed"),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just realized we have a td.skip_if_no mark as well

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this work inside of pytest.param as well?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should, yeah. It returns a skipif marker

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, thx

lithomas1
lithomas1 reviewed Oct 24, 2023
View reviewed changes
pandas/compat/pyarrow.py Outdated
Comment on lines 19 to 22
pa_version_under7p0 = True
pa_version_under8p0 = True
pa_version_under9p0 = True
pa_version_under10p0 = True
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need these?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got in while merging main, sorry

lithomas1
lithomas1 approved these changes Oct 25, 2023
View reviewed changes
Copy link
Member

@lithomas1 lithomas1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this LGTM.

I'll merge this tomorrow morning assuming no other comments.

@mroeschke mroeschke merged commit bb2d2e0 into pandas-dev:main Oct 25, 2023
39 checks passed
@mroeschke
Copy link
Member

Thanks @phofl

@lumberbot-app
Copy link

lumberbot-app bot commented Oct 25, 2023

Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

  1. Checkout backport branch and update it.
git checkout 2.1.x
git pull
  1. Cherry pick the first parent branch of the this PR on top of the older branch:
git cherry-pick -x -m1 bb2d2e00524fb31d08a43fc54706035f810d2489
  1. You will likely have some merge/cherry-pick conflict here, fix them and commit:
git commit -am 'Backport PR #55621: BUG: mode not sorting values for arrow backed strings'
  1. Push to a named branch:
git push YOURFORK 2.1.x:auto-backport-of-pr-55621-on-2.1.x
  1. Create a PR against branch 2.1.x, I would have named this PR:

"Backport PR #55621 on branch 2.1.x (BUG: mode not sorting values for arrow backed strings)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the Still Needs Manual Backport label once the PR gets merged.

If these instructions are inaccurate, feel free to suggest an improvement.

@phofl phofl deleted the string_array_mode branch October 25, 2023 07:34
phofl added a commit to phofl/pandas that referenced this pull request Oct 25, 2023
@phofl
BUG: mode not sorting values for arrow backed strings (pandas-dev#55621)
ba40bda 
* BUG: mode not sorting values for arrow backed strings

* Fix tests

* Change to pa_installed variable

* Update pyarrow.py

* Fix

* Fix

(cherry picked from commit bb2d2e0)
@phofl phofl mentioned this pull request Oct 25, 2023
Merged
mroeschke pushed a commit that referenced this pull request Oct 25, 2023
@phofl
Backport PR #55621 on branch 2.1.x (BUG: mode not sorting values for …
b7aaaf9 
…arrow backed strings) (#55680)

BUG: mode not sorting values for arrow backed strings (#55621)

* BUG: mode not sorting values for arrow backed strings

* Fix tests

* Change to pa_installed variable

* Update pyarrow.py

* Fix

* Fix

(cherry picked from commit bb2d2e0)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mroeschke mroeschke mroeschke approved these changes

@lithomas1 lithomas1 lithomas1 approved these changes

Assignees
No one assigned
Labels
Arrow pyarrow functionality Strings String extension data type and string data
Projects
None yet
Milestone
2.1.2
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@phofl @mroeschke @lithomas1