Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

REGR: Arrow backed objects not propagating exceptions #54952

Merged
merged 2 commits into from Sep 20, 2023
Merged

REGR: Arrow backed objects not propagating exceptions #54952

merged 2 commits into from Sep 20, 2023
+18 −7

Conversation

phofl
Copy link
Member

@phofl phofl commented Sep 2, 2023

@phofl phofl added Error Reporting Incorrect or improved errors from pandas Arrow pyarrow functionality labels Sep 2, 2023
@phofl phofl added this to the 2.1.1 milestone Sep 2, 2023
mroeschke
mroeschke reviewed Sep 5, 2023
View reviewed changes
pandas/core/arrays/arrow/array.py
@@ -626,20 +626,22 @@ def __setstate__(self, state) -> None:

def _cmp_method(self, other, op):
pc_func = ARROW_CMP_FUNCS[op.__name__]
try:
if isinstance(other, (ArrowExtensionArray, np.ndarray, list, BaseMaskedArray)):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to split the cases when ArrowNotImplementedError vs ArrowInvalid is raised? I'm a little nervous of restricting the types of other?

Additionally maybe self._box_pa should be called first before going down the array like vs scalar path

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this basically restores the logic we had before the last refactor that introduced the regression. I think calling box_pa on everything raised an exception, but can retry

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay if this was the logic before I'm okay having this as a patch for 2.1.1, thought I think this should be more generic in the future.

@lithomas1
Copy link
Member

Hm, this is kinda annoying so I think it should go into the release.

I'm not familiar enough to review this part of the code, though.

@mroeschke can you take another quick look

@mroeschke mroeschke merged commit 49c89d2 into pandas-dev:main Sep 20, 2023
39 checks passed
@mroeschke
Copy link
Member

Thanks @phofl

@meeseeksmachine meeseeksmachine mentioned this pull request Sep 20, 2023
Merged
meeseeksmachine pushed a commit to meeseeksmachine/pandas that referenced this pull request Sep 20, 2023
@phofl @meeseeksmachine
Backport PR pandas-dev#54952: REGR: Arrow backed objects not propagat…
34fb435 
…ing exceptions
hedeershowk pushed a commit to hedeershowk/pandas that referenced this pull request Sep 20, 2023
@phofl @mroeschke @hedeershowk
REGR: Arrow backed objects not propagating exceptions (pandas-dev#54952)
8933b71 
Co-authored-by: Matthew Roeschke <10647082+mroeschke@users.noreply.github.com>
phofl added a commit that referenced this pull request Sep 20, 2023
@meeseeksmachine @phofl
Backport PR #54952 on branch 2.1.x (REGR: Arrow backed objects not pr…
7acebe4 
…opagating exceptions) (#55209)

Backport PR #54952: REGR: Arrow backed objects not propagating exceptions

Co-authored-by: Patrick Hoefler <61934744+phofl@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mroeschke mroeschke mroeschke approved these changes

Assignees
No one assigned
Labels
Arrow pyarrow functionality Error Reporting Incorrect or improved errors from pandas
Projects
None yet
Milestone
2.1.1
Development

Successfully merging this pull request may close these issues.

BUG: Overflow in arrow comparison operator hides actual error behind NotImplementedError
3 participants
@phofl @lithomas1 @mroeschke