Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(python): enhance expression-level filter syntax with support for multiple predicates and kwargs #12689

Merged
merged 1 commit into from Dec 28, 2023

Conversation

alexander-beedie
Copy link
Collaborator

@alexander-beedie alexander-beedie commented Nov 26, 2023

Closes #12577 (following up #12603).

Adds the same support for multiple predicates and kwargs as recently added to frame-level filter and when/then.

Example

import polars as pl

df = pl.DataFrame({
    "key": ["a", "a", "a", "a", "b", "b", "b", "b", "b"],
    "n": [1, 2, 2, 3, 1, 3, 3, 2, 3],
})

df.group_by("key").agg(
    n_0 = pl.col("n").filter( n=1 ).sum(),
    n_1 = pl.col("n").filter( n=2 ).sum(),
    n_2 = pl.col("n").filter( n=3 ).sum(),
).sort(by="key")

# shape: (2, 4)
# ┌─────┬─────┬─────┬─────┐
# │ key ┆ n_0 ┆ n_1 ┆ n_2 │
# │ --- ┆ --- ┆ --- ┆ --- │
# │ str ┆ i64 ┆ i64 ┆ i64 │
# ╞═════╪═════╪═════╪═════╡
# │ a   ┆ 1   ┆ 4   ┆ 3   │
# │ b   ┆ 1   ┆ 2   ┆ 9   │
# └─────┴─────┴─────┴─────┘

@github-actions github-actions bot added enhancement New feature or an improvement of an existing feature python Related to Python Polars labels Nov 26, 2023
py-polars/polars/expr/expr.py Outdated Show resolved Hide resolved
py-polars/polars/expr/expr.py Outdated Show resolved Hide resolved
@ritchie46 ritchie46 changed the title feat(python): enhance expression-level filter syntax with support for multiple predicates and kwargs feat(python!): enhance expression-level filter syntax with support for multiple predicates and kwargs Nov 27, 2023
@github-actions github-actions bot added the breaking python Change that breaks backwards compatibility for the Python package label Nov 27, 2023
@ritchie46
Copy link
Member

Nice! I wait merging this when we merge the breaking changes ( soon ).

@alexander-beedie
Copy link
Collaborator Author

alexander-beedie commented Nov 30, 2023

Nice! I wait merging this when we merge the breaking changes ( soon ).

No problem to wait, but this change isn't actually breaking ;)

@thomasfrederikhoeck
Copy link
Contributor

This new style is very convinient but does it translate to the same filter? See #12932

@alexander-beedie alexander-beedie removed the breaking python Change that breaks backwards compatibility for the Python package label Dec 18, 2023
@alexander-beedie alexander-beedie changed the title feat(python!): enhance expression-level filter syntax with support for multiple predicates and kwargs feat(python): enhance expression-level filter syntax with support for multiple predicates and kwargs Dec 18, 2023
@alexander-beedie
Copy link
Collaborator Author

This new style is very convinient but does it translate to the same filter? See #12932

Logically it did, but the optimiser didn't realise it could push the components down; should be fixed by #12976 👌

@alexander-beedie
Copy link
Collaborator Author

@ritchie46: rebased and added an "explain" check to confirm that we reduce all_horizontal nodes 👍

@ritchie46
Copy link
Member

Great!

@ritchie46 ritchie46 merged commit 00f0f91 into pola-rs:main Dec 28, 2023
14 checks passed
@alexander-beedie alexander-beedie deleted the expr-filter-kwargs branch December 28, 2023 20:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or an improvement of an existing feature python Related to Python Polars
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow kwargs constraints in pl.when and expr.filter
4 participants