Skip to content

[Enh]: Add Expr.any_value? #3298

@danielgafni

Description

@danielgafni

We would like to learn about your use case. For example, if this feature is needed to adopt Narwhals in an open source project, could you please enter the link to it below?

https://github.com/anam-org/metaxy

Please describe the purpose of the new feature or describe the problem to solve.

I would like to perform a group by on a DataFrame and take a random (any) row from each group.

I could have used df.group_by().agg(nw.all().first()), because rows within groups are not sorted already, but this currently fails with:

E           narwhals.exceptions.InvalidOperationError: Order-dependent expressions are not supported for use in LazyFrame.
E
E           Hint: To make the expression valid, use `.over` with `order_by` specified.
E
E           For example, if you wrote `nw.col('price').cum_sum()` and you have a column
E           `'date'` which orders your data, then replace:
E
E              nw.col('price').cum_sum()
E
E            with:
E
E              nw.col('price').cum_sum().over(order_by='date')
E                                       ^^^^^^^^^^^^^^^^^^^^^^
E
E           See https://narwhals-dev.github.io/narwhals/concepts/order_dependence/.

In my case performing a sort would introduce a performance penalty, so I would like to avoid it.

It seems like the best I could do now is with .is_first_distinct() over a dummy column:

def group_and_take_any(df: FrameT, cols: Sequence[str]) -> FrameT:
     return (
            df.with_columns(nw.lit(True).alias("_dummy"))
            .filter(
                nw.col("_dummy")
                .is_first_distinct()
                .over(*cols, order_by="_dummy")
            )
            .drop("_dummy")
        )

Suggest a solution if possible.

No response

If you have tried alternatives, please describe them below.

No response

Additional information that may help us understand your needs.

No response

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions