-
Notifications
You must be signed in to change notification settings - Fork 175
Open
Labels
Description
We would like to learn about your use case. For example, if this feature is needed to adopt Narwhals in an open source project, could you please enter the link to it below?
https://github.com/anam-org/metaxy
Please describe the purpose of the new feature or describe the problem to solve.
I would like to perform a group by on a DataFrame and take a random (any) row from each group.
I could have used df.group_by().agg(nw.all().first()), because rows within groups are not sorted already, but this currently fails with:
E narwhals.exceptions.InvalidOperationError: Order-dependent expressions are not supported for use in LazyFrame.
E
E Hint: To make the expression valid, use `.over` with `order_by` specified.
E
E For example, if you wrote `nw.col('price').cum_sum()` and you have a column
E `'date'` which orders your data, then replace:
E
E nw.col('price').cum_sum()
E
E with:
E
E nw.col('price').cum_sum().over(order_by='date')
E ^^^^^^^^^^^^^^^^^^^^^^
E
E See https://narwhals-dev.github.io/narwhals/concepts/order_dependence/.
In my case performing a sort would introduce a performance penalty, so I would like to avoid it.
It seems like the best I could do now is with .is_first_distinct() over a dummy column:
def group_and_take_any(df: FrameT, cols: Sequence[str]) -> FrameT:
return (
df.with_columns(nw.lit(True).alias("_dummy"))
.filter(
nw.col("_dummy")
.is_first_distinct()
.over(*cols, order_by="_dummy")
)
.drop("_dummy")
)Suggest a solution if possible.
No response
If you have tried alternatives, please describe them below.
No response
Additional information that may help us understand your needs.
No response
MarcoGorelli