segfault in polars lazyframe.collect() surfaced in tpch_q1_test.py

[cameronraysmith]

Apologies for being terse, but, I just wanted to reach out to determine if the developers have ever run across something like what's described here NixOS/nixpkgs#456547 (review) .

Quoted below for convenience.

Details

Regarding the failures appearing in NixOS/nixpkgs#456547 (comment), the logs show narwhals tpch_q1_test.py, stochastically but with macroscopic probability, segfaults in polars' lazyframe.collect() (frame.py:2332, exit 139) on all platforms,

narwhals 2.9.0 checkPhase segfault via polars

2025-10-29T22:54:46.7225630Z error: Cannot build '/nix/store/fndvkmw95zjpabh46dvsj0z4q5jwwwq2-python3.12-narwhals-2.9.0.drv'.
2025-10-29T22:54:46.7333970Z        Reason: builder failed with exit code 139.
2025-10-29T22:54:46.7442220Z        Output paths:
2025-10-29T22:54:46.7558020Z          /nix/store/46d4ymywaj091ky7dzya6m5pvsp9bggx-python3.12-narwhals-2.9.0
2025-10-29T22:54:46.7663590Z          /nix/store/vgczkahsl2bkm3vfr9laqrdj9ym2zcrs-python3.12-narwhals-2.9.0-dist
2025-10-29T22:54:46.7769350Z        Last 25 log lines:
2025-10-29T22:54:46.7874420Z        > tests/series_only/to_arrow_test.py ....x...                              [ 92%]
2025-10-29T22:54:46.7976200Z        > tests/series_only/to_dummy_test.py ................ssssssss........      [ 93%]
2025-10-29T22:54:46.8080980Z        > tests/series_only/to_frame_test.py ....                                  [ 93%]
2025-10-29T22:54:46.8191710Z        > tests/series_only/to_list_test.py ....                                   [ 93%]
2025-10-29T22:54:46.8306320Z        > tests/series_only/to_native_test.py ....                                 [ 93%]
2025-10-29T22:54:46.8314810Z        > tests/series_only/to_numpy_test.py x.......                              [ 93%]
2025-10-29T22:54:46.8315310Z        > tests/series_only/to_pandas_test.py .x..                                 [ 93%]
2025-10-29T22:54:46.8315770Z        > tests/series_only/to_polars_test.py ....                                 [ 93%]
2025-10-29T22:54:46.8316490Z        > tests/series_only/value_counts_test.py ................                  [ 93%]
2025-10-29T22:54:46.8316970Z        > tests/series_only/zip_with_test.py ........                              [ 93%]
2025-10-29T22:54:46.8317810Z        > tests/stable_api_test.py ....                                            [ 93%]
2025-10-29T22:54:46.8318290Z        > tests/stable_v1_dependencies_test.py .                                   [ 93%]
2025-10-29T22:54:46.8440270Z        > tests/system_info_test.py ...                                            [ 93%]
2025-10-29T22:54:46.8538820Z        > tests/testing/assert_series_equal_test.py .............................. [ 93%]
2025-10-29T22:54:46.8640110Z        > ........................................................................ [ 94%]
2025-10-29T22:54:46.8742990Z        > .......x                                                                 [ 94%]
2025-10-29T22:54:46.8845420Z        > tests/tpch_q1_test.py .Fatal Python error: Segmentation fault
2025-10-29T22:54:46.8949260Z        >
2025-10-29T22:54:46.9066500Z        > Thread 0x000000020b552140 (most recent call first):
2025-10-29T22:54:46.9171910Z        >   File "/nix/store/f1vafsbx177b4lm9322dgcfr3d0x2vmv-python3.12-polars-1.31.0/lib/python3.12/site-packages/polars/lazyframe/frame.py", line 2332 in collect
2025-10-29T22:54:46.9280250Z        >   File "/nix/store/f1vafsbx177b4lm9322dgcfr3d0x2vmv-python3.12-polars-1.31.0/lib/python3.12/site-packages/polars/lazyframe/opt_flags.py", line 330 in wrapper
2025-10-29T22:54:46.9364190Z        >   File "/nix/store/f1vafsbx177b4lm9322dgcfr3d0x2vmv-python3.12-polars-1.31.0/lib/python3.12/site-packages/polars/_utils/deprecation.py", line 97 in wrapper
2025-10-29T22:54:46.9467840Z        >   File "/nix/build/nix-7776-2159791391/source/narwhals/_polars/dataframe.py", line 687 in collect
2025-10-29T22:54:46.9596070Z        >   File "/nix/build/nix-7776-2159791391/source/narwhals/dataframe.py", line 2472 in collect
2025-10-29T22:54:46.9707720Z        >   File "/nix/build/nix-7776-2159791391/source/tests/utils.py", line 106 in assert_equal_data/nix/store/ckxyzx7c3syrv84w58mrm0n1r5m94vih-pytest-check-hook/nix-support/setup-hook: line 23: 36059 Segmentation fault: 11     /nix/store/amnm5l9wmqkzk2paibhy81vp690dhrdk-python3-3.12.11/bin/python3.12 "${flagsArray[@]}"
2025-10-29T22:54:46.9811160Z        For full logs, run:
2025-10-29T22:54:46.9915940Z          nix log /nix/store/fndvkmw95zjpabh46dvsj0z4q5jwwwq2-python3.12-narwhals-2.9.0.drv

cascading through imageio → plotly → optuna → resampy → librosa → beets.

Something like this

diff --git a/pkgs/development/python-modules/narwhals/default.nix b/pkgs/development/python-modules/narwhals/default.nix
index a33dad202948..xxxxxxxxxxxx 100644
--- a/pkgs/development/python-modules/narwhals/default.nix
+++ b/pkgs/development/python-modules/narwhals/default.nix
@@ -80,6 +80,10 @@ buildPythonPackage rec {
     # ibis improvements cause strict XPASS failures (tests expected to fail now pass)
     "test_empty_scalar_reduction_with_columns"
     "test_collect_empty"
+    # segfaults in polars' lazyframe.collect()
+    "test_q1"
+    "test_q1_w_generic_funcs"
+    "test_q1_w_pandas_agg_generic_path"
   ];

   pytestFlags = [

should probably be applied to disable the tests from that module to minimize noise from this issue elsewhere so long as it persists in narwhals 2.10.0 (this PR is not quite rebased past the most recent upgrade in NixOS/nixpkgs#456823) and unless it somehow proves to be idiosyncratic to the above three runs (GHA runners?). Could also be reported to the upstreams if it really is a robust failure mode. Quick search didn't suggest others have been experiencing and reporting this, so I'd hesitate to do so without a little more evidence since the tests work fine for me locally

narwhals pytest tests/tpch_q1_test.py

❯ pytest tests/tpch_q1_test.py -v
========================================================= test session starts =========================================================
platform darwin -- Python 3.13.6, pytest-8.4.2, pluggy-1.6.0
cachedir: .pytest_cache
Using --randomly-seed=1205150323
hypothesis profile 'default'
configfile: pyproject.toml
plugins: xdist-3.8.0, randomly-4.0.1, hypothesis-6.142.4, env-1.2.0, cov-7.0.0
collected 7 items

tests/tpch_q1_test.py::test_q1_w_generic_funcs[polars] PASSED                                                                   [ 14%]
tests/tpch_q1_test.py::test_q1_w_pandas_agg_generic_path PASSED                                                                 [ 28%]
tests/tpch_q1_test.py::test_q1_w_generic_funcs[pandas] PASSED                                                                   [ 42%]
tests/tpch_q1_test.py::test_q1[dask] PASSED                                                                                     [ 57%]
tests/tpch_q1_test.py::test_q1[pandas] PASSED                                                                                   [ 71%]
tests/tpch_q1_test.py::test_q1[polars] PASSED                                                                                   [ 85%]
tests/tpch_q1_test.py::test_q1[pyarrow] PASSED                                                                                  [100%]

========================================================== 7 passed in 0.54s ==========================================================

Not sure how robust or persistent it might be after 2.9.0; which is one among the reasons I would not immediately create an issue. Obviously, if it's truly robust, would be more reportable via polars than here, but would require some work to extract a MWE from the test.

[cameronraysmith]

Nevermind this must have been due to failures idiosyncratic to github actions runners.

NixOS/nixpkgs#456547 (comment)

[MarcoGorelli]

sure, thanks for resolving!