-
Notifications
You must be signed in to change notification settings - Fork 4k
[ci] move Windows Python jobs to GitHub Actions #7086
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
7e8f4e8
4aed19f
5a56a64
a12fa98
f497648
4fdbf2d
56ada20
23d4e8b
44f4305
62097d0
b6f5ddf
4d09e82
64a2ca7
77fe1e1
57f6251
d339e48
23f5e77
c81729c
f34e8b9
74b150d
c88a0c8
a9aca4d
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -66,10 +66,13 @@ if ($env:TASK -eq "swig") { | |
| } | ||
|
|
||
| # setup for Python | ||
| conda init powershell | ||
| conda activate | ||
| conda config --set always_yes yes --set changeps1 no | ||
| conda install -q -y conda "python=$env:PYTHON_VERSION[build=*_cp*]" | ||
| conda activate ; Assert-Output $? | ||
| conda config --set always_yes yes --set changeps1 no ; Assert-Output $? | ||
| conda config --remove channels defaults ; Assert-Output $? | ||
| conda config --add channels nodefaults ; Assert-Output $? | ||
| conda config --add channels conda-forge ; Assert-Output $? | ||
| conda config --set channel_priority strict ; Assert-Output $? | ||
| conda install -q -y conda "python=$env:PYTHON_VERSION[build=*_cp*]" ; Assert-Output $? | ||
|
|
||
| # print output of 'conda info', to help in submitting bug reports | ||
| Write-Output "conda info:" | ||
|
|
@@ -141,7 +144,7 @@ if (($env:TASK -eq "sdist") -or (($env:APPVEYOR -eq "true") -and ($env:TASK -eq | |
| if ($env:TASK -eq "bdist") { | ||
| # Make sure we can do both CPU and GPU; see tests/python_package_test/test_dual.py | ||
| # TODO: set LIGHTGBM_TEST_DUAL_CPU_GPU back to "1" as part of https://github.com/microsoft/LightGBM/issues/6968 | ||
| env:LIGHTGBM_TEST_DUAL_CPU_GPU = "0" | ||
| $env:LIGHTGBM_TEST_DUAL_CPU_GPU = "0" | ||
|
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not sure how this was working before on Azure DevOps, but it's failing here.
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This change exposed something else...
Pushed f34e8b9 proposing that we treat "set to 0" and "unset" as both meaning "skip the test". |
||
| } | ||
|
|
||
| pytest $tests ; Assert-Output $? | ||
|
|
||
| Original file line number | Diff line number | Diff line change | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -17,6 +17,8 @@ env: | |||||||||
| # tell scripts where to put artifacts | ||||||||||
| # (this variable name is left over from when jobs ran on Azure DevOps) | ||||||||||
| BUILD_ARTIFACTSTAGINGDIRECTORY: '${{ github.workspace }}/artifacts' | ||||||||||
| # where repo sources are cloned to | ||||||||||
| BUILD_SOURCESDIRECTORY: '${{ github.workspace }}' | ||||||||||
| CMAKE_BUILD_PARALLEL_LEVEL: 4 | ||||||||||
| SKBUILD_STRICT_CONFIG: true | ||||||||||
|
|
||||||||||
|
|
@@ -69,7 +71,7 @@ jobs: | |||||||||
| with: | ||||||||||
| name: linux-aarch64-wheel | ||||||||||
| path: artifacts/*.whl | ||||||||||
| test-macos: | ||||||||||
| test: | ||||||||||
| # yamllint disable-line rule:line-length | ||||||||||
| name: ${{ matrix.task }} ${{ matrix.method }} (${{ matrix.os }}, ${{ matrix.compiler }}, Python ${{ matrix.python_version }}) | ||||||||||
| runs-on: ${{ matrix.os }} | ||||||||||
|
|
@@ -94,6 +96,13 @@ jobs: | |||||||||
| task: bdist | ||||||||||
| compiler: gcc | ||||||||||
| python_version: '3.9' | ||||||||||
| - os: windows-2022 | ||||||||||
| task: sdist | ||||||||||
| compiler: MSVC | ||||||||||
| python_version: '3.10' | ||||||||||
| ############ | ||||||||||
| # MPI jobs # | ||||||||||
| ############ | ||||||||||
| - os: macos-15-intel | ||||||||||
| task: mpi | ||||||||||
| compiler: gcc | ||||||||||
|
|
@@ -133,14 +142,28 @@ jobs: | |||||||||
| produces-artifacts: 'true' | ||||||||||
| artifact-name: 'macosx-amd64-liblightgbm' | ||||||||||
| python_version: '3.11' | ||||||||||
| - os: windows-2022 | ||||||||||
| task: bdist | ||||||||||
| compiler: MSVC | ||||||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. To match other similar job names.
Suggested change
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sure we can do this and add the But note that nothing in https://github.com/microsoft/LightGBM/blob/master/.ci/test-windows.ps1 uses it.
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ah but we do use it in the job name for the GitHub Actions UI
I forgot about that.
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. did this in c88a0c8 |
||||||||||
| method: wheel | ||||||||||
| produces-artifacts: 'true' | ||||||||||
| artifact-name: 'windows-amd64-wheel' | ||||||||||
| python_version: '3.13' | ||||||||||
| - os: windows-2022 | ||||||||||
| task: regular | ||||||||||
| compiler: MSVC | ||||||||||
| produces-artifacts: 'true' | ||||||||||
| artifact-name: 'windows-amd64-cli-and-liblightgbm' | ||||||||||
| python_version: '3.11' | ||||||||||
| steps: | ||||||||||
| - name: Checkout repository | ||||||||||
| uses: actions/checkout@v5 | ||||||||||
| with: | ||||||||||
| fetch-depth: 5 | ||||||||||
| persist-credentials: false | ||||||||||
| submodules: true | ||||||||||
| - name: Setup and run tests | ||||||||||
| - name: Setup and run tests on Linux and macOS | ||||||||||
|
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This will be expanded with Linux jobs as well in the next PR. This name matches the step name in the R-package workflow: LightGBM/.github/workflows/r_package.yml Line 178 in 3f7db2b
|
||||||||||
| if: startsWith(matrix.os, 'macos') || startsWith(matrix.os, 'ubuntu') | ||||||||||
| shell: bash | ||||||||||
| run: | | ||||||||||
| export COMPILER="${{ matrix.compiler }}" | ||||||||||
|
|
@@ -154,14 +177,39 @@ jobs: | |||||||||
| export PRODUCES_ARTIFACTS="${{ matrix.produces-artifacts }}" | ||||||||||
| $GITHUB_WORKSPACE/.ci/setup.sh || exit 1 | ||||||||||
| $GITHUB_WORKSPACE/.ci/test.sh || exit 1 | ||||||||||
| - name: Install OpenCL on Windows | ||||||||||
| if: startsWith(matrix.os, 'windows') && (matrix.task == 'bdist') | ||||||||||
| shell: pwsh -command ". {0}" | ||||||||||
| run: | | ||||||||||
| & "$env:GITHUB_WORKSPACE/.ci/install-opencl.ps1" | ||||||||||
| # 'conda init powershell' needs to be run in a separate process | ||||||||||
| # ref: https://docs.conda.io/projects/conda/en/stable/dev-guide/deep-dives/activation.html | ||||||||||
| - name: Initialize conda on Windows | ||||||||||
| if: startsWith(matrix.os, 'windows') | ||||||||||
| shell: pwsh -command ". {0}" | ||||||||||
| run: | | ||||||||||
| $env:PATH = "$env:CONDA/Scripts;$env:PATH" | ||||||||||
| conda init powershell | ||||||||||
| - name: Setup and run tests on Windows | ||||||||||
| if: startsWith(matrix.os, 'windows') | ||||||||||
| shell: pwsh -command ". {0}" | ||||||||||
| run: | | ||||||||||
| $env:COMPILER = "${{ matrix.compiler }}" | ||||||||||
| $env:METHOD = "${{ matrix.method }}" | ||||||||||
| $env:PRODUCES_ARTIFACTS = "${{ matrix.produces-artifacts }}" | ||||||||||
| $env:PYTHON_VERSION = "${{ matrix.python_version }}" | ||||||||||
| $env:TASK = "${{ matrix.task }}" | ||||||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I guess we need more env variables here, e.g. https://github.com/microsoft/LightGBM/actions/runs/19627711345/job/56199935267?pr=7086#step:5:4848
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ah you're completely right. I was following patterns from other jobs like SWIG and the R-package, but I guess those don't need to set any of these things (they only affect Python tests). Will push a fix.
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. My changes are not quite working:
Moving this back to draft while I work on that.
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Alright I think this is now working and all Windows Python-package jobs are using the correct Python version.
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sure we can do this. I'd omitted it because it isn't used in the test script: #7086 (comment) I'll push a commit with it though, to preserve the alphabetical order.
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. did this in c88a0c8 |
||||||||||
| & "$env:GITHUB_WORKSPACE/.ci/test-windows.ps1" | ||||||||||
| - name: Upload artifacts | ||||||||||
| if: ${{ matrix.produces-artifacts == 'true' }} | ||||||||||
| uses: actions/upload-artifact@v5 | ||||||||||
| with: | ||||||||||
| name: ${{ matrix.artifact-name }} | ||||||||||
| path: | | ||||||||||
| artifacts/*.whl | ||||||||||
| artifacts/*.dll | ||||||||||
| artifacts/*.dylib | ||||||||||
| artifacts/*.exe | ||||||||||
| artifacts/*.whl | ||||||||||
|
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||||||||||
| if-no-files-found: error | ||||||||||
| test-latest-versions: | ||||||||||
| name: Python - latest versions (ubuntu-latest) | ||||||||||
|
|
@@ -231,7 +279,7 @@ jobs: | |||||||||
| runs-on: ubuntu-latest | ||||||||||
| needs: | ||||||||||
| - test-latest-versions | ||||||||||
| - test-macos | ||||||||||
| - test | ||||||||||
| - test-linux-aarch64 | ||||||||||
| - test-old-versions | ||||||||||
| steps: | ||||||||||
|
|
||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also added
Assert-Outputto all of these, to get louder and quicker failures