Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
7e8f4e8
[ci] move Windows Python jobs to GitHub Actions
jameslamb Nov 22, 2025
4aed19f
more artifacts stuff
jameslamb Nov 22, 2025
5a56a64
set BUILD_SOURCESDIRECTORY correctly
jameslamb Nov 22, 2025
a12fa98
is conda pre-installed?
jameslamb Nov 22, 2025
f497648
move conda stuff around
jameslamb Nov 22, 2025
4fdbf2d
mirror conda strategy from Azure DevOps
jameslamb Nov 22, 2025
56ada20
update artifact name, restore all CI
jameslamb Nov 23, 2025
23d4e8b
Merge branch 'master' into ci/windows-jobs
StrikerRUS Nov 24, 2025
44f4305
merge master
jameslamb Nov 24, 2025
62097d0
remove duplicated code, set all env variables
jameslamb Nov 24, 2025
b6f5ddf
Merge branch 'ci/windows-jobs' of github.com:microsoft/LightGBM into …
jameslamb Nov 24, 2025
4d09e82
do not use cmd
jameslamb Nov 25, 2025
64a2ca7
fail on failures and get more feedback
jameslamb Nov 25, 2025
77fe1e1
conda init should be run outside of the script
jameslamb Nov 25, 2025
57f6251
more debugging
jameslamb Nov 25, 2025
d339e48
move conda init powershell to a separate process
jameslamb Nov 25, 2025
23f5e77
need PATH update
jameslamb Nov 25, 2025
c81729c
remove debugging stuff, fix env variable setting
jameslamb Nov 25, 2025
f34e8b9
treat LIGHTGBM_TEST_DUAL_CPU_GPU=0 and that var being unset similarly
jameslamb Nov 25, 2025
74b150d
Update .github/workflows/python_package.yml
jameslamb Nov 26, 2025
c88a0c8
set METHOD
jameslamb Nov 26, 2025
a9aca4d
Merge branch 'ci/windows-jobs' of github.com:microsoft/LightGBM into …
jameslamb Nov 26, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions .ci/download-artifacts.sh
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,11 @@
# The following are not yet handled by this script.
#
# - commit.txt
# - lib_lightgbm.dll
# - lib_lightgbm.so
# - lightgbm-4.6.0-py3-none-manylinux_2_28_x86_64.whl
# - lightgbm-4.6.0-py3-none-win_amd64.whl
# - lightgbm-4.6.0.tar.gz
# - LightGBM-complete_source_code_tar_gz.tar.gz
# - LightGBM.4.6.0.nupkg
# - lightgbm.exe
#

set -e -u -E -o pipefail
Expand Down
13 changes: 8 additions & 5 deletions .ci/test-windows.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -66,10 +66,13 @@ if ($env:TASK -eq "swig") {
}

# setup for Python
conda init powershell
conda activate
conda config --set always_yes yes --set changeps1 no
conda install -q -y conda "python=$env:PYTHON_VERSION[build=*_cp*]"
conda activate ; Assert-Output $?
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also added Assert-Output to all of these, to get louder and quicker failures

conda config --set always_yes yes --set changeps1 no ; Assert-Output $?
conda config --remove channels defaults ; Assert-Output $?
conda config --add channels nodefaults ; Assert-Output $?
conda config --add channels conda-forge ; Assert-Output $?
conda config --set channel_priority strict ; Assert-Output $?
conda install -q -y conda "python=$env:PYTHON_VERSION[build=*_cp*]" ; Assert-Output $?

# print output of 'conda info', to help in submitting bug reports
Write-Output "conda info:"
Expand Down Expand Up @@ -141,7 +144,7 @@ if (($env:TASK -eq "sdist") -or (($env:APPVEYOR -eq "true") -and ($env:TASK -eq
if ($env:TASK -eq "bdist") {
# Make sure we can do both CPU and GPU; see tests/python_package_test/test_dual.py
# TODO: set LIGHTGBM_TEST_DUAL_CPU_GPU back to "1" as part of https://github.com/microsoft/LightGBM/issues/6968
env:LIGHTGBM_TEST_DUAL_CPU_GPU = "0"
$env:LIGHTGBM_TEST_DUAL_CPU_GPU = "0"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure how this was working before on Azure DevOps, but it's failing here.

env:LIGHTGBM_TEST_DUAL_CPU_GPU: D:\a\LightGBM\LightGBM.ci\test-windows.ps1:150
Line |
150 | env:LIGHTGBM_TEST_DUAL_CPU_GPU = "0"
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| The term 'env:LIGHTGBM_TEST_DUAL_CPU_GPU' is not recognized as a name of a cmdlet, function, script file, or
| executable program. Check the spelling of the name, or if a path was included, verify that the path is correct
| and try again.
Error: Process completed with exit code 1.

(build link)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change exposed something else... test_dual.py is skipped only if LIGHTGBM_TEST_DUAL_CPU_GPU is unset but runs if it's set to any value.

FAILED ..\tests\python_package_test\test_dual.py::test_cpu_and_gpu_work - lightgbm.basic.LightGBMError: GPU Tree Learner was not enabled in this build.
Please recompile with CMake option -DUSE_GPU=1

(build link)

Pushed f34e8b9 proposing that we treat "set to 0" and "unset" as both meaning "skip the test".

}

pytest $tests ; Assert-Output $?
Expand Down
56 changes: 52 additions & 4 deletions .github/workflows/python_package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ env:
# tell scripts where to put artifacts
# (this variable name is left over from when jobs ran on Azure DevOps)
BUILD_ARTIFACTSTAGINGDIRECTORY: '${{ github.workspace }}/artifacts'
# where repo sources are cloned to
BUILD_SOURCESDIRECTORY: '${{ github.workspace }}'
CMAKE_BUILD_PARALLEL_LEVEL: 4
SKBUILD_STRICT_CONFIG: true

Expand Down Expand Up @@ -69,7 +71,7 @@ jobs:
with:
name: linux-aarch64-wheel
path: artifacts/*.whl
test-macos:
test:
# yamllint disable-line rule:line-length
name: ${{ matrix.task }} ${{ matrix.method }} (${{ matrix.os }}, ${{ matrix.compiler }}, Python ${{ matrix.python_version }})
runs-on: ${{ matrix.os }}
Expand All @@ -94,6 +96,13 @@ jobs:
task: bdist
compiler: gcc
python_version: '3.9'
- os: windows-2022
task: sdist
compiler: MSVC
python_version: '3.10'
############
# MPI jobs #
############
- os: macos-15-intel
task: mpi
compiler: gcc
Expand Down Expand Up @@ -133,14 +142,28 @@ jobs:
produces-artifacts: 'true'
artifact-name: 'macosx-amd64-liblightgbm'
python_version: '3.11'
- os: windows-2022
task: bdist
compiler: MSVC
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To match other similar job names.

Suggested change
compiler: MSVC
compiler: MSVC
method: wheel

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure we can do this and add the METHOD environment variable for consistency.

But note that nothing in https://github.com/microsoft/LightGBM/blob/master/.ci/test-windows.ps1 uses it.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah but we do use it in the job name for the GitHub Actions UI

name: ${{ matrix.task }} ${{ matrix.method }} (${{ matrix.os }}, ${{ matrix.compiler }}, Python ${{ matrix.python_version }})

I forgot about that.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did this in c88a0c8

method: wheel
produces-artifacts: 'true'
artifact-name: 'windows-amd64-wheel'
python_version: '3.13'
- os: windows-2022
task: regular
compiler: MSVC
produces-artifacts: 'true'
artifact-name: 'windows-amd64-cli-and-liblightgbm'
python_version: '3.11'
steps:
- name: Checkout repository
uses: actions/checkout@v5
with:
fetch-depth: 5
persist-credentials: false
submodules: true
- name: Setup and run tests
- name: Setup and run tests on Linux and macOS
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will be expanded with Linux jobs as well in the next PR.

This name matches the step name in the R-package workflow:

- name: Setup and run tests on Linux and macOS

if: startsWith(matrix.os, 'macos') || startsWith(matrix.os, 'ubuntu')
shell: bash
run: |
export COMPILER="${{ matrix.compiler }}"
Expand All @@ -154,14 +177,39 @@ jobs:
export PRODUCES_ARTIFACTS="${{ matrix.produces-artifacts }}"
$GITHUB_WORKSPACE/.ci/setup.sh || exit 1
$GITHUB_WORKSPACE/.ci/test.sh || exit 1
- name: Install OpenCL on Windows
if: startsWith(matrix.os, 'windows') && (matrix.task == 'bdist')
shell: pwsh -command ". {0}"
run: |
& "$env:GITHUB_WORKSPACE/.ci/install-opencl.ps1"
# 'conda init powershell' needs to be run in a separate process
# ref: https://docs.conda.io/projects/conda/en/stable/dev-guide/deep-dives/activation.html
- name: Initialize conda on Windows
if: startsWith(matrix.os, 'windows')
shell: pwsh -command ". {0}"
run: |
$env:PATH = "$env:CONDA/Scripts;$env:PATH"
conda init powershell
- name: Setup and run tests on Windows
if: startsWith(matrix.os, 'windows')
shell: pwsh -command ". {0}"
run: |
$env:COMPILER = "${{ matrix.compiler }}"
$env:METHOD = "${{ matrix.method }}"
$env:PRODUCES_ARTIFACTS = "${{ matrix.produces-artifacts }}"
$env:PYTHON_VERSION = "${{ matrix.python_version }}"
$env:TASK = "${{ matrix.task }}"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we need more env variables here, e.g. METHOD, OS_NAME, PYTHON_VERSION, because right now wrong Python version is used:

============================= test session starts =============================
platform win32 -- Python 3.14.0, pytest-9.0.1, pluggy-1.6.0

https://github.com/microsoft/LightGBM/actions/runs/19627711345/job/56199935267?pr=7086#step:5:4848

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah you're completely right. I was following patterns from other jobs like SWIG and the R-package, but I guess those don't need to set any of these things (they only affect Python tests).

Will push a fix.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My changes are not quite working:

[INFO] --- building sdist ---
C:\hostedtoolcache\windows\Python\3.9.13\x64\python.exe: No module named build
Error: Process completed with exit code -1.

(build link)

Moving this back to draft while I work on that.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright I think this is now working and all Windows Python-package jobs are using the correct Python version.

platform win32 -- Python 3.9.23, pytest-7.4.4, pluggy-1.0.0

(Appveyor - MSVC)

platform win32 -- Python 3.9.23, pytest-7.4.4, pluggy-1.0.0

(Appveyor - MINGW)

platform win32 -- Python 3.10.19, pytest-9.0.1, pluggy-1.6.0

(GitHub Actions - sdist)

platform win32 -- Python 3.13.9, pytest-9.0.1, pluggy-1.6.0

(GitHub Actions - bdist)

platform win32 -- Python 3.11.14, pytest-9.0.1, pluggy-1.6.0

(GitHub Actions - regular)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
$env:TASK = "${{ matrix.task }}"
$env:TASK = "${{ matrix.task }}"
$env:METHOD = "${{ matrix.method }}"

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure we can do this. I'd omitted it because it isn't used in the test script: #7086 (comment)

I'll push a commit with it though, to preserve the alphabetical order.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did this in c88a0c8

& "$env:GITHUB_WORKSPACE/.ci/test-windows.ps1"
- name: Upload artifacts
if: ${{ matrix.produces-artifacts == 'true' }}
uses: actions/upload-artifact@v5
with:
name: ${{ matrix.artifact-name }}
path: |
artifacts/*.whl
artifacts/*.dll
artifacts/*.dylib
artifacts/*.exe
artifacts/*.whl
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

*.whl shows up in the diff here only because I'm taking this opportunity to alphabetize this list.

if-no-files-found: error
test-latest-versions:
name: Python - latest versions (ubuntu-latest)
Expand Down Expand Up @@ -231,7 +279,7 @@ jobs:
runs-on: ubuntu-latest
needs:
- test-latest-versions
- test-macos
- test
- test-linux-aarch64
- test-old-versions
steps:
Expand Down
47 changes: 0 additions & 47 deletions .vsts-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -203,52 +203,6 @@ jobs:
inputs:
filePath: $(Build.SourcesDirectory)/.ci/test.sh
targetType: 'filePath'
###########
# Windows #
###########
- job: Windows
pool:
vmImage: 'windows-2022'
strategy:
matrix:
regular:
TASK: regular
PYTHON_VERSION: '3.11'
sdist:
TASK: sdist
PYTHON_VERSION: '3.10'
bdist:
TASK: bdist
steps:
- powershell: |
Write-Host "##vso[task.prependpath]$env:CONDA\Scripts"
displayName: 'Set Variables'
- script: |
git clean -d -f -x
displayName: 'Clean source directory'
- script: |
cmd /c "powershell -ExecutionPolicy Bypass -File %BUILD_SOURCESDIRECTORY%/.ci/install-opencl.ps1"
condition: eq(variables['TASK'], 'bdist')
displayName: 'Install OpenCL'
- script: |
cmd /c "conda config --remove channels defaults"
cmd /c "conda config --add channels nodefaults"
cmd /c "conda config --add channels conda-forge"
cmd /c "conda config --set channel_priority strict"
cmd /c "conda init powershell"
cmd /c "powershell -ExecutionPolicy Bypass -File %BUILD_SOURCESDIRECTORY%/.ci/test-windows.ps1"
displayName: Test
- task: PublishBuildArtifacts@1
condition: >
and(
succeeded(),
in(variables['TASK'], 'regular', 'bdist'),
not(startsWith(variables['Build.SourceBranch'], 'refs/pull/'))
)
inputs:
pathtoPublish: '$(Build.ArtifactStagingDirectory)'
artifactName: PackageAssets
artifactType: container

###########
# Package #
Expand All @@ -257,7 +211,6 @@ jobs:
dependsOn:
- Linux
- Linux_latest
- Windows
condition: and(succeeded(), not(startsWith(variables['Build.SourceBranch'], 'refs/pull/')))
pool:
vmImage: 'ubuntu-22.04'
Expand Down
4 changes: 2 additions & 2 deletions tests/python_package_test/test_dual.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@


@pytest.mark.skipif(
os.environ.get("LIGHTGBM_TEST_DUAL_CPU_GPU", None) is None,
reason="Only run if appropriate env variable is set",
os.environ.get("LIGHTGBM_TEST_DUAL_CPU_GPU", "0") != "1",
reason="Set LIGHTGBM_TEST_DUAL_CPU_GPU=1 to test using CPU and GPU training from the same package.",
)
def test_cpu_and_gpu_work():
# If compiled appropriately, the same installation will support both GPU and CPU.
Expand Down
Loading