Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP][docs] update Python-package installation guide #6767

Draft
wants to merge 112 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
112 commits
Select commit Hold shift + click to select a range
1f6c773
Update README.rst
StrikerRUS Dec 20, 2024
dc765b0
Update README.rst
StrikerRUS Dec 20, 2024
339cc84
Update pyproject.toml
StrikerRUS Dec 20, 2024
3e841e0
Update README.rst
StrikerRUS Dec 20, 2024
deb2d98
Update README.rst
StrikerRUS Dec 20, 2024
afdc323
Update README.rst
StrikerRUS Dec 20, 2024
e8f52a8
Update README.rst
StrikerRUS Dec 20, 2024
1f1d88f
Update README.rst
StrikerRUS Dec 20, 2024
b537c7e
Update README.rst
StrikerRUS Dec 20, 2024
fefaded
Update README.rst
StrikerRUS Dec 20, 2024
449292f
Update README.rst
StrikerRUS Dec 20, 2024
dd6e17a
Update README.rst
StrikerRUS Dec 20, 2024
9d93ca9
Update README.rst
StrikerRUS Dec 20, 2024
7141270
Update README.rst
StrikerRUS Dec 20, 2024
4ea935f
Update README.rst
StrikerRUS Dec 20, 2024
ce0b19e
Update README.rst
StrikerRUS Dec 20, 2024
eb8d2d3
Update README.rst
StrikerRUS Dec 20, 2024
c55af10
Update README.rst
StrikerRUS Dec 20, 2024
9d44d69
Update pyproject.toml
StrikerRUS Dec 20, 2024
fb90500
Update README.rst
StrikerRUS Dec 20, 2024
6790bdd
Update README.rst
StrikerRUS Dec 20, 2024
8c58b4f
Update README.rst
StrikerRUS Dec 20, 2024
6f961d9
Update README.rst
StrikerRUS Dec 20, 2024
e271a29
Update README.rst
StrikerRUS Dec 20, 2024
b747e4d
Update pyproject.toml
StrikerRUS Dec 20, 2024
80a7c20
Update README.rst
StrikerRUS Dec 20, 2024
8f6cb05
Update README.rst
StrikerRUS Dec 20, 2024
26c921a
Update README.rst
StrikerRUS Dec 20, 2024
e50acb5
Update README.rst
StrikerRUS Dec 20, 2024
8eb7005
Update README.rst
StrikerRUS Dec 20, 2024
0dd5983
Update README.rst
StrikerRUS Dec 20, 2024
d818fb8
Update README.rst
StrikerRUS Dec 21, 2024
4eaf198
Update README.rst
StrikerRUS Dec 21, 2024
ca2fee1
Update README.rst
StrikerRUS Dec 21, 2024
c09506a
Update README.rst
StrikerRUS Dec 21, 2024
ec9cd50
Update README.rst
StrikerRUS Dec 21, 2024
8148b69
Update README.rst
StrikerRUS Dec 22, 2024
acfe30d
Update README.rst
StrikerRUS Dec 22, 2024
8ed111f
Update pyproject.toml
StrikerRUS Dec 22, 2024
02f4b95
Update README.rst
StrikerRUS Dec 22, 2024
ec24127
Update README.rst
StrikerRUS Dec 22, 2024
5f29431
Update build-python.sh
StrikerRUS Dec 22, 2024
6555396
Update build-python.sh
StrikerRUS Dec 22, 2024
d0fe127
Update build-python.sh
StrikerRUS Dec 22, 2024
ff3e4c7
Update build-python.sh
StrikerRUS Dec 22, 2024
73334ca
Update build-python.sh
StrikerRUS Dec 22, 2024
726ef96
Update build-python.sh
StrikerRUS Dec 22, 2024
a9a534c
Update build-python.sh
StrikerRUS Dec 22, 2024
848a536
Update build-python.sh
StrikerRUS Dec 22, 2024
784fcba
Update build-python.sh
StrikerRUS Dec 22, 2024
cc8a24e
Update build-python.sh
StrikerRUS Dec 22, 2024
03bd3c9
Update build-python.sh
StrikerRUS Dec 22, 2024
4cd32ce
Update build-python.sh
StrikerRUS Dec 22, 2024
19a07e2
Update build-python.sh
StrikerRUS Dec 22, 2024
e9cced7
Update build-python.sh
StrikerRUS Dec 22, 2024
ac34770
Update build-python.sh
StrikerRUS Dec 22, 2024
f7b8e7c
Update build-python.sh
StrikerRUS Dec 22, 2024
ddb52b9
Update build-python.sh
StrikerRUS Dec 22, 2024
b07feda
Update build-python.sh
StrikerRUS Dec 22, 2024
9d065ce
Update build-python.sh
StrikerRUS Dec 23, 2024
53b8e19
Update build-python.sh
StrikerRUS Dec 23, 2024
4b3c98a
Update build-python.sh
StrikerRUS Dec 23, 2024
e3a4836
Update build-python.sh
StrikerRUS Dec 23, 2024
d99d0cf
Update build-python.sh
StrikerRUS Dec 23, 2024
a592c54
Update build-python.sh
StrikerRUS Dec 23, 2024
df3f69d
Update build-python.sh
StrikerRUS Dec 24, 2024
dd710f5
Update build-python.sh
StrikerRUS Dec 24, 2024
3fc761f
Update build-python.sh
StrikerRUS Dec 24, 2024
3bee691
Update build-python.sh
StrikerRUS Dec 24, 2024
f160bcf
Update build-python.sh
StrikerRUS Dec 24, 2024
866b46f
Update build-python.sh
StrikerRUS Dec 24, 2024
e7c9a8e
Update build-python.sh
StrikerRUS Dec 24, 2024
b85cc0f
Update build-python.sh
StrikerRUS Dec 25, 2024
c9c5c04
Update build-python.sh
StrikerRUS Dec 25, 2024
dbc9400
Update build-python.sh
StrikerRUS Dec 25, 2024
00c31e3
Update build-python.sh
StrikerRUS Dec 25, 2024
88d586b
Update build-python.sh
StrikerRUS Dec 25, 2024
9ae9c28
Update build-python.sh
StrikerRUS Dec 25, 2024
9f056cf
Update build-python.sh
StrikerRUS Dec 25, 2024
2bd90f7
Update build-python.sh
StrikerRUS Dec 25, 2024
3f043dd
Update build-python.sh
StrikerRUS Dec 25, 2024
c9f8483
Update build-python.sh
StrikerRUS Dec 25, 2024
532121f
Update build-python.sh
StrikerRUS Dec 25, 2024
f4fc100
Update build-python.sh
StrikerRUS Dec 25, 2024
652ada1
Update build-python.sh
StrikerRUS Dec 25, 2024
b467e78
Update README.rst
StrikerRUS Dec 25, 2024
f722634
Update README.rst
StrikerRUS Dec 25, 2024
0a858fe
Update build-python.sh
StrikerRUS Dec 25, 2024
a212e1a
Update README.rst
StrikerRUS Dec 25, 2024
92db16e
Update README.rst
StrikerRUS Dec 25, 2024
7a80b77
Update build-python.sh
StrikerRUS Dec 25, 2024
078481d
Update build-python.sh
StrikerRUS Dec 26, 2024
08a98a3
Update build-python.sh
StrikerRUS Dec 26, 2024
fb27330
Update README.rst
StrikerRUS Dec 26, 2024
1e7e2ef
Update build-python.sh
StrikerRUS Dec 26, 2024
0d055ad
Update build-python.sh
StrikerRUS Dec 26, 2024
7a3ba7f
Update build-python.sh
StrikerRUS Dec 26, 2024
c13aa46
Update README.rst
StrikerRUS Dec 26, 2024
2a85784
Update build-python.sh
StrikerRUS Dec 26, 2024
0736db2
Update README.rst
StrikerRUS Dec 26, 2024
f0a541f
Update README.rst
StrikerRUS Dec 26, 2024
722f928
Update README.rst
StrikerRUS Dec 26, 2024
56cc500
Update build-python.sh
StrikerRUS Dec 26, 2024
da0ebc9
Update build-python.sh
StrikerRUS Dec 26, 2024
ace2590
Update build-python.sh
StrikerRUS Dec 26, 2024
c38187c
Update build-python.sh
StrikerRUS Dec 26, 2024
ed4c453
Update LightGBM.vcxproj
StrikerRUS Dec 26, 2024
599561a
Update LightGBM.vcxproj
StrikerRUS Dec 26, 2024
0247c6b
Update README.rst
StrikerRUS Dec 26, 2024
dd1a94a
Update README.rst
StrikerRUS Dec 26, 2024
1367902
Update README.rst
StrikerRUS Dec 26, 2024
bb39ff5
Update README.rst
StrikerRUS Dec 26, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 55 additions & 27 deletions build-python.sh
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 +34,14 @@
# OpenCL include directory.
# --opencl-library=FILEPATH
# Path to OpenCL library.
# --sanitizers=LIST_OF_SANITIZERS
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

# Semicolon separated list with chosen compiler sanitizers.
# --bit32
# Compile 32-bit version.
# --cuda
# Compile CUDA version.
# --debug
# Compile in debug mode.
# --gpu
# Compile GPU version.
# --integrated-opencl
Expand All @@ -49,20 +53,22 @@
# --no-isolation
# Assume all build and install dependencies are already installed,
# don't go to the internet to get them.
# --nohomebrew
# Compile version ignoring Homebrew standard folders for finding dependencies.
# --nomp
# Compile version without OpenMP support.
# --precompile
# Use precompiled library.
# Only used with 'install' command.
# --time-costs
# Output time costs for different internal routines.
# Compile version that outputs time costs for different internal routines.
# --user
# Install into user-specific instead of global site-packages directory.
# Only used with 'install' command.

set -e -u

echo "building lightgbm"
echo "[INFO] building lightgbm"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the consistency with

echo "[INFO] Attempting to build 32-bit version of LightGBM, which is only supported on Windows with generator '${CMAKE_GENERATOR}'."

and to better distinguish between our own logs and scikit-build-core ones.


# Default values of arguments
INSTALL="false"
Expand Down Expand Up @@ -132,27 +138,38 @@ while [ $# -gt 0 ]; do
OPENCL_LIBRARY="${1#*=}"
BUILD_ARGS="${BUILD_ARGS} --config-setting=cmake.define.OpenCL_LIBRARY='${OPENCL_LIBRARY}'"
;;
--sanitizers|--sanitizers=*)
if echo "$1" | grep -q '^*=*$';
then shift;
fi
SANITIZERS="${1#*=}"
BUILD_ARGS="${BUILD_ARGS} --config-setting=cmake.define.USE_SANITIZER=ON"
BUILD_ARGS="${BUILD_ARGS} --config-setting=cmake.define.ENABLED_SANITIZERS='${SANITIZERS}'"
;;
#########
# flags #
#########
--bit32)
export CMAKE_GENERATOR="Visual Studio 17 2022"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Allow to use any default Visual Studio version.

export CMAKE_GENERATOR_PLATFORM="Win32"
echo "[INFO] Attempting to build 32-bit version of LightGBM, which is only supported on Windows with generator '${CMAKE_GENERATOR}'."
echo "[INFO] Attempting to build 32-bit version of LightGBM, which is only supported on Windows with Visual Studio."
BUILD_ARGS="${BUILD_ARGS} --config-setting=cmake.args=-AWin32"
;;
--cuda)
BUILD_ARGS="${BUILD_ARGS} --config-setting=cmake.define.USE_CUDA=ON"
;;
--debug)
BUILD_ARGS="${BUILD_ARGS} --config-setting=cmake.define.USE_DEBUG=ON"
BUILD_ARGS="${BUILD_ARGS} --config-setting=cmake.build-type=Debug"
;;
--gpu)
BUILD_ARGS="${BUILD_ARGS} --config-setting=cmake.define.USE_GPU=ON"
;;
--integrated-opencl)
BUILD_ARGS="${BUILD_ARGS} --config-setting=cmake.define.__INTEGRATE_OPENCL=ON"
;;
--mingw)
export CMAKE_GENERATOR='MinGW Makefiles'
# ref: https://stackoverflow.com/a/45104058/3986677
BUILD_ARGS="${BUILD_ARGS} --config-setting=cmake.define.CMAKE_SH=CMAKE_SH-NOTFOUND"
BUILD_ARGS="${BUILD_ARGS} --config-setting=cmake.args=-G'MinGW Makefiles'"
;;
--mpi)
BUILD_ARGS="${BUILD_ARGS} --config-setting=cmake.define.USE_MPI=ON"
Expand All @@ -161,6 +178,9 @@ while [ $# -gt 0 ]; do
BUILD_ARGS="${BUILD_ARGS} --no-isolation"
PIP_INSTALL_ARGS="${PIP_INSTALL_ARGS} --no-build-isolation"
;;
--nohomebrew)
BUILD_ARGS="${BUILD_ARGS} --config-setting=cmake.define.USE_HOMEBREW_FALLBACK=OFF"
;;
--nomp)
BUILD_ARGS="${BUILD_ARGS} --config-setting=cmake.define.USE_OPENMP=OFF"
;;
Expand All @@ -174,7 +194,7 @@ while [ $# -gt 0 ]; do
PIP_INSTALL_ARGS="${PIP_INSTALL_ARGS} --user"
;;
*)
echo "invalid argument '${1}'"
echo "[ERROR] invalid argument '${1}'. Aborting"
exit 1
;;
esac
Expand Down Expand Up @@ -315,18 +335,28 @@ if test "${INSTALL}" = true; then
echo "" >> ./MANIFEST.in
mkdir -p ./lightgbm/lib
if test -f ../lib_lightgbm.so; then
echo "found pre-compiled lib_lightgbm.so"
echo "[INFO] found pre-compiled lib_lightgbm.so"
cp ../lib_lightgbm.so ./lightgbm/lib/lib_lightgbm.so
elif test -f ../lib_lightgbm.dylib; then
echo "found pre-compiled lib_lightgbm.dylib"
echo "[INFO] found pre-compiled lib_lightgbm.dylib"
cp ../lib_lightgbm.dylib ./lightgbm/lib/lib_lightgbm.dylib
elif test -f ../lib_lightgbm.dll; then
echo "[INFO] found pre-compiled lib_lightgbm.dll"
cp ../lib_lightgbm.dll ./lightgbm/lib/lib_lightgbm.dll
Comment on lines +343 to +345
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

elif test -f ../Release/lib_lightgbm.dll; then
echo "found pre-compiled Release/lib_lightgbm.dll"
echo "[INFO] found pre-compiled Release/lib_lightgbm.dll"
cp ../Release/lib_lightgbm.dll ./lightgbm/lib/lib_lightgbm.dll
elif test -f ../windows/x64/DLL/lib_lightgbm.dll; then
echo "found pre-compiled windows/x64/DLL/lib_lightgbm.dll"
echo "[INFO] found pre-compiled windows/x64/DLL/lib_lightgbm.dll"
cp ../windows/x64/DLL/lib_lightgbm.dll ./lightgbm/lib/lib_lightgbm.dll
cp ../windows/x64/DLL/lib_lightgbm.lib ./lightgbm/lib/lib_lightgbm.lib
elif test -f ../windows/x64/Debug_DLL/lib_lightgbm.dll; then
echo "[INFO] found pre-compiled windows/x64/Debug_DLL/lib_lightgbm.dll"
cp ../windows/x64/Debug_DLL/lib_lightgbm.dll ./lightgbm/lib/lib_lightgbm.dll
cp ../windows/x64/Debug_DLL/lib_lightgbm.lib ./lightgbm/lib/lib_lightgbm.lib
Comment on lines +353 to +356
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can be located there after compilation with Debug_DLL config:

<ProjectConfiguration Include="Debug_DLL|x64">
<Configuration>Debug_DLL</Configuration>
<Platform>x64</Platform>

else
echo "[ERROR] cannot find pre-compiled library. Aborting"
exit 1
Comment on lines +357 to +359
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without this code, script simply builds empty sdist if no pre-compiled file was found.

fi
rm -f ./*.bak
else
Expand All @@ -336,29 +366,27 @@ if test "${INSTALL}" = true; then
fi

if test "${BUILD_SDIST}" = true; then
echo "--- building sdist ---"
echo "[INFO] --- building sdist ---"
rm -f ../dist/*.tar.gz
# shellcheck disable=SC2086
python -m build \
--sdist \
--outdir ../dist \
${BUILD_ARGS} \
.
# use xargs to work with args that contain whitespaces
# note that empty echo string leads to that xargs doesn't run the command
# in some implementations of xargs
# ref: https://stackoverflow.com/a/8296746
echo "--sdist --outdir ../dist ${BUILD_ARGS} ." | xargs python -m build
Comment on lines +371 to +375
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use xargs to handle args with whitespaces, e.g. -G'MinGW Makefiles' or -DOpenCL_INCLUDE_DIR='C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.0/include'

fi

if test "${BUILD_WHEEL}" = true; then
echo "--- building wheel ---"
echo "[INFO] --- building wheel ---"
rm -f ../dist/*.whl || true
# shellcheck disable=SC2086
python -m build \
--wheel \
--outdir ../dist \
${BUILD_ARGS} \
.
# use xargs to work with args that contain whitespaces
# note that empty echo string leads to that xargs doesn't run the command
# in some implementations of xargs
# ref: https://stackoverflow.com/a/8296746
echo "--wheel --outdir ../dist ${BUILD_ARGS} ." | xargs python -m build
fi

if test "${INSTALL}" = true; then
echo "--- installing lightgbm ---"
echo "[INFO] --- installing lightgbm ---"
cd ../dist
if test "${BUILD_WHEEL}" = true; then
PACKAGE_NAME="$(echo lightgbm*.whl)"
Expand All @@ -377,5 +405,5 @@ if test "${INSTALL}" = true; then
cd ../
fi

echo "cleaning up"
echo "[INFO] cleaning up"
rm -rf ./lightgbm-python
Loading
Loading