Create wheels for pypi

[brunsgaard]

Is this a thing that could be done?

[gfairchild]

I need to look into it. I believe these docs describe how to do it. Because this project contains a compiled extension (i.e., it contains C code that needs to be compiled), I need to make sure I get the wheel correct. It looks like I just need to create platform wheels.

[brunsgaard]

I know about the C code, that is why I really would like a wheel, I have a developer on my team that would like to use your package. But as of now I am blocking it as the production servers should not compile C code. We only use wheels in productions.

[brunsgaard]

It is just best practice to upload wheels today instead of source distributions. :) It makes it a little more hard for you, because you have to create the wheels, for all the different platforms. But it is better you do it right once. Instead like in this situation where the users (like us) have to build their own wheel.

[gfairchild]

I'll definitely look into it. I think I can set aside some time Sunday to deal with PR #7 and this issue.

[brunsgaard]

@gfairchild, I can help you if you want. I have access to osx, freebsd and linux platforms.

[gfairchild]

Hm, looking at the docs:

PyPI currently only allows uploads of platform wheels for Windows and OS X, NOT linux. Currently, the wheel tag specification (PEP 425) does not handle the variation that can exist across linux distros.

I've got access to Windows and OS X. I have access to Linux as well, but it doesn't look like that'll be useful.

[brunsgaard]

tell that to the numpy people :) https://pypi.python.org/pypi/numpy there is alot of linux wheel out there. I will look into it too :)

[gfairchild]

Ok! In that case, I can totally look into it.

[brunsgaard]

I think we should look into this project https://github.com/pypa/manylinux

[gfairchild]

Nice find. I hadn't heard of that before.

[gfairchild]

I just built and uploaded an OS X wheel (my work laptop is a Mac):

python setup.py bdist_wheel upload

That's all it took (well, after running pip install wheel).

Looking into manylinux.

[CJ-Wright]

Potentially related, there is a conda-forge recipe/build for this so you should be able to conda install this across platforms.

[gfairchild]

Cool! Thanks for bringing that to my attention. Really neat for that to exist.

[CJ-Wright]

Feel free to PR/issue against it. I went from the PyPi release, which is missing the tests and licence.

[Stranger6667]

Hello! Any news on this topic?

[gfairchild]

Unfortunately not. Work has been absolutely crazy this year. However, I'd love it if someone would like to help me out by issuing a PR.

[gfairchild]

FYI, I'm actively working on this now. I'm trying to figure out how to build manylinux containers manually (without Travis). pypa/manylinux#73 is where this is going down right now.

[gfairchild]

Making some progress. This primarily comes from https://github.com/pypa/python-manylinux-demo/blob/master/travis/build-wheels.sh.

> docker pull quay.io/pypa/manylinux1_x86_64
> docker run -it -v $(pwd):/io quay.io/pypa/manylinux1_x86_64
[root@1c3dba71b425 /]# # Compile wheels
for PYBIN in /opt/python/*/bin; do
    "${PYBIN}/pip" wheel /io/ -w wheelhouse/
done

# Bundle external shared libraries into the wheels
for whl in wheelhouse/*.whl; do
    auditwheel repair "$whl" -w /io/wheelhouse/
done

# Install packages and test
for PYBIN in /opt/python/*/bin/; do
    "${PYBIN}/pip" install pyxDamerauLevenshtein --no-index -f /io/wheelhouse
done

With this, I end up with a new directory in the root pyxDamerauLevenshtein directory called wheelhouse, and it contains this:

[root@f9689e43204c /]# cd io/wheelhouse/
[root@f9689e43204c wheelhouse]# ls -la
total 86928
drwxr-xr-x 14 root root      448 Jan  7 22:56 .
drwxr-xr-x 19 root root      608 Jan  7 22:56 ..
-rw-r--r--  1 root root 13835276 Jan  7 23:04 numpy-1.15.4-cp27-cp27m-manylinux1_x86_64.whl
-rw-r--r--  1 root root 13834837 Jan  7 23:04 numpy-1.15.4-cp27-cp27mu-manylinux1_x86_64.whl
-rw-r--r--  1 root root 13831861 Jan  7 23:04 numpy-1.15.4-cp34-cp34m-manylinux1_x86_64.whl
-rw-r--r--  1 root root 13835425 Jan  7 23:04 numpy-1.15.4-cp35-cp35m-manylinux1_x86_64.whl
-rw-r--r--  1 root root 13855020 Jan  7 23:04 numpy-1.15.4-cp36-cp36m-manylinux1_x86_64.whl
-rw-r--r--  1 root root 13845388 Jan  7 23:05 numpy-1.15.4-cp37-cp37m-manylinux1_x86_64.whl
-rw-r--r--  1 root root    77187 Jan  7 23:05 pyxDamerauLevenshtein-1.5.2-cp27-cp27m-manylinux1_x86_64.whl
-rw-r--r--  1 root root    77211 Jan  7 23:05 pyxDamerauLevenshtein-1.5.2-cp27-cp27mu-manylinux1_x86_64.whl
-rw-r--r--  1 root root    84204 Jan  7 23:05 pyxDamerauLevenshtein-1.5.2-cp34-cp34m-manylinux1_x86_64.whl
-rw-r--r--  1 root root    86550 Jan  7 23:05 pyxDamerauLevenshtein-1.5.2-cp35-cp35m-manylinux1_x86_64.whl
-rw-r--r--  1 root root    88682 Jan  7 23:05 pyxDamerauLevenshtein-1.5.2-cp36-cp36m-manylinux1_x86_64.whl
-rw-r--r--  1 root root    90239 Jan  7 23:05 pyxDamerauLevenshtein-1.5.2-cp37-cp37m-manylinux1_x86_64.whl

I don't quite understand the wheel-making process well enough yet to understand why this script is placing NumPy wheels in wheelhouse and if/how I can tell it to not do that, but this is definitely progress!

[erb13020]

@gfairchild Has there been any progress on this issue? I'm still trying to figure out how to get this installed on my local machine.

[gfairchild]

@gfairchild Has there been any progress on this issue? I'm still trying to figure out how to get this installed on my local machine.

Unfortunately not. I haven't had the time to tackle this issue, and I'm not currently using the library myself at this time, so it hasn't been a priority. I'd love a contribution from the community, though!

As long as you have a C compiler, you should be able to pip install without much fuss. It'll take maybe an extra minute to install, but it'll work just the same as the pre-compiled wheel.