Installing 3rd-party libraries

To create deep learning training runs and experiments, you can use the coding libraries that already come installed with IBM Watson Machine Learning Deep Learning. However, you might also want to install additional libraries to support your work. To do this, you must create a wheel (.whl) file and include this with the .zip file that you submit as part of your training run or experiment. You can then execute them by running the pip install /<wheel_file_name>.whl command.

Getting started

By creating a wheel for your project you save time for your end user because there is no need to go through a build process. Your wheel file is already and built package that can be included as part of the training .zip file.

There are several different types of wheels:

  • A Universal Wheel has the following characteristics:

    • It contains no compiled extensions, which means that it is pure Python code with no compiled extension.
    • It natively supports both Python 2 and Python 3.
    • It does not have any C extensions.
  • A Pure Python Wheel has the following characteristics:

    • It contains no compiled extensions, which means that it is pure Python code with no compiled extension.
    • It does not natively support both Python 2 and Python 3.
  • A Platform Wheel is for a project that contains compiled extensions.

Before you can build any of these wheels for your deep learning .zip file, you’ll need to install the wheel package by running the following command:

pip install wheel

Building a universal wheel

Universal Wheels are wheels that are pure Python (i.e. contain no compiled extensions) and support Python 2 and 3. This is a wheel that can be installed anywhere by pip.

python setup.py bdist_wheel --universal

Tip: You can also permanently set the —universal flag in the setup.cfg file:

[bdist_wheel]
universal=1

Only use the --universal setting, if the following restrictions are true:

  • Your project runs on Python 2 and 3 with no changes (i.e. it does not require 2to3).
  • Your project does not have any C extensions. Beware that bdist_wheel does not currently have any checks to warn if you use the setting inappropriately.

If your project has optional C extensions, it is recommended not to publish a universal wheel, because pip will prefer the wheel over a source installation, and prevent the possibility of building the extension.

Building a pure Python wheel

Remember, although the pure Python wheel contains only pure Python and no compiled extensions, it is not truly “universal” because it don’t natively support both Python 2 and Python 3.

To build a pure Python wheel, run the following command:

python setup.py bdist_wheel

The bdist_wheel method detects that the code is pure Python. As such, it’s usable on any Python installation with the same major version (Python 2 or Python 3) as the version you used to build the wheel.

Tip: You might want to produce wheels for each version of Python by running this set up twice.

Building a platform wheel

Platform Wheels are wheels that are specific to a certain platform, such as the Linux, macOS, or Microsoft Windows operating systems. What makes them specific? It's because they contain compiled extensions that are unique to those operating systems.

To build a platform wheel, run the following command:

python setup.py bdist_wheel

The bdist_wheel method detects that the code is not pure Python. As such, it’s only usable on the platform that it was built on.

Because PyPI supports uploads of platform wheels for Windows, macOS, and the multi-distro manylinux1 ABI, when you finish creating a platform wheel, you'll need to upload the project to PyPI.

When you ran the command to create your distribution, a new directory dist/ was created under your project’s root directory. That’s where you’ll find your distribution file(s) to upload.

Note These files are only created when you run the command to create your distribution. This means that any time you change the source of your project or the configuration in your setup.py file, you will need to rebuild these files again before you can distribute the changes to PyPI.

Note: Before releasing on main PyPI repo, you might prefer training with the PyPI test site which is cleaned on a semi regular basis. See Using TestPyPI on how to setup your configuration in order to use it.

Warning: In other resources you may encounter references to using python setup.py register and python setup.py upload. These methods of registering and uploading a package are strongly discouraged as it may use a plaintext HTTP or unverified HTTPS connection on some Python versions, allowing your username and password to be intercepted during transmission.

Tip: The reStructuredText parser used on PyPI is not Sphinx! Furthermore, to ensure safety of all users, certain kinds of URLs and directives are forbidden or stripped out (e.g., the .. raw:: directive). Before trying to upload your distribution, you should check to see if your brief / long descriptions provided in setup.py are valid. You can do this by following the instructions for the pypa/readme_renderer tool.