Supported frameworks

You can use popular tools, libraries, and frameworks to train and deploy machine learning models and functions using IBM Watson Machine Learning. This topic lists supported versions and features.

See Machine Learning samples and examples for links to sample notebooks that demonstrate creating batch deployments using the Watson Machine Learning REST API and Watson Machine Learning Python client library.

Supported machine learning frameworks

Table 1. Supported machine learning tools, libraries, and frameworks
Framework Features supported in IBM Watson Studio
Spark MLlib
Spark 2.4
  • Deployment: online and batch.
  • Only classification and regression models are supported.
  • Custom transformers, user-defined functions, and classes are not supported.
scikit-learn
  • scikit-learn 0.23 with Python 3.7
  • scikit-learn 0.22.1 on Anaconda 2019.03 for Python 3.6 Runtime (deprecated)
  • scikit-learn 0.20.3 on Anaconda 2019.03 for Python 3.6 Runtime (deprecated)
  • scikit-learn 0.19 on Anaconda 2019.03 for Python 3.6 Runtime (deprecated)
  • Deployment: online and batch.
  • Core ML
  • Only classification and regression models are supported.
  • Models that use custom components cannot be trained using Watson Machine Learning training runs or experiments.
  • Models can contain references to only those packages (and package versions) that are available by default in the Anaconda installer of the Anaconda distribution used.
  • To deploy models with references to user-defined transformers/estimators, see: Using custom components
XGBoost
  • XGBoost 0.90 with scikit-learn 0.23 for Python 3.7 Runtime
  • XGBoost 0.90 in an Anaconda 2019.03 with scikit-learn 0.22.1 for Python 3.6 Runtime(deprecated)
  • XGBoost 0.82 in an Anaconda 2019.03 with scikit-learn 0.20.3 for Python 3.6 Runtime(deprecated)
  • XGBoost 0.80 in an Anaconda 2019.03 environment with scikit-learn 0.19 for Python 3.6 Runtime (deprecated)
  • Deployment: online and batch.
  • Core ML (XGBoost 0.80, 0.82, and 0.90)
  • You can define custom scikit-learn transformers and then add them as a stage in a scikit-learn Pipeline with XGBoostClassifier or XGBRegressor.
  • Probability of prediction is not returned in results.
  • Models can contain references to only those packages (and package versions) that are available by default in the Anaconda installer of corresponding Anaconda Distribution version.
  • To deploy models with references to user-defined transformers/estimators, see: Using custom components
  • Online deployment uses Python 3.6.
  • To use XGBoost versions >= 0.80, you must use the Watson Machine Learning Python client version >= 1.0.338
TensorFlow
  • Version 1.15 in an Anaconda 2019.03 environment
  • Version 2.1 with Python 3.7
  • Deployment: online and batch.
  • Only JSON content is supported as the payload for scoring.
  • tf.estimator is not supported.
  • Custom component support:
    • User-defined tensor operations that are defined using the tf.py_func API are supported with online deployment.
    • Custom C++ operations are not supported.
    • Models that use custom components cannot be trained using Watson Machine Learning training runs or experiments.
    • See: Using custom operators
Keras
  • Keras 2.2.5 with TensorFlow 1.15 in an Anaconda 2019.03 environment
  • Deployment: online, batch and Core ML.
  • Only JSON content is supported as the payload for scoring.
  • tf.estimator is not supported.
  • User-defined layers with trainable weights are not supported. However, stateless custom operations defined using layers.core.Lambda layers are supported with online deployment.
PyTorch
Versions: 1.2 (deployment) 1.1, 1.3
  • Deployment: online and batch.
IBM SPSS Modeler
  • IBM SPSS Modeler 17.1
  • IBM SPSS Modeler 18.1 (deprecated), 18.2
  • Deployment: online and batch.
  • A branch cannot require a connection to an external service (such as references to rules stored in an IBM SPSS Collaboration and Deployment Services repository.)
  • When using IBM SPSS Modeler embedded Python scripting, the only supported way to reference a node is by its unique ID. (See: Finding nodes external link)
  • The following features are not supported:
    • DB data/function
    • Hadoop
    • Analytic Server
    • R/Python extensions
    • Social Network Analysis
    • Entity Analytics
    • Text Analytics Japanese edition
  • If you export a stream file (.str) from the Watson Studio flow editor, it can be imported back into the flow editor (to back up or share a design, for example) and it can be imported into SPSS Modeler. But that stream file cannot itself be deployed to Watson Machine Learning; you must train the stream in the flow editor.
Predictive Model Markup Language (PMML)
Version 3.0 to 4.3
  • Deployment: online only.
  • Supported models: R, SAS, SPSS
Decision Optimization runtime
  • opl (do-opl_12.9, do-opl_12.10)
  • cplex (do-cplex_12.9. do-cplex_12.10 )
  • cpo (do-cpo_12.9, do-cpo_12.10)
  • docplex (do-docplex_12.9, do-docplex_12.10)

Note: GPU-accelerated computing is the employment of a graphics processing unit (GPU) along with a computer processing unit (CPU) in order to facilitate processing-intensive operations such as deep learning experiments. GPU acceleration is limited to training and is only available with the following frameworks:

  • Keras
  • PyTorch
  • Tensorflow

Managing assets that refer to discontinued software or frameworks

Note these guidelines for when assets that rely on discontinued software specifications or frameworks are handled. In some cases, the migration is seamless. In other cases, your action is required to retrain or redeploy assets.

For details on managing assets that rely on deprecated or discontinued frameworks and software specifications, see Specifying a model type and software specification.

Framework notes

  • Support for Python 3.6 is deprecated in favor of Python 3.7. Starting on November 20, 2020, new training based on Python 3.6 will be blocked. Support for Python 3.6 frameworks and related software specifications will be discontinued on Jan 20, 2021 for V4-GA API and on March 1, 2021 for v3 APIs and V4-beta API. For example:
    • Scikit-learn 0.22/XGBoost 0.90 framework with Python3.6 for Watson Machine Learning is deprecated and will be removed. Use Scikit-learn 0.23/XGBoost 0.90 with default_py3.7 instead.
    • Software specification ‘ai-function_0.1-py3.6 is deprecated and will be removed. Use default_py3.7` software specification instead when saving a Python function.
  • To deploy with these Python 3.7-based frameworks you must be using the V4 GA version of Watson Machine Learning APIs and a current, V2 machine learning service instance.
    • scikit-learn 0.23
    • xgboost 0.90 with python 3.7
    • TF 2.1
    • PyTorch 1.3
  • Spark 2.3 framework for Watson Machine Learning is deprecated and will be removed on December 1, 2020. Use Spark 2.4 instead.
  • If you are creating a CoreML deployment for a logistic regression model using Scikit-learn 0.23 with Python 3.7, you must explicitly specify a value to override the default Scikit-learn package with Scikit-learn 0.23 during model training. See this article for details of specifying a package version when you are training a Scikit-learn pipeline.
  • If you are using Pytorch 1.3 with Python 3.7 to train a model, you must explicitly set keep_initializers_as_inputs=True when exporting your model.
  • In order for a Scikit-learn or Xgboost model to be converted to CoreML format, it must contain only supported transformer algorithms. Refer to this documentation for lists of supported CoreML transformers for Scikit-learn and Xgboost frameworks:
    • https://apple.github.io/coremltools/generated/coremltools.converters.sklearn.convert.html
    • https://apple.github.io/coremltools/generated/coremltools.converters.xgboost.convert.html
  • Due to security vulnerabilities with several TensorFlow versions, Watson Machine Learning has added support for TensorFlow version 1.15 and removed support for all unsecure TensorFlow versions, including 1.14 and 1.13. For details, read the blog announcement. For help changing to a supported runtime, see the TensorFlow compatibility guide.
  • If a framework is marked as deprecated then support for the framework will be removed in a future release.
  • If you trained a model in a GPU Notebook, only TensorFlow is supported as a deployment framework.

SPSS Modeler flows might need retraining

  • Due to an upgrade of the Python version in supported SPSS Modeler runtimes, some models that were built using SPSS Modeler in IBM Watson Studio Cloud prior to this upgrade can no longer be deployed using Watson Machine Learning. If you’re using one of the following 6 nodes in your SPSS Modeler Flow, you must rebuild and redeploy your model(s) with SPSS Modeler and Watson Machine Learning starting on September 1, 2020. The affected models include these nodes:

    • XGBoost Tree
    • XGBoost Linear
    • One-Class SVM
    • HDBSCAN
    • KDE Modeling
    • Gaussian Mixture

If your SPSS model uses any of these SPSS Modeler nodes, take the following action:

  • If you’re using the Watson Studio user interface, open the SPSS Modeler flow in Watson Studio, retrain, and save the model to Watson Machine Learning. After saving the model to the project, you can promote it to a deployment space and create a new deployment.
  • If you’re using REST APIs or Python Client, retrain the model using the latest SPSS Modeler and save the model to the Watson Machine Learning repository with the model type ‘spss-modeler-18.2.’