2020 What's new
Here are the new features for Cloud Pak for Data as a Service, the core services of Watson Studio, Watson Machine Learning, and Watson Knowledge Catalog, and other services for the year 2020.
Week ending 18 December 2020
Deprecation and removal of Streams flows (Streaming Analytics)
Streams flows is being deprecated on January 31, 2021. See Service plan changes and deprecations.
Week ending 4 December 2020
IBM Watson Visual Recognition is discontinued in Watson Machine Learning
IBM Watson Visual Recognition is discontinued. See Service plan changes and deprecations.
Week ending 27 November 2020
Open beta for Federated Learning (Watson Machine Learning)
Federated Learning provides the tools for training a model collaboratively, using a federated set of secure, remote data sources. The data sources are never moved or combined, but they each contribute to training and improving the quality of the common model. The high-level steps are:
- Define the parties for federated learning and create remote training systems
- Create a common model and configure how to aggregate the data.
- Train the model with the federated data sources.
- Deploy and score the resulting model.
For details on setting up remote parties and training a common model, see Federated Learning.
Hybrid Subscription Advantage
The IBM Hybrid Subscription Advantage program is a licensing benefit that applies existing on-premises Cloud Pak for Data software entitlements within the Cloud Pak for Data as a Service portfolio. To learn more, see Activating the Hybrid Subscription Advantage.
Week of 20 November 2020
HIPAA readiness for Watson Knowledge Catalog
Watson Knowledge Catalog meets the required IBM controls that are commensurate with the Health Insurance Portability and Accountability Act of 1996 (HIPAA) Security and Privacy Rule requirements. HIPAA readiness applies to only certain plans and regions. For more information, see Keeping your data secure and compliant.
Week of 13 November 2020
Reminder: Upcoming changes to Watson Machine Learning frameworks
All frameworks that are built with Python 3.6 are deprecated in favor of frameworks that are built with Python 3.7. Spark 2.3 frameworks are deprecated in favor of Spark 2.4. Support for Spark 2.3 will be discontinued on December 1, 2020. See Service plan changes and deprecations.
Week of 28 October 2020
Legacy APIs no longer supported for Watson Machine Learning Lite plan users
The migration period for Watson Machine Learning Lite plan users to migrate assets to the V2 machine learning service instance and the V4 Watson Machine Learning APIs is ended. See Service plan changes and deprecations.
IBM Match 360 with Watson service beta
The IBM Match 360 with Watson service is now in beta. This new IBM Match 360 with Watson experience seamlessly consolidates data from the disparate sources that fuel your business to establish a single, trusted, 360-degree view of your customers.
The beta release of IBM Match 360 with Watson includes two user experiences:
-
Master data configuration for data engineers to prepare and configure master data. This experience enables you to:
- Configure the master data system.
- Refine the generated data model.
- Upload or connect data assets and sources.
- Map data into the model.
- Run the IBM Match 360 with Watson service's powerful matching capability to create master data entities.
- Configuring and tuning the matching algorithm to meet your organization's requirements.
-
Master data explorer for business analysts and data stewards to search, view, analyze, and export master data entities.
The IBM Match 360 with Watson service on Cloud Pak for Data as a Service also includes a rich set of APIs that empower your business applications with direct access to trusted data.
For more information about IBM Match 360 with Watson, see Managing master data (Beta).
Week ending 23 October 2020
New way of adding data (Watson Knowledge Catalog)
Create metadata import assets to configure and run the metadata import for a selected set of data assets into a project or a catalog. For details, see Managing metadata imports.
SJIS encoding available in Data Refinery for input and output
SJIS ("Shift JIS" or Shift Japanese Industrial Standards) encoding is an encoding for the Japanese language.
To change the encoding of the input file, open the file in Data Refinery, go to the Data tab, scroll down to the SOURCE FILE information, and then click the “Specify data format” icon .
To change the encoding of the output (target) file, open the Information pane and click the Details tab. Click the Edit button. In the DATA REFINERY FLOW OUTPUT pane, click the Edit icon to change the encoding.
The SJIS encoding is supported only for CSV and delimited files.
New visualization charts for Data Refinery and SPSS Modeler (Watson Studio)
To access the charts in Data Refinery, click the Visualizations tab and then select the columns to visualize. The chart automatically updates as you refine the data.
To access the charts in SPSS Modeler, use a Charts node. The Charts node is available under the Graphs section on the node palette. Double-click the Charts node to open the properties pane. Then click Launch Chart Builder to create one or more chart definitions to associate with the node.
For the full list of available charts, see Visualizing your data.
-
Evaluation charts are combination charts that measure the quality of a binary classifier. You need three columns for input: actual (target) value, predict value, and confidence (0 or 1). Move the slider in the Cutoff chart to dynamically update the other charts. The ROC and other charts are standard measurements of the classifier.
-
Math curve charts display a group of curves based on equations that you enter. You do not use a data set with this chart. Instead, you use it to compare the results with the data set in another chart, like the scatter plot chart.
-
Sunburst charts display different depths of hierarchical groups. The Sunburst chart was formerly an option in the Treemap chart.
-
Tree charts represent a hierarchy in a tree-like structure. The Tree chart consists of a root node, line connections called branches that represent the relationships and connections between the members, and leaf nodes that do not have child nodes. The Tree chart was formerly an option in the Treemap chart.
Week ending 16 October 2020
Change to Watson Machine Learning deployment frameworks
The following changes to deployment frameworks might require user action.
Support for Python 3.7
You can now select Python 3.7 frameworks to train models and run Watson Machine Learning deployments. See Service plan changes and deprecations.
Deprecation of Python 3.6
Python 3.6 is being deprecated. Support will be discontinued on April 8, 2021. See Service plan changes and deprecations.
Support for Spark 3.0 and new language versions
-
Spark 3.0
- You can now select a Spark 3.0 environment to run notebooks with Python 3.7, R 3.6 and Scala 2.12 or to run notebook jobs.
-
New languages
- Python 3.7
You can select Python 3.7 environments to run Jupyter notebooks (including with GPU) in Watson Studio.
Python 3.6 is being deprecated. You can continue to use the Python 3.6 environments; however you will be notified that you should move to a Python 3.7 environment. When you switch to Python 3.7, you might need to update code in notebooks if the versions of open source libraries that you used are different in Python 3.7.
- Scala 2.12
With the introduction of Spark 3.0, you can start using Spark with Scala 2.12 in notebooks and jobs. Again, you might need to update code in your notebooks if library versions that you used with Scala 2.11 are not compatible in Scala 2.12.
Support ends for SPSS Modeler runtime 18.1 and certain Python nodes
Support for SPSS Modeler flows trained with 18.1 and containing certain Python nodes is discontinued as of October 14, 2020. See Service plan changes and deprecations.
Support ends for deployments based on deprecated AutoAI images (Watson Machine Learning)
Due to a known security vulnerability, AutoAI model deployments created using Watson Machine Learning on IBM Cloud prior to August 1, 2020 will be removed on November 1, 2020. See Service plan changes and deprecations.
Week ending 9 October 2020
Time series library for notebooks (Watson Studio)
You can now use the time series library to perform operations on time series data, including segmentation, forecasting, joins, transforms and reducers. You can use the time series library functions in Python notebooks that run with Spark. See Time series library.
Week ending 2 October 2020
Updates to Watson Knowledge Catalog offering plans
Starting 1 October, 2020, the Watson Knowledge Catalog offering plans have changes. See Service plan changes and deprecations.
Week ending 25 September 2020
Batch deployment available for AutoAI experiments (Watson Machine Learning)
Starting with the V2 Watson Machinne Learning service instance and the V4 Watson Machine Learning APIs, rolled out on September 1, 2020, batch deployment is supported for AutoAI experiments. For details, see Creating a batch deployment.
New Databases for EDB service
You can now create the Databases for EDB service from the Cloud Pak for Data as a Service services catalog to access EDB Postegres Advanced Server. See Databases for EDB.
Week ending 18 September 2020
New jobs user interface for running and scheduling Data Refinery flows and notebooks (Watson Studio)
The user interface gives you a unified view of the job information.
You create the job for a Data Refinery flow or for a notebook directly in the user interface for each tool or from the Assets page of a project. See Jobs in a project.
Week ending 11 September 2020
Change to Watson Machine Learning service credentials
The new V2 Watson Machine Learning service instance rolled out on September 1 uses new, simplified authentication. Obtaining bearer tokens from IAM is now performed using a generic user apikey instead of a Watson Machine Learning specific apikey. It is no longer necessary to create specific credentials on the Watson Machine Learning instance, so the Credentials page was removed from the IBM Cloud services catalog. For details, see Authentication.
During the migration period, you can use existing Watson Machine Learning service credentials to access your legacy V1 service instance and assets. Lite users of instances provisioned before Sept 1st 2020 can keep using existing credentials during the migration period but cannot generate new credentials.
Spark 2.3 deprecation (Watson Studio)
Starting 1 October, 2020, you must select a Spark 2.4 environment instead of a Spark 2.3 environment to run a notebook or job. See Service plan changes and deprecations.
Week ending 4 September 2020
New Watson Machine Learning service plans
Watson Machine Learning released new plans on IBM Cloud. These new plans accommodate and provide entitlements for the newest features and patterns available to Watson Machine Learning users, starting on September 1, 2020. See Service plan changes and deprecations.
Upgrading to Watson Machine Learning "V2" Instances
All Lite plan users are automatically upgraded from v1 to v2 service plans. Lite plan users can now call the v4 APIs or use the v4 Python client library to conduct machine learning model training, model saving, and deployment, and access the newest features such as runtime software specifications for your deployments. See Service plan changes and deprecations.
Full support for v4 APIs and an updated Python client library (Watson Machine Learning)
The v4 APIs and Python client library are now generally available for use with the v2 service plans. The new APIs support features such as deployment spaces for organizing all of the assets required for running and managing deployment jobs, software specifications, and updated authentication. Note that support for v3 and v4 beta APIs ends on March 1, 2021.
Introducing deployment spaces (Watson Machine Learning)
Deployment spaces let you deploy and manage models and other operational assets such as data sources and software specifications in a separate environment from your projects. When your project assets are ready to deploy, you promote assets to your deployment space to configure deployments, test models and functions, consume scoring endpoints, and manage production jobs. Spaces, like projects, are collaborative, so you can invite others to collaborate and manage access for a space.
Watson Studio enhancements
Watson Studio now leverages the newest Watson Machine Learning APIs. Consequently, to take actions from the Watson Studio interface that require Watson Machine Learning, such as triggering AutoAI experiments, you must have a "v2" Watson Machine Learning instance associated with the project. Watson Studio projects that are still associated with an older Watson Machine Learning instance will display a message that instructs you to migrate your assets and associate a new v2 Watson Machine Learning instance.
Additionally, starting on September 1st, Watson Studio users will be able to save models and other artifacts produced through use of Watson Machine Learning alongside other assets like notebooks in their Watson Studio project. For details on machine learning tools you can use to create project assets, see Machine Learning Overview.
Decision Optimization support for Watson Machine Learning
Decision Optimization supports all of the new features available with Watson Machine Learning, including using software and hardware specifications to configure optimization models and using deployment spaces to organize the assets required for deployment. For a complete list of changes, see Migrating from Watson Machine Learning API V4 Beta.
Decision Optimization enhancements (Watson Studio)
You can now use these enhancements to Decision Optimization:
- The Decision Optimization model builder now contains a new Overview pane which provides you with model, data and solution summary information for all your scenarios at a glance. From this view you can also open an information pane where you can create or choose your deployment space. See Overview pane.
- To create and run Optimization models you must have both a Machine Learning service added to your project and a deployment space associated with your experiment.
- You can now deploy models using Watson Machine Learning from the the Decision Optimization model builder scenario pane. See Scenario pane and Deploying a model using the user interface.
- A new sample Extend software specification notebook is now available which shows you how to extend the Decision Optimization software specification (runtimes with additional Python libraries for docplex models). See Python client examples and download the sample from Decision Optimization sample repository.
- The Explore solution view of the model builder has been updated to show more information about the objectives/KPIs, solution tables, constraint or bounds relaxations or conflicts, engine statistics and log. See Explore solution view.
- The Visualization view of the model builder now enables you to create Gantt charts for any type of data where it is meaningful and is no longer restricted to scheduling models only. See Visualizations view.
Translation of documentation
You can now read this documentation in these languages by setting your browser locale:
- Brazilian Portuguese
- Simplified Chinese
- Traditional Chinese
- French
- German
- Italian
- Spanish
- Japanese
- Korean
- Russian
Not all documentation topics are translated into all of these languages.
Jupyter notebook editor upgraded (Watson Studio)
The Jupyter notebook editor in Watson Studio is upgraded from Jupyter notebook version 6.0.3 to version 6.1.1. For a list of changes, including keyboard short-cut key changes, see Jupyter Notebook Changelog.
Week ending 21 August 2020
Enhanced Cognos Dashboards
You can now use these enhancements to the dashboards in a project:
- New visualizations, including Waterfall, KPI widget, and an enhanced cross-tab.
- A contextual toolbar and a data mapping panel.
See Cognos Dashboards.
IBM Cloud Databases for MongoDB service
You can now provision a IBM Cloud Databases for MongoDB service from the Services catalog.
Use Data Refinery to change the decimal and thousands grouping symbols in all applicable columns (Watson Studio, Watson Knowledge Catalog)
When you use the Convert column type GUI operation to detect and convert the data types for all the columns in a data asset, you can now also choose the decimal symbol and the thousands grouping symbol if the data is converted to an Integer or to a Decimal data type. Previously you had to select individual columns to specify the symbols.
See Convert column type in GUI operations in Data Refinery, under the FREQUENTLY USED category.
Week ending 14 August 2020
Google Cloud Platform integration
You can now configure an integration with the Google Cloud Platform (GCP) to access data sources from GCP.
Week ending 31 July 2020
Security update for AutoAI deployments (Watson Machine Learning)
There is a known security vulnerability with the image used for AutoAI model deployments created using Watson Machine Learning on IBM Cloud prior to August 1, 2020. The image vulnerability has been addressed, so deployments of models created with AutoAI experiment after August 1, 2020 are not impacted. See Service plan changes and deprecations.
Removal of Neural Network Modeler and SparkML modeler
Both the beta Neural Network Modeler and the beta SparkML modeler tools are removed from Watson Studio.
Week ending 24 July 2020
Cloud Pak for Data as a Service is GA
Cloud Pak for Data as a Service is now generally available. Sign up for Cloud Pak for Data as a Service at dataplatform.cloud.ibm.com.
Learn more about Cloud Pak for Data as a Service.
Read the Making IBM Cloud Pak for Data more accessible—as a service blog post.
Subscribe to Cloud Pak for Data as a Service
You can now upgrade your IBM Cloud account from a Lite plan by subscribing to Cloud Pak for Data as a Service. With a subscription, you commit to a minimum spending amount for a certain period of time and receive a discount on the overall cost compared to a Pay-As-You-Go account.
See Upgrading to a Cloud Pak for Data as a Service subscription account.
Learn quickly with a guided tutorial
You can quickly learn how to use tools in projects by taking a guided tutorial. A guided tutorial starts with a sample project that contains the data and anything else you need. After you create the project, the tutorial starts and guides you through the steps to solve a specific business problem.
Click the Take a guided tutorial link on your home page.
New services catalog
You can now create services IBM Cloud services that work with Cloud Pak for Data as a Service from the new services catalog. Select Services catalog from the main menu. You can see all your services by selecting the Your services option.
See IBM Cloud services.
Integrate with other cloud platforms
You can now configure integration with other cloud platforms to simplify creating connections to data sources in those cloud platforms in projects and catalogs. Select Cloud integration from the main menu.
Automatically detect and convert date and timestamp data types (Watson Studio, Watson Knowledge Catalog)
When you open a file in Data Refinery, the Convert column type GUI operation is automatically applied as the first step if it detects any non-string data types in the data. Now date and timestamp data are detected and are automatically converted to inferred data types. You can change the automatic conversion for selected columns or undo the step. For information about the supported inferred date and timestamp formats, see Convert column type in GUI operations in Data Refinery, under the FREQUENTLY USED category.
Week ending 17 July 2020
Week ending 10 July 2020
Upcoming removal of Neural Network Modeler and SparkML modeler
Both the beta Neural Network Modeler and the beta SparkML modeler tools will be removed from Watson Studio on July 31.
Easily add data from a Cognos Analytics connection to a notebook (Watson Studio)
You can now add data from a Cognos Analytics connection by using the Insert to code function for the connection within a notebook. See Data load support.
The Lineage page is renamed to Activities (Watson Knowledge Catalog)
The Lineage page that you can see when you open a data asset in a catalog or project is now called Activities. The information shown on this page remains the same.
Week ending 12 June 2020
Perform aggregate calculations on multiple columns in Data Refinery
Now you can select multiple columns in the Aggregate operation. Previously all aggregate calculations applied to one column.
For more information, see Aggregate in GUI operations in Data Refinery, under the ORGANIZE category.
Filter values in a Boolean column in Data Refinery
You can now use these operators in the Filter GUI operation to filter Boolean (logical) data:
Is false
Is true
For more information, see Filter in GUI operations in Data Refinery, under the FREQUENTLY USED category.
In addition, a new template for filtering by Boolean values has been added to the filter coding operation.
filter(`<column>`== <logical>)
For more information about the filter templates, see Interactive code templates in Data Refinery.
Week ending 05 June 2020
SPSS Modeler flow properties (Watson Studio)
You can now set flow properties. For details, see Setting properties for flows.
Week ending 22 May 2020
AutoAI Auto-generated WML notebooks and SDK available in beta
You now have two options for saving an AutoAI pipeline as a notebook:
- WML notebook - Work with a trained model in an annotated notebook. You can review and update the code, view visualization, and deploy the model with Watson Machine Learning.
- AutoAI_lib notebook - View the Scikit-Learn source code for the trained model in a notebook. Does not require Watson Machine Learning.
Additionally, the Watson Machine Learning Python client has been extended to include an SDK for the WML notebook. For details, see Saving an AutoAI generated notebook. Note: These features are being offered as a beta and are subject to change.
Changes to the Watson Studio plans
Starting on May 19, 2020, the Watson Studio plans has changes. See Service plan changes and deprecations.
Week ending 01 May 2020
More Decision Optimization compute options (Watson Studio})
You now have more options that cost less when you run Decision Optimization jobs. You can choose from new, more powerful Decision Optimization environments. The basic Decision Optimization compute environment now consumes only five capacity unit hours (CUH) instead of 20 CUH. The new environments consume 6-13 CUH.
Read the More Decision Optimization Compute on Watson Studio at No Additional Cost blog post.
“PureData System for Analytics” connection renamed to “Netezza (PureData System for Analytics)”
The PureData System for Analytics connection is now the Netezza (PureData System for Analytics) connection. This change is to reflect the announcement of the new Netezza Performance Server for on premises and Cloud. Your previous settings for a connection to PureData System for Analytics remain the same. Only the connection name changed.
New visualization charts in Data Refinery (Watson Studio, Watson Knowledge Catalog)
Data Refinery introduces six new charts. To access the charts, click the Visualizations tab in Data Refinery, and then select the columns to visualize. The chart automatically updates as you refine the data.
-
Bubble charts display each category in the groups as a bubble.
-
Circle packing charts display hierarchical data as a set of nested areas.
-
Multi-charts display up to four combinations of Bar, Line, Pie, and Scatter plot charts. You can show the same kind of chart more than once with different data. For example, two pie charts with data from different columns.
-
Radar charts integrate three or more quantitative variables that are represented on axes (radii) into a single radial figure. Data is plotted on each axis and joined to adjacent axes by connecting lines. Radar charts are useful to show correlations and compare categorized data.
-
Theme river charts use a specialized flow graph that shows changes over time.
-
Time plot charts illustrate data points at successive intervals of time.
Week ending 24 April 2020
Synchronizing assets with Information Governance Catalog is discontinued (Watson Knowledge Catalog)
You can no longer automatically synchronize data assets between Information Governance Catalog and Watson Knowledge Catalog.
Week ending 17 April 2020
GPU environments for running notebooks are GA (Watson Studio)
GPU environments for running Jupyter notebooks with Python 3.6 are now generally available for the Watson Studio Standard and Enterprise plans. GPU environments are available in the Dallas IBM Cloud service region only.
With GPU environments, you can reduce the training time needed for compute-intensive machine learning models you create in a Jupyter notebook with Python 3.6. With more compute power, you can run more training iterations while fine-tuning your machine learning models.
See GPU environments.
Week ending 3 April 2020
Changes to Watson Studio Enterprise plan
On April 1, 2020, the Watson Studio Enterprise plan has changes. See Service plan changes and deprecations.
Week ending 27 March 2020
AutoAI Auto-generated notebooks available in beta
Save an AutoAI pipeline as a notebook so you can view all of the transformations that went into creating the pipeline. Use the autoai-lib reference as a guide to the transformations. This feature is being offered as a beta and is subject to change. For details, see Saving an AutoAI generated notebook.
Week ending 20 March 2020
Upcoming changes to Watson Machine Learning GPU plans
Starting on May 1, 2020, Watson Machine Learning will update the capacity units per hour for GPU capacity types. See Service plan changes and deprecations.
Week ending 13 March 2020
Custom security policies available for restricting downloads (Watson Machine Learning)
By default, Watson Machine Learning does not restrict external sites users can access as part of operations such as downloading data source files or installing Python library packages. If you would like to limit access to a list of approved sites, contact IBM Cloud support to request a custom network policy for your organization.
New capabilities in AutoAI Watson Machine Learning
The following features are new or enhanced in AutoAI:
- The limit on the size of a data source for an AutoAI experiment is increased from 100 MB to 1 GB.
- The number of pipelines generated for an experiment is increasing from four to eight, based on the two top performing algorithms. You can now also increase the number of top performing algorithms to use for generating pipelines if you want to view and compare more pipelines. Each algorithm creates four optimized pipelines. For details, see Building an AutoAI model.
Week ending 06 March 2020
Updates to Watson Machine Learning frameworks
Support is now available for TensorFlow 1.15 and Keras version 2.2.5 for training and deploying models. See Service plan changes and deprecations.
Week ending 07 February 2020
New Spark and R runtime enabled for jobs in Data Refinery
You can now select Default Spark 2.4 & R 3.6
when you select an environment template for a new job. The new runtime uses the same capacity unit hours (CUHs) as the Default Spark R 3.4
(which is Spark 2.3) runtime.
SAV files
SPSS Statistics .sav data files are now supported for import or export in SPSS Modeler.
Exercise more control over pipeline creation for an AutoAI experiment
Week ending 10 January 2020
"Hortonworks HDFS" connection renamed to "Apache HDFS"
The Hortonworks HDFS connection is now the Apache HDFS via the WebHDFS API connection. Your previous settings for connections to Hortonworks HDFS remain the same. Only the connection name has changed.
Geospatio-temporal library for notebooks (Watson Studio)
You can use the geospatio-temporal library to expand your data science analysis to include location analytics in your Python notebooks that run with Spark. See Using the geospatio-temporal library.
Older what's new entries
Parent topic: What's new