2019 What’s New

Here are the new features for Watson Studio, Watson Machine Learning, and Watson Knowledge Catalog for the year 2019.

Week ending 13 December 2019

“Object Storage OpenStack Swift (Infrastructure)” connection is discontinued

Support for the Object Storage OpenStack Swift (Infrastructure) connection is discontinued. The Object Storage OpenStack Swift (Infrastructure) connection is no longer in the user interface.

Week ending 6 December 2019

Updates to supported frameworks for Watson Machine Learning

Support is now available for PyTorch version 1.1. For the complete list of supported frameworks, see this topic.

Due to security vulnerabilities with several TensorFlow versions, Watson Machine Learning has added support for TensorFlow version 1.14 as well as 1.13 and has removed support for all unsecure TensorFlow versions, including 1.5 and 1.11. For details, read the blog announcement. For help changing to a supported runtime, see the TensorFlow compatibility guide.

Synthesized Neural Networks (NeuNetS) beta tool removed

Synthesized Neural Networks (NeuNetS) model building tool is removed from Watson Studio until it gets merged with AutoAI. For details, see this blog post.

Week ending 22 November 2019

Synthesized Neural Networks (NeuNetS) merging with AutoAI

In 2020, the Synthesized Neural Networks (NeuNetS) model building tool (currently in beta) will be merged with AutoAI for a unified, automated model-building experience. Starting on December 6, 2019, the NeuNetS tool will be removed from the Watson Studio interface until the merge is complete. Please remove your NeuNetS models prior to that date and migrate them to newer versions of Keras models. For details on the merge of NeuNetS with AutoAI, see this blog post.

Data Refinery removes a restriction on source data

  • Column names can now include periods.

Week ending 15 November 2019

Multibyte character support

Multibyte characters are now fully supported in these areas of Watson Knowledge Catalog:

  • The names or descriptions of data policies, rules, or categories.
  • The names, business definitions, or descriptions of business glossary terms.
  • Asset tags.

However, Data Refinery does not support multibyte characters in user-input fields and some fields allow multibyte characters but do not display them correctly.

Week ending 8 November 2019

AutoAI enhancements

New features in AutoAI give you greater control over how your model pipelines are generated and greater insight into the automated process. For example, a new visualization shows you the relationships between pipelines as well as what makes each one unique. New experiment settings let you choose specific algorithms for AutoAI to consider for model selection. You can also exercise more control over how your data is used to train the pipelines. For details, see Building an AutoAI model.

Week ending 1 November 2019

Watson Machine Learning support for TensorFlow 1.14

Due to security vulnerabilities with several TensorFlow versions, Watson Machine Learning has added support for TensorFlow version 1.14 as well as 1.13 and is deprecating support for all unsecure TensorFlow versions, including 1.5 and 1.11. For details, read the blog announcement. For help changing to a supported runtime, see the TensorFlow compatibility guide.

Week ending 18 October 2019

Object detection tool

The Object Detection tool for the Visual Recognition service is now generally available. To view a video that introduces object detection, see Custom object detection models.

Week ending 04 October 2019

Data Refinery automatically detects and converts data types

Previously when you opened a file in Data Refinery, for most file types all the columns were interpreted as the string data type. Now the Convert column type GUI operation is automatically applied as the first step in the Data Refinery flow. The operation automatically detects and converts the data types to inferred data types (for example, to Integer, Boolean, etc.) as needed. This enhancement will save you a lot of time, particularly if the data has many columns. It is easy to undo the automatic conversion or to edit the operation for selected columns.

Automatic detection and conversion of data type

For more information, see Convert column type in GUI operations in Data Refinery, under the FREQUENTLY USED category.

Select the runtime for a Data Refinery flow with the Jobs interface

Previously you could select the default runtime for a Data Refinery flow in the DATA REFINERY FLOW DETAILS pane in Data Refinery (accessed from the Info pane Details tab). The SELECT RUNTIME selection has been removed. Instead, select the runtime when you save a job to run the Data Refinery flow. The runtime for any previously scheduled jobs remains unchanged. For information about jobs, see Jobs in a project.

Confirm the stop words removed in a Data Refinery flow

Use the Tokenize GUI operation to test the words you remove from a selected column with the Remove stop words GUI operation. For information, see Remove stop words in GUI operations in Data Refinery, under the NATURAL LANGUAGE category.

Week ending 20 September 2019

Decision optimization available on all plans

Decision Optimization is now available on the Standard plan as well as the Lite and Enterprise plans. For details, see the announcement.

Removal of Apache Spark as a Service

If you were using Spark as a Service Enterprise plan or Lite plan from Watson Studio, you must switch to using built-in Spark environments. Spark as a Service is no longer supported.

For details, read this blog post: Deprecation of Apache Spark (Lite Plan).

Use built-in Spark environments instead. See Spark environments.

Week ending 6 September 2019

Changes to the Community

The Watson Studio Community is now split into two sites to better serve your needs:

  • The Gallery contains sample data sets, notebooks, and projects that you can add to Watson Studio directly. You can access the Gallery from the main menu.
  • The Community contains articles, blog posts, tutorials, events, and discussions. You can access it here.

Week ending 30 August 2019

Decision Optimization model builder beta

The Decision Optimization model builder is now in beta. With the Decision Optimization model builder, you can create several scenarios, using different data sets and optimization models. This allows you to create and compare different scenarios and see how big an impact changes can have on a given problem.

The model builder helps you:

  • Select and edit the data relevant for your optimization problem.
  • Run optimization models
  • Investigate and compare solutions for multiple scenarios.
  • Create, import, edit, and solve Python and OPL models.
  • Import and export Python models to and from Jupyter notebooks.
  • Easily create and share reports with tables, charts and notes using widgets provided in the visualization editor.

See Decision Optimization.

Support for R 3.6 and deprecation of R 3.4

You can now use R 3.6 runtimes in Watson Studio for notebooks and AutoAI. Support for R 3.4 in Watson Studio is ending on October 30, 2019. When you upgrade a notebook from R 3.4 to R 3.6, you might need to make code changes because some open source libraries versions might be different.

Read the announcement.

Reminder: Support ending for Python versions 3.5 and 2.7

Support for Python versions 3.5 and 2.7 in Watson Studio ended on August 28, 2019. Support in Watson Machine Learning is ending on September 9, 2019. If you have not already done so, migrate your assets and models to run with Python version 3.6. For more information, see the announcements for Watson Studio and Watson Machine Learning.

Week ending 16 August 2019

Open beta for Object Detection service

A new component for the Watson Studio Visual Recognition service lets you build a model that can identify objects within images. For details, see Creating custom Object Detection models.

Reminder: Support ending for Python versions 3.5 and 2.7

Support for Python versions 3.5 and 2.7 is ending on August 28, 2019. If you have not already done so, migrate your assets and models to run with Python version 3.6. For more information, see the announcements for Watson Studio and Watson Machine Learning.

Support for TensorFlow 1.13

Due to security vulnerabilities with several TensorFlow versions, Watson Machine Learning has added support for TensorFlow version 1.13 and is deprecating support for all unsecure TensorFlow versions, including 1.5 and 1.11. For details, read the blog announcement.

Change to Watson Machine Learning V4 API date/time format

The date/time format returned from the Watson Machine Learning version 4 API has changed. This change will impact users who are using the V4 API-supported Watson Machine Learning Python client for creating deployments or jobs and parsing the date/time fields in deployment or jobs-related metadata.

The date format previously returned in a GET response of /v4/deployments was:

yyyy-MM-dd’T’HH:mm:ssZZZZ

The new format is:

yyyy-MM-dd’T’HH:mm:ss.SSS’Z’

Faster SPSS Modeler flows

SPSS Modeler flows now run faster because their environment runtime is more powerful. The environment runtime for running SPSS Modeler flows is now 4 vCPU and 16 GB RAM instead of 2 vCPU and 8 GB RAM. The new environment runtime consumes 2 capacity units per hour.

RStudio XXS environment runtime removed

The smallest RStudio environment runtime, Default RStudio XXS, with 1 vCPU and 5 GB RAM, is no longer available. Use the more powerful RStudio environment runtimes.

Week ending 09 August 2019

Deadline for migrating notebook schedules to jobs extended to August 30

You now have until Friday, August 30, 2019 to migrate your notebook schedules to the new jobs interface.

Week ending 02 August 2019

End of support for Watson Machine Learning JSON Token Authentication service

Deprecation of the Watson Machine Learning JSON Token Authentication service was announced on April 23, 2019. If you interact with the Watson Machine Learning service programmatically, via API, Python client, or command line interface, you should be using IBM Cloud VCAP credentials, as described in Watson Machine Learning authentication.

Retirement of Watson Machine Learning Model Builder

Watson Machine Learning Model Builder is no longer available for training machine learning models. Models trained with Model Builder and deployed to Watson Machine Learning will continue to be supported, but no new models can be trained using Model Builder. Instead, use AutoAI for training classification and regression models. Read about the announcement, or learn more about AutoAI.

Reminder: Data Refinery schedules will be discontinued August 12

You must migrate Data Refinery flow schedules to the new jobs interface before August 12, 2019.

Week ending 26 July 2019

General availability for Decision Optimization in notebooks

Decision Optimization is now generally available in Watson Studio notebooks when you select Python runtime environments. See Notebook environments.

AutoAI updates

These enhancements are new for AutoAI:

  • The data source you use to create an AutoAI model can now be output from Watson Studio Data Refinery.
  • After adding data to the AutoAI Experiment builder, you can preview the data without leaving the tool. You can also adust the percentage of data that is held out to test the performance of the model, from 0 to thirty percent.
  • For binary classification models, you can edit the positive class.

For details on these updates, see Building an AutoAI model

Details on how feature engineering transformations are applied are documented in AutoAI implementation details

Week ending 19 July 2019

Switch to Python 3.6 environments

The default Python environment version in Watson Studio is now 3.6. Python 2.7 and 3.5 are being deprecated and will no longer be available after August 28, 2019. When you switch from Python 3.5 or 2.7 to Python 3.6, you might need to update your code if the versions of open source libraries that you use are different in Python 3.6. See Changing the environment.

Read this blog post: Python version upgrade in Watson Studio Cloud

Use a form to test an AutoAI model deployment

You can now test a deployed AutoAI model using an input form as an alternative to entering JSON code. Enter values the form fields, then click Predict to see the prediction.
Prediction from test data

For details see Deploying an AutoAI model

Simplified project creation

When you create a project, you can now choose from creating an empty project and creating a project from a file or a sample. All tools are available in all projects.

See Set up a project and Importing a project.

Add dashboards to project export file

You can now include dashboards when you export a project ZIP file to your desktop. See Exporting projects.

RStudio environments

When you launch RStudio in a Watson Studio project, you can now select the RStudio environment runtime in which to launch RStudio by hardware size. For information about the RStudio environments, see RStudio jobs in a project.

Week ending 12 July 2019

New jobs user interface for running and scheduling Data Refinery flows and scheduling notebooks

The jobs user interface provides a new way to run or schedule a Data Refinery flow or to schedule a notebook. From the project page, click the Jobs tab to view all the jobs in a project and their run details. Now you can create multiple jobs for the same asset, for example a job with different runtimes or different schedules. You can also create a job from Data Refinery or a notebook. For information about jobs, see Jobs in a project.

Important
You must manually migrate your current Data Refinery flow schedules to the new jobs interface before August 12, 2019. Migrate your notebook schedules before August 30, 2019.

New default runtime for Data Refinery

The new default runtime for Data Refinery is Default Data Refinery XS. Any Data Refinery flow runs previously set to None - Use Data Refinery flow Default will now use this new runtime. Like the Spark R 3.4 runtime, the Default Data Refinery XS runtime is HIPAA ready.

Default Data Refinery XS environment

You can also select this runtime when you create a job.

Default Data Refinery XS in a job

See Data Refinery environments.

Working in Data Refinery consumes CUH

When you create or edit a Data Refinery flow, the runtime consumes capacity units per hour. The runtime automatically stops after an hour of inactivity.
Important: You can manually stop the runtime on the Environments page in your project to stop consuming CUH. See Data Refinery environments.

New way to open a Data Refinery flow from the project page

To access a Data Refinery flow from the project’s Assets page, click the Data Refinery flow’s name. (Previously, you accessed the Data Refinery flow from a Refine option under the ACTIONS menu.)

Changing the source of a Data Refinery flow is now in the Data Refinery steps

To change the source of a Data Refinery flow, click the edit icon next to Data Source in the Steps pane. (Previously, you changed the source from the Summary page.)

Edit source

As before, for best results, the new data set should have a schema that is compatible to the original data set (for example, column names, number of columns, and data types). If the new data set has a different schema, operations that won’t work with the schema will show errors. You can edit or delete the operations, or change the source to one that has a more compatible schema.

Project readme included in project export and import

Before you export a project, you can add a brief description of the analytics use case of the included assets and the applied analysis methods to the project readme and this readme is now included in the project export. When you import a project, you can check the readme for a short description of the project’s intent on the project’s Overview page.

Watson Analytics connector is discontinued

The Watson Analytics connector has been removed from the list of data sources on the New connections page.

Snowflake connector available

Projects and catalogs now support connections to a Snowflake database, enabling you to store and retrieve data there.

Week ending 05 July 2019

Creating a project from a sample

If you are new to Watson Studio and are looking for how to use data assets in tools, such as notebooks to prepare data, analyze data, build and train models, and vizualize analysis results, you can now create a project from a sample. See Importing a project from a sample.

Week ending 28 June 2019

How to choose a tool in Watson Studio

You can now find which tool you need to use by matching your type of data, what you want to do with your data, and how much automation you want.

See Choosing a tool.

Streams flow

MQTT source operator, in addition to the message, now also provides the metadata attribute event_topic, for each event.

Week ending 21 June 2019

Decision Optimization

Decision Optimization gives you access to IBM’s world-leading solution engines for mathematical programming and constraint programming. Use this sophisticated prescriptive analytics technology, which can explore a huge range of possible solutions before suggesting the best way to respond to a present or future situation. With Decision Optimization, you can:

  • Start with a business problem, such as planning, scheduling, pricing, inventory, or resource management.
  • Create an optimization model, which is the mathematical formulation of the problem that can be interpreted and solved by an optimization engine. The optimization model plus the input data creates a scenario instance.
  • Run the Decision Optimization engine (or solver) to find a solution, a set of decisions that achieves the best values of the goals and respects limits and constraints imposed. Metrics measure the quality of the solution in terms of the business goals.
  • Use Watson Machine Learning to deploy the solution and make it available to business users via a business application. Usually, the solution and goals are summarized in tabular or graphical views that provide understanding and insight.

For details on creating a prescriptive analytics model, see Decision Optimization.

For details on deploying a solutions, see Decision Optimization Deployment.

AutoAI experiments

  • You can now create an AutoAI experiment from a sample file so you can see how AutoAI analyzes and transforms data, then creates model candidate pipelines for you to review and compare without having to upload your own data.
  • Follow the steps in Creating an AutoAI experiment from sample data to learn how to deploy and score a model created from the Bank marketing sample data set.
  • AutoAI models saved as Watson Machine Learning assets are now only available in the project in which they were created. AutoAI models created prior to this update are available in other projects that share the same machine learning instance.

Preview of new Python client library with Watson Machine Learning v4 API

A new version of the Python client library is in the works to support new features in Watson Machine Learning. This Python client, built on version 4 of the Watson Machine Learning APIs, is available as a preview to support Decision Optimization and AutoAI experiments.

For details on installing the new Python client and accessing the associated documentation, see Python client.

Streams flow

In addition to the JSON message, Watson IoT source operator now also provides the following metadata attributes for each event: event_typeId, event_deviceId, and event_eventId.

Week ending 07 June 2019

Cognos Analytics connector available

Projects and catalogs now support connections to Cognos Analytics, enabling you to store and retrieve data there.

Enhancements to AutoAI Experiments

A new tutorial walks you through the process of building, deploying, and scoring a binary classification model using AutoAI.

The following enhancements make it easier for you to review AutoAI model pipelines:

  • Updated designs make it faster for you to see the details for a pipeline
  • When you expand a pipeline in the leaderboard to review details you can now view hold-out and training data scores

Week ending 31 May 2019

Build models with AutoAI

AutoAI in Watson Studio automatically analyzes your data, selects a model type, applies estimators, and generates candidate model pipelines customized for your predictive modeling problem. Results are dislayed in a leaderboard, which is a table showing the list of automatically generated candidate models, as pipelines, ranked according to the specified criteria.

AutoAI provides a view into the estimators and hyper-parameter optimization applied to each pipeline so you can have confidence in how each pipeline is generated. After viewing and comparing pipelines, you can save one as a model that can be deployed and tested. For details on building models using AutoAI, see AutoAI overview

Week ending 24 May 2019

New SPSS modeler nodes

You can use these new nodes in SPSS Modeler:

  • Reprojection node: A field operations node that changes the coordinate system of fields in the geographical coordinate system to the projected coordinate system. See Reproject node.
  • Space-Time-Boxes node: A record operations node that represents a regularly shaped region of space and time as an alphanumeric string. See Space-Time-Boxes node.
  • Spatio-Temporal Prediction node: A modeling node that provides statistical techniques that you can use to forecast future values at different locations and explicitly model adjustable factors to perform what-if analysis. See Spatio-Temporal Prediction node.

New right-click option for SPSS Modeler nodes

Previously, when you right-clicked a node and selected Preview, a Data tab, Profile tab, and Visualizations tab opened—allowing you to examine your flow’s data in various ways. Now when you select Preview, you get a snapshot of your data that loads more quickly. Use the new right-click option called Profile to work with the full features such as the Visualizations tab.

Profile and Preview selections

Week ending 17 May 2019

Spark runtimes for Data Refinery flows

You can now use Spark runtimes with Data Refinery:

  • Spark R 3.4 environments for running Data Refinery flows are now generally available and are HIPAA ready. When you run a Data Refinery flow, you can select to use the preset Default Spark R 3.4 environment or configure your own Spark environment with the hardware size you need for your workload. You can’t select a Spark R 3.4 environment to schedule a Data Refinery flow run.

    Spark environment selection in the Data Refinery flow details page

    For information and instructions, see Spark environments for Data Refinery.

  • The None - Use Data Refinery Default runtime is deprecated. However, you can still use this runtime to run Data Refinery flows that operate on small data sets and to schedule Data Refinery flow runs.

Week ending 10 May 2019

Streams flow

  • Added the Binary data type to support use cases, such as processing, scoring and classifying binary data, images, video, and audio using pre-trained machine learning models.
  • Option of ingesting raw data without built-in JSON parsing in Event Streams, Kafka, and HTTP source operators.
    • Ability to have your own custom parsing by appending a Code operator.
    • Ability to ingest binary data (for example, images, video, and audio).
  • Option of writing raw data without built-in formatting as JSON, in Event Streams and Kafka target operators.
    • Ability to have your own custom formatting by inserting a Code operator.
    • Ability to write binary data (for example, images, video, and audio).
  • Optional metadata attributes in Event Streams and Kafka source operators: event_topic, event_offset, event_timestamp, event_partition.

Week ending 3 May 2019

Watson Studio Local 2.0 GA

Watson Studio Local 2.0 is now generally available for when you need the functionality of Watson Studio on your private cloud. See Watson Studio Local 2.0 documentation.

For a comparison of features between different deployment environments of Watson Studio, Watson Machine Learning, and Watson Knowledge Catalog, see Feature differences between deployment environments.

Deprecation of Apache Spark Lite service

You can no longer associate an Apache Spark Lite service with a project. Apache Spark Lite services will be deleted on June 28, 2019. Read this blog post: Deprecation of Apache Spark (Lite Plan).

Use built-in Spark environments instead. See Spark environments.

Week ending 25 April 2019

Migration of Object Storage OpenStack Swift projects

You might not be able to download assets from any remaining projects that use Object Storage OpenStack Swift. If you have trouble downloading data from a project with Object Storage OpenStack Swift so that you can migrate the project to IBM Cloud Object Storage, open a ticket with IBM Support.

Attribute classifiers renamed to data classes

The attribute classifiers that characterize the contents of columns in relational data assets are now called data classes. Data classes are automatically assigned to columns during profiling. See Profiles.

Week ending 12 April 2019

Export a project to desktop

You can now share project assets with others by exporting your project. The project assets that you select are downloaded as a project ZIP file to your desktop.

Import a project from desktop

When you create a project, you can select the import project starter that enables importing assets from another project to use in a new project.

Improved search filtering in Watson Knowledge Catalog

When you search for assets in a catalog, the search filters are now directly below the search field. You can filter on tags, business terms that are assigned to data assets, and asset types. The list of tags is now sorted alphabetically.

Searching for assets in a catalog with filters

Week ending 5 April 2019

Data Refinery: Column-action operations remain focused on the column

When you click a column in Data Refinery and select an operation from the actions menu (three vertical dots), the focus remains on that column. Previously the focus always shifted to the first column. This enhancement is especially useful when you are working on wide tables.

Week ending 22 March 2019

Import a streams flow from URL

You can now also import a streams flow from a URL, in addition to importing from a file.

Week ending 15 March 2019

Google Cloud Storage connector available

Projects and catalogs now support connections to Google Cloud Storage, enabling you to store and retrieve data there.

Week ending 1 March 2019

Migrate Watson Studio from Cloud Foundry to a resource group

You can migrate your Watson Studio service instance from a Cloud Foundry org and space to a resource group in IBM Cloud. Resource groups include finer-grained access control by using IBM Cloud Identity and Access Management (IAM), the ability to connect service instances to apps and service across different regions, and an easy way to view usage per group.

For instructions, see IBM Cloud: Migrating Cloud Foundry service instances to a resource group.

Secure Gateway service for Watson Studio instances in the Tokyo region

The Secure Gateway service is not yet available in the Tokyo (AP-North) service region on IBM Cloud. However, you can now provision a Secure Gateway service in any other region and use it when you create a connection in a Watson Studio instance from any region, including Tokyo.

Assign terms and tags to columns in Watson Knowledge Catalog

You can now assign business terms and tags to columns in relational data assets in catalogs. See Editing assets in catalogs.

Publish and subscribe messages to topics with the Streams operator in streams flow

In streams flows, you can now subscribe to topics with the Source opearator and publish to topics with the Target operator in the Streaming Analytics service.

Week ending 22 February 2019

Data Refinery flow runs now consume CUHs which are tracked

The capacity unit hours (CUHs) that are consumed when you run a Data Refinery flow in a Spark R 3.4 environment are now tracked.

For information, see Spark environments for Data Refinery.

Week ending 8 February 2019

Run Data Refinery flows in a Spark R environment (open beta)

You can now select a Spark R environment for your Data Refinery flows. You can select the Default Spark R 3.4 environment or you can create your own Spark R environment definition that is customized for the size of your data set. Each Data Refinery flow runs in a dedicated Spark cluster.

Select the Spark environment from the Data Refinery flow details page when you save and run the flow.

Spark environment selection in the Data Refinery flow details page

For information and instructions, see Spark environments for Data Refinery.

Week ending 1 February 2019

New navigation menu

You can easily find all your menu options in one place on the new navigation menu. Click The navigation menu icon to expand the menu.

The navigation menu

HIPAA readiness for Watson Studio and Watson Machine Learning

Watson Studio and Watson Machine Learning meet the required IBM controls that are commensurate with the Health Insurance Portability and Accountability Act of 1996 (HIPAA) Security and Privacy Rule requirements. HIPAA readiness applies to only certain plans and regions.

See HIPAA readiness and read this blog.

Week ending 25 January 2019

New SPSS modeler node

SPSS Modeler flows now support the Set Globals node to compute summary values for CLEM expressions. See SPSS Modeler nodes.

Python model operator now supports Watson Machine Learning in streams flows

In addition to loading models from IBM Cloud Object Storage, the Python model operator now also supports loading models from the Watson Machine Learning service.

Week ending 18 January 2019

SPSS Modeler flows now support more of the SPSS Modeler desktop nodes.

There are more modeling nodes. Check it out.

Week ending 11 January 2019

Easy upgrade with the Upgrade button

When you’re ready to upgrade your Watson Studio, Watson Knowledge Catalog, or Watson Machine Learning service from a Lite plan, just click the Upgrade button. You’ll be guided through the upgrade in just a few clicks. See Upgrade your Watson services.

Streams flows now support User Defined Parallelism (UDP) for Event Streams

With User Defined Parallelism, multiple workers help to increase the ingestion rate for topics with multiple partitions.