2019 What’s New
Here are the new features for Watson Studio, Watson Machine Learning, and Watson Knowledge Catalog for the year 2019.
Week ending 13 December 2019
“Object Storage OpenStack Swift (Infrastructure)” connection is discontinued
Support for the Object Storage OpenStack Swift (Infrastructure) connection is discontinued. The Object Storage OpenStack Swift (Infrastructure) connection is no longer in the user interface.
Week ending 6 December 2019
Updates to supported frameworks for Watson Machine Learning
Support is now available for PyTorch version 1.1. For the complete list of supported frameworks, see this topic.
Due to security vulnerabilities with several TensorFlow versions, Watson Machine Learning has added support for TensorFlow version 1.14 as well as 1.13 and has removed support for all unsecure TensorFlow versions, including 1.5 and 1.11. For details, read the blog announcement. For help changing to a supported runtime, see the TensorFlow compatibility guide.
Synthesized Neural Networks (NeuNetS) beta tool removed
Synthesized Neural Networks (NeuNetS) model building tool is removed from Watson Studio until it gets merged with AutoAI. For details, see this blog post.
Week ending 22 November 2019
Synthesized Neural Networks (NeuNetS) merging with AutoAI
In 2020, the Synthesized Neural Networks (NeuNetS) model building tool (currently in beta) will be merged with AutoAI for a unified, automated model-building experience. Starting on December 6, 2019, the NeuNetS tool will be removed from the Watson Studio interface until the merge is complete. Please remove your NeuNetS models prior to that date and migrate them to newer versions of Keras models. For details on the merge of NeuNetS with AutoAI, see this blog post.
Data Refinery removes a restriction on source data
- Column names can now include periods.
Week ending 15 November 2019
Multibyte character support
Multibyte characters are now fully supported in these areas of Watson Knowledge Catalog:
- The names or descriptions of data policies, rules, or categories.
- The names, business definitions, or descriptions of business glossary terms.
- Asset tags.
However, Data Refinery does not support multibyte characters in user-input fields and some fields allow multibyte characters but do not display them correctly.
Week ending 8 November 2019
New features in AutoAI give you greater control over how your model pipelines are generated and greater insight into the automated process. For example, a new visualization shows you the relationships between pipelines as well as what makes each one unique. New experiment settings let you choose specific algorithms for AutoAI to consider for model selection. You can also exercise more control over how your data is used to train the pipelines. For details, see Building an AutoAI model.
Week ending 1 November 2019
Watson Machine Learning support for TensorFlow 1.14
Due to security vulnerabilities with several TensorFlow versions, Watson Machine Learning has added support for TensorFlow version 1.14 as well as 1.13 and is deprecating support for all unsecure TensorFlow versions, including 1.5 and 1.11. For details, read the blog announcement. For help changing to a supported runtime, see the TensorFlow compatibility guide.
Week ending 18 October 2019
Object detection tool
The Object Detection tool for the Visual Recognition service is now generally available. To view a video that introduces object detection, see Custom object detection models.
Week ending 04 October 2019
Data Refinery automatically detects and converts data types
Previously when you opened a file in Data Refinery, for most file types all the columns were interpreted as the string data type. Now the Convert column type GUI operation is automatically applied as the first step in the Data Refinery flow. The operation automatically detects and converts the data types to inferred data types (for example, to Integer, Boolean, etc.) as needed. This enhancement will save you a lot of time, particularly if the data has many columns. It is easy to undo the automatic conversion or to edit the operation for selected columns.
For more information, see Convert column type in GUI operations in Data Refinery, under the FREQUENTLY USED category.
Select the runtime for a Data Refinery flow with the Jobs interface
Previously you could select the default runtime for a Data Refinery flow in the DATA REFINERY FLOW DETAILS pane in Data Refinery (accessed from the Info pane Details tab). The SELECT RUNTIME selection has been removed. Instead, select the runtime when you save a job to run the Data Refinery flow. The runtime for any previously scheduled jobs remains unchanged. For information about jobs, see Jobs in a project.
Confirm the stop words removed in a Data Refinery flow
Use the Tokenize GUI operation to test the words you remove from a selected column with the Remove stop words GUI operation. For information, see Remove stop words in GUI operations in Data Refinery, under the NATURAL LANGUAGE category.
Week ending 20 September 2019
Decision optimization available on all plans
Decision Optimization is now available on the Standard plan as well as the Lite and Enterprise plans. For details, see the announcement.
Removal of Apache Spark as a Service
If you were using Spark as a Service Enterprise plan or Lite plan from Watson Studio, you must switch to using built-in Spark environments. Spark as a Service is no longer supported.
For details, read this blog post: Deprecation of Apache Spark (Lite Plan).
Use built-in Spark environments instead. See Spark environments.
Week ending 6 September 2019
Changes to the Community
The Watson Studio Community is now split into two sites to better serve your needs:
- The Gallery contains sample data sets, notebooks, and projects that you can add to Watson Studio directly. You can access the Gallery from the main menu.
- The Community contains articles, blog posts, tutorials, events, and discussions. You can access it here.
Week ending 30 August 2019
Decision Optimization model builder beta
The Decision Optimization model builder is now in beta. With the Decision Optimization model builder, you can create several scenarios, using different data sets and optimization models. This allows you to create and compare different scenarios and see how big an impact changes can have on a given problem.
The model builder helps you:
- Select and edit the data relevant for your optimization problem.
- Run optimization models
- Investigate and compare solutions for multiple scenarios.
- Create, import, edit, and solve Python and OPL models.
- Import and export Python models to and from Jupyter notebooks.
- Easily create and share reports with tables, charts and notes using widgets provided in the visualization editor.
Support for R 3.6 and deprecation of R 3.4
You can now use R 3.6 runtimes in Watson Studio for notebooks and AutoAI. Support for R 3.4 in Watson Studio is ending on October 30, 2019. When you upgrade a notebook from R 3.4 to R 3.6, you might need to make code changes because some open source libraries versions might be different.
Read the announcement.
Reminder: Support ending for Python versions 3.5 and 2.7
Support for Python versions 3.5 and 2.7 in Watson Studio ended on August 28, 2019. Support in Watson Machine Learning is ending on September 9, 2019. If you have not already done so, migrate your assets and models to run with Python version 3.6. For more information, see the announcements for Watson Studio and Watson Machine Learning.
Week ending 16 August 2019
Open beta for Object Detection service
A new component for the Watson Studio Visual Recognition service lets you build a model that can identify objects within images. For details, see Creating custom Object Detection models.
Reminder: Support ending for Python versions 3.5 and 2.7
Support for Python versions 3.5 and 2.7 is ending on August 28, 2019. If you have not already done so, migrate your assets and models to run with Python version 3.6. For more information, see the announcements for Watson Studio and Watson Machine Learning.
Support for TensorFlow 1.13
Due to security vulnerabilities with several TensorFlow versions, Watson Machine Learning has added support for TensorFlow version 1.13 and is deprecating support for all unsecure TensorFlow versions, including 1.5 and 1.11. For details, read the blog announcement.
Change to Watson Machine Learning V4 API date/time format
The date/time format returned from the Watson Machine Learning version 4 API has changed. This change will impact users who are using the V4 API-supported Watson Machine Learning Python client for creating deployments or jobs and parsing the date/time fields in deployment or jobs-related metadata.
The date format previously returned in a GET response of /v4/deployments was:
The new format is:
Faster SPSS Modeler flows
SPSS Modeler flows now run faster because their environment runtime is more powerful. The environment runtime for running SPSS Modeler flows is now 4 vCPU and 16 GB RAM instead of 2 vCPU and 8 GB RAM. The new environment runtime consumes 2 capacity units per hour.
RStudio XXS environment runtime removed
The smallest RStudio environment runtime, Default RStudio XXS, with 1 vCPU and 5 GB RAM, is no longer available. Use the more powerful RStudio environment runtimes.
Week ending 09 August 2019
Deadline for migrating notebook schedules to jobs extended to August 30
You now have until Friday, August 30, 2019 to migrate your notebook schedules to the new jobs interface.
Week ending 02 August 2019
End of support for Watson Machine Learning JSON Token Authentication service
Deprecation of the Watson Machine Learning JSON Token Authentication service was announced on April 23, 2019. If you interact with the Watson Machine Learning service programmatically, via API, Python client, or command line interface, you should be using IBM Cloud VCAP credentials, as described in Watson Machine Learning authentication.
Retirement of Watson Machine Learning Model Builder
Watson Machine Learning Model Builder is no longer available for training machine learning models. Models trained with Model Builder and deployed to Watson Machine Learning will continue to be supported, but no new models can be trained using Model Builder. Instead, use AutoAI for training classification and regression models. Read about the announcement, or learn more about AutoAI.
Reminder: Data Refinery schedules will be discontinued August 12
You must migrate Data Refinery flow schedules to the new jobs interface before August 12, 2019.
Week ending 26 July 2019
General availability for Decision Optimization in notebooks
Decision Optimization is now generally available in Watson Studio notebooks when you select Python runtime environments. See Notebook environments.
These enhancements are new for AutoAI:
- The data source you use to create an AutoAI model can now be output from Watson Studio Data Refinery.
- After adding data to the AutoAI Experiment builder, you can preview the data without leaving the tool. You can also adust the percentage of data that is held out to test the performance of the model, from 0 to thirty percent.
- For binary classification models, you can edit the positive class.
For details on these updates, see Building an AutoAI model
Details on how feature engineering transformations are applied are documented in AutoAI implementation details
Week ending 19 July 2019
Switch to Python 3.6 environments
The default Python environment version in Watson Studio is now 3.6. Python 2.7 and 3.5 are being deprecated and will no longer be available after August 28, 2019. When you switch from Python 3.5 or 2.7 to Python 3.6, you might need to update your code if the versions of open source libraries that you use are different in Python 3.6. See Changing the environment.
Read this blog post: Python version upgrade in Watson Studio Cloud
Use a form to test an AutoAI model deployment
You can now test a deployed AutoAI model using an input form as an alternative to entering JSON code. Enter values the form fields, then click Predict to see the prediction.
For details see Deploying an AutoAI model
Simplified project creation
When you create a project, you can now choose from creating an empty project and creating a project from a file or a sample. All tools are available in all projects.
Add dashboards to project export file
You can now include dashboards when you export a project ZIP file to your desktop. See Exporting projects.
When you launch RStudio in a Watson Studio project, you can now select the RStudio environment runtime in which to launch RStudio by hardware size. For information about the RStudio environments, see RStudio jobs in a project.
Week ending 12 July 2019
New jobs user interface for running and scheduling Data Refinery flows and scheduling notebooks
The jobs user interface provides a new way to run or schedule a Data Refinery flow or to schedule a notebook. From the project page, click the Jobs tab to view all the jobs in a project and their run details. Now you can create multiple jobs for the same asset, for example a job with different runtimes or different schedules. You can also create a job from Data Refinery or a notebook. For information about jobs, see Jobs in a project.
You must manually migrate your current Data Refinery flow schedules to the new jobs interface before August 12, 2019. Migrate your notebook schedules before August 30, 2019.
New default runtime for Data Refinery
The new default runtime for Data Refinery is
Default Data Refinery XS. Any Data Refinery flow runs previously set to
None - Use Data Refinery flow Default will now use this new runtime. Like the
Spark R 3.4 runtime, the
Default Data Refinery XS runtime is HIPAA ready.
You can also select this runtime when you create a job.
Working in Data Refinery consumes CUH
When you create or edit a Data Refinery flow, the runtime consumes capacity units per hour. The runtime automatically stops after an hour of inactivity.
Important: You can manually stop the runtime on the Environments page in your project to stop consuming CUH. See Data Refinery environments.
New way to open a Data Refinery flow from the project page
To access a Data Refinery flow from the project’s Assets page, click the Data Refinery flow’s name. (Previously, you accessed the Data Refinery flow from a Refine option under the ACTIONS menu.)
Changing the source of a Data Refinery flow is now in the Data Refinery steps
To change the source of a Data Refinery flow, click the edit icon next to Data Source in the Steps pane. (Previously, you changed the source from the Summary page.)
As before, for best results, the new data set should have a schema that is compatible to the original data set (for example, column names, number of columns, and data types). If the new data set has a different schema, operations that won’t work with the schema will show errors. You can edit or delete the operations, or change the source to one that has a more compatible schema.
Project readme included in project export and import
Before you export a project, you can add a brief description of the analytics use case of the included assets and the applied analysis methods to the project readme and this readme is now included in the project export. When you import a project, you can check the readme for a short description of the project’s intent on the project’s Overview page.
Watson Analytics connector is discontinued
The Watson Analytics connector has been removed from the list of data sources on the New connections page.
Snowflake connector available
Projects and catalogs now support connections to a Snowflake database, enabling you to store and retrieve data there.
Week ending 05 July 2019
Creating a project from a sample
If you are new to Watson Studio and are looking for how to use data assets in tools, such as notebooks to prepare data, analyze data, build and train models, and vizualize analysis results, you can now create a project from a sample. See Importing a project from a sample.
Week ending 28 June 2019
How to choose a tool in Watson Studio
You can now find which tool you need to use by matching your type of data, what you want to do with your data, and how much automation you want.
See Choosing a tool.
MQTT source operator, in addition to the message, now also provides the metadata attribute
event_topic, for each event.
Week ending 21 June 2019
Decision Optimization gives you access to IBM’s world-leading solution engines for mathematical programming and constraint programming. Use this sophisticated prescriptive analytics technology, which can explore a huge range of possible solutions before suggesting the best way to respond to a present or future situation. With Decision Optimization, you can:
- Start with a business problem, such as planning, scheduling, pricing, inventory, or resource management.
- Create an optimization model, which is the mathematical formulation of the problem that can be interpreted and solved by an optimization engine. The optimization model plus the input data creates a scenario instance.
- Run the Decision Optimization engine (or solver) to find a solution, a set of decisions that achieves the best values of the goals and respects limits and constraints imposed. Metrics measure the quality of the solution in terms of the business goals.
- Use Watson Machine Learning to deploy the solution and make it available to business users via a business application. Usually, the solution and goals are summarized in tabular or graphical views that provide understanding and insight.
For details on creating a prescriptive analytics model, see Decision Optimization.
For details on deploying a solutions, see Decision Optimization Deployment.
- You can now create an AutoAI experiment from a sample file so you can see how AutoAI analyzes and transforms data, then creates model candidate pipelines for you to review and compare without having to upload your own data.
- Follow the steps in Creating an AutoAI experiment from sample data to learn how to deploy and score a model created from the Bank marketing sample data set.
- AutoAI models saved as Watson Machine Learning assets are now only available in the project in which they were created. AutoAI models created prior to this update are available in other projects that share the same machine learning instance.
Preview of new Python client library with Watson Machine Learning v4 API
A new version of the Python client library is in the works to support new features in Watson Machine Learning. This Python client, built on version 4 of the Watson Machine Learning APIs, is available as a preview to support Decision Optimization and AutoAI experiments.
For details on installing the new Python client and accessing the associated documentation, see Python client.
In addition to the JSON message, Watson IoT source operator now also provides the following metadata attributes for each event:
Week ending 07 June 2019
Cognos Analytics connector available
Projects and catalogs now support connections to Cognos Analytics, enabling you to store and retrieve data there.
Enhancements to AutoAI Experiments
A new tutorial walks you through the process of building, deploying, and scoring a binary classification model using AutoAI.
The following enhancements make it easier for you to review AutoAI model pipelines:
- Updated designs make it faster for you to see the details for a pipeline
- When you expand a pipeline in the leaderboard to review details you can now view hold-out and training data scores
Week ending 31 May 2019
Build models with AutoAI
AutoAI in Watson Studio automatically analyzes your data, selects a model type, applies estimators, and generates candidate model pipelines customized for your predictive modeling problem. Results are dislayed in a leaderboard, which is a table showing the list of automatically generated candidate models, as pipelines, ranked according to the specified criteria.
AutoAI provides a view into the estimators and hyper-parameter optimization applied to each pipeline so you can have confidence in how each pipeline is generated. After viewing and comparing pipelines, you can save one as a model that can be deployed and tested. For details on building models using AutoAI, see AutoAI overview
Week ending 24 May 2019
New SPSS modeler nodes
You can use these new nodes in SPSS Modeler:
- Reprojection node: A field operations node that changes the coordinate system of fields in the geographical coordinate system to the projected coordinate system. See Reproject node.
- Space-Time-Boxes node: A record operations node that represents a regularly shaped region of space and time as an alphanumeric string. See Space-Time-Boxes node.
- Spatio-Temporal Prediction node: A modeling node that provides statistical techniques that you can use to forecast future values at different locations and explicitly model adjustable factors to perform what-if analysis. See Spatio-Temporal Prediction node.
New right-click option for SPSS Modeler nodes
Previously, when you right-clicked a node and selected Preview, a Data tab, Profile tab, and Visualizations tab opened—allowing you to examine your flow’s data in various ways. Now when you select Preview, you get a snapshot of your data that loads more quickly. Use the new right-click option called Profile to work with the full features such as the Visualizations tab.
Week ending 17 May 2019
Spark runtimes for Data Refinery flows
You can now use Spark runtimes with Data Refinery:
Spark R 3.4 environments for running Data Refinery flows are now generally available and are HIPAA ready. When you run a Data Refinery flow, you can select to use the preset
Default Spark R 3.4environment or configure your own Spark environment with the hardware size you need for your workload. You can’t select a Spark R 3.4 environment to schedule a Data Refinery flow run.
For information and instructions, see Spark environments for Data Refinery.
None - Use Data Refinery Defaultruntime is deprecated. However, you can still use this runtime to run Data Refinery flows that operate on small data sets and to schedule Data Refinery flow runs.
Week ending 10 May 2019
- Added the Binary data type to support use cases, such as processing, scoring and classifying binary data, images, video, and audio using pre-trained machine learning models.
- Option of ingesting raw data without built-in JSON parsing in Event Streams, Kafka, and HTTP source operators.
- Ability to have your own custom parsing by appending a Code operator.
- Ability to ingest binary data (for example, images, video, and audio).
- Option of writing raw data without built-in formatting as JSON, in Event Streams and Kafka target operators.
- Ability to have your own custom formatting by inserting a Code operator.
- Ability to write binary data (for example, images, video, and audio).
- Optional metadata attributes in Event Streams and Kafka source operators:
Week ending 3 May 2019
Watson Studio Local 2.0 GA
Watson Studio Local 2.0 is now generally available for when you need the functionality of Watson Studio on your private cloud. See Watson Studio Local 2.0 documentation.
For a comparison of features between different deployment environments of Watson Studio, Watson Machine Learning, and Watson Knowledge Catalog, see Feature differences between deployment environments.
Deprecation of Apache Spark Lite service
You can no longer associate an Apache Spark Lite service with a project. Apache Spark Lite services will be deleted on June 28, 2019. Read this blog post: Deprecation of Apache Spark (Lite Plan).
Use built-in Spark environments instead. See Spark environments.
Week ending 25 April 2019
Migration of Object Storage OpenStack Swift projects
You might not be able to download assets from any remaining projects that use Object Storage OpenStack Swift. If you have trouble downloading data from a project with Object Storage OpenStack Swift so that you can migrate the project to IBM Cloud Object Storage, open a ticket with IBM Support.
Attribute classifiers renamed to data classes
The attribute classifiers that characterize the contents of columns in relational data assets are now called data classes. Data classes are automatically assigned to columns during profiling. See Profiles.
Week ending 12 April 2019
Export a project to desktop
You can now share project assets with others by exporting your project. The project assets that you select are downloaded as a project ZIP file to your desktop.
Import a project from desktop
When you create a project, you can select the import project starter that enables importing assets from another project to use in a new project.
Improved search filtering in Watson Knowledge Catalog
When you search for assets in a catalog, the search filters are now directly below the search field. You can filter on tags, business terms that are assigned to data assets, and asset types. The list of tags is now sorted alphabetically.
Week ending 5 April 2019
Data Refinery: Column-action operations remain focused on the column
When you click a column in Data Refinery and select an operation from the actions menu (three vertical dots), the focus remains on that column. Previously the focus always shifted to the first column. This enhancement is especially useful when you are working on wide tables.
Week ending 22 March 2019
Import a streams flow from URL
You can now also import a streams flow from a URL, in addition to importing from a file.
Week ending 15 March 2019
Google Cloud Storage connector available
Projects and catalogs now support connections to Google Cloud Storage, enabling you to store and retrieve data there.
Week ending 1 March 2019
Migrate Watson Studio from Cloud Foundry to a resource group
You can migrate your Watson Studio service instance from a Cloud Foundry org and space to a resource group in IBM Cloud. Resource groups include finer-grained access control by using IBM Cloud Identity and Access Management (IAM), the ability to connect service instances to apps and service across different regions, and an easy way to view usage per group.
For instructions, see IBM Cloud: Migrating Cloud Foundry service instances to a resource group.
Secure Gateway service for Watson Studio instances in the Tokyo region
The Secure Gateway service is not yet available in the Tokyo (AP-North) service region on IBM Cloud. However, you can now provision a Secure Gateway service in any other region and use it when you create a connection in a Watson Studio instance from any region, including Tokyo.
Assign terms and tags to columns in Watson Knowledge Catalog
You can now assign business terms and tags to columns in relational data assets in catalogs. See Editing assets in catalogs.
Publish and subscribe messages to topics with the Streams operator in streams flow
In streams flows, you can now subscribe to topics with the Source opearator and publish to topics with the Target operator in the Streaming Analytics service.
Week ending 22 February 2019
Data Refinery flow runs now consume CUHs which are tracked
The capacity unit hours (CUHs) that are consumed when you run a Data Refinery flow in a Spark R 3.4 environment are now tracked.
For information, see Spark environments for Data Refinery.
Week ending 8 February 2019
Run Data Refinery flows in a Spark R environment (open beta)
You can now select a Spark R environment for your Data Refinery flows. You can select the Default Spark R 3.4 environment or you can create your own Spark R environment definition that is customized for the size of your data set. Each Data Refinery flow runs in a dedicated Spark cluster.
Select the Spark environment from the Data Refinery flow details page when you save and run the flow.
For information and instructions, see Spark environments for Data Refinery.
Week ending 1 February 2019
New navigation menu
You can easily find all your menu options in one place on the new navigation menu. Click to expand the menu.
HIPAA readiness for Watson Studio and Watson Machine Learning
Watson Studio and Watson Machine Learning meet the required IBM controls that are commensurate with the Health Insurance Portability and Accountability Act of 1996 (HIPAA) Security and Privacy Rule requirements. HIPAA readiness applies to only certain plans and regions.
Week ending 25 January 2019
New SPSS modeler node
SPSS Modeler flows now support the Set Globals node to compute summary values for CLEM expressions. See SPSS Modeler nodes.
Python model operator now supports Watson Machine Learning in streams flows
In addition to loading models from IBM Cloud Object Storage, the Python model operator now also supports loading models from the Watson Machine Learning service.
Week ending 18 January 2019
SPSS Modeler flows now support more of the SPSS Modeler desktop nodes.
There are more modeling nodes. Check it out.
Week ending 11 January 2019
Easy upgrade with the Upgrade button
When you’re ready to upgrade your Watson Studio, Watson Knowledge Catalog, or Watson Machine Learning service from a Lite plan, just click the Upgrade button. You’ll be guided through the upgrade in just a few clicks. See Upgrade your Watson services.
Streams flows now support User Defined Parallelism (UDP) for Event Streams
With User Defined Parallelism, multiple workers help to increase the ingestion rate for topics with multiple partitions.