Check out the new features for Cloud Pak for Data as a Service and the core services of Watson Studio, Watson Machine Learning, and Watson Knowledge Catalog each week.
Week ending 22 January 2021
Open beta for building AutoAI experiments with joined data sets
You can now build an AutoAI experiment using up to 5 data sets joined by common keys into a single data set. Use the canvas tool to configure how the data is joined before you run the experiment. When you deploy a resulting model, specify input data that matches the schema of your experiment. For details, see Building an AutoAI experiment using joined data.
- The Lookup stage is now supported. Note: Reject links and the ability to define Range Lookups on the Stream Link are not yet supported within the Lookup stage.
- The Amazon S3 Connector has been added to the Existing Asset experience. In total, you can now use IBM Cloud Object Storage, Google Cloud Storage, HTTP Connections, and Amazon S3 with the Existing Asset experience.
Week ending 15 January 2021
New organization for building and deploying assets
The content for building models has been separated from the content for deploying and managing models, to make it easier for you to find tools and processes. The updated sections are:
- Analyzing data and building models which documents all of the tools you can use to build models and solutions.
- Deploying and managing models which documents frameworks, tools, and processes for deploying, evaluating, and updating assets.
- Corrected the Asset Browser behavior where the Add button could be selected before the asset was chosen.
- Optimized job startup and improved the backend of the runtime service.
Week ending 8 January 2021
- You can use the Existing Asset experience in the palette to browse existing connections, pick the data you’re interested in, preview it, and then add it to your flow design. Adding connections this way greatly speeds up designing a flow. This feature not only brings in and configures the stage with the connection details, but also automatically infers and populates the schema and column definitions.
- Support for Google Cloud Storage and HTTP Connections is added to the Existing Asset connector.
- When you used the Existing Asset connector previously, the Existing Asset icon was added to the canvas. Now when you finish your flow with the Existing Asset connector, the underlying connector icon, such as HTTP Connector or IBM Cloud Object Store, is added to the canvas.
- The Existing Asset browser now enables only connectors that are supported.
- The Tail stage is now supported.
Week ending 18 December 2020
Deprecation and removal of Streams flows
Streams flows is being deprecated. Starting on January 31, 2021, you will no longer be able to create or run streams flows. To preserve your code in an archive package, see Downloading streams flow code in a code archive package. For future support for streams flow use cases, see Streaming Analytics on IBM Cloud.
Week ending 4 December 2020
IBM Watson Visual Recognition is discontinued in Watson Machine Learning
IBM Watson Visual Recognition is discontinued. Existing instances are supported until 1 December 2021, but as of 7 January 2021, you can’t create instances. Any instance that is provisioned on 1 December 2021 will be deleted.
DataStage flows (Beta)
You can now create DataStage flows in IBM Watson Studio by using the beta version of DataStage. DataStage is an ETL tool that is available in a true SaaS, cloud-native, cloud-first environment. You can use prebuilt transformations such as joins, lookups, and aggregations to process data and load it to targets with a rich library of connections.
To learn more, see DataStage (Beta).
Week ending 27 November 2020
Open beta for Federated Learning
Federated Learning provides the tools for training a model collaboratively, using a federated set of secure, remote data sources. The data sources are never moved or combined, but they each contribute to training and improving the quality of the common model. The high-level steps are:
- Define the parties for federated learning and create remote training systems
- Create a common model and configure how to aggregate the data.
- Train the model with the federated data sources.
- Deploy and score the resulting model.
For details on setting up remote parties and training a common model, see Federated Learning.
Hybrid Subscription Advantage
The IBM Hybrid Subscription Advantage program is a licensing benefit that applies existing on-premises Cloud Pak for Data software entitlements within the Cloud Pak for Data as a Service portfolio. To learn more, see Activating the Hybrid Subscription Advantage.
Week of 20 November 2020
HIPAA readiness for Watson Knowledge Catalog
Watson Knowledge Catalog meets the required IBM controls that are commensurate with the Health Insurance Portability and Accountability Act of 1996 (HIPAA) Security and Privacy Rule requirements. HIPAA readiness applies to only certain plans and regions. For more information, see Keeping your data secure and compliant.
Week of 13 November 2020
Reminder: Upcoming changes to Watson Machine Learning frameworks
Note the following upcoming changes to Watson Machine Learning frameworks:
- All frameworks that are built with Python 3.6 are deprecated in favor of frameworks that are built with Python 3.7. Starting on November 20, 2020, new training based on Python 3.6 will be blocked. Support for Python 3.6 frameworks and related software specifications will be discontinued on January 20, 2021 for assets built with V4-GA API and on March 1, 2021 for assets built with v3 or v4-beta API.
- Spark 2.3 frameworks are deprecated in favor of Spark 2.4. Support for Spark 2.3 will be discontinued on December 1, 2020.
Week of 28 October 2020
Legacy APIs no longer supported for Watson Machine Learning Lite plan users
The migration period for Watson Machine Learning Lite plan users to migrate assets to the V2 machine learning service instance and the V4 Watson Machine Learning APIs is ended. As a result, Lite users will get errors when they score deployed assets that rely on deprecated APIs. Create new deployments based on V4 Watson Machine Learning v4 APIs. For details, see API changes for Watson Machine Learning.
Master Data Management service beta
The Master Data Management service is now in beta. This new Master Data Management experience seamlessly consolidates data from the disparate sources that fuel your business to establish a single, trusted, 360-degree view of your customers.
The beta release of Master Data Management includes two user experiences:
- Master data configuration for data engineers to prepare and configure master data. This experience enables you to:
- Configure the master data system.
- Refine the generated data model.
- Upload or connect data assets and sources.
- Map data into the model.
- Run the Master Data Management service’s powerful matching capability to create master data entities.
- Configuring and tuning the matching algorithm to meet your organization’s requirements.
- Master data explorer for business analysts and data stewards to search, view, analyze, and export master data entities.
The Master Data Management service on Cloud Pak for Data as a Service also includes a rich set of APIs that empower your business applications with direct access to trusted data.
For more information about Master Data Management, see Managing master data (Beta).
Week ending 23 October 2020
New way of adding data
Create metadata import assets to configure and run the metadata import for a selected set of data assets into a project or a catalog. For details, see Managing metadata imports.
SJIS encoding available in Data Refinery for input and output
SJIS (“Shift JIS” or Shift Japanese Industrial Standards) encoding is an encoding for the Japanese language.
To change the encoding of the input file, open the file in Data Refinery, go to the Data tab, scroll down to the SOURCE FILE information, and then click the “Specify data format” icon .
To change the encoding of the output (target) file, open the Information pane and click the Details tab. Click the Edit button. In the DATA REFINERY FLOW OUTPUT pane, click the Edit icon to change the encoding.
The SJIS encoding is supported only for CSV and delimited files.
New visualization charts for Data Refinery and SPSS Modeler
To access the charts in Data Refinery, click the Visualizations tab and then select the columns to visualize. The chart automatically updates as you refine the data.
To access the charts in SPSS Modeler, use a Charts node. The Charts node is available under the Graphs section on the node palette. Double-click the Charts node to open the properties pane. Then click Launch Chart Builder to create one or more chart definitions to associate with the node.
For the full list of available charts, see Visualizing your data.
Evaluation charts are combination charts that measure the quality of a binary classifier. You need three columns for input: actual (target) value, predict value, and confidence (0 or 1). Move the slider in the Cutoff chart to dynamically update the other charts. The ROC and other charts are standard measurements of the classifier.
Math curve charts display a group of curves based on equations that you enter. You do not use a data set with this chart. Instead, you use it to compare the results with the data set in another chart, like the scatter plot chart.
Sunburst charts display different depths of hierarchical groups. The Sunburst chart was formerly an option in the Treemap chart.
Tree charts represent a hierarchy in a tree-like structure. The Tree chart consists of a root node, line connections called branches that represent the relationships and connections between the members, and leaf nodes that do not have child nodes. The Tree chart was formerly an option in the Treemap chart.
Week ending 16 October 2020
Change to Watson Machine Learning deployment frameworks
The following changes to deployment frameworks might require user action.
Support for Python 3.7
You can now select Python 3.7 frameworks to train models and run Watson Machine Learning deployments.
Deprecation of Python 3.6
Python 3.6 is being deprecated. Support will be discontinued on January 20, 2021. You can continue to use the Python 3.6 frameworks; however you will be notified that you should move to a Python 3.7 framework. For details on migrating an asset to a supported framework, see Supported frameworks.
Support for Spark 3.0 and new language versions
- You can now select a Spark 3.0 environment to run notebooks with Python 3.7, R 3.6 and Scala 2.12 or to run notebook jobs.
You can select Python 3.7 environments to run Jupyter notebooks (including with GPU) in Watson Studio.
Python 3.6 is being deprecated. You can continue to use the Python 3.6 environments; however you will be notified that you should move to a Python 3.7 environment. When you switch to Python 3.7, you might need to update code in notebooks if the versions of open source libraries that you used are different in Python 3.7.
With the introduction of Spark 3.0, you can start using Spark with Scala 2.12 in notebooks and jobs. Again, you might need to update code in your notebooks if library versions that you used with Scala 2.11 are not compatible in Scala 2.12.
Support ends for SPSS Modeler runtime 18.1 and certain Python nodes
Support for SPSS Modeler flows trained with 18.1 and containing certain Python nodes is discontinued as of October 14, 2020. If your SPSS models uses any of these Python nodes, then you must retrain your models using Watson Studio Canvas or any tool that uses SPSS Modeler 18.2 version. For details, see Supported frameworks.
Support ends for deployments based on deprecated AutoAI images
Due to a known security vulnerability, AutoAI model deployments created using Watson Machine Learning on IBM Cloud prior to August 1, 2020 will be removed on November 1, 2020. If you have not already migrated and redeployed your AutoAI models, do so prior to November 1, 2020. For details, see Migrating Assets.
Data Refinery flows are supported in deployment spaces
You can now promote a Data Refinery flow from a project to a deployment space. Deployment spaces are used to manage a set of related assets in a separate environment from your projects. You can promote Data Refinery flows from multiple projects to a space. You run a job for the Data Refinery flow in the space and then use the shaped output as input for deployment jobs in Watson Machine Learning. For instructions, see Promote a Data Refinery flow to a space in Managing Data Refinery flows.
Week ending 9 October 2020
Time series library for notebooks
You can now use the time series library to perform operations on time series data, including segmentation, forecasting, joins, transforms and reducers. You can use the time series library functions in Python notebooks that run with Spark. See Time series library.
Week ending 2 October 2020
Updates to Watson Knowledge Catalog offering plans
Starting 1 October, 2020, the Watson Knowledge Catalog offering plans have these changes:
- Lite plan: The maximum number of users is reduced to 2.
- Standard plan: The maximum number of assets is increased to 1000.
- Standard and Professional plans: The cost for the plan, extra users, and extra CUH is changing.
If you already have the Lite or Standard plan, your existing assets and catalog users remain unchanged.
Read the blog post.
End of migration period for Watson Machine Learning Lite plans
On October 1, 2020, the migration period ended for Watson Machine Learning Lite plan users with V1 machine learning service plans created before September 1, 2020. For details on the migration to new plans and service instances, see the what’s new entry for September 4, 2020.
Week ending 25 September 2020
Batch deployment available for AutoAI experiments
Starting with the V2 Watson Machinne Learning service instance and the V4 Watson Machine Learning APIs, rolled out on September 1, 2020, batch deployment is supported for AutoAI experiments. For details, see Creating a batch deployment.
New Databases for EDB service
You can now create the Databases for EDB service from the Cloud Pak for Data as a Service services catalog to access EDB Postegres Advanced Server. See Databases for EDB.
Week ending 18 September 2020
New jobs user interface for running and scheduling Data Refinery flows and notebooks
The user interface gives you a unified view of the job information.
Data Refinery flow job
You create the job for a Data Refinery flow or for a notebook directly in the user interface for each tool or from the Assets page of a project. See Jobs in a project.
Week ending 11 September 2020
Change to Watson Machine Learning service credentials
The new V2 Watson Machine Learning service instance rolled out on September 1 uses new, simplified authentication. Obtaining bearer tokens from IAM is now performed using a generic user apikey instead of a Watson Machine Learning specific apikey. It is no longer necessary to create specific credentials on the Watson Machine Learning instance, so the Credentials page was removed from the IBM Cloud services catalog. For details, see Authentication.
During the migration period, you can use existing Watson Machine Learning service credentials to access your legacy V1 service instance and assets. Lite users of instances provisioned before Sept 1st can keep using existing credentials during the migration period but cannot generate new credentials. Standard and Professional plan users can follow the steps in Generating legacy Watson Machine Learning credentials to create new credentials.
Spark 2.3 deprecation
Starting 1 October, 2020, you can no longer select a Spark 2.3 environment to run a notebook or job. Select a Spark 2.4 environment instead. Existing notebooks and jobs with Spark 2.3 environments will continue running until 30 November, 2020. After that you must select a different environment for the affected notebooks and jobs.
Week ending 4 September 2020
New Watson Machine Learning service plans
Watson Machine Learning released new plans on IBM Cloud. These new plans accommodate and provide entitlements for the newest features and patterns available to Watson Machine Learning users, starting on September 1, 2020.
New sign-ups will receive the latest plans and API entitlements; no Watson Machine Learning instances that correspond to the older plans can be provisioned. By March 1st 2021, only Watson Machine Learning instances which correspond to the updated plans will be supported. The following sections introduce the new features and describe how to best plan your migration.
Upgrading to Watson Machine Learning “v2” Instances
All Lite plan users are automatically upgraded from v1 to v2 service plans. Lite plan users can now call the v4 APIs or use the v4 Python client library to conduct machine learning model training, model saving, and deployment, and access the newest features such as runtime software specifications for your deployments.
For Standard plan and Professional plan users, you can choose when to migrate your assets for use with the v2 machine learning service instance. Users of these plan instances will have more time to work with both the older and the newer API sets and plan instances until March 1 2021. For details on working with a deprecated service instance, see Generating legacy Watson Machine Learning credentials.
Note: During the migration period, you will not be charged for your usage associated with the new plan instances while your v1 plan instances are still active.
Get details about the Watson Machine Learning plans.
Full support for v4 APIs and an updated Python client library
The v4 APIs and Python client library are now generally available for use with the v2 service plans. The new APIs support features such as deployment spaces for organizing all of the assets required for running and managing deployment jobs, software specifications, and updated authentication. Note that support for v3 and v4 beta APIs ends on March 1, 2021. Review the differences between the v3, v4 beta, and v4 APIs.
Migration assistance for Watson Machine Learning
Watson Machine Learning users can easily migrate their Watson Machine Learning repository assets, such as machine learning models to Watson Studio Projects with automated assistance from a graphical migration tool, or programmatically, using a dedicated set of APIs.
Introducing deployment spaces
Deployment spaces let you deploy and manage models and other operational assets such as data sources and software specifications in a separate environment from your projects. When your project assets are ready to deploy, you promote assets to your deployment space to configure deployments, test models and functions, consume scoring endpoints, and manage production jobs. Spaces, like projects, are collaborative, so you can invite others to collaborate and manage access for a space.
Watson Studio enhancements
Watson Studio now leverages the newest Watson Machine Learning APIs. Consequently, to take actions from the Watson Studio interface that require Watson Machine Learning, such as triggering AutoAI experiments, you must have a “v2” Watson Machine Learning instance associated with the project. Watson Studio projects that are still associated with an older Watson Machine Learning instance will display a message that instructs you to migrate your assets and associate a new v2 Watson Machine Learning instance.
Additionally, starting on September 1st, Watson Studio users will be able to save models and other artifacts produced through use of Watson Machine Learning alongside other assets like notebooks in their Watson Studio project. For details on machine learning tools you can use to create project assets, see Machine Learning Overview.
Decision Optimization support for Watson Machine Learning
Decision Optimization supports all of the new features available with Watson Machine Learning, including using software and hardware specifications to configure optimization models and using deployment spaces to organize the assets required for deployment. For a complete list of changes, see Migrating from Watson Machine Learning API V4 Beta.
Watson Machine Learning migration action plan
Follow these steps to migrate assets and upgrade your service instance to take advantage of the new Watson Machine Learning plans and features.
- Review the updated Watson Machine Learning plans and consider which level of service is right for you.
- Migrate your assets, using the Migration Assistant tool from Watson Studio or using a programmatic solution.
- Start using your new machine learning service instance.
- Retrain models or update your Python functions, as needed.
- Create a deployment space and start to work with your migrated assets.
- Delete your old v1 service instance.
Spark 2.3 framework for Watson Machine Learning deprecated
Spark 2.3 framework for Watson Machine Learning is deprecated and will be removed on December 1, 2020. Use Spark 2.4 instead. For details, see Supported frameworks.
Compatibility issue for SPSS Modeler runtime 18.1 and older Python version
Support for SPSS Modeler flows trained with 18.1 and containing certain Python nodes is deprecated. For existing deployments which are using these nodes, you can continue to score the deployments till October 1, 2020. If the SPSS models uses any of these Python nodes, then it will require retraining the model using Watson Studio Canvas or any tool that uses SPSS Modeler 18.2 version. For details, see Supported frameworks.
Decision Optimization enhancements
You can now use these enhancements to Decision Optimization:
- The Decision Optimization model builder now contains a new Overview pane which provides you with model, data and solution summary information for all your scenarios at a glance. From this view you can also open an information pane where you can create or choose your deployment space. See Overview pane.
- To create and run Optimization models you must have both a Machine Learning service added to your project and a deployment space associated with your experiment.
- You can now deploy models using Watson Machine Learning from the the Decision Optimization model builder scenario pane. See Scenario pane and Deploying a model using the user interface.
- A new sample Extend software specification notebook is now available which shows you how to extend the Decision Optimization software specification (runtimes with additional Python libraries for docplex models). See Python client examples and download the sample from https://github.com/IBMDecisionOptimization/DO-Samples/tree/watson_studio_cloud/jupyter.
- The Explore solution view of the model builder has been updated to show more information about the objectives/KPIs, solution tables, constraint or bounds relaxations or conflicts, engine statistics and log. See Explore solution view.
- The Visualization view of the model builder now enables you to create Gantt charts for any type of data where it is meaningful and is no longer restricted to scheduling models only. See Visualizations view.
Translation of documentation
You can now read this documentation in these languages by setting your browser locale:
- Brazilian Portuguese
- Simplified Chinese
- Traditional Chinese
Not all documentation topics are translated into all of these languages.
Jupyter notebook editor upgraded
The Jupyter notebook editor in Watson Studio is upgraded from Jupyter notebook version 6.0.3 to version 6.1.1. For a list of changes, including keyboard short-cut key changes, see Jupyter Notebook Changelog.
Week ending 21 August 2020
Enhanced Cognos Dashboards
You can now use these enhancements to the dashboards in a project:
- New visualizations, including Waterfall, KPI widget, and an enhanced cross-tab.
- A contextual toolbar and a data mapping panel.
See Cognos Dashboards.
Databases for MongoDB service
You can now provision a Databases for MongoDB service from the Services catalog.
Use Data Refinery to change the decimal and thousands grouping symbols in all applicable columns
When you use the Convert column type GUI operation to detect and convert the data types for all the columns in a data asset, you can now also choose the decimal symbol and the thousands grouping symbol if the data is converted to an Integer or to a Decimal data type. Previously you had to select individual columns to specify the symbols.
See Convert column type in GUI operations in Data Refinery, under the FREQUENTLY USED category.
Week ending 14 August 2020
Google Cloud Platform integration
You can now configure an integration with the Google Cloud Platform (GCP) to access data sources from GCP.
Week ending 31 July 2020
Security update for AutoAI deployments
There is a known security vulnerability with the image used for AutoAI model deployments created using Watson Machine Learning on IBM Cloud prior to August 1, 2020. The image vulnerability has been addressed, so deployments of models created with AutoAI experiment after August 1, 2020 are not impacted. The following remedies are available:
For Lite plan users
Impacted AutoAI deployments will be deprecated (stop working) on the September 1st, 2020. You can redeploy your models in August, then migrate them to a new deployment space in September, 2020.
For Standard and Professional plans users
Impacted AutoAI deployments will be deprecated (stop working) on November 1st, 2020. You can redeploy your models in August, then migrate them to a new deployment space in September-October, 2020.
Removal of Neural Network Modeler and SparkML modeler
Both the beta Neural Network Modeler and the beta SparkML modeler tools are removed from Watson Studio.
Week ending 24 July 2020
Cloud Pak for Data as a Service is GA!
Cloud Pak for Data as a Service is now generally available. Sign up for Cloud Pak for Data as a Service at dataplatform.cloud.ibm.com.
Read the blog.
Subscribe to Cloud Pak for Data as a Service
You can now upgrade your IBM Cloud account from a Lite plan by subscribing to Cloud Pak for Data as a Service. With a subscription, you commit to a minimum spending amount for a certain period of time and receive a discount on the overall cost compared to a Pay-As-You-Go account.
Learn quickly with a guided tutorial
You can quickly learn how to use tools in projects by taking a guided tutorial. A guided tutorial starts with a sample project that contains the data and anything else you need. After you create the project, the tutorial starts and guides you through the steps to solve a specific business problem.
Click the Take a guided tutorial link on your home page.
New services catalog
You can now create services IBM Cloud services that work with Cloud Pak for Data as a Service from the new services catalog. Select Services catalog from the main menu. You can see all your services by selecting the Your services option.
See IBM Cloud services.
Integrate with other cloud platforms
You can now configure integration with other cloud platforms to simplify creating connections to data sources in those cloud platforms in projects and catalogs. Select Cloud integration from the main menu.
Use the new Data Refinery “Union” operation to combine the rows from two data sets that share the same schema
The Union operation is in the ORGANIZE category. For more information, see GUI operations in Data Refinery.
Automatically detect and convert date and timestamp data types
When you open a file in Data Refinery, the Convert column type GUI operation is automatically applied as the first step if it detects any non-string data types in the data. Now date and timestamp data are detected and are automatically converted to inferred data types. You can change the automatic conversion for selected columns or undo the step. For information about the supported inferred date and timestamp formats, see Convert column type in GUI operations in Data Refinery, under the FREQUENTLY USED category.
Week ending 17 July 2020
New look, new features, new Cloud Pak for Data as a Service brand coming soon!
Starting on 21 July, you’ll see some changes to the Watson Studio, Watson Machine Learning, and Watson Knowledge Catalog services.
- What is changing
- Your home page will look different and shows more information.
You’ll have some new options on the main menu:
- A Services catalog option where you can create IBM Cloud services that work with Watson Studio and Watson Knowledge Catalog. See IBM Cloud services.
- A Cloud integration option for configuring integrations to other cloud platforms to simplify creating connections to data sources in those cloud platforms. See Integrating with other cloud platforms.
- What might change
- Your product brand might change to Cloud Pak for Data. If you provisioned any services that work with Watson Studio besides Watson Machine Learning and Watson Knowledge Catalog, you’ll see Cloud Pak for Data as the product brand. See Relationships between the Watson Studio and Watson Knowledge Catalog services and Cloud Pak for Data as a Service.
- What isn’t changing
- Your service plans and billing for your IBM Cloud services remain the same.
Week ending 10 July 2020
Upcoming removal of Neural Network Modeler and SparkML modeler
Both the beta Neural Network Modeler and the beta SparkML modeler tools will be removed from Watson Studio on July 31.
If you use the Neural Network Modeler, similar functionality will be added to AutoAI in the future. Download your existing Neural Network Modeler flows as TensorFlow, Keras, Caffe, or PyTorch code.
If you use the SparkML modeler, you can find similar drag and drop visual modeling functionality in the SPSS Modeler. Alternatively, you can to process big data with powerful Spark environments in Jupyter notebooks. Export your existing SparkML modeler flows as compressed files.
Easily add data from a Cognos Analytics connection to a notebook
You can now add data from a Cognos Analytics connection by using the Insert to code function for the connection within a notebook. See Data load support.
The Lineage page is renamed to Activities
The Lineage page that you can see when you open a data asset in a catalog or project is now called Activities. The information shown on this page remains the same.
Week ending 3 July 2020
Upcoming Watson Machine Learning plan changes
As part of a broader Watson Machine Learning update coming in September, Lite plan users will be automatically upgraded to new “v2” plan instances on September 1. With this automatic upgrade, Lite plan users can use the new v4 Watson Machine Learning APIs to support new features for building and deploying assets. Additionally, Lite plan users can upgrade to the Standard plan instances free of charge before September 1 to retain entitlement to the older API set during the migration period. With the Standard plan, you pay only for capacity unit hours and predictions, but you can take other actions, such as reading model information, free of charge.
Starting September 1, Watson Machine Learning Lite plans will include a maximum of 20 capacity unit hours, instead of 50.
Week ending 12 June 2020
Perform aggregate calculations on multiple columns in Data Refinery
Now you can select multiple columns in the Aggregate operation. Previously all aggregate calculations applied to one column.
For more information, see Aggregate in GUI operations in Data Refinery, under the ORGANIZE category.
Filter values in a Boolean column in Data Refinery
You can now use these operators in the Filter GUI operation to filter Boolean (logical) data:
For more information, see Filter in GUI operations in Data Refinery, under the FREQUENTLY USED category.
In addition, a new template for filtering by Boolean values has been added to the filter coding operation.
For more information about the filter templates, see Interactive code templates in Data Refinery.
Week ending 05 June 2020
SPSS Modeler flow properties
You can now set flow properties. For details, see Setting properties for flows.
Week ending 22 May 2020
AutoAI Auto-generated WML notebooks and SDK available in beta
You now have two options for saving an AutoAI pipeline as a notebook:
- WML notebook - Work with a trained model in an annotated notebook. You can review and update the code, view visualization, and deploy the model with Watson Machine Learning.
- AutoAI_lib notebook - View the Scikit-Learn source code for the trained model in a notebook. Does not require Watson Machine Learning.
Additionally, the Watson Machine Learning Python client has been extended to include an SDK for the WML notebook. For details, see Saving an AutoAI generated notebook. Note: These features are being offered as a beta and are subject to change.
Changes to the Watson Studio plans
Starting on May 19, 2020, the Watson Studio plans has the following changes:
- All plans: The free compute environment is no longer available. All your compute usage now consumes capacity unit hours. The Lite plan has a limit of 50 capacity unit hours per month.
- Lite and Standard plans: Compute environments provided by associated services, such as IBM Analytics Engine, are now available only with the Enterprise plan.
- Lite plan: Only the smallest size Spark environments are now available for Lite plans, with 2 executors that each have 1 vCPU and 4 GB RAM, and one driver that has 1 vCPU and 4 GB RAM. Large compute environments with 8 or more vCPU are no longer available for the Lite plan.
- Lite plan: The ability to export projects now requires the Standard or Enterprise plan.
If you have an operational asset, for example, a notebook, or a job that uses an environment that is no longer available, you will see a message to select a different environment. See Changing your environment.
If you need more compute resources, upgrade to the Watson Studio Standard or Enterprise plan. See Upgrading Watson Studio.
This change was first announced on March 17 here and in this blog post.
Week ending 01 May 2020
More Decision Optimization compute options
You now have more options that cost less when you run Decision Optimization jobs. You can choose from new, more powerful Decision Optimization environments. The basic Decision Optimization compute environment now consumes only five capacity unit hours (CUH) instead of 20 CUH. The new environments consume 6-13 CUH.
Read the blog.
“PureData System for Analytics” connection renamed to “Netezza (PureData System for Analytics)”
The PureData System for Analytics connection is now the Netezza (PureData System for Analytics) connection. This change is to reflect the announcement of the new Netezza Performance Server for on premises and Cloud. Your previous settings for a connection to PureData System for Analytics remain the same. Only the connection name changed.
New visualization charts in Data Refinery
Data Refinery introduces six new charts. To access the charts, click the Visualizations tab in Data Refinery, and then select the columns to visualize. The chart automatically updates as you refine the data.
Bubble charts display each category in the groups as a bubble.
Circle packing charts display hierarchical data as a set of nested areas.
Multi-charts display up to four combinations of Bar, Line, Pie, and Scatter plot charts. You can show the same kind of chart more than once with different data. For example, two pie charts with data from different columns.
Radar charts integrate three or more quantitative variables that are represented on axes (radii) into a single radial figure. Data is plotted on each axis and joined to adjacent axes by connecting lines. Radar charts are useful to show correlations and compare categorized data.
Theme river charts use a specialized flow graph that shows changes over time.
Time plot charts illustrate data points at successive intervals of time.
Week ending 24 April 2020
Synchronizing assets with Information Governance Catalog is discontinued
You can no longer automatically synchronize data assets between Information Governance Catalog and Watson Knowledge Catalog.
Week ending 17 April 2020
GPU environments for running notebooks are GA
GPU environments for running Jupyter notebooks with Python 3.6 are now generally available for the Watson Studio Standard and Enterprise plans. GPU environments are available in the Dallas IBM Cloud service region only.
With GPU environments, you can reduce the training time needed for compute-intensive machine learning models you create in a Jupyter notebook with Python 3.6. With more compute power, you can run more training iterations while fine-tuning your machine learning models.
See GPU environments.
Week ending 3 April 2020
Changes to Watson Studio Enterprise plan
On April 1, 2020, the Watson Studio Enterprise plan has the following changes:
- The number of free authorized users is now 10.
- The cost of adding extra authorized users is reduced by 50%.
- The compute usage rate is reduced to $0.40 USD per capacity unit hour used beyond the 5000 CUH per month that are included in the plan.
Read the blog post.
Week ending 27 March 2020
AutoAI Auto-generated notebooks available in beta
Save an AutoAI pipeline as a notebook so you can view all of the transformations that went into creating the pipeline. Use the autoai-lib reference as a guide to the transformations. This feature is being offered as a beta and is subject to change. For details, see Saving an AutoAI generated notebook.
Week ending 20 March 2020
Upcoming changes to Watson Studio Lite and Enterprise plans
On May 17, 2020, the Watson Studio Lite plan will have the following changes:
- Free compute environment will not be available. All Lite plan users will have a limit of 50 capacity unit hours of compute usage per month.
- Large compute environments with 8 or more vCPU will not be available.
- Only the smallest size Spark environments will be available, with 2 executors that each have 1 vCPU and 4 GB RAM, and one driver that has 1 vCPU and 4 GB RAM.
- Compute environments provided by associated services, such as IBM Analytics Engine, will be available only with the Enterprise plan.
- The ability to export projects will not be available.
If you need more compute resources, upgrade to the Watson Studio Standard or Enterprise plan. See Upgrading Watson Studio.
On April 1, 2020, the Watson Studio Enterprise plan will have the following changes:
- The number of free authorized users will be doubled, to 10.
- The cost of adding extra authorized users will be reduced by 50%.
- The compute usage rate will be reduced from $0.50 USD to $0.40 USD per capacity unit hour used beyond the 5000 CUH per month that are included in the plan.
Read the blog post.
Upcoming changes to Watson Machine Learning GPU plans
Starting on May 1, 2020, Watson Machine Learning will update the capacity units per hour for GPU capacity types, as follows:
|Capacity Type||Capacity units required per hour|
|1 NVIDIA K80 GPU||3|
|1 NVIDIA V100 GPU||10|
Capacity units required per hour of multiple GPUs is calculated by the capacity units per hour on single GPU times the total number of GPUs. For details, read the blog post.
Week ending 13 March 2020
Custom security policies available for restricting downloads
By default, Watson Machine Learning does not restrict external sites users can access as part of operations such as downloading data source files or installing Python library packages. If you would like to limit access to a list of approved sites, contact IBM Cloud support to request a custom network policy for your organization.
New capabilities in AutoAI
The following features are new or enhanced in AutoAI:
- The limit on the size of a data source for an AutoAI experiment is increased from 100 MB to 1 GB.
- The number of pipelines generated for an experiment is increasing from four to eight, based on the two top performing algorithms. You can now also increase the number of top performing algorithms to use for generating pipelines if you want to view and compare more pipelines. Each algorithm creates four optimized pipelines. For details, see Building an AutoAI model.
Week ending 06 March 2020
Updates to Watson Machine Learning frameworks
Support is now available for TensorFlow 1.15 and Keras version 2.2.5 for training and deploying models. Due to a security vunerability with certain TensorFlow versions, support for TensorFlow 1.13 and 1.14 along with Keras 2.1.6 and Keras 2.2.4 will be deprecated. Users will need to upgrade to Keras 2.2.5 and switch to TensorFlow 1.15 backend. For details on the changes, view this announcement. For the complete list of supported frameworks, see this topic.
Week ending 07 February 2020
New Spark and R runtime enabled for jobs in Data Refinery
You can now select
Default Spark 2.4 & R 3.6 when you select an environment definition for a new job. The new runtime uses the same capacity unit hours (CUHs) as the
Default Spark R 3.4 (which is Spark 2.3) runtime.
SPSS Statistics .sav data files are now supported for import or export in SPSS Modeler.
Exercise more control over pipeline creation for an AutoAI experiment
You now have the option of specifying which algorithms AutoAI should consider for an experiment and how many of the top performing algorithms to use for creating pipelines. For details, see Building an AutoAI model.
Week ending 10 January 2020
“Hortonworks HDFS” connection renamed to “Apache HDFS”
The Hortonworks HDFS connection is now the Apache HDFS via the WebHDFS API connection. Your previous settings for connections to Hortonworks HDFS remain the same. Only the connection name has changed.
Geospatio-temporal library for notebooks
You can use the geospatio-temporal library to expand your data science analysis to include location analytics in your Python notebooks that run with Spark. See Using the geospatio-temporal library.