Troubleshoot Machine Learning

Here are the answers to common troubleshooting questions about using IBM Watson Machine Learning.

Getting help and support for Machine Learning

If you have problems or questions when using Machine Learning, you can get help by searching for information or by asking questions through a forum. You can also open a support ticket.

When using the forums to ask a question, tag your question so that it is seen by the Machine Learning development teams.

If you have technical questions about Machine Learning, post your question on Stack Overflow External link icon and tag your question with "ibm-bluemix" and "machine-learning".

For questions about the service and getting started instructions, use the IBM developerWorks dW Answers External link icon forum. Include the "machine-learning" and "bluemix" tags. See Getting help for more details about using the forums.

For information about opening an IBM support ticket, or about support levels and ticket severities, see Contacting support.

Contents

Authorization token has not been provided.

What's happening

The REST API cannot be invoked successfully.

Why it's happening

Authorization token has not been provided in the Authorization header.

How to fix it

Pass authorization token in the Authorization header.

Invalid authorization token.

What's happening

The REST API cannot be invoked successfully.

Why it's happening

Authorization token which has been provided cannot be decoded or parsed.

How to fix it

Pass correct authorization token in the Authorization header.

Authorization token and instance_id which was used in the request are not the same.

What's happening

The REST API cannot be invoked successfully.

Why it's happening

The Authorization token which has been used is not generated for the service instance against which it was used.

How to fix it

Pass authorization token in the Authorization header which corresponds to the service instance which is being used.

Authorization token is expired.

What's happening

The REST API cannot be invoked successfully.

Why it's happening

Authorization token is expired.

How to fix it

Pass not expired authorization token in the Authorization header.

Public key needed for authentication is not available.

What's happening

The REST API cannot be invoked successfully.

Why it's happening

This is internal service issue.

How to fix it

The issue needs to be fixed by support team.

Operation timed out after {{timeout}}

What's happening

The REST API cannot be invoked successfully.

Why it's happening

The timeout occurred during performing requested operation.

How to fix it

Try to invoke desired operation again.

Unhandled exception of type {{type}} with {{status}}

What's happening

The REST API cannot be invoked successfully.

Why it's happening

This is internal service issue.

How to fix it

Try to invoke desired operation again. If it occurs more times than it needs to be fixed by support team.

Unhandled exception of type {{type}} with {{response}}

What's happening

The REST API cannot be invoked successfully.

Why it's happening

This is internal service issue.

How to fix it

Try to invoke desired operation again. If it occurs more times than it needs to be fixed by support team.

Unhandled exception of type {{type}} with {{json}}

What's happening

The REST API cannot be invoked successfully.

Why it's happening

This is internal service issue.

How to fix it

Try to invoke desired operation again. If it occurs more times than it needs to be fixed by support team.

Unhandled exception of type {{type}} with {{message}}

What's happening

The REST API cannot be invoked successfully.

Why it's happening

This is internal service issue.

How to fix it

Try to invoke desired operation again. If it occurs more times than it needs to be fixed by support team.

Requested object could not be found.

What's happening

The REST API cannot be invoked successfully.

Why it's happening

The request resource could not be found.

How to fix it

Ensure that you are referring to the existing resource.

Underlying database reported too many requests.

What's happening

The REST API cannot be invoked successfully.

Why it's happening

The user has sent too many requests in a given amount of time.

How to fix it

Try to invoke desired operation again.

The definition of the evaluation is not defined neither in the artifactModelVersion nor in the deployment. It needs to be specified \" +\n \"at least in one of the places.

What's happening

The REST API cannot be invoked successfully.

Why it's happening

Learning Configuration does not contain all required information

How to fix it

Provide definition in learning configuration

Evaluation requires learning configuration specified for the model.

What's happening

There is no possibility to create learning iteration.

Why it's happening

There is no learning configuration defined for the model.

How to fix it

Create learning configuration and try to create learning iteration again.

Evaluation requires spark instance to be provided in X-Spark-Service-Instance header

What's happening

The REST API cannot be invoked successfully.

Why it's happening

There is no all required information in learning configuration

How to fix it

Provide spark_service in Learning Configuration or in X-Spark-Service-Instance header

Model does not contain any version.

What's happening

There is no possibility to create neither deployment nor set learning configuration.

Why it's happening

There is inconsistency related to the persistence of the model.

How to fix it

Try to persist the model again and try perform the action again.

Patch operation can only modify existing learning configuration.

What's happening

There is no possibility to invoke patch REST API method to patch learning configuration.

Why it's happening

There is no learning configuration set for this model or model does not exist.

How to fix it

Endure that model exists and has already learning configuration set.

Patch operation expects exactly one replace operation.

What's happening

The deployment cannot be patched.

Why it's happening

The patch payload contains more than one operation or the patch operation is different than replace.

How to fix it

Use only one operation in the patch payload which is replace operation

The given payload is missing required fields: FIELD or the values of the fields are corrupted.

What's happening

There is no possibility to process action which is related to access to the underlying data set.

Why it's happening

The access to the data set is not properly defined.

How to fix it

Correct the access definition for the data set.

Provided evaluation method: METHOD is not supported. Supported values: VALUE.

What's happening

There is no possibility to create learning configuration.

Why it's happening

The wrong evaluation method was used to create learning configuration.

How to fix it

Use supported evaluation method which is one of: regression, binary, multiclass.

There can be only one active evaluation per model. Request could not be completed because of existing active evaluation: {{url}}

What's happening

There is no possibility to create another learning iteration

Why it's happening

There can be only one running evaluation for the model.

How to fix it

See the already running evaluation or wait till it ends and start the new one.

The deployment type {{type}} is not supported.

What's happening

There is no possibility to create deployment.

Why it's happening

Not supported deployment type was used.

How to fix it

Supported deployment type should be used.

Incorrect input: ({{message}})

What's happening

The REST API cannot be invoked successfully.

Why it's happening

There is an issue with parsing json.

How to fix it

Ensure that the correct json is passed in the request.

Insufficient data - metric {{name}} could not be calculated

What's happening

Learning iteration has failed

Why it's happening

Value for metric with defined threshold could not be calculated because of insufficient feedback data

How to fix it

Review and improve data in data source feedback_data_ref in learning configuration

For type {{type}} spark instance must be provided in X-Spark-Service-Instance header

What's happening

Deployment cannot be created

Why it's happening

batch and streaming deployments require spark instance to be provided

How to fix it

Provide spark instance in X-Spark-Service-Instance header

Action {{action}} has failed with message {{message}}

What's happening

The REST API cannot be invoked successfully.

Why it's happening

There was an issue with invoking underlying service.

How to fix it

If there is suggestion how to fix the issue than follow it. Contact the support team if there is no suggestion in the message or the suggestion does not solve the issue.

Path {{path}} is not allowed. Only allowed path for patch stream is /status

What's happening

There is no possibility to patch the stream deployment.

Why it's happening

The wrong path was used to patch the stream deployment.

How to fix it

Patch the stream deployment with supported path option which is /status (it allows to start/stop stream processing.

Patch operation is not allowed for instance of type {{$type}}

What's happening

There is no possibility to patch the deployment.

Why it's happening

The wrong deployment type is being patched.

How to fix it

Patch the stream deployment type.

Data connection {{data}} is invalid for feedback_data_ref

What's happening

There is no possibility to create learning configuration for the model.

Why it's happening

Not supported data source was used when defining feedback_data_ref.

How to fix it

Use only supported data source type which is dashdb

Path {{path}} is not allowed. Only allowed path for patch model is /deployed_version/url or /deployed_version/href for V2

What's happening

There is no option to patch model.

Why it's happening

The wrong path was used during patching of the model.

How to fix it

Patch model with supported path which allows to update the version of deployed model.

Parsing failure: {{msg}}

What's happening

The REST API cannot be invoked successfully.

Why it's happening

The requested payload could not be parsed successfully.

How to fix it

Ensure that your request payload is correct and can be parsed correctly.

Runtime environment for selected model: {{env}} is not supported for learning configuration. Supported environments: [{{supported_envs}}].

What's happening

There is no option to create learning configuration

Why it's happening

The model for which the learning_configuration was tried to be created is not supported.

How to fix it

Create learning configuration for model which has supported runtime.

Current plan \'{{plan}}\' only allows {{limit}} deployments

What's happening

There is no possibility to create deployment.

Why it's happening

The limit for number of deployments was reached for the current plan.

How to fix it

Upgrade to the plan which does not have such limitation.

Database connection definition is not valid ({{code}})

What's happening

There is no possibility utilize the learning configuration functionality.

Why it's happening

Database connection definition is not valid.

How to fix it

Try to fix the issue which is described by code returned by underlying database.

There were problems while connecting underlying {{system}}

What's happening

The REST API cannot be invoked successfully.

Why it's happening

There was an issue during connection to the underlying system. It might be temporary network issue.

How to fix it

Try to invoke desired operation again. If it occurs more times than contact support team.

Error extracting X-Spark-Service-Instance header: ({{message}})

What's happening

There is no possibility to invoke REST API which requires Spark credentials

Why it's happening

There is an issue with base-64 decoding or parsing Spark credentials.

How to fix it

Ensure that the correct Spark credentials were correctly base-64 encoded. For more information, see the documentation.

This functionality is forbidden for non beta users.

What's happening

The desired REST API cannot be invoked successfully.

Why it's happening

REST API which was invoked is currently in beta.

How to fix it

If you are interested in participating, add yourself to the wait list. The details can be found in documentation.

{{code}} {{message}}

What's happening

The REST API cannot be invoked successfully.

Why it's happening

There was an issue with invoking underlying service.

How to fix it

If there is suggestion how to fix the issue then follow it. Contact the support team if there is no suggestion in the message or the suggestion does not solve the issue.

Rate limit exceeded.

What's happening

Rate limit exceeded.

Why it's happening

Rate limit for current plan has been exceeded.

How to fix it

To solve this problem, acquire another plan with a greater rate limit

Invalid query parameter {{paramName}} value: {{value}}

What's happening

Validation error as passed incorrect value for query parameter.

Why it's happening

Error in getting result for query.

How to fix it

Correct query parameter value. The details can be found in documentation.

Invalid token type: {{type}}

What's happening

Error regarding token type.

Why it's happening

Error in authorization.

How to fix it

Token should be started with Bearer prefix

Invalid token format. Bearer token format should be used.

What's happening

Error regarding token format.

Why it's happening

Error in authorization.

How to fix it

Token should be bearer token and should start with Bearer prefix

Input JSON file is missing or invalid: 400

What's happening

The following message displays when you try to score online: Input JSON file is missing or invalid.

Why it's happening

This message displays when the scoring input payload doesn't match the expected input type that is required for scoring the model. Specifically, the following reasons may apply:

  • The input payload is empty.
  • The input payload schema is not valid.
  • The input datatypes does not match the expected datatypes.

How to fix it

Correct the input payload. Make sure that the payload has correct syntax, a valid schema, and proper data types. After you make corrections, try to score online again. For syntax issues, verify the JSON file by using the jsonlint command.

Authorization token has expired: 401

What's happening

The following message displays when you try to score online: Authorization failed.

Why it's happening

This message displays when the token that is used for scoring has expired.

How to fix it

Re-generate the token for this IBM Watson Machine Learning instance and then retry. If you still see this issue contact IBM Support.

Unknown deployment identification:404

What's happening

The following message displays when you try to score online Unknown deployment identification.

Why it's happening

This message displays when the deployment ID that is used for scoring does not exists.

How to fix it

Make sure you are providing the correct deployment ID. If not, deploy the model with the deployment ID and then try scoring it again.

Internal server error:500

What's happening

The following message displays when you try to score online: Internal server error

Why it's happening

This message displays if the downstream data flow on which the online scoring depends fails.

How to fix it

After waiting for a period of time, try to score online again. If it fails again then contact IBM Support.

Invalid type for ml_artifact: Pipeline

What's happening

The following message displays when you try to publish Spark model using Common API client library on your workstation.

Why it's happening

This message displays if you have invalid pyspark setup in operating system.

How to fix it

Set up system environment paths according to the instruction:

SPARK_HOME={installed_spark_path}
JAVA_HOME={installed_java_path}
PYTHONPATH=$SPARK_HOME/python/

ValueError: Training_data_ref name and connection cannot be None, if Pipeline Artifact is not given.

What's happening

The training data set is missing or has not been properly referenced.

Why it's happening

The Pipeline Artifact is a training data set in this instance.

How to fix it

When persisting a Spark PipelineModel you MUST supply a training data set, if you don't the client says it doesn't support PipelineModels, rather than saying a PipelineModel must be accompanied by the training set.