Custom machine learning frameworks

When is a custom machine learning engine the best choice for me?

A custom machine learning engine is the best choice when the following situations are true:

• You are not using any immediately available products to serve your machine learning models. You have a system to serve your models and no direct support exists in Watson OpenScale for that function.
• The serving engine that you are using from a 3rd-party supplier is not supported by Watson OpenScale yet. In this case, consider developing a custom machine learning engine as a wrapper to your original or native deployments.

How it works

The following image shows the custom environment support:

You can also reference the following links:

Watson OpenScale payload logging API

Python client SDK

Python SDK for IBM Watson OpenScale

• Input criteria for model to support monitors

  In the following example, your model takes a feature vector, which is essentially a collection of named fields and their values, as an input.

  {
  "fields": [
      "name",
      "age",
      "position"
  ],
  "values": [
      [
          "john",
          33,
          "engineer"
      ],
      [
          "mike",
          23,
          "student"
      ]
  ]
  
  

  The “age” field can be evaluated for fairness.

  If the input is a tensor or matrix, which is transformed from the input feature space, that model cannot be handled by the Watson OpenScale platform in the current release. By extension, deep learning models with text or image inputs cannot be handled for bias detection and mitigation.

  Additionally, training data must be loaded to support Explainability.

  For explainability on text, the full text should be one of the features. Explainability on images for a Custom model is not supported in the current release.

• Output criteria for model to support monitors

  Your model outputs the input feature vector alongside the prediction probabilities of various classes in that model.

  {
  "fields": [
      "name",
      "age",
      "position",
      "prediction",
      "probability"
  ],
  "labels": [
      "personal",
      "camping"
  ],
  "values": [
      [
          "john",
          33,
          "engineer",
          "personal",
          [
              0.6744664422398081,
              0.3255335577601919
          ]
      ],
      [
          "mike",
          23,
          "student"
          "camping",
          [
              0.2794765664946941,
              0.7205234335053059
          ]
      ]
  ]
  }
  

  In this example, "personal” and “camping” are the possible classes, and the scores in each scoring output are assigned to both classes. If the prediction probabilities are missing, bias detection works, but auto-debias does not.

  You can access the scoring output from a live scoring endpoint that Watson OpenScale can call with the REST API. For CUSTOMML, SageMaker, and IBM Watson Machine Learning, Watson OpenScale directly connects to the native scoring endpoints.

Custom machine learning engine

A custom machine learning engine provides the infrastructure and hosting capabilities for machine learning models and web applications. Custom machine learning engines that are supported by Watson OpenScale must conform to the following requirements:

• Expose two types of REST API endpoints:

  • discovery endpoint (GET list of deployments and details)
  • scoring endpoints (online and real-time scoring)

• All endpoints need to be compatible with the swagger specification to be supported.

• Input payload and output to or from the deployment must be compliant with the JSON file format that is described in the specification.

Watson OpenScale supports only the BasicAuth, none, or apiKey authentication formats.

To see the REST API endpoints specification, see Watson OpenScale REST API.

Adding a custom machine learning engine to Watson OpenScale

You can configure Watson OpenScale to work with a custom machine learning provider by using one of the following methods:

• You can use the configuration interface to add your first custom machine learning provider to Watson OpenScale. For more information, see Specifying a custom machine learning instance.
• You can also add your machine learning provider by using the Python SDK. You must use this method if you want to have more than one provider. For more information, see Add your custom machine learning engine.

Explore further

Use a custom machine learning monitor to create a way to interact with other services:

• Monitor custom machine learning engine with Watson OpenScale

Specifying a Custom ML service instance

Your first step in the Watson OpenScale tool is to specify a service instance. Your service instance is where you store your AI models and deployments.

Connect your Custom service instance

Watson OpenScale connects to AI models and deployments in a service instance. You can connect a custom service. To connect your service to Watson OpenScale, go to the Configure tab, add a machine learning provider, and click the Edit icon. In addition to a name, description and specifying the Pre-production or Production environment type, you must provide the following information that is specific to this type of service instance:

• Username
• Password
• API endpoint that uses the format https://host:port, such as https://custom-serve-engine.example.net:8443

Choose whether to connect to your deployments by requesting a list or by entering individual scoring endpoints.

Custom machine learning engine examples

Use the following ideas to set up your own custom machine learning engine.

Python and flask

You can use Python and flask to serve scikit-learn model.

To generate the drift detection model, you must use scikit-learn version 0.20.2 in the notebook.

The app can be deployed locally for testing purposes and as an application on IBM Cloud.

Node.js

You can also find an example of a custom machine learning engine that is written in Node.js here.

End2end code pattern

Code pattern showing end2end example of custom engine deployment and integration with Watson OpenScale.

Payload logging with the Custom machine learning engine

To configure payload logging for a non-IBM Watson Machine Learning or custom machine learning engine, you must bind the ML engine as custom.

CUSTOM_ENGINE_CREDENTIALS = {
    "url": "***",
    "username": "***",
    "password": "***",
}
 
wos_client.service_providers.add(
        name=SERVICE_PROVIDER_NAME,
        description=SERVICE_PROVIDER_DESCRIPTION,
        service_type=ServiceTypes.CUSTOM_MACHINE_LEARNING,
        credentials=CustomCredentials(
            url= CUSTOM_ENGINE_CREDENTIALS['url'],
            username= CUSTOM_ENGINE_CREDENTIALS['username'],
            password= CUSTOM_ENGINE_CREDENTIALS['password'],
        ),
        background_mode=False
    ).result

client.service_providers.get(service_provider_id).result.to_dict()

{
	"input_data": [{
		"fields": [
			"name",
			"age",
			"position"
		],
		"values": [
			[
			"john",
			 33,
			"engineer"
			],
			[
			"mike",
			 23,
			"student"
			]
		]
	}]
}

{
	"predictions": [{
		"fields": [
			"name",
			"age",
			"position",
			"prediction",
			"probability"
		],
		"labels": [
			"personal",
			"camping"
		],
		"values": [
			[
			"john",
			 33,
			"engineer",
			"personal",
			[
			0.6744664422398081,
			0.32553355776019194
			]
			],
			[
			"mike",
			 23,
			"student",
			"camping",
			[
			0.2794765664946941,
			0.7205234335053059
			]
			]
		]
	}]
}

custom_asset = Asset(
        asset_id=asset['entity']['asset']['asset_id'],
        name=asset['entity']['asset']['name'],
        url = "dummy_url",
        asset_type=asset['entity']['asset']['asset_type'] if 'asset_type' in asset['entity']['asset'] else 'model',
        problem_type=ProblemType.MULTICLASS_CLASSIFICATION,
        input_data_type=InputDataType.STRUCTURED,
    )
deployment = AssetDeploymentRequest(
        deployment_id=asset['metadata']['guid'],
        url=asset['metadata']['url'],
        name=asset['entity']['name'],
        deployment_type=asset['entity']['type'],
        scoring_endpoint =  scoring_endpoint
    )
asset_properties = AssetPropertiesRequest(
        prediction_field='predicted_label',
        probability_fields = ["probability"],
        training_data_reference=None,
        training_data_schema=None,
        input_data_schema=None,
        output_data_schema=output_schema,
    )
result = ai_client.subscriptions.add(
        data_mart_id=cls.datamart_id,
        service_provider_id=cls.service_provider_id,
        asset=custom_asset,
        deployment=deployment,
        asset_properties=asset_properties,
        background_mode=False
    ).result

subscription_id = subscription_details.metadata.id
subscription_id
 
details: wos_client.subscriptions.get(subscription_id).result.to_dict()

request_data = {'fields': feature_columns, 
                'values': [[payload_values]]}

response_data = {'fields': list(result['predictions'][0]),
                 'values': [list(x.values()) for x in result['predictions']]}