Payload logging | IBM Cloud Pak for Data as a Service

Payload logging

You can enable payload logging in Watson OpenScale to configure model evaluations.

To manage payload data for configuring fairness and drift evaluations and explainability, Watson OpenScale must log your payload data in the data mart. When you add deployments for evaluations or provide model details, Watson OpenScale can automatically send a scoring request to log your model transactions if all of the required model details are detected. If Watson OpenScale doesn't detect all of the required model details or if you're evaluating image or unstructured text models, you must manually enable payload logging.

Manual payload logging

To enable payload logging, you can use the payload logging API or a JSON file to send a scoring request. When you use the payload logging API, you can send scoring requests with cURL or with the Watson OpenScale Python SDK.

Payload logging with the Python SDK

You can send a scoring request with the Watson OpenScale Python SDK as shown in the following example:

from ibm_cloud_sdk_core.authenticators import IAMAuthenticator
from ibm_watson_openscale import APIClient

service_credentials = {
                  "apikey": "*****",
                  "url": "https://api.aiopenscale.cloud.ibm.com"
               }

authenticator = IAMAuthenticator(
        apikey=service_credentials['apikey']
    )
client = APIClient(
    service_url=service_credentials['url'],
    service_instance_id="230a8e9f-2453-4c2d-a560-2a75399210bf"
    authenticator=authenticator
)

from ibm_watson_openscale.data_sets import DataSetTypes, TargetTypes

# Put your subscription ID here
SUBSCRIPTION_ID = "<subscription_id>"
payload_logging_data_set_id = wos_client.data_sets.list(type=DataSetTypes.PAYLOAD_LOGGING, target_target_id=SUBSCRIPTION_ID, target_target_type=TargetTypes.SUBSCRIPTION).result.data_sets[0].metadata.id

from ibm_watson_openscale.supporting_classes.payload_record import PayloadRecord

# Put your scoring ID here
SCORING_ID = "<scoring_id>"
REQUEST_DATA = <put_your_data_here>
RESPONSE_DATA = <put_your_data_here>
wos_client.data_sets.store_records(data_set_id=payload_logging_data_set_id, request_body=[PayloadRecord(scoring_id=SCORING_ID, request=REQUEST_DATA, response=RESPONSE_DATA, response_time=460)])

Payload logging with cURL

You can use the following cURL command to send a scoring request:

curl -k -X POST https://$ICP_HOSTNAME:$PORT/openscale/00000000-0000-0000-0000-000000000000/v2/data_sets/<dataset_id>/records -d "$SCORING_PAYLOAD" \
--header 'Content-Type: application/json' --header 'Accept: application/json' --header "Authorization: Bearer $ICP_TOKEN"

For more information, see the Watson OpenScale v2 API.

Payload logging with JSON

You can send a scoring request with JSON as shown in the following example:

[
  {
    "request": {
      "fields": [
        "AGE",
        "BP",
        "CHOLESTEROL",
        "NA",
        "K"
      ],
      "values": [
        [
          28,
          "LOW",
          "HIGH",
          0.61,
          0.026
        ]
      ],
      "meta": {
        "fields": [
          "SEX"
        ],
        "values": [
          [
            "M"
          ]
        ]
      }
    },
    "response": {
      "fields": [
        "AGE",
        "BP",
        "CHOLESTEROL",
        "NA",
        "K",
        "probability",
        "prediction",
        "predictedLabel"
      ],
      "values": [
        [
          28,
          "LOW",
          "HIGH",
          0.61,
          0.026,
          [
            0.82,
            0.07,
            0,
            0.05,
            0.03
          ],
          0,
          "drugY"
        ]
      ]
    }
  }
]

Parent topic: Managing payload data in Watson OpenScale