watsonx.ai Runtime compute usage and pricing

watsonx.ai Runtime compute usage is calculated by the number of capacity unit hours (CUH) consumed by an active machine learning instance. The rate of capacity units per hour consumed is determined by the computing requirements of your machine learning assets and models. For example, a model with a large, complex data set will consume more training resources than a model with a smaller, simpler data set. Note that scaling a deployment to support more concurrent users and requests also increases CUH consumption.

For all plans:

• Capacity-unit-hour (CUH) rate consumption for training is based on training tool, hardware specification, and runtime environment.
• Capacity-unit-hour (CUH) rate consumption for deployment is based on deployment type, hardware specification, and software specification.
• watsonx.ai Runtime places limits on the number of deployment jobs retained for each single deployment space. If you exceed your limit, you cannot create new deployment jobs until you delete existing jobs or upgrade your plan. By default, jobs metadata will be auto-delete after 30 days. You can override this value when creating a job. See Managing jobs.
• Time to idle refers to the amount of time to consider a deployment active between scoring requests. If a deployment does not receive scoring requests for a given duration, it is treated as inactive, or idle, and billing stops for all frameworks other than SPSS.
• A plan allows for at least the stated rate limit, and the actual rate limit can be higher than the stated limit. For example, the Lite plan might process more than 2 requests per second without issuing an error. If you have a paid plan and believe you are reaching the rate limit in error, contact IBM Support for assistance.
• Compute time is calculated to the millisecond. However, there is a one-minute minimum for each distinct operation. That is, a training run that takes 12 seconds is billed as one minute toward the capacity unit hour quota, while a training run that takes 83.555 seconds is billed exactly as calculated.
• The way that online deployments consume capacity units is based on framework. For some frameworks, CUH is charged for the number of hours the deployment asset is active in a deployment space. For example, SPSS models in online deployment mode that run 24 hours a day for seven days a week consume CUH and are charged for that period. There is no idle time for an active online deployment. For other frameworks, CUH is charged according to scoring duration. See the CUH consumption table for details on how CUH is calculated.

CUH consumption rates by asset type

CUH consumption by deployment and framework type

CUH consumption is calculated using these formulas:

For example, consider a Decision Optimization batch deployment job that runs for 15 minutes. Resource consumption is calculated this way: 15 minutes = 0.25 hours, on 2 nodes, and with 2 vCPU and 8 GB RAM. This combination results in a CUH rate of 30, so every time the job runs it consumes 0.25 * 2 * 30, which equals 15 CUH.

These tables show the capacity units per hour calculation for predefined machine learning environments, by usage type.

Monitoring resource usage

You can track resource usage for assets you own or collaborate on in a project or space. If you are an account owner or administrator, you can track CUH for an entire account. For more information, see Monitoring account resource usage.

You can track the runtime usage for an account on the Environment Runtimes page if you are the IBM Cloud account owner or administrator or the watsonx.ai Runtime service owner. For more information, see Monitoring resources.

CP =  client.service_instance.get_details()
CUH = CUH["entity"]["usage"]["capacity_units"]["current"]/(3600*1000)
print(CUH)

'capacity_units': {'current': 19773430}

19773430/(3600*1000)

Learn more

• Billing details for generative AI assets
• Compute options for AutoAI experiments
• Compute options for model training and scoring

Parent topic: watsonx.ai Runtime plans