Tuning a foundation model programmatically (Python)
Last updated: Feb 21, 2025
Tuning a foundation model programmatically (Python)
You can tune foundation models in IBM watsonx.ai programmatically by using the Python library.
To prompt-tune a foundation model, you run an experiment that uses training data provided by you. The experiment is a machine learning process that shows the foundation model the outputs you expect the model to return for your prompt inputs. The
tuning process is complex and involves a data asset, a training asset, and a deployment asset.
The python library has methods and helper classes for tuning foundation models. For more information about the library, see Tuning.
Use functions that are available in the watsonx.ai Python library from notebooks in watsonx.ai to prompt-tune foundation models.
The sample notebook is designed to prompt-tune the granite-13b-instruct-v2 model. But you can use it for tuning other foundation models also. To do so, replace the base_model references as follows:
base_model='google/flan-t5-xl'
If you change the foundation model, you must replace the training data also. Replace the file path in the Data loading section of the notebook.
url = "{path to your file}"
if not os.path.isfile(filename):
wget.download(url)
Copy to clipboardCopied to clipboard
You can also use a sample notebook that tunes the other foundation models that can be prompt-tuned.
Using the sample notebook to optimize tuning parameter values
Copy link to section
The sample notebook has code that optimizes the learning_rate parameter value. The sample notebook systematically changes the learning rate value and reruns the experiment 10 times, so the loss can be compared across the 10 runs.
The sample notebook calculates the optimal learning rate value for you.
The sample notebook generates 10 separate experiments; it does not run the same experiment 10 times.
The parameter to optimize is defined in the Search space and optimization section of the notebook.
You can edit or add to the sample notebook to run automated code to optimize the following parameters in addition to learning rate:
accumulate_steps
batch_size
num_epochs
To check for the optimal values for many parameters at once, you can change the sample notebook to use code like this, for example:
Optimizing many parameters at once can save time because the parameters work together. Their values affect one another and the right balance of values among them leads to the best results.
The sample notebook uses methods from the scikit-optimize library. For more information, see the scikit-optimize API reference.
Using the sample notebook to evaluate the tuned model
Copy link to section
The sample notebook has code that deploys the tuned model, inferences the tuned model, and then calculates the accuracy of the tuned model output. It also inferences the underlying foundation model and calculates the accuracy of the base model
output, so that you can see a comparison.
If you want to use the sample notebook to tune and assess other models, you can replace the value of the model_id parameter in the following code.
Remember, optimization of the tuning parameters is specific to the model and training data that you are using. If you change either the model or the training data, you need to reassess the tuning experiment. Adjust the tuning parameters again
to optimize them for your augmented data set.
About cookies on this siteOur websites require some cookies to function properly (required). In addition, other cookies may be used with your consent to analyze site usage, improve the user experience and for advertising.For more information, please review your cookie preferences options. By visiting our website, you agree to our processing of information as described in IBM’sprivacy statement. To provide a smooth navigation, your cookie preferences will be shared across the IBM web domains listed here.