Supported foundation models available with watsonx.ai
Supported foundation models available with watsonx.ai
A collection of open source and IBM foundation models are deployed in IBM watsonx.ai. You can prompt the deployed foundation models in the Prompt Lab or programmatically.
To understand how the model provider, instruction tuning, token limits, and other factors can affect which model you choose, see Choosing a model.
IBM foundation models
The following table lists the supported foundation models that IBM provides for inferencing. All IBM models are instruction-tuned.
Some IBM foundation models are also available from Hugging Face. License terms for IBM models that you access from Hugging Face are available from the Hugging Face website. For more information about contractual protections related to IBM indemnification
for IBM foundation models that you access in watsonx.ai, see the IBM Client Relationship Agreement and IBM watsonx.ai service description.
In addition to working with foundation models that are curated by IBM, you can upload and deploy your own foundation models. After the custom models are deployed and registered with watsonx.ai, you can create prompts that inference the custom
models from the Prompt Lab.
The available foundation models support a range of use cases for both natural languages and programming languages. To see the types of tasks that these models can do, review and try the sample prompts.
allam-1-13b-instruct
The allam-1-13b-instruct foundation model is a bilingual large language model for Arabic and English provided by the National Center for Artificial Intelligence and supported by the Saudi Authority for Data and Artificial Intelligence that
is fine-tuned to support conversational tasks. The ALLaM series is a collection of powerful language models designed to advance Arabic language technology. These models are initialized with Llama-2 weights and undergo training on both Arabic
and English languages.
Note:
This foundation model is available only in the Frankfurt data center. When you inference this model from the Prompt Lab, disable AI guardrails.
Usage
Supports Q&A, summarization, classification, generation, extraction, and translation in Arabic.
allam-1-13b-instruct is based on the Allam-13b-base model, which is a foundation model that is pre-trained on a total of 3 trillion tokens in English and Arabic, including the tokens seen from its initialization. The Arabic data set contains
500 billion tokens after cleaning and deduplication. The additional data is collected from open-source collections and web crawls. The allam-1-13b-instruct foundation model is fine-tuned with a curated set of 4 million Arabic and 6 million
English prompt-and-response pairs.
Note: The maximum new tokens, which means the tokens generated by the foundation model, is limited to 8192.
Supported natural languages
English
Supported programming languages
The codellama-34b-instruct-hf foundation model supports many programming languages, including Python, C++, Java, PHP, Typescript (Javascript), C#, Bash, and more.
Instruction tuning information
The instruction fine-tuned version was fed natural language instruction input and the expected output to guide the model to generate helpful and safe answers in natural language.
The elyza-japanese-llama-2-7b-instruct model is provided by ELYZA, Inc on Hugging Face. The elyza-japanese-llama-2-7b-instruct foundation model is a version of the Llama 2 model from Meta that is trained to understand
and generate Japanese text. The model is fine-tuned for solving various tasks that follow user instructions and for participating in a dialog.
Note:
This foundation model is available only in the Tokyo data center. When you inference this model from the Prompt Lab, disable AI guardrails.
Usage
General use with zero- or few-shot prompts. Works well for classification and extraction in Japanese and for translation between English and Japanese. Performs best when prompted in
Japanese.
For Japanese language training, Japanese text from many sources were used, including Wikipedia and the Open Super-large Crawled ALMAnaCH coRpus (a multilingual corpus that is generated by classifying and filtering language in the Common
Crawl corpus). The model was fine-tuned on a data set that was created by ELYZA. The ELYZA Tasks 100 data set contains 100 diverse and complex tasks that were created manually and evaluated by humans. The ELYZA Tasks 100 data
set is publicly available from HuggingFace.
The flan-t5-xl-3b model is provided by Google on Hugging Face. This model is based on the pretrained text-to-text transfer transformer (T5) model and uses instruction fine-tuning methods to achieve better zero- and few-shot performance. The model is also fine-tuned with chain-of-thought data to improve its ability to perform reasoning tasks.
Note:
This foundation model can be tuned by using the Tuning Studio.
The model was fine-tuned on tasks that involve multiple-step reasoning from chain-of-thought data in addition to traditional natural language processing tasks. Details about the training data sets used are published.
The flan-t5-xxl-11b model is provided by Google on Hugging Face. This model is based on the pretrained text-to-text transfer transformer (T5) model and uses instruction fine-tuning methods to achieve better zero- and few-shot performance. The model is also fine-tuned with chain-of-thought data to improve its ability to perform reasoning tasks.
The model was fine-tuned on tasks that involve multiple-step reasoning from chain-of-thought data in addition to traditional natural language processing tasks. Details about the training data sets used are published.
The flan-ul2-20b model is provided by Google on Hugging Face. This model was trained by using the Unifying Language Learning Paradigms (UL2). The model is optimized for language generation, language understanding, text classification, question
answering, common sense reasoning, long text reasoning, structured-knowledge grounding, and information retrieval, in-context learning, zero-shot prompting, and one-shot prompting.
The flan-ul2-20b model is pretrained on the colossal, cleaned version of Common Crawl's web crawl corpus. The model is fine-tuned with multiple pretraining objectives to optimize it for various natural language processing tasks. Details
about the training data sets used are published.
The granite-13b-chat-v2 model is provided by IBM. This model is optimized for dialog use cases and works well with virtual agent and chat applications.
Usage: Generates dialog output like a chatbot. Uses a model-specific prompt format. Includes a keyword in its output that can be used as a stop sequence to produce succinct answers. Follow the prompting guidelines for tips
on usage. For more information, see Prompting granite-13b-chat-v2.
Note:
This foundation model supports skills contributed by the open source community from InstructLab.
The Granite family of models is trained on enterprise-relevant data sets from five domains: internet, academic, code, legal, and finance. Data used to train the models first undergoes IBM data governance reviews and is filtered of text
that is flagged for hate, abuse, or profanity by the IBM-developed HAP filter. IBM shares information about the training methods and data sets used.
The granite-13b-instruct-v2 model is provided by IBM. This model was trained with high-quality finance data, and is a top-performing model on finance tasks. Financial tasks evaluated include: providing sentiment scores for stock and earnings
call transcripts, classifying news headlines, extracting credit risk assessments, summarizing financial long-form text, and answering financial or insurance-related questions.
Note:
This foundation model can be tuned by using the Tuning Studio.
Usage
Supports extraction, summarization, and classification tasks. Generates useful output for finance-related tasks. Uses a model-specific prompt format. Accepts special characters, which can be used for generating structured output.
The Granite family of models is trained on enterprise-relevant data sets from five domains: internet, academic, code, legal, and finance. Data used to train the models first undergoes IBM data governance reviews and is filtered of text
that is flagged for hate, abuse, or profanity by the IBM-developed HAP filter. IBM shares information about the training methods and data sets used.
The granite-7b-lab foundation model is provided by IBM. The granite-7b-lab foundation model uses a novel alignment tuning method from IBM Research. Large-scale Alignment for chatBots, or LAB is a method for adding new skills to existing foundation
models by generating synthetic data for the skills, and then using that data to tune the foundation model.
Usage
Supports general purpose tasks, including extraction, summarization, classification, and more. Follow the prompting guidelines for tips on usage. For more information, see Prompting granite-7b-lab.
Note:
This foundation model supports skills contributed by the open source community from InstructLab.
IBM-developed foundation models are considered part of the IBM Cloud Service. When you use the granite-7b-lab foundation model that is provided in watsonx.ai the contractual protections related to IBM indemnification apply. See the
IBM Client Relationship Agreement and IBM watsonx.ai service description.
The granite-8b-japanese model is provided by IBM. The granite-8b-japanese foundation model is based on the IBM Granite Instruct foundation model and is trained to understand and generate Japanese text.
Note:
This foundation model is available only in the Tokyo data center. When you inference this model from the Prompt Lab, disable AI guardrails.
Usage
Useful for general purpose tasks in the Japanese language, such as classification, extraction, question-answering, and for language translation between Japanese and English.
The Granite family of models is trained on enterprise-relevant data sets from five domains: internet, academic, code, legal, and finance. The granite-8b-japanese model was pretrained on 1 trillion tokens of English and 0.5 trillion tokens
of Japanese text.
A foundation model from the IBM Granite family. The granite-20b-multilingual foundation model is based on the IBM Granite Instruct foundation model and is trained to understand and generate text in English, German, Spanish, French, and Portuguese.
Usage
English, German, Spanish, French, and Portuguese closed-domain question answering, summarization, generation, extraction, and classification.
Note:
This foundation model supports skills contributed by the open source community from InstructLab.
The Granite family of models is trained on enterprise-relevant data sets from five domains: internet, academic, code, legal, and finance. Data used to train the models first undergoes IBM data governance reviews and is filtered of text
that is flagged for hate, abuse, or profanity by the IBM-developed HAP filter. IBM shares information about the training methods and data sets used.
Foundation models from the IBM Granite family. The Granite code foundation models are instruction-following models fine-tuned using a combination of Git commits paired with human instructions and open-source synthetically
generated code instruction data sets.
Note:
These foundation models are available only in the Dallas data center. When you inference these models from the Prompt Lab, disable AI guardrails.
Usage
Granite code foundation models are designed to respond to coding-related instructions and can be used to build coding assitants. For more information and sample prompts, see Prompts for code.
granite-3b-code-instruct : 128,000 The maximum new tokens, which means the tokens generated by the foundation model, is limited to 4096.
granite-8b-code-instruct : 128,000 The maximum new tokens, which means the tokens generated by the foundation model, is limited to 8192.
granite-20b-code-instruct : 8192 The maximum new tokens, which means the tokens generated by the foundation model, is limited to 4096.
granite-34b-code-instruct : 8192
Supported natural languages
English
Supported programming languages
The Granite code foundation models support 116 programming languages including Python, Javascript, Java, C++, Go, and Rust. For the full list, see IBM foundation models.
Instruction tuning information
These models were fine-tuned from Granite code base models on a combination of permissively licensed instruction data to enhance instruction-following capabilities including logical reasoning and problem-solving
skills.
Jais-13b-chat is based on the Jais-13b model, which is a foundation model that is trained on 116 billion Arabic tokens and 279 billion English tokens. Jais-13b-chat is fine-tuned with a curated set of 4 million Arabic and 6 million English
prompt-and-response pairs.
The Meta Llama 3 family of foundation models are accessible, open large language models that are built with Meta Llama 3 and provided by Meta on Hugging Face.
The Llama 3 foundation models are instruction fine-tuned language models that can support various use cases.
Note: The maximum new tokens, which means the tokens generated by the foundation model, is limited to 4096.
Supported natural languages
English
Instruction tuning information
Llama 3 features improvements in post-training procedures that reduce false refusal rates, improve alignment, and increase diversity in the foundation model output. The result is better reasoning, code generation, and instruction-following
capabilities. Llama 3 has more training tokens (15T) that result in better language comprehension.
The Meta Llama 3.1 collection of foundation models are provided by Meta. The Llama 3.1 foundation models are pretrained and instruction tuned text-only generative models that are optimized
for multilingual dialogue use cases. The models use supervised fine-tuning and reinforcement learning with human feedback to align with human preferences for helpfulness and safety.
The llama-3-405b-instruct model is Meta's largest open-sourced foundation model to date. This foundation model can also be used as a synthetic data generator, post-training data ranking judge, or model teacher/supervisor
that can improve specialized capabilities in more inference-friendly, derivative models.
Usage
Generates dialog output like a chatbot. Uses a model-specific prompt format.
Although the model supports a context window length of 131,072, the window is limited to 16,384 to reduce the time it takes for the model to generate a response.
Supported natural languages
English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai
Instruction tuning information
Llama 3.1 was pretrained on 15 trillion tokens of data from publicly available sources. The fine-tuning data includes publicly available instruction datasets, as well as over 25 million synthetically generated examples.
The Llama 2 Chat model is provided by Meta on Hugging Face. The fine-tuned model is useful for chat generation. The model is pretrained with publicly available online data and fine-tuned using reinforcement learning
from human feedback.
You can choose to use the 13 billion parameter or 70 billion parameter version of the model.
Note: The 13 billion parameter version of this foundation model can be tuned by using the Tuning Studio.
Usage
Generates dialog output like a chatbot. Uses a model-specific prompt format.
Llama 2 was pretrained on 2 trillion tokens of data from publicly available sources. The fine-tuning data includes publicly available instruction data sets and more than one million new examples that were annotated by humans.
The llama2-13b-dpo-v7 foundation model is provided by Minds & Company. The llama2-13b-dpo-v7 foundation model is a version of llama2-13b foundation model from Meta that is instruction-tuned and fine-tuned by
using the direct preference optimzation method to handle Korean.
Note:
This foundation model is available only in the Tokyo data center. When you inference this model from the Prompt Lab, disable AI guardrails.
Usage
Suitable for many tasks, including classification, extraction, summarization, code creation and conversion, question-answering, generation, and retreival-augmented generation in Korean.
Direct preference optimzation (DPO) is an alternative to reinforcement learning from human feedback. With reinforcement learning from human feedback, responses must be sampled from a language model and an intermediate step of training a
reward model is required. The direct preference optimzation uses a binary method of reinforcement learning where the model chooses the best of two answers based on preference data.
Mistral Large 2 is a large language model developed by Mistral Al. The mistral-large foundation model is fluent in and understands the grammar and cultural context of English, French, Spanish, German, and Italian. The foundation model can
also understand dozens of other languages. The model has a large context window, which means you can add large documents as contextual information in prompts that you submit for retrieval-augmented generation (RAG) use cases. The mistral-large
foundation model is effective at programmatic tasks, such as generating, reviewing, and commenting on code, and can generate results in JSON format.
Usage
Suitable for complex multilingual reasoning tasks, including text understanding, transformation, and code generation. Due to the model's large context window, use the max tokens parameter to specify a token limit when prompting the model.
Although the model supports a context window length of 128,000, the window is limited to 32,768 to reduce the time it takes for the model to generate a response.
The maximum new tokens, which means the tokens generated by the foundation model, is limited to 16,384.
Supported natural languages
English, French, German, Italian, Spanish, and dozens of other languages.
Supported programming languages
The mistral-large model has been trained on over 80 programming languages including Python, Java, C, C++, JavaScript, Bash, Swift, and Fortran.
Instruction tuning information
The mistral-large foundation model is pre-trained on diverse data sets like text, codebases, and mathematical data from various domains.
Model architecture
Decoder-only
License
For terms of use, including information about contractual protections related to capped indemnification, see Terms of use.
The mixtral-8x7b-instruct-v01 foundation model is provided by Mistral AI. The mixtral-8x7b-instruct-v01 foundation model is a pretrained generative sparse mixture-of-experts network that groups the model parameters,
and then for each token chooses a subset of groups (referred to as experts) to process the token. As a result, each token has access to 47 billion parameters, but only uses 13 billion active parameters for inferencing, which reduces
costs and latency.
Usage
Suitable for many tasks, including classification, summarization, generation, code creation and conversion, and language translation. Due to the model's unusually large context window, use the max tokens parameter to specify a token limit
when prompting the model.
The mt0-xxl-13b model is provided by BigScience on Hugging Face. The model is optimized to support language generation and translation tasks with English, languages other than English, and multilingual prompts.
Usage: General use with zero- or few-shot prompts. For translation tasks, include a period to indicate the end of the text you want translated or the model might continue
the sentence rather than translate it.
Any deprecated foundation models are highlighted with a deprecated warning icon . For more information about deprecation, including foundation
model withdrawal dates, see Foundation model lifecycle.