Models

Jurassic-1 Jumbo (178B parameters) (docs, tech report).

Jurassic-1 Large (7.5B parameters) (docs, tech report).

Jurassic-1 Grande (17B parameters) with a "few tweaks" to the training process (docs, tech report).

Jurassic-1 Grande v2 beta (17B parameters)

Jurassic-2 Jumbo (178B parameters) (docs)

Jurassic-2 Grande (17B parameters) (docs)

Jurassic-2 Large (7.5B parameters) (docs)

Luminous Base (13B parameters) (docs

Luminous Extended (30B parameters) (docs

Luminous Supreme (70B parameters) (docs

A 52B parameter language model, trained using reinforcement learning from human feedback paper.

A model trained using reinforcement learning from human feedback (docs).

A lightweight version of Claude, a model trained using reinforcement learning from human feedback (docs).

Koala (13B) is a chatbot fine-tuned from Llama (13B) on dialogue data gathered from the web. (blog post)

BLOOM (176B parameters) is an autoregressive model trained on 46 natural languages and 13 programming languages (paper).

BLOOMZ (176B parameters) is BLOOM that has been fine-tuned on natural language instructions (details).

T0pp (11B parameters) is an encoder-decoder model trained on a large set of different tasks specified in natural language prompts (paper).

SantaCoder (1.1B parameters) model trained on the Python, Java, and JavaScript subset of The Stack (v1.1) (model card).

The StarCoder (15.5B parameter) model trained on 80+ programming languages from The Stack (v1.2) (model card).

Cerebras GPT is a family of open compute-optimal language models scaled from 111M to 13B parameters trained on the Eleuther Pile. (paper)

Cerebras GPT is a family of open compute-optimal language models scaled from 111M to 13B parameters trained on the Eleuther Pile. (paper)

Cohere xlarge v20220609 (52.4B parameters)

Cohere large v20220720 (13.1B parameters), which is deprecated by Cohere as of December 2, 2022.

Cohere medium v20220720 (6.1B parameters)

Cohere small v20220720 (410M parameters), which is deprecated by Cohere as of December 2, 2022.

Cohere xlarge v20221108 (52.4B parameters)

Cohere medium v20221108 (6.1B parameters)

Cohere Command beta (6.1B parameters) is fine-tuned from the medium model to respond well with instruction-like prompts (details).

Cohere Command beta (52.4B parameters) is fine-tuned from the XL model to respond well with instruction-like prompts (details).

Dolly V2 (3B) is an instruction-following large language model trained on the Databricks machine learning platform. It is based on pythia-12b.

Dolly V2 (7B) is an instruction-following large language model trained on the Databricks machine learning platform. It is based on pythia-12b.

Dolly V2 (12B) is an instruction-following large language model trained on the Databricks machine learning platform. It is based on pythia-12b.

Gopher (540B parameters) (paper).

Chinchilla (70B parameters) (paper).

GPT-J (6B parameters) autoregressive language model trained on The Pile (details).

GPT-NeoX (20B parameters) autoregressive language model trained on The Pile (paper).

Pythia (3B parameters). The Pythia project combines interpretability analysis and scaling laws to understand how knowledge develops and evolves during training in autoregressive transformers.

Pythia (7B parameters). The Pythia project combines interpretability analysis and scaling laws to understand how knowledge develops and evolves during training in autoregressive transformers.

Pythia (12B parameters). The Pythia project combines interpretability analysis and scaling laws to understand how knowledge develops and evolves during training in autoregressive transformers.

T5 (11B parameters) is an encoder-decoder model trained on a multi-task mixture, where each task is converted into a text-to-text format (paper).

UL2 (20B parameters) is an encoder-decoder model trained on the C4 corpus. It's similar to T5 but trained with a different objective and slightly different scaling knobs (paper).

Flan-T5 (11B parameters) is T5 fine-tuned on 1.8K tasks (paper).

Pathways Language Model (540B parameters) is trained using 6144 TPU v4 chips (paper).

H3 (2.7B parameters) is a decoder-only language model based on state space models (paper).

OPT-IML (175B parameters) is a suite of decoder-only transformer LMs that are multi-task fine-tuned on 2000 datasets (paper).

OPT-IML (30B parameters) is a suite of decoder-only transformer LMs that are multi-task fine-tuned on 2000 datasets (paper).

Open Pre-trained Transformers (175B parameters) is a suite of decoder-only pre-trained transformers that are fully and responsibly shared with interested researchers (paper).

Open Pre-trained Transformers (66B parameters) is a suite of decoder-only pre-trained transformers that are fully and responsibly shared with interested researchers (paper).

Open Pre-trained Transformers (6.7B parameters) is a suite of decoder-only pre-trained transformers that are fully and responsibly shared with interested researchers (paper).

Open Pre-trained Transformers (1.3B parameters) is a suite of decoder-only pre-trained transformers that are fully and responsibly shared with interested researchers (paper).

Galactica (120B parameters) is trained on 48 million papers, textbooks, lectures notes, compounds and proteins, scientific websites, etc. (paper).

Galactica (30B parameters) is trained on 48 million papers, textbooks, lectures notes, compounds and proteins, scientific websites, etc. (paper).

LLaMA is a collection of foundation language models ranging from 7B to 65B parameters.

Alpaca 7B is a model fine-tuned from the LLaMA 7B model on 52K instruction-following demonstrations.

LLaMA is a collection of foundation language models ranging from 7B to 65B parameters.

LLaMA is a collection of foundation language models ranging from 7B to 65B parameters.

LLaMA is a collection of foundation language models ranging from 7B to 65B parameters.

LLaMA is a collection of foundation language models ranging from 7B to 65B parameters.

StableLM-Base-Alpha is a suite of 3B and 7B parameter decoder-only language models pre-trained on a diverse collection of English datasets with a sequence length of 4096 to push beyond the context window limitations of existing open-source language models.

Alpaca 7B is a model fine-tuned from the LLaMA 7B model on 52K instruction-following demonstrations

Alpaca 13B is a model fine-tuned from the LLaMA 13B model on 52K instruction-following demonstrations

Alpaca 30B is a model fine-tuned from the LLaMA 30B model on 52K instruction-following demonstrations

Vicuna-13B is an open-source chatbot trained by fine-tuning LLaMA on user-shared conversations collected from ShareGPT.

TNLG v2 (530B parameters) autoregressive language model trained on a filtered subset of the Pile and CommonCrawl (paper).

TNLG v2 (6.7B parameters) autoregressive language model trained on a filtered subset of the Pile and CommonCrawl (paper).

Original GPT-3 (175B parameters) autoregressive language model (paper, docs).

Original GPT-3 (6.7B parameters) autoregressive language model (paper, docs).

Original GPT-3 (1.3B parameters) autoregressive language model (paper, docs).

Original GPT-3 (350M parameters) autoregressive language model (paper, docs).

text-davinci-003 model that involves reinforcement learning (PPO) with reward models. Derived from text-davinci-002 (docs).

text-davinci-002 model that involves supervised fine-tuning on human-written demonstrations. Derived from code-davinci-002 (docs).

text-davinci-001 model that involves supervised fine-tuning on human-written demonstrations (docs).

text-curie-001 model that involves supervised fine-tuning on human-written demonstrations (docs).

text-babbage-001 model that involves supervised fine-tuning on human-written demonstrations (docs).

text-ada-001 model that involves supervised fine-tuning on human-written demonstrations (docs).

GPT-4 is a large multimodal model (currently only accepting text inputs and emitting text outputs) that is optimized for chat but works well for traditional completions tasks. Snapshot of gpt-4 from March 14th 2023.

GPT-4 is a large multimodal model (currently only accepting text inputs and emitting text outputs) that is optimized for chat but works well for traditional completions tasks. Snapshot of gpt-4 with a longer context length of 32,768 tokens from March 14th 2023.

Codex-style model that is designed for pure code-completion tasks (docs).

code-davinci-001 model

Codex-style model that is a stronger, multilingual version of the Codex (12B) model in the Codex paper.

Sibling model Sibling model of text-davinci-003 is optimized for chat but works well for traditional completions tasks as well. Snapshot from 2023-03-01.

Sibling model Sibling model of text-davinci-003 is optimized for chat but works well for traditional completions tasks as well. Snapshot from 2023-06-13.

Sibling model to InstructGPT which interacts in a conversational way. See OpenAI's announcement. The size of the model is unknown.

GPT-JT (6B parameters) is a fork of GPT-J (blog post).

GPT-NeoXT-Chat-Base (20B) is fine-tuned from GPT-NeoX, serving as a base model for developing open-source chatbots.

RedPajama-INCITE-Base-v1 (3B parameters) is a 3 billion base model that aims to replicate the LLaMA recipe as closely as possible.

RedPajama-INCITE-Instruct-v1 (3B parameters) is a model fine-tuned for few-shot applications on the data of GPT-JT. It is built from RedPajama-INCITE-Base-v1 (3B), a 3 billion base model that aims to replicate the LLaMA recipe as closely as possible.

RedPajama-INCITE-Chat-v1 (3B parameters) is a model fine-tuned on OASST1 and Dolly2 to enhance chatting ability. It is built from RedPajama-INCITE-Base-v1 (3B), a 3 billion base model that aims to replicate the LLaMA recipe as closely as possible.

RedPajama-INCITE-Base-v1 (7B parameters) is a 7 billion base model that aims to replicate the LLaMA recipe as closely as possible.

MPT-7B is a Transformer trained from scratch on 1T tokens of text and code.

MPT-Chat (7B) is a chatbot-like model for dialogue generation. It is built by finetuning MPT-7B, a Transformer trained from scratch on 1T tokens of text and code.

MPT-Instruct (7B) is a model for short-form instruction following. It is built by finetuning MPT (7B), a Transformer trained from scratch on 1T tokens of text and code.

CodeGen (16B parameters) is an open dense code model trained for multi-turn program synthesis (blog).

GLM (130B parameters) is an open bilingual (English & Chinese) bidirectional dense model that was trained using General Language Model (GLM) procedure (paper).

CodeGeeX (13B parameters) is an open dense code model trained on more than 20 programming languages on a corpus of more than 850B tokens (blog).

Palmyra Base (5B)

Palmyra Large (20B)

InstructPalmyra (30B)

Palmyra E (30B)

Silk Road (35B)

Palmyra X (43B)

YaLM (100B parameters) is an autoregressive language model trained on English and Russian text ([GitHub](From https://github.com/yandex/YaLM-100B)).

GPT-2 implemented in Megatron-LM (paper).