What are generative (and discriminative) models?

If the model learns a distribution of the form or , where are the inputs and the outputs/labels, from which you can sample data, then it's a generative model. An example of a generative model: variational autoencoder (VAE).

Bishop also defines generative models in this way (p. 43)

Approaches that explicitly or implicitly model the distribution of inputs as well as outputs are known as generative models, because by sampling from them it is possible to generate synthetic data points in the input space

If it learns a distribution of the form , then it's a discriminative model - many/most classifiers learn this distribution, but you can also derive the conditional given the the joint and prior (that's why above Bishop uses implicitly or explicitly).

Bishop also defines discriminative models in this way (p. 43)

Approaches that model the posterior probabilities directly are called discriminative models

The related Wikipedia article claims that people have not always been using these terms consistently (which is common in machine learning), so one should always keep that in mind.

GPTs are autoregressive

As far as I know, GPTs are autoregressive models. Here is another potentially useful post that explains what autoregressive models are.

My understanding of autoregressive models, at least based on neural networks, is that they are also generative models - the linked articles and even the GPT-2 paper seem to start the descriptions from the assumption that you can factorize some joint distribution like into conditional distributions.

ChatGPT is based on a GPT model, so it's probably considered a generative model too, but there are several steps involved to create this model, so it may not be super clear how to categorise this model.

Moreover, the authors of the transformer, which GPT models are based on, claim that the transformer is an autoregressive model.

Conclusion

It seems to me that many people in ML refer to any model that generates data as a generative model, even if there's no written theoretical formulation of it as a generative model, which doesn't mean that you cannot formulate these models as generative models, i.e. a model that learns some distribution that you can use to sample data from data distribution.

I am currently not familiar enough with the details of the GPT models to say if they have been mathematically formulated as generative models of the form , but they model some distribution of the form , from which you can sample, otherwise, how could you even sample data (words)?

Answer from nbro on Stack Exchange
🌐
Bernard Marr
bernardmarr.com › home › the difference between chatgpt and generative ai
The Difference Between ChatGPT And Generative AI | Bernard Marr
July 5, 2024 - ChatGPT: A specific type of generative AI, ChatGPT is specialized in text generation. Think of it as a specialized tool within the larger AI toolkit, specifically sharpened for generating readable, coherent text.
🌐
Reddit
reddit.com › r/chatgpt › stop confusing “ai” with “generative ai”
r/ChatGPT on Reddit: Stop Confusing “AI” with “Generative AI”
October 13, 2024 - Like, okay, let’s say that artists ... cancers, optimizing engineering solutions, or other technical applications. ... ChatGPT is generative AI trained on unauthorized use of copyrighted material so I’m not sure how you disentangle ...
Discussions

machine learning - What makes ChatGPT a generative model? - Artificial Intelligence Stack Exchange
Bring the best of human thought and AI automation together at your work. Explore Stack Internal ... I'm working my way through how ChatGPT works. So I read that ChatGPT is a generative model. More on ai.stackexchange.com
🌐 ai.stackexchange.com
February 2, 2023
ELI5: What is the difference between predictive AI and generative AI?
A predictive AI predicts things like weather or traffic. A generative AI generates new content, like images or text. You could phrase it in a way that mixes them; "can you predict what a kangaroo cooking pizza would look like?" And some use cases are bit of a combination, eg AI rendering a red circle around a fracture in a X-ray. A possible difference is in how they are trained. For predictive AI, you usually feed past data with a known outcome, and then the network is modified with what is called backpropagation to change the network so that its output matches more closely with the known correct answer. Generative AI is usually taught unsupervised, meaning that it is fed data without knowing what the output should be. There is usually also a supervised learning step, for example, the AI might generate two outputs and humans then label one as better than the other. Especially in image generation, an adversial network can be used in training. Here, the adversial network tries to determine how good the output of the generative network is, and the two networks essentially compete. But, lot of techniques can really be shared between predictive and generative AI, so it's more about their purpose really. These terms aren't fully descriptive of the technical implementations, as most modern bleeding edge AI solutions tend to be a combination of many learning techniques and even many network architectures. And e.g. for weather modelling, you might use a generative network which then produces your weather prediction. More on reddit.com
🌐 r/explainlikeimfive
15
0
May 22, 2024
Benchmarking generative Ai, ChatGPT agrees my method is better than what's currently used, and why.

I see merit in when something that is humorous or creative pushes the needle toward intelligence. But what if it is not? Are humorless non creatives not intelligent? Also in regard of AGI, do our models need to be more intelligent? There are plenty of non intelligent humans that if were an AI they would be considered AGI.

More on reddit.com
🌐 r/OpenAI
8
0
November 6, 2023
What is Generative AI?
Yo dawg, I heard you like generative AI, so I got a generative AI to write about generative AI, so you can generative AI while you generative AI. More on reddit.com
🌐 r/generativeAI
12
4
August 8, 2024
🌐
McKinsey
mckinsey.com › featured-insights › mckinsey-explainers › what-is-generative-ai
What is ChatGPT, DALL-E, and generative AI? | McKinsey
April 2, 2024 - Generative artificial intelligence (AI) describes algorithms (such as ChatGPT) that can be used to create new content, including audio, code, images, text, simulations, and videos.
🌐
Upwork
upwork.com › resources › articles › chatgpt vs. generative ai: definitions and distinctions
ChatGPT vs. Generative AI: Definitions and Distinctions - Upwork
May 8, 2025 - Keep reading to learn more about generative AI and ChatGPT, and how these technologies shape digital interactions. Yes, ChatGPT is a generative AI tool.
🌐
OpenAI
openai.com › index › chatgpt
Introducing ChatGPT | OpenAI
We randomly selected a model-written message, sampled several alternative completions, and had AI trainers rank them. Using these reward models, we can fine-tune the model using Proximal Policy Optimization⁠. We performed several iterations of this process. ChatGPT is fine-tuned from a model in the GPT‑3.5 series, which finished training in early 2022.
🌐
Wesleyan
libguides.wesleyan.edu › chatgpt
What Is Generative AI? - ChatGPT and other Generative AI - LibGuides at Wesleyan University
Generative AI can be used in innumerable ways from content creation to problem solving. ChatGPT (Chat Generative Pre-trained Transformer) is, according to ChatGPT itself, “a computer program created by OpenAI that can understand and generate text like a human.
Top answer
1 of 3
4

What are generative (and discriminative) models?

If the model learns a distribution of the form or , where are the inputs and the outputs/labels, from which you can sample data, then it's a generative model. An example of a generative model: variational autoencoder (VAE).

Bishop also defines generative models in this way (p. 43)

Approaches that explicitly or implicitly model the distribution of inputs as well as outputs are known as generative models, because by sampling from them it is possible to generate synthetic data points in the input space

If it learns a distribution of the form , then it's a discriminative model - many/most classifiers learn this distribution, but you can also derive the conditional given the the joint and prior (that's why above Bishop uses implicitly or explicitly).

Bishop also defines discriminative models in this way (p. 43)

Approaches that model the posterior probabilities directly are called discriminative models

The related Wikipedia article claims that people have not always been using these terms consistently (which is common in machine learning), so one should always keep that in mind.

GPTs are autoregressive

As far as I know, GPTs are autoregressive models. Here is another potentially useful post that explains what autoregressive models are.

My understanding of autoregressive models, at least based on neural networks, is that they are also generative models - the linked articles and even the GPT-2 paper seem to start the descriptions from the assumption that you can factorize some joint distribution like into conditional distributions.

ChatGPT is based on a GPT model, so it's probably considered a generative model too, but there are several steps involved to create this model, so it may not be super clear how to categorise this model.

Moreover, the authors of the transformer, which GPT models are based on, claim that the transformer is an autoregressive model.

Conclusion

It seems to me that many people in ML refer to any model that generates data as a generative model, even if there's no written theoretical formulation of it as a generative model, which doesn't mean that you cannot formulate these models as generative models, i.e. a model that learns some distribution that you can use to sample data from data distribution.

I am currently not familiar enough with the details of the GPT models to say if they have been mathematically formulated as generative models of the form , but they model some distribution of the form , from which you can sample, otherwise, how could you even sample data (words)?

2 of 3
2

They both refer to the same type of models. However, the second definition is a more 'intuitive' explanation of what generative AI does, while the first is a definition that refers more to what a generative model is.

To generate new data similar to some training data (definition 2), a model needs to learn the training data distributions (definition 1). Only if the model has learned that distribution it can use that distribution to sample (generate) new data from that distribution.

During training, ChatGPT also learned the distribution of the training data that OpenAI provided the model with. After training, the model simply takes in the input and uses the input to sample from the learned distribution to generate an output. So ChatGPT also follows both of your definitions.

Find elsewhere
🌐
Kommunicate
kommunicate.io › blog › generative ai › 19 generative ai tools like chatgpt that you cannot ignore
19 Generative AI Tools Like ChatGPT That You Cannot Ignore
3 weeks ago - ChatGPT has carved out a path for other generative AI-based tools to tread. This is leading to innovation on a scale that is increasing on an exponential scale with generative AI tools for every application from content creation and scheduling ...
🌐
Coursera
coursera.org › coursera articles › data › ai and machine learning › is chatgpt generative ai: understanding its functioning, capabilities, and limitations
Is ChatGPT Generative AI: Understanding Its Functioning, Capabilities, and Limitations | Coursera
April 11, 2025 - Because ChatGPT is built on OpenAI’s neural network specifically designed for natural language processing (NLP) known as generative pre-trained transformer (GPT) architecture—it is generative AI.
🌐
MIT LIDS
lids.mit.edu › news-and-events › news › explained-generative-ai-how-do-powerful-generative-ai-systems-chatgpt-work-and
Explained: Generative AI - How do powerful generative AI systems like ChatGPT work, and what makes them different from other types of artificial intelligence? | MIT LIDS
November 9, 2023 - In fact, some of those headlines may actually have been written by generative AI, like OpenAI’s ChatGPT, a chatbot that has demonstrated an uncanny ability to produce text that seems to have been written by a human.
🌐
Call Centre Helper
callcentrehelper.com › the-difference-between-chatgpt-llms-and-generative-ai-222433.htm
The Difference Between ChatGPT, LLMs, and Generative AI
May 30, 2023 - Generative AI covers a wide range of creative and functional applications, with ChatGPT being one example among many, and is a broad term for AI models that produce unique and novel outputs.
🌐
ChatGPT
chatgpt.com › g › g-XNOaitB9h-generative-ai
ChatGPT - Generative AI
ChatGPT is your AI chatbot for everyday use. Chat with the most advanced AI to explore ideas, solve problems, and learn faster.
🌐
Cuny
libguides.citytech.cuny.edu › genai › chatgpt
WHAT ABOUT CHATGPT? - Generative AI - Subject Guides at New York City College of Technology
November 19, 2024 - Scroll down or click HERE to learn about other forms of generative AI. ... ChatGPT is a Large Language Model (LLM) chatbot, created by OpenAI, which responds to prompts with natural language.
🌐
LinkedIn
linkedin.com › pulse › understanding-distinction-generative-ai-vs-chatgpt-ragu
Understanding the Distinction: Generative AI vs. ChatGPT
June 10, 2023 - Generative AI has found applications in diverse areas such as art, design, and content creation. ... ChatGPT, on the other hand, is a specific implementation of generative AI designed explicitly for conversational purposes.
🌐
Wikipedia
en.wikipedia.org › wiki › ChatGPT
ChatGPT - Wikipedia
16 hours ago - ChatGPT is a generative artificial intelligence chatbot developed by OpenAI, and released in November 2022. It uses generative pre-trained transformers (GPTs), such as GPT-5, to generate text, speech, and images in response to user prompts.
🌐
Coursera
coursera.org › coursera articles › data › data science › what is chatgpt? how it works, how to use it, and more
What Is ChatGPT? How It Works, How to Use It, and More | Coursera
The intuitive, free tool has already gained popularity as an alternative to traditional search engines and as a tool for AI writing, among other things. The "GPT" in ChatGPT is short for generative pre-trained transformer.
Published   October 13, 2025
Views   214
🌐
PubMed Central
pmc.ncbi.nlm.nih.gov › articles › PMC10337400
The ChatGPT (Generative Artificial Intelligence) Revolution Has Made Artificial Intelligence Approachable for Medical Professionals - PMC
In November 2022, OpenAI publicly launched its large language model (LLM), ChatGPT, and reached the milestone of having over 100 million users in only 2 months. LLMs have been shown to be useful in a myriad of health care–related tasks and processes. In this paper, I argue that attention ...
🌐
Libguides
loomis.libguides.com › generativeai
About Generative AI and ChatGPT - ChatGPT and other Generative AI tools - LibGuides at Loomis Chaffee School
January 31, 2025 - This technology finds applications ... AI doesn't truly understand or think—it replicates patterns it has learned. ChatGPT is probably the most popular generative AI tool at this time....
🌐
Pluralsight
pluralsight.com › blog › ai & data
What are ChatGPT and Generative AI (and how can I use them)? | Online Courses, Learning Paths, and Certifications - Pluralsight
March 15, 2023 - Discriminative AI is normally used for supervised machine learning. ... ChatGPT stands for “Chat Generative Pre-Trained Transformer”, and it’s a generative AI language model that acts in a conversational way.
🌐
CNET
cnet.com › tech › services & software › ai › generative ai: everything to know about the tech behind chatbots like chatgpt
Generative AI: Everything to Know About the Tech Behind Chatbots Like ChatGPT - CNET
May 16, 2025 - Some of the most popular generative AI tools on the market include: ... Foremost among its abilities, ChatGPT can craft human-like conversations or essays based on a few simple prompts.