Generative and Discriminative Models
Class 12 · Artificial Intelligence
7.3 Generative and Discriminative Models
Machine learning models can be broadly categorized into Generative Models and Discriminative Models, based on how they learn from data and the type of tasks they perform.
Generative models learn the underlying distribution of data and can generate new samples, whereas discriminative models learn to distinguish between different classes or categories of data.
7.3.1. Discriminative Models
Discriminative models focus on distinguishing between different classes or categories. They learn the relationship between the input data and its corresponding output or class.
These models learn class boundaries that help them decide which category a particular data item belongs to.
| Aspect | Discriminative Models |
|---|---|
| Purpose | To distinguish between different classes or categories. |
| Focus | Learning class boundaries and making predictions. |
| Output | A class, category or prediction. |
| Example | Determining whether an email is spam or not spam. |
Examples of Discriminative Models
- Logistic Regression
- Decision Trees
- Support Vector Machines (SVM)
- Random Forest
Consider an email classification system. The model is trained using examples of spam and legitimate emails. It learns characteristics that distinguish the two categories.
When a new email is received, the model determines whether it belongs to the spam or not spam category.
Since the model is distinguishing between predefined classes, this is an example of a discriminative task.
7.3.2. Generative Models
Generative models learn the underlying distribution and patterns present in a dataset. After learning these patterns, they can generate new data samples that resemble the original training data.
Generative models are therefore useful for creating new text, images, audio, video and other forms of content.
| Aspect | Generative Models |
|---|---|
| Purpose | To understand the underlying data distribution and generate new samples. |
| Focus | Learning patterns and characteristics of the training data. |
| Output | New data or content similar to the training data. |
| Example | Generating a new image of a cat after learning patterns from thousands of cat images. |
Examples of Generative Models
- Generative Adversarial Networks (GANs)
- Variational Autoencoders (VAEs)
- Large Language Models (LLMs)
- Naïve Bayes
Suppose an AI model is trained using thousands of images of cats. Instead of simply identifying whether a given image contains a cat, a generative model can learn the patterns present in those images and create a new image of a cat.
The generated image is not simply a copy of one of the training images. It is created using patterns learned from the training data.
7.3.3. Generative Models vs Discriminative Models
| Aspect | Generative Models | Discriminative Models |
|---|---|---|
| Purpose | Generate new data or content. | Distinguish between different classes. |
| Learning | Learn the underlying data distribution. | Learn class boundaries. |
| Main Task | Content generation. | Classification or prediction. |
| Output | New data samples. | Class labels or predictions. |
| Examples | GANs, VAEs, LLMs. | Logistic Regression, Decision Trees, SVM, Random Forest. |
| Example Application | Generating an image, story or audio clip. | Identifying spam or classifying an image. |
Generative → Creates
Discriminative → Distinguishes
7.3.4. Training Focus
The major difference between the two types of models lies in what they learn from the training data.
| Model Type | What It Learns | What It Does |
|---|---|---|
| Generative | Underlying data distribution and patterns. | Generates new samples. |
| Discriminative | Class boundaries and relationships between inputs and classes. | Classifies or predicts. |
7.3.5. Real-World Applications
| Task | Model Type | Example |
|---|---|---|
| Generating a story | Generative | Creating a new story using an LLM. |
| Creating an image | Generative | Generating a new image from a prompt. |
| Spam detection | Discriminative | Classifying an email as spam or not spam. |
| Image classification | Discriminative | Classifying an image into a predefined category. |
Board Exam & SQP Questions
1. What distinguishes generative models from discriminative models?
Generative models aim to understand and replicate the underlying data distribution to generate new samples, whereas discriminative models focus on distinguishing between different data classes.
2. What is the difference between the training focus of generative and discriminative models?
Generative models learn to understand the underlying data distribution, while discriminative models focus on identifying class boundaries.
3. State whether the following statement is True or False:
Discriminative models generate new data samples similar to the training data.
Answer: False.
Generative models generate new samples, whereas discriminative models distinguish between classes.
4. Name two algorithms used in discriminative modelling.
Two examples are Logistic Regression and Decision Trees.
5. Which type of model would be suitable for a facial recognition system?
A discriminative model would be suitable because the task involves distinguishing and identifying faces based on their features.
6. Give one example of a generative model and one example of a discriminative model.
Generative model: Generative Adversarial Network (GAN).
Discriminative model: Logistic Regression.
7. A model is trained to identify whether an email is spam or not spam. Is this a generative or discriminative task?
It is a discriminative task because the model distinguishes between two predefined classes: spam and not spam.
8. A model generates a new image based on patterns learned from thousands of existing images. Which type of model is being used?
A generative model is being used because the model generates a new sample based on patterns learned from existing data.
The most important distinction to remember is:
Generative Models → Learn data distribution → Generate new data
Discriminative Models → Learn class boundaries → Classify data
Generative: Creates new content such as text, images, audio and video.
Discriminative: Distinguishes between categories or classes.
Generative Examples: GANs, VAEs, LLMs.
Discriminative Examples: Logistic Regression, Decision Trees, SVM, Random Forest.