Practical: Calculate Precision, Recall, F1-Score and Accuracy from a Confusion Matrix (CBSE Class 12 Artificial Intelligence)
Class 12 · Artificial Intelligence
Practical: Calculate Precision, Recall, F1-Score and Accuracy from a Confusion Matrix
Classification is one of the most widely used Machine Learning techniques. It is used whenever the output belongs to predefined categories such as Yes/No, Spam/Not Spam, Disease/No Disease, or Pass/Fail.
After developing a classification model, it is important to determine how accurately it classifies new data. Instead of relying only on overall accuracy, data scientists use several evaluation metrics such as Accuracy, Precision, Recall, and F1-Score. These metrics are calculated using a Confusion Matrix.
In this practical activity, you will learn how to calculate these evaluation metrics manually as well as using Microsoft Excel.
Practical Aim
To calculate Accuracy, Precision, Recall, and F1-Score from a given Confusion Matrix using manual calculations and Microsoft Excel.
Learning Outcomes
After completing this practical, you will be able to:
- Understand the structure of a Confusion Matrix.
- Identify TP, TN, FP, and FN values.
- Calculate Accuracy.
- Calculate Precision.
- Calculate Recall.
- Calculate F1-Score.
- Interpret the performance of a classification model.
Software Required
| Software | Purpose |
|---|---|
| Microsoft Excel | Perform calculations |
| Calculator (Optional) | Verify calculations |
Theory
A Confusion Matrix summarizes the performance of a classification model by comparing the actual class labels with the predicted class labels.
It contains four important values:
- True Positive (TP)
- True Negative (TN)
- False Positive (FP)
- False Negative (FN)
Using these four values, several performance evaluation metrics can be calculated.
Sample Confusion Matrix
| Predicted Positive | Predicted Negative | |
|---|---|---|
| Actual Positive | 80 | 10 |
| Actual Negative | 5 | 105 |
Identify the Values
| Parameter | Value |
|---|---|
| True Positive (TP) | 80 |
| False Negative (FN) | 10 |
| False Positive (FP) | 5 |
| True Negative (TN) | 105 |
Formula for Accuracy
Accuracy
=
(TP + TN)
──────────────────────
TP + TN + FP + FN
Manual Calculation
TP = 80
TN = 105
FP = 5
FN = 10
Accuracy
=
(80 + 105)
────────────
80 +105 +5 +10
=
185
────
200
=
0.925
=
92.5%
Formula for Precision
Precision
=
TP
──────
TP + FP
Manual Calculation
Precision
=
80
──────
80 + 5
=
80
───
85
=
0.941
=
94.1%
Formula for Recall
Recall
=
TP
──────
TP + FN
Manual Calculation
Recall
=
80
──────
80 +10
=
80
───
90
=
0.889
=
88.9%
Formula for F1-Score
2 × Precision × Recall
F1 Score = ──────────────────────
Precision + Recall
Manual Calculation
Precision = 0.941
Recall = 0.889
F1 Score
=
2 × 0.941 × 0.889
────────────────────
0.941 +0.889
=
0.914
=
91.4%
Worksheet Layout in Excel
-----------------------------------------------
A B
-----------------------------------------------
TP 80
TN 105
FP 5
FN 10
Accuracy
Precision
Recall
F1-Score
-----------------------------------------------
Excel Formula for Accuracy
Assume:
- TP → B2
- TN → B3
- FP → B4
- FN → B5
=(B2+B3)/(B2+B3+B4+B5)
Excel Formula for Precision
=B2/(B2+B4)
Excel Formula for Recall
=B2/(B2+B5)
Excel Formula for F1-Score
Assume:
- Precision → B7
- Recall → B8
=(2*B7*B8)/(B7+B8)
Flow of Evaluation
Prediction Results
│
▼
Confusion Matrix
│
▼
Identify
TP TN FP FN
│
▼
Calculate
Accuracy
Precision
Recall
F1 Score
│
▼
Evaluate Model
Example Output
| Metric | Value |
|---|---|
| Accuracy | 92.5% |
| Precision | 94.1% |
| Recall | 88.9% |
| F1-Score | 91.4% |
Formula Explanation
| Metric | Formula | Purpose |
|---|---|---|
| Accuracy | (TP + TN) / (TP + TN + FP + FN) |
Measures the overall correctness of the classification model. |
| Precision | TP / (TP + FP) |
Measures how many predicted positive cases are actually positive. |
| Recall | TP / (TP + FN) |
Measures how many actual positive cases are correctly identified. |
| F1-Score | 2 × Precision × Recall / (Precision + Recall) |
Provides a balanced measure of Precision and Recall. |
Interpretation of Results
After calculating all four evaluation metrics, the obtained values are:
| Metric | Calculated Value | Interpretation |
|---|---|---|
| Accuracy | 92.5% | The model correctly predicts 92.5% of all observations. |
| Precision | 94.1% | Among all predicted positive cases, 94.1% are actually positive. |
| Recall | 88.9% | The model correctly detects 88.9% of actual positive cases. |
| F1-Score | 91.4% | The model has a good balance between Precision and Recall. |
Observation
- The Confusion Matrix successfully summarized the prediction results.
- All evaluation metrics were calculated correctly.
- The model achieved high Accuracy and Precision.
- The Recall value indicates that most positive cases were identified correctly.
- The F1-Score confirms that the model performs consistently.
Result
The Accuracy, Precision, Recall, and F1-Score of the classification model were successfully calculated using both manual calculations and Microsoft Excel. These evaluation metrics help determine the effectiveness of a Machine Learning classification model.
Real-Life Applications
- Spam email detection.
- Disease diagnosis systems.
- Face recognition systems.
- Credit card fraud detection.
- Fake news detection.
- Customer churn prediction.
- Online product recommendation systems.
- Sentiment analysis.
Case Study
A hospital develops an Artificial Intelligence system to identify patients suffering from pneumonia using chest X-ray images.
The hospital evaluates the model using a Confusion Matrix.
- High Recall ensures that very few pneumonia patients are missed.
- High Precision ensures that healthy patients are not incorrectly diagnosed.
- F1-Score provides a balanced evaluation of both metrics.
After evaluating these metrics, the hospital deploys the model for clinical use.
Think Like a Data Scientist
An AI model used in airport security correctly identifies almost every dangerous object but also incorrectly flags many harmless objects.
Which evaluation metric should be improved to reduce unnecessary alarms?
Click to View Answer
The Precision of the model should be improved because a low Precision indicates a large number of False Positives.
Competency-Based Question
An Artificial Intelligence model predicts whether a patient has a serious disease.
- Model A has very high Accuracy but low Recall.
- Model B has slightly lower Accuracy but very high Recall.
Which model should be preferred by the hospital? Justify your answer.
Practical File Observation
- The Confusion Matrix is the basis for evaluating classification models.
- Accuracy alone is not sufficient for every application.
- Precision becomes important when False Positives are costly.
- Recall becomes important when False Negatives are dangerous.
- F1-Score balances both Precision and Recall.
Common Errors
- Interchanging False Positive and False Negative values.
- Using incorrect formulas for Precision and Recall.
- Using percentages instead of decimal values while calculating F1-Score.
- Ignoring the importance of Recall in medical applications.
- Relying only on Accuracy for model evaluation.
Troubleshooting Tips
- Verify the TP, TN, FP, and FN values before calculations.
- Ensure the Excel formulas refer to the correct cells.
- Convert percentages to decimal values while calculating the F1-Score.
- Cross-check manual calculations with Excel results.
- Interpret each metric based on the application.
Viva Questions
- What is a Confusion Matrix?
- What does TP stand for?
- Differentiate between False Positive and False Negative.
- How is Accuracy calculated?
- How is Precision calculated?
- How is Recall calculated?
- What is the purpose of the F1-Score?
- Which metric is most important in disease diagnosis?
- Which metric is most important in spam email detection?
- Why is Accuracy alone not sufficient for evaluating all Machine Learning models?
Quick Revision
- TP = Correct Positive Prediction.
- TN = Correct Negative Prediction.
- FP = Incorrect Positive Prediction.
- FN = Incorrect Negative Prediction.
- Accuracy measures overall correctness.
- Precision measures prediction quality.
- Recall measures detection capability.
- F1-Score balances Precision and Recall.
Memory Trick
Accuracy → Overall Performance
Precision → Predicted Positive
Recall → Actual Positive
F1-Score → Balance of Precision & Recall
Exam Tips
- Memorize all four evaluation formulas.
- Learn the difference between TP, TN, FP, and FN.
- Practice solving Confusion Matrix numericals.
- Know where Precision, Recall, and Accuracy are preferred.
- Remember that F1-Score combines Precision and Recall into one metric.
Summary
- Classification models are evaluated using a Confusion Matrix.
- Accuracy measures overall prediction correctness.
- Precision measures the correctness of positive predictions.
- Recall measures the ability to detect actual positive cases.
- F1-Score balances Precision and Recall.
- These metrics help compare and improve Machine Learning classification models.
Next Topic: Practical: Python Code to Evaluate a Machine Learning Model