Practical: Python Code to Evaluate a Machine Learning Model

Developing a Machine Learning model is only the first step in solving an Artificial Intelligence problem. After training the model, it is essential to evaluate its performance to determine whether it can make accurate predictions on unseen data.

Python provides several libraries that simplify the evaluation process. One of the most widely used libraries is Scikit-learn (sklearn), which offers built-in functions to calculate various performance metrics such as Accuracy, Precision, Recall, F1-Score, and Confusion Matrix.

In this practical, you will learn how to evaluate a Machine Learning classification model using Python. You will also understand how to interpret the generated evaluation metrics.

---

Practical Aim

To evaluate a Machine Learning classification model using Python and the Scikit-learn library.

---

Learning Outcomes

After completing this practical, you will be able to:

• Import Machine Learning evaluation libraries.
• Understand Actual and Predicted values.
• Generate a Confusion Matrix.
• Calculate Accuracy.
• Calculate Precision.
• Calculate Recall.
• Calculate F1-Score.
• Interpret evaluation results.

---

Software Requirements

Software	Purpose
Python 3.x	Programming Language
Google Colab	Online Python IDE
Jupyter Notebook	Interactive Programming
VS Code / PyCharm	Desktop Python IDE
Scikit-learn Library	Machine Learning Evaluation

---

Theory

Machine Learning models make predictions based on previously learned patterns.

After making predictions, these predicted values must be compared with the actual values to determine how accurately the model performs.

Python's Scikit-learn library provides ready-made functions for evaluating classification models.

---

Evaluation Metrics Used

• Confusion Matrix
• Accuracy
• Precision
• Recall
• F1-Score

---

Required Python Module

Import the required evaluation functions.

from sklearn.metrics import confusion_matrix
from sklearn.metrics import accuracy_score
from sklearn.metrics import precision_score
from sklearn.metrics import recall_score
from sklearn.metrics import f1_score

---

Sample Dataset

Suppose the actual and predicted values are:

Actual Values

[1,1,1,0,0,1,0,1,0,0]

Predicted Values

[1,1,0,0,0,1,1,1,0,0]

Here,

• 1 = Positive
• 0 = Negative

---

Complete Python Program

from sklearn.metrics import confusion_matrix
from sklearn.metrics import accuracy_score
from sklearn.metrics import precision_score
from sklearn.metrics import recall_score
from sklearn.metrics import f1_score

# Actual values
actual = [1,1,1,0,0,1,0,1,0,0]

# Predicted values
predicted = [1,1,0,0,0,1,1,1,0,0]

# Confusion Matrix
cm = confusion_matrix(actual, predicted)

print("Confusion Matrix")
print(cm)

# Accuracy
accuracy = accuracy_score(actual, predicted)

# Precision
precision = precision_score(actual, predicted)

# Recall
recall = recall_score(actual, predicted)

# F1 Score
f1 = f1_score(actual, predicted)

print("Accuracy :", accuracy)

print("Precision :", precision)

print("Recall :", recall)

print("F1 Score :", f1)

---

Sample Output

Confusion Matrix

[[4 1]
 [1 4]]

Accuracy : 0.80

Precision : 0.80

Recall : 0.80

F1 Score : 0.80

---

Understanding the Confusion Matrix

The output

[[4 1]
 [1 4]]

represents:

Value	Meaning
TN = 4	Correctly predicted negative cases.
FP = 1	Negative cases predicted as positive.
FN = 1	Positive cases predicted as negative.
TP = 4	Correctly predicted positive cases.

---

Program Explanation

Statement	Purpose
confusion_matrix()	Generates the Confusion Matrix.
accuracy_score()	Calculates Accuracy.
precision_score()	Calculates Precision.
recall_score()	Calculates Recall.
f1_score()	Calculates F1-Score.

---

Workflow

Train Model

      │

      ▼

Generate Predictions

      │

      ▼

Compare Actual & Predicted Values

      │

      ▼

Generate Confusion Matrix

      │

      ▼

Calculate Accuracy

Precision

Recall

F1 Score

      │

      ▼

Evaluate Model

---

Dry Run Table

Step	Operation	Result
1	Import sklearn.metrics	Functions Loaded
2	Create Actual List	10 Values
3	Create Predicted List	10 Values
4	Generate Confusion Matrix	2 × 2 Matrix
5	Calculate Metrics	Accuracy, Precision, Recall, F1
6	Display Results	Evaluation Completed

---

Trace Table

Metric	Calculated Value
Accuracy	0.80
Precision	0.80
Recall	0.80
F1-Score	0.80

---

Think Like an AI Engineer

An AI model predicts whether a transaction is fraudulent. After evaluating the model using Python, the Recall value is very low.

What does this indicate?

Click to View Answer

A low Recall indicates that the model is missing many actual fraudulent transactions (False Negatives). The model should be improved to correctly identify more positive cases.

---

Observation

• The Machine Learning model was successfully evaluated using Python.
• The Confusion Matrix displayed the number of correct and incorrect predictions.
• Accuracy, Precision, Recall, and F1-Score were calculated using Scikit-learn functions.
• The evaluation metrics helped measure the overall performance of the classification model.
• The Python program produced results quickly without performing manual calculations.

---

Result

The Machine Learning classification model was successfully evaluated using Python and the Scikit-learn library. The Confusion Matrix, Accuracy, Precision, Recall, and F1-Score were generated successfully and used to assess the model's performance.

---

Real-Life Applications

• Email Spam Detection.
• Credit Card Fraud Detection.
• Medical Disease Diagnosis.
• Face Recognition Systems.
• Customer Churn Prediction.
• Fake News Detection.
• Sentiment Analysis.
• Cyber Security Threat Detection.

---

Case Study

An online banking company develops an Artificial Intelligence model to detect fraudulent credit card transactions.

After training the model, Python is used to generate the Confusion Matrix and calculate Accuracy, Precision, Recall, and F1-Score.

The evaluation shows that the model has a high Accuracy of 98% but a comparatively lower Recall value. Since missing fraudulent transactions is risky, the bank improves the model and retrains it until the Recall value also improves.

---

Think Like an AI Engineer

A hospital develops a Machine Learning model to detect cancer from medical reports. The evaluation results are:

• Accuracy = 97%
• Precision = 95%
• Recall = 72%

Is this model suitable for deployment? Explain your answer.

Click to View Answer

No. Although the Accuracy and Precision are high, the Recall value is comparatively low. This means the model is missing many actual cancer patients (False Negatives), which can have serious medical consequences. The Recall should be improved before deployment.

---

Competency-Based Question

An Artificial Intelligence company develops two classification models.

Metric	Model A	Model B
Accuracy	95%	92%
Recall	74%	94%

If these models are being developed for disease detection, which model should be selected? Justify your answer.

---

Practical File Observation

• Scikit-learn provides ready-made functions for evaluating Machine Learning models.
• The Confusion Matrix summarizes prediction results.
• Accuracy measures overall correctness.
• Precision measures prediction quality.
• Recall measures the ability to detect actual positive cases.
• F1-Score balances Precision and Recall.
• Python significantly reduces manual calculations.

---

Common Errors

• Forgetting to install the Scikit-learn library.
• Using Actual and Predicted lists of different lengths.
• Importing incorrect functions.
• Misspelling the evaluation function names.
• Interpreting the Confusion Matrix incorrectly.
• Ignoring the importance of Precision and Recall.

---

Troubleshooting Tips

• Ensure that the Actual and Predicted lists contain the same number of elements.
• Verify that the Scikit-learn library is installed.
• Import the required evaluation functions correctly.
• Check the syntax carefully while calling the evaluation functions.
• Interpret the Confusion Matrix before calculating the evaluation metrics.

---

Best Practices

• Always evaluate the model using unseen testing data.
• Use multiple evaluation metrics instead of Accuracy alone.
• Interpret the evaluation metrics based on the application.
• Use Recall for medical diagnosis and fraud detection applications.
• Document the evaluation results for future comparison.

---

Viva Questions

1. What is Scikit-learn?
2. Which Python library is commonly used to evaluate Machine Learning models?
3. Which function generates the Confusion Matrix?
4. How is Accuracy calculated in Python?
5. Which function calculates Precision?
6. Which function calculates Recall?
7. Which function calculates the F1-Score?
8. Why is Recall important in disease detection?
9. Can Accuracy alone determine the quality of a Machine Learning model?
10. Why is the Confusion Matrix important?

---

Quick Revision

---

Memory Trick

---

Exam Tips

• Remember the names of all Scikit-learn evaluation functions.
• Practice writing the complete Python program without referring to notes.
• Understand the interpretation of the Confusion Matrix.
• Know the importance of each evaluation metric.
• Be able to explain which metric is most suitable for different real-world applications.

---

Frequently Asked Questions (FAQs)

1. Which Python library is commonly used for evaluating Machine Learning models?

The Scikit-learn (sklearn) library is the most commonly used Python library for evaluating Machine Learning models.

2. Which function generates the Confusion Matrix?

The confusion_matrix() function generates the Confusion Matrix.

3. Why are multiple evaluation metrics used instead of only Accuracy?

Accuracy alone may be misleading, especially for imbalanced datasets. Precision, Recall, and F1-Score provide a more complete evaluation of the model.

4. Can these evaluation functions be used for regression models?

No. These functions are primarily used for classification models. Regression models use metrics such as MSE, RMSE, MAE, and R² Score.

5. Why should a model be evaluated before deployment?

Evaluation ensures that the model performs accurately on unseen data and is reliable enough for real-world applications.

---

Summary

• Python simplifies Machine Learning model evaluation through the Scikit-learn library.
• The Confusion Matrix summarizes the prediction results.
• Accuracy, Precision, Recall, and F1-Score provide different perspectives on model performance.
• Choosing the appropriate evaluation metric depends on the application.
• Proper model evaluation ensures that Artificial Intelligence systems are accurate, reliable, and ready for deployment.