Working of a Neural Network
Class 12 · Artificial Intelligence
6.3 Working of a Neural Network
A neural network works by processing input data through a series of interconnected neurons arranged in different layers. Each neuron performs mathematical calculations and passes its output to the next layer.
The network learns from data by adjusting its weights and biases based on the errors in its predictions. This enables the neural network to improve its performance over repeated training iterations.
How Does a Neuron Work?
Each neuron acts like a simple mathematical calculator. It receives one or more inputs, assigns importance to each input using weights, adds a bias, and then passes the result through an activation function.
| Step | What Happens? |
|---|---|
| 1. Receive Inputs | The neuron receives numerical inputs from the input layer or from neurons in the previous layer. |
| 2. Apply Weights | Each input is multiplied by a corresponding weight that represents its importance. |
| 3. Add Bias | A bias is added to the weighted sum. It helps the neuron adjust its output appropriately. |
| 4. Apply Activation Function | The calculated value is passed through an activation function to determine the neuron's output. |
| 5. Produce Output | The output is passed to the next neuron or layer. |
Weighted Inputs
Every input to a neuron has an associated weight. A weight represents the importance or influence of an input feature on the neuron's output.
The input value is multiplied by its corresponding weight before all the weighted inputs are added together.
Higher weight → Greater influence on the neuron's output.
Lower weight → Smaller influence on the neuron's output.
Weighted Sum and Bias
After multiplying the inputs by their respective weights, the neuron calculates their sum and adds a bias.
The basic mathematical calculation can be represented as:
Weighted Sum + Bias
w1x1 + w2x2 + w3x3 + b
Here, x represents the input, w represents the corresponding weight, and b represents the bias.
What is Bias?
Bias is a constant value added to the weighted sum before the activation function is applied.
Bias allows a neuron to adjust its output and helps the network learn patterns more effectively.
Activation Function
The result of the weighted sum and bias is passed through an activation function.
An activation function acts as a decision-making mechanism for the neuron. It determines the output of the neuron and introduces non-linearity, allowing the neural network to learn complex patterns.
| Activation Function | Purpose / Characteristic |
|---|---|
| Sigmoid | Produces values between 0 and 1 and is commonly useful when interpreting an output as a probability. |
| Tanh | Produces values between -1 and 1. |
| ReLU | Outputs zero for negative input values and the input value for positive values. |
Sigmoid, Tanh and ReLU are examples of activation functions. Linear Regression is a machine learning algorithm, not an activation function.
Forward Propagation
Forward Propagation is the process in which input data moves forward through the neural network, from the input layer through the hidden layers to the output layer.
During forward propagation, each neuron performs its mathematical calculation and passes the resulting output to the next layer. Finally, the network produces a prediction.
| Stage | Process |
|---|---|
| Input Layer | Receives the input features. |
| Hidden Layer(s) | Processes the inputs using weights, biases and activation functions. |
| Output Layer | Produces the final prediction or output. |
Prediction and Error
After forward propagation, the neural network produces a predicted output. This prediction is compared with the actual target value.
The difference between the predicted output and the actual output represents the error or loss.
Suppose a neural network is trained to identify whether an email is spam or not spam.
The network receives information about the email, processes the inputs through its layers and predicts: Spam.
If the actual answer is Not Spam, the prediction contains an error. The network uses this error during training to adjust its weights and biases.
Back Propagation
Back Propagation is the process used to improve a neural network by adjusting its weights and biases according to the error produced by the prediction.
After the network calculates the error, the information about the error is propagated backward through the network. The weights and biases are then fine-tuned to reduce the error in future predictions.
Forward Propagation → Makes the prediction.
Back Propagation → Uses the error to improve the network.
Forward Propagation vs Back Propagation
| Feature | Forward Propagation | Back Propagation |
|---|---|---|
| Direction | Input layer → Hidden layer(s) → Output layer | Error information moves backward through the network. |
| Purpose | Generate a prediction. | Reduce prediction error. |
| Uses | Calculates the output of the network. | Adjusts weights and biases during training. |
How Does a Neural Network Learn?
A neural network learns through repeated training iterations. The basic learning process can be represented as:
Input → Forward Propagation → Prediction → Error → Back Propagation → Weight/Bias Adjustment → Improved Prediction
This process is repeated over the training data so that the network gradually improves its ability to make accurate predictions.
Numerical Example
Consider a simple neuron with the following values:
- Input values: (1, 0, 1)
- Weights: (6, 3, 2)
- Bias: -2
The weighted sum is calculated as:
(1 × 6) + (0 × 3) + (1 × 2) + (-2)
= 6 + 0 + 2 - 2
= 6
Therefore, the value obtained before applying an activation function is 6.
Complete Working of a Neural Network
| Step | What Happens? |
|---|---|
| 1. Input | Input features are provided to the input layer. |
| 2. Weighting | Each input is multiplied by its corresponding weight. |
| 3. Summation | The weighted inputs are added together. |
| 4. Bias | A bias value is added to the weighted sum. |
| 5. Activation | The result is passed through an activation function. |
| 6. Forward Propagation | The processed information moves through the network and produces a prediction. |
| 7. Error Calculation | The prediction is compared with the actual target to determine the error. |
| 8. Back Propagation | The network uses the error to adjust weights and biases. |
| 9. Learning | The process is repeated so that the network can improve its predictions. |
Competency-Based Question
A school develops a neural network to predict whether a student is likely to perform well in an examination. The network receives inputs such as attendance percentage, previous examination marks and assignment performance.
During training, the network makes a prediction. The predicted result is different from the student's actual result.
Explain how the neural network can use this error to improve its future predictions.
Click to View Answer
The network first performs forward propagation to generate a prediction. The prediction is compared with the actual result to determine the error. During back propagation, the error is propagated backward and the network adjusts its weights and biases. Repeating this process helps reduce the error and improve future predictions.
Think Like an AI Engineer
A neural network is being trained to classify emails as Spam or Not Spam. After several training iterations, the model continues to incorrectly classify some emails.
What part of the neural network learning process should be used to adjust the network and reduce these errors?
Click to View Answer
Back Propagation should be used. It uses the error in the network's predictions to fine-tune the weights and biases, helping the network improve its accuracy during subsequent training iterations.
Common Beginner Mistakes
- Confusing weights with bias.
- Thinking that the activation function simply adds the inputs together.
- Confusing Forward Propagation with Back Propagation.
- Assuming that a neural network learns without comparing its predictions with actual results.
- Forgetting that weights and biases are adjusted during training.
- Thinking that back propagation generates the original prediction.
- Confusing activation functions such as ReLU and Sigmoid with machine learning algorithms.
Quick Revision
- A neuron receives one or more numerical inputs.
- Each input is multiplied by a corresponding weight.
- The weighted inputs are added together.
- A bias is added to the weighted sum.
- The result is passed through an activation function.
- Forward Propagation produces a prediction.
- The prediction is compared with the actual result to calculate the error.
- Back Propagation uses the error to adjust weights and biases.
- Repeated training helps the neural network improve its predictions.
Memory Trick
I → W → S → B → A → P → E → BP → L
Remember:
- I → Input
- W → Weights
- S → Sum
- B → Bias
- A → Activation
- P → Prediction
- E → Error
- BP → Back Propagation
- L → Learning
Easy Flow: Input → Weight → Sum → Bias → Activation → Prediction → Error → Back Propagation → Learning
Exam Tips
- Learn the basic calculation: Weighted Sum + Bias.
- Remember that weights represent importance or influence of inputs.
- Remember that bias is added to the weighted sum.
- Learn the purpose of an activation function: it introduces non-linearity and determines the neuron's output.
- Clearly differentiate between Forward Propagation and Back Propagation.
- For numerical questions, carefully multiply each input by its corresponding weight before adding the bias.
- In descriptive questions, write the learning sequence in order: Input → Prediction → Error → Back Propagation → Weight/Bias Adjustment.
Frequently Asked Questions (FAQs)
1. What is the role of weights in a neural network?
Weights represent the importance or influence of individual input features on the output of a neuron. The network adjusts weights during training to improve its predictions.
2. What is bias in a neural network?
Bias is a constant value added to the weighted sum before the activation function is applied. It helps the neuron adjust its output.
3. What is an activation function?
An activation function determines the output of a neuron after the weighted sum and bias have been calculated. It also introduces non-linearity, allowing the network to learn complex patterns.
4. What is Forward Propagation?
Forward Propagation is the process of moving input data from the input layer through the hidden layers to the output layer to generate a prediction.
5. What is Back Propagation?
Back Propagation is the process of using the error in a prediction to adjust the weights and biases of a neural network during training.
6. How does a neural network learn from its mistakes?
The network compares its prediction with the actual result, calculates the error and uses back propagation to adjust its weights and biases. Repeated training reduces the error and improves performance.
7. What is the difference between Forward Propagation and Back Propagation?
Forward Propagation moves information from the input layer toward the output layer to generate a prediction. Back Propagation moves error information backward through the network to adjust weights and biases.
8. Name some common activation functions.
Common activation functions include Sigmoid, Tanh and ReLU.
9. What happens after a neural network makes a prediction?
The prediction is compared with the actual target to determine the error or loss. During training, this error is then used in back propagation to improve the network.
Summary
- A neural network processes information through interconnected neurons arranged in layers.
- Each neuron receives inputs and multiplies them by corresponding weights.
- The weighted inputs are added and a bias is included.
- The result is passed through an activation function.
- Forward Propagation moves information from the input layer towards the output layer and produces a prediction.
- The prediction is compared with the actual target to determine the error.
- Back Propagation uses the error to adjust weights and biases.
- Repeated training enables the neural network to learn patterns and improve its predictions.
- Sigmoid, Tanh and ReLU are examples of activation functions.