Types of Neural Networks

6.4 Types of Neural Networks

Neural networks are classified into different types based on their architecture, flow of information, and the type of problem they are designed to solve.

Different neural networks are suitable for different kinds of data. For example, CNNs are widely used for images, while RNNs are designed for sequential data such as speech and time-series information.

1. Standard Neural Network (Perceptron)

A Perceptron is one of the simplest forms of an artificial neural network. It was developed by Frank Rosenblatt in 1958.

A perceptron consists of input nodes connected to output nodes. Each connection has a weight, and the neuron uses these weighted inputs to produce an output.

Perceptrons are mainly used for simple binary classification problems.

Example:

• Spam or Not Spam
• Pass or Fail
• Yes or No

2. Feed Forward Neural Network (FFNN)

A Feed Forward Neural Network (FFNN) is a neural network in which information moves in only one direction, from the input layer towards the output layer.

It may contain an input layer, one or more hidden layers, and an output layer. There are no feedback loops in a basic feed forward neural network.

Feed Forward Neural Networks are also commonly associated with Multi-Layer Perceptrons (MLPs).

Applications:

• Image recognition
• Classification
• Regression
• Natural Language Processing
• Prediction problems

3. Convolutional Neural Network (CNN)

A Convolutional Neural Network (CNN) is a specialised type of neural network mainly designed for processing visual and image data.

CNNs use filters to identify important features such as edges, shapes, textures and patterns in an image. These features are then used to recognise objects or classify images.

CNNs are particularly effective because they can automatically learn important visual features from images.

Applications:

• Facial recognition
• Object detection
• Handwritten digit recognition
• Medical image analysis
• Autonomous vehicles
• Image classification

4. Recurrent Neural Network (RNN)

A Recurrent Neural Network (RNN) is designed to process sequential data.

Unlike a basic feed forward neural network, an RNN contains feedback connections that allow information from previous steps to influence the processing of later steps.

This ability to retain information about previous inputs makes RNNs useful for data where the order or sequence of information is important.

Applications:

• Speech recognition
• Machine translation
• Chatbots
• Sentiment analysis
• Language modelling
• Time-series forecasting

5. Generative Adversarial Network (GAN)

A Generative Adversarial Network (GAN) consists of two neural networks that work against each other in a competitive learning process.

The two networks are called the Generator and the Discriminator.

During training, the Generator tries to create increasingly realistic data, while the Discriminator becomes better at identifying whether the data is real or generated.

Applications:

• Generating realistic images
• Image and video generation
• Style transfer
• Data augmentation
• Creating synthetic data

Quick Comparison of Neural Networks

Competency-Based Question

A company wants to develop three AI systems:

• System A recognises handwritten digits from images.
• System B predicts the next word in a sentence.
• System C generates realistic artificial photographs.

Identify the most suitable neural network for each system and give a reason for your choice.

Click to View Answer

System A → CNN: CNNs are specialised for processing visual and image data and can identify important visual features.

System B → RNN: RNNs are suitable for sequential data because previous information can influence the processing of later inputs.

System C → GAN: GANs use a Generator and a Discriminator to create and evaluate synthetic data, making them suitable for generating realistic images.

Common Beginner Mistakes

• Confusing CNN with RNN. CNNs are mainly associated with visual data, whereas RNNs are designed for sequential data.
• Thinking that all neural networks have the same architecture.
• Forgetting that an RNN uses information from previous steps in a sequence.
• Confusing the Generator and Discriminator in a GAN.
• Assuming that a Perceptron is suitable for every complex classification problem.
• Choosing a neural network without considering the type of data being processed.

Memory Trick

Exam Tips

• Remember the main application of each type of neural network.
• For image-related questions, think of CNN.
• For sequential or time-series data, think of RNN.
• For generating realistic synthetic data, think of GAN.
• Remember that a GAN contains two networks: Generator and Discriminator.
• In comparison questions, clearly mention the architecture, type of data and application.

Frequently Asked Questions (FAQs)

1. Which neural network is commonly used for image recognition?

A Convolutional Neural Network (CNN) is commonly used for image recognition because it can automatically identify important visual features such as edges, shapes and textures.

2. Which neural network is suitable for sequential data?

A Recurrent Neural Network (RNN) is suitable for sequential data because it can use information from previous steps while processing new inputs.

3. What are the two networks in a GAN?

A GAN consists of a Generator and a Discriminator.

4. What does a Generator do in a GAN?

The Generator creates new synthetic data, such as artificial images, that attempts to resemble real data.

5. What does a Discriminator do in a GAN?

The Discriminator evaluates data and tries to distinguish between real data and data generated by the Generator.

6. What is a Feed Forward Neural Network?

A Feed Forward Neural Network is a neural network in which information moves only in one direction, from the input layer through the hidden layers to the output layer.

7. Which neural network is suitable for handwritten digit recognition?

A Convolutional Neural Network (CNN) is suitable because handwritten digits are visual data and CNNs are designed to identify patterns and features in images.

Summary

Neural networks have different architectures designed for different types of problems. A Perceptron is suitable for simple classification, while a Feed Forward Neural Network processes information in one direction. CNNs are particularly useful for visual data, RNNs are suitable for sequential data, and GANs use a Generator and Discriminator to create realistic synthetic data.