The most common jargon in the field of Artificial Intelligence is the ‘Neural Network’. This is a part of Artificial Intelligence where predictive models by Machine Learning are based on the network of a human brain.
Table of Contents:
- What is a Neural Network?
- What Frameworks do Deep Neural Networks run on?
- The Activation Function
What is a Neural Network?
Before we move any further into the topic of Deep Neural Networks, we need to understand what Neural Networks are. There is a term called Layers in Neural Networks.
A Layer is the basic building block of any Neural Network. Let us take the human brain for example. We know that all brains are made up of neurons. These neurons, also called nodes in the field of Artificial Intelligence, are the computational units. When these computational units are stacked together, a single layer formation occurs. It is possible to form multiple layers if a single layer is stacked with others.
The layer present at the beginning is called the input layer. This input layer consists of the already pre-processed data that we intend to work with. Next, we run our algorithms to obtain an output that is used to feed the computation for the next layer.
A single neuron or node as we call it is one layer (let us say the input layer), and it is connected to all the neurons or nodes of the succeeding layer (this is called Hidden Layer 1). Now, all the nodes of Hidden Layer 1 are connected to each of the neurons or nodes of the succeeding layer (let us call this Hidden Layer 2), which again joins to the next layer (let’s say Hidden Layer 3) in the same manner. This process keeps on recurring till we reach our output layer. It is this Layer where we obtain the results we were looking for, on the particular pre-processed data we were building our model on. If you do this on a piece of paper by drawing lines and connecting the nodes of one layer with the successive layer, then you will see that there is a formation of a network. When there is a presence of more than two Hidden Layers in a Neural Network, it is called a Deep Neural Network.
What Frameworks do Deep Neural Networks run on?
When we talk about frameworks on which Deep Neural Networks run, we’re basically talking about the frameworks on which Deep Learning can be executed. When the development of models occurs on Deep Neural Networks, it is said to be based on Deep Learning.
Some frameworks used for Deep Learning are:
- TensorFlow: TensorFlow is probably the most used framework for Deep Learning. Released by Google at the end of 2015, TensorFlow is a framework that is not only used in Deep Neural Networks but also in Convolutional and Recurrent Neural Networks. It aids the fields of NLP or Natural Language Processing, classification of text, recognising speech/image/handwriting (Google Lens for example). TensorFlow uses both the GPU and CPU to compute for models. You can also line GPUs together, as it supports the usage of multiple GPUs, increasing the computing power significantly.
- Keras: Keras is a wrapper for TensorFlow. In simpler words, Keras written in Python is a high-level deep learning Application Programming Interface, also known as API. Since it was comparatively difficult for programmers to completely understand and write code for TensorFlow, Keras was developed as an experiment. This Neural Network Library mainly uses TensorFlow as its backend but can run on Theano as well. With Keras, one can have access to numerous Neural Network Libraries which not only include Deep Neural Networks but also Convolutional and Recurrent Neural Networks. It is pretty lightweight and is really easy to use when compared to frameworks like TensorFlow.
- Apache MXNet: Developed by the Apache Software Foundation, released in the year 2021, Apache MXNet or MXNet is an open-source framework for Deep Learning that is used for training and deploying models based on Deep Neural Networks. The Unique Selling Point or USP of MXNet is its functionality of Distributed Training. It supports many programming languages like Python, R, C++, Java, Go, Julia, Scala and even Perl.
The Activation Function
The Activation Function, also known as the Transfer Function, is used to get the output of a node which constitutes the Layers of a Deep Neural Network. The activation function allows the computer to learn patterns of complex data being fed to it and decides what output to release to the node of the next layer (this includes the Output Layer as well as the Hidden Layers). There are mainly two categories of activation functions, they are Non-Linear and Linear Activation functions.
Though there isn’t much use of Linear Activation functions in a Deep Neural Network as the models are based on more than one input variable being fed to the Network, Non-Linear Activation Functions are used widely in Deep Neural Networks. A few Non-Linear Activation Functions are Softmax Activation Function, Sigmoid Activation Function, ReLu or Rectified Linear Unit Activation Function, tanh (Hyperbolic Tangent) Activation Function, Leaky ReLU or Leaky Rectified Linear Unit Activation Function.
The usage of Deep Neural Networks has almost taken over the automation industry, starting with building models, Deep Learning and Deep Neural Networks have found their use in many fields such as Fraud Detection, Computer Vision, Natural Language Processing, refining of data and cognitive computation.
Automotive companies which offer you self-driven cars, Tesla for example, also use Deep Neural Networks to make their cars auto-driven. Deep Neural Networks and Deep Learning are radical innovation that has taken over almost all of the processes that can be automated and are here to stay. With time, it will improve more in terms of efficiency and speed moving us to a future where humans just have to command the machines and computers to do all the work for them.