Deep learning falls under the umbrella term of machine learning that imitates the workings of the human brain in processing data and learning from examples to ultimately apply in decision making. Deep learning, also known as deep neural learning and sometimes deep neural network, is a sub-field of machine learning in artificial intelligence which uses networks capable of learning unsupervised from unstructured or unlabeled data. The field of deep learning is so vast that the scope of its applications is still being realized in many sectors. With that let us dive into the top trends in deep learning trends to watch for in 2023.

Improved Natural Language Processing

Although Natural Language Processing has its roots in the 1950s, there have been significant developments since then. One of the breakthroughs was achieved recently in May 2020 when Open AI released GPT3. GPT3 is an auto-regressive language model that can produce human-like text.

Why is this deep learning trend gaining popularity? It can write stories, scripts, poems, articles, and even code for that matter. It could take in instructions in plain English of what needs to be created and GPT3 can write the entire code for that application. Many language models have been released to date. GPT3 is different and the most powerful among all. One of the reasons for that is its sheer size. 

• It has 175 billion parameters which make it the largest built neural network to date. That is why it easily beats the other language models by a fair margin. 
• The other reason for the success of GPT 3 is training. GPT3 has been trained on 45 TB of text data. It employs Transformer Neural Networks. 
• Nowadays, Transformer Neural Networks are being extensively used in Language Models as they can process each word in a sequence parallelly unlike RNNs and LSTMs that process each word in a sentence one by one.

With the launch of ChatGPT, Natural Language Processing is for sure going to be in focus in 2023.

Better Gameplay Using AI

The long term goal of AI is to solve advanced real-world challenges and games have served as a stepping stone in this path for decades from Chess in 1997 to Atari in 2013. However, yet again the credit of taking AI in gaming to the next level goes to Open AI when on April 13, 2019, Open AI Five became the first AI system to defeat the world champions at an Esports game.

Why is it is included in deep learning trends? It is being considered a breakthrough because the game was DOTA 2 which runs at 30 frames per second for about 40 to 45 minutes which makes the input given to the model very complex. The second reason is the partial observability. 

• The mod model only has partial observability of the area within its proximity, unlike Chess where the model can see the entire game. 
• The third reason is the high dimensionality and the fourth is the complex rules of the game. 
• Open AI used reinforcement learning for training Open AI Five.

Deep-fakes

Recently Deep-fakes are making headlines. In this deep learning trend, Deep-fakes likeness of one person is replaced with another person in a recorded video. The recent advancements in this trend have blurred the lines of distinction between a real video and a Deep-fake even more.

Why is it is included in deep learning trends? The state of the art model for creating Deep-fakes is the First Order Motion Model. This model takes two inputs. One is the source image and the other is the driving video that contains the desired motion. 

• It creates a Deep-fake using these two inputs. 
• The First Order Motion Model works in two parts. First is the extraction module which extracts the motion from the driving video. 
• The second is the generation module which generates the motion in the source image. 

Initially, consecutive frames of videos are passed to an unsupervised key point detector to obtain sparse key points for each frame. Additionally, local affine transformations for these key points are also found.

Efficient Neural Network for Embedded Devices

When we want to scale up any model for better accuracy, we can do three things – we can either increase the number of layers, that is, the depth of the model or increase the number of units per layer, that is the width of the model or we can increase the resolution of the image. In the research on the Efficient Net paper, the researchers concluded that we can scale up the model by maintaining the efficiency by compound scaling, that is, increasing all three dimensions in a fixed ratio.

Why is it is included in deep learning trends? The efficient DET paper on the other hand was created specifically for object detection because the state of the art object detectors are expensive and given the real-life use cases of object detection like self-driving cars, expensive models can be dangerous. The idea behind DET was that most object detectors have three components which are:

1. Backbone network that extracts features from the given image.
2. Feature network that takes multiple levels of features from Backbone as inputs and outputs the list of fused features that represents salient features of the image.
3. Class/base network which uses the fused features to predict the class and location of each object.

Conclusion

Governments, global health organizations, academic research centers, and businesses have joined together to develop new techniques for collecting, aggregating, and working with data. We’ve grown accustomed to viewing the findings on news channels every day.

Technological advancement is the main reason that this pandemic is yet to kill as many people as it would without it, for example, the 1918 Spanish Flu outbreak that killed 50 million people. From advancements in medical technology and standards of care to advances in communication technology have all enabled outbreaks to be spotted more quickly and efficiently.

On a concluding note, the last trend to watch for is smarter Big Data Analytics and Insights. During the ongoing pandemic, we have seen first-hand the urgent need to quickly analyze and interpret data on the spread of viruses around the world. To learn about deep learning, you need to learn to code. For this, you can learn python from websites like python for beginners.

Disclosure of Material Connection: Some of the links in the post above are “affiliate links.” This means if you click on the link and purchase the item, I will receive an affiliate commission. Regardless, I only recommend products or services I use personally and believe will add value to my readers.