Difference between traditional and deep learning methods

1) Deep Learning methods such as CNN mostly improve prediction performance with big data sets.
According to my knowledge, millions of data records are often required. When big datasets or high
computing facility are unavailable , traditional methods will come into play.

2] DL is doing very good for solving a number of image/video related problems. But, one can boost the
power of DL using image processing/traditional techniques. One has to put image processing techniques
in several layers of DL when he/she is not using CNN/LSTM etc. as black box.

4] Deep Learning really shines when it comes to complex problems such as image classification, natural
language processing, and speech recognition. [https://towardsdatascience.com/why-deep-learning-is-
needed-over-traditional-machine-learning-1b6a99177063]

5]  Deep Learning out perform other techniques if the data size is large. But with small data
size, traditional Machine Learning algorithms are preferable.

 Deep Learning techniques need to have high end infrastructure to train in reasonable time.[
https://towardsdatascience.com/why-deep-learning-is-needed-over-traditional-machine-learning-
1b6a99177063]

Why Deep Learning over Traditional Machine Learning?

Artificial Intelligence is on a rage! All of a sudden every one, whether understands or not, is
talking about it. Understanding the latest advancements in artificial intelligence can seem
overwhelming, but it really boils down to two very popular concepts Machine Learning and
Deep Learning. But lately, Deep Learning is gaining much popularity due to it’s supremacy in
terms of accuracy when trained with huge amount of data.

Just to show you the kind of attention Deep Learning is getting, here is the Google trend for the
keywrd:

Trend of “Deep Learning” in google

The software industry now-a-days moving towards machine intelligence. Machine Learning has
become necessary in every sector as a way of making machines intelligent. In a simpler way,
Machine Learning is set of algorithms that parse data, learn from them, and then apply what
they’ve learned to make intelligent decisions.

Examples of Machine Learning are everywhere. It’s how Netflix knows which show you’ll
want to watch next or how Facebook recognises your friend’s face in a digital photo. Or how a
customer service representative will know if you’ll be satisfied with their support before you
even take a CSAT survey.

The thing about traditional Machine Learning algorithms is that as complex as they may seem,
they’re still machine like. They need lot of domain expertise, human intervention only capable of
what they’re designed for; nothing more, nothing less. For AI designers and the rest of the world,
that’s where deep learning holds a bit more promise.

What is Deep Learning?

Practically, Deep Learning is a subset of Machine Learning that achieves great power and
flexibility by learning to represent the world as nested hierarchy of concepts, with each
concept defined in relation to simpler concepts, and more abstract representations computed
in terms of less abstract ones.

Elaborately, a deep learning technique learn categories incrementally through it’s hidden layer
architecture, defining low-level categories like letters first then little higher level categories like
words and then higher level categories like sentences. In the example of image recognition it
means identifying light/dark areas before categorizing lines and then shapes to allow face
recognition. Each neuron or node in the network represents one aspect of the whole and together
they provide a full representation of the image. Each node or hidden layer is given a weight that
represents the strength of its relationship with the output and as the model develops the weights
are adjusted.

Deep Learning Architecture

Distinctive Features Of Deep Learning

A big advantage with deep learning, and a key part in understanding why it’s becoming popular,
is that it’s powered by massive amounts of data. The “Big Data Era” of technology will provide
huge amounts of opportunities for new innovations in deep learning. As per Andrew Ng, the
chief scientist of China’s major search engine Baidu and one of the leaders of the Google Brain
Project, “The analogy to deep learning is that the rocket engine is the deep learning models
and the fuel is the huge amounts of data we can feed to these algorithms.”

Deep Learning requires high-end machines contrary to traditional Machine Learning

algorithms. GPU has become a integral part now to execute any Deep Learning algorithm.
In traditional Machine learning techniques, most of the applied features need to be identified by
an domain expert in order to reduce the complexity of the data and make patterns more visible to
learning algorithms to work. The biggest advantage Deep Learning algorithms as discussed
before are that they try to learn high-level features from data in an incremental manner. This
eliminates the need of domain expertise and hard core feature extraction.

Another major difference between Deep Learning and Machine Learning technique is the
problem solving approach. Deep Learning techniques tend to solve the problem end to end,
where as Machine learning techniques need the problem statements to break down to different
parts to be solved first and then their results to be combine at final stage. For example for a
multiple object detection problem, Deep Learning techniques like Yolo net take the image as
input and provide the location and name of objects at output. But in usual Machine Learning
algorithms like SVM, a bounding box object detection algorithm is required first to identify all
possible objects to have the HOG as input to the learning algorithm in order to recognize
relevant objects.

Usually, a Deep Learning algorithm takes a long time to train due to large number of
parameters. Popular ResNet algorithm takes about two weeks to train completely from scratch.
Where as, traditional Machine Learning algorithms take few seconds to few hours to train. The
scenario is completely reverse in testing phase. At test time, Deep Learning algorithm takes
much less time to run. Whereas, if you compare it with k-nearest neighbors (a type of machine
learning algorithm), test time increases on increasing the size of data. Although this is not
applicable on all machine learning algorithms, as some of them have small testing times too.

Interpretability is the main issue why many sectors using other Machine Learning techniques
over Deep Learning. Let’s take an example. Suppose we use deep learning to calculate the
relevance score of a document. The performance it gives is quite excellent and is near human
performance. But there’s is an issue. It does not reveal why it has given that score. Indeed
mathematically you can find out which nodes of a deep neural network were activated, but we
don’t know what there neurons were supposed to model and what these layers of neurons were
doing collectively. So we fail to interpret the results. Which is not in case of Machine Learning
algorithms like decision trees, logistic regression etc.

The Deep Learning approach –

Deep learning, which is a subset of machine learning has shown a significant performance and
accuracy gain in the field of computer vision. Arguably one of the most influential papers in
applying deep learning to computer vision, in 2012, a neural network containing over 60 million
parameters significantly beat previous state-of-the-art approaches to image recognition in a
popular ImageNet computer vision competition: ISVRC-2012
The boom started with the convolutional neural networks and the modified architectures of
ConvNets. By now it is said that some convNet architectures are so close to 100% accuracy of
image classification challenges, sometimes beating the human eye!

The main difference in deep learning approach of computer vision is the concept of end-to-end
learning. There’s no longer need of defining the features and do feature engineering. The neural
do that for you. It can simply put in this way.

If you want to teach a [deep] neural network to recognize a cat, for instance, you don’t tell it to
look for whiskers, ears, fur, and eyes. You simply show it thousands and thousands of photos of
cats, and eventually it works things out. If it keeps misclassifying foxes as cats, you don’t rewrite
the code. You just keep coaching it.

Though deep neural networks has its major drawbacks like, need of having huge amount of
training data and need of large computation power, the field of computer vision has already
conquered by this amazing tool already![ https://naadispeaks.wordpress.com/2018/08/12/deep-
learning-vs-traditional-computer-vision/]

The key differences can be illustrated through an example problem of vehicle

number plate interpretation:
1. Conventional image processing: A domain expert will analyze the images, identify a set
of filters, transformations and pre processing steps that would be best to uniquely identify
each digit. An algorithm (series of such transformations) would be devised that would do
a fairly good job of identifying a digit given a number plate that adheres to a fixed
format. Essentially, codify human thought process into a computer vision algorithm.
2. Machine Learning: You would get a training set (hundred thousand images maybe) of
number plates. Convert all the images into feature vectors and train a model. This model
could be chosen from an umbrella of algorithms that fall under ‘ML’: SVM (support
vector machine) to a 3 layer neural network with a softmax classifier. No clear way to
identify which algorithm to use.
3. Deep learning: It is essentially a neural network with multiple layers (usually more than
3) that would require a fairly robust dataset of images to make very accurate predictions.
Convolutional Neural Nets (CNN) deep learning models have proven to get very good
results. The plus side is that this approach provides a one-for-all architecture to solve
 such problems. However, it would require more powerful machines than a simple 3 layer
neural network.

Deep learning gives very good results for problems where sufficiently large datasets are present.
Conventional image processing would still be necessary in cases where the image dataset is not
vast enough.[ https://www.quora.com/What-is-the-different-between-using-deep-learning-
machine-learning-or-image-processing-to-classify-an-image]

How Do Machine Learning Algorithms Differ From

Traditional Algorithms?
Machine learning is an algorithm or model that learns patterns in data and then predicts similar
patterns in new data. For example, if you want to classify children’s books, it would mean that
instead of setting up precise rules for what constitutes a children’s book, developers can feed the
computer hundreds of examples of children’s books. The computer finds the patterns in these
books and uses that pattern to identify future books in that category.

Essentially, ML is a subset of artificial intelligence that enables computers to learn without being
explicitly programmed with predefined rules. It focuses on the development of computer
programs that can teach themselves to grow and change when exposed to new data. This
predictive ability, in addition to the computer’s ability to process massive amounts of data,
enables ML to handle complex business situations with efficiency and accuracy.

Traditionally, applications are programmed to make particular decisions, for example there may be a
scenario based on predefined rules. These rules are based on human experience of the frequently-
occurring scenarios. However, as the number of scenarios increases significantly, it would demand
massive investment to define rules to accurately address all scenarios, and either efficiency or accuracy
is sacrificed.

A traditional algorithm takes some input and some logic in the form of code and drums up the output.
As opposed to this, a Machine Learning Algorithm takes an input and an output and gives the some logic
which can then be used to work with new input to give one an output. The logic generated is what
makes it ML.[ https://www.analyticsindiamag.com/how-do-machine-learning-algorithms-differ-from-
traditional-algorithms/]