Deep Learning

Deep Learning (Deep Machine Learning) is a class of Machine Learning algorithms based on learning representations of data.
One of the goals of DL is to make better representations and create models to learn these representations from large-scale unlabeled data.
Most of the DL algorithms are applied to unsupervised learning tasks and that is the big advantage of these algorithms because unlabeled data occurs more frequently than labeled data.
Deep Learning Strengths:
  • features are automatically learned to be optimal for the task at hand
  • robustness to natural variations in the data is automatically learned
  • generalizable - the same NN approach can be used for many different applications and data types
  • scalable - performance increases with more data, this method is parallelizable
Deep Learning Weaknesses:
  • required a large dataset
  • long training period (due to large dataset)
  • the learned features are usually difficult to understand (concatenations/combinations of different features)
  • requires a good understanding of how to model multiple modalities with traditional tools
  • this method tend to learn everything - it's better to encode prior knowledge about scructure of images  
Typical Deep Learning architectures:
  • deep neural networks (DNN)
  • convolutional deep neural networks (CNN)
  • recurrent neural networks (RNN)
  • deep belief networks (DBN)
 Deep Machine Learning:
  • use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised).
  • are based on the unsupervised learning of multiple levels of features or representations of the data. 
  • higher level features are derived from lower level features to form a hierarchical representation.
Examples of Deep Learning applications - automatic:
  • object classification and detection in photographs (ImageNet classification problem)
  • automatic image caption generation
  • colorization of black and white images
  • machine translation (translation of: text, images) - instant visual translation
  • text generation
  • game playing
List of Deep Learning resources for computer vision
Best performing algorithms for computer vision tasks