7 October 2018

Types of Deep Learning

TypeGroup
Attentional InterfaceAttention-Memory
Memory-Attention NetworksAttention-Memory
One-Shot Associative MemoryAttention-Memory
KeyValue Memory NetworksAttention-Memory
Compositional Attention NetworkAttention-Memory
Deep Memory NetworkAttention-Memory
Structured Attention NetworkAttention-Memory
Hyperbolic Attention NetworkAttention-Memory
Multi-Cast Attention NetworkAttention-Memory
Bi-Directional Attention FlowAttention-Memory
Variational AutoencoderAutoencoder
AutoencoderAutoencoder
Denoising AutoencoderAutoencoder
Sparse AutoencoderAutoencoder
Contrastive AutoencoderAutoencoder
FeedforwardBasic
PerceptronBasic
Multilayer PerceptronBasic
Deep Convolutional NetworkCNN
Convolutional Deep Belief NetworkCNN
Convolutional GANCNN
DeConvolutional NetworkCNN
Deep Convolutional Inverse Graphics NetworkCNN
Geometric Deep LearningCNN
Convolutional Kernel NetworksCNN
Convolutional AutoencoderCNN
Hierarchical Convolutional Deep Maxout NetworkCNN
Deep Belief NetworkDBN
Continuous DQNDQN
Deep Q NetworkDQN
Dueling DQNDQN
Episodic-Memory DQNDQN
Bidirectional LSTMLSTM
Convolutional LSTMLSTM
Grid LSTMLSTM
Long Short Term MemoryLSTM
Peephole LSTMLSTM
Phrasal LSTMLSTM
Hierarchical LSTMLSTM
Gated Recurrent UnitLSTM
Adaptive Resonance TheoryModular
Maximum EntropyModular
CounterpropogationModular
SplineModular
GaussianModular
NeocognitronNeural
Neural ProgrammerNeural
Neural Turing MachineNeural
Neuro-FuzzyNeural
NeuroevolutionNeural
Neural Associative MemoryNeural
Neural Hawkes Process MemoryNeural
Sequence-2-SequenceOther
Deep FeedforwardOther
Deep Neural NetworkOther
Helmholtz MachineOther
Hopfield NetworkOther
Kohonen NetworkOther
Compound Hierarchical Deep ModelOther
Dense Associative MemoryOther
Hierarchical Temporal MemoryOther
Large Memory Storage and Retrieval NetworkOther
Generative Adversarial NetworkOther
Associative Neural NetworkOther
Adaptive Computation TimeOther
Deep Coding NetworkOther
Deep Deterministic Policy GradientOther
Deep Predictive Coding NetworkOther
Deep Reservoir ComputingOther
Deep Residual NetworkOther
Deep Stacking NetworkOther
Diffusion NetworkOther
Echo state NetworkOther
Elman Jordan NetworkOther
Extreme Learning MachineOther
Instantaneously Trained Neural NetworkOther
Learning Vector QuantizationOther
Liquid State MachinesOther
Spiking Neural NetworkOther
Tensor Deep Stacking NetworkOther
Radial Basis FunctionOther
Recursive Neural NetworkOther
Markov ChainProbabilistic
Deep Bayesian Neural NetworkProbabilistic
Deep Markov ModelProbabilistic
Stochastic Neural NetworkProbabilistic
Spike and Slab RBMRBM
Boltzmann MachineRBM
Restricted Bolzmann MachineRBM
Bidirectional RNNRNN
Clockwork RNNRNN
Continuous Time RNNRNN
Dilated RNNRNN
Hierarchical RNNRNN
Recurrent Neural NetworkRNN
Second Order RNNRNN
Multi-Time Scales RNNRNN
Recurrent Multilayer PerceptronRNN
Deep Kernel MachineSVM
Support Vector MachineSVM
Shallow Neural NetworksThoughtVectors/WordVectors

*Shallow = one hidden layer in NN
*Deep = more than one hidden layer in NN