Deep Learning

Content

1. Neural Networks
   • Foundations
     • Neuron structure
     • Activation functions
     • Network topology
   • Representation & stability
     • Embedding layer
     • Parameter initialization
     • Normalization layer
     • Residual connections
   • Learning objective
     • Cost functions
     • Regularization
     • Training vs inference
   • Optimization
     • Backpropagation
     • Optimization algorithms
     • Gradient instability
2. Convolutional Neural Networks
   • Image processing
     • Preprocessing phase
     • Feature extraction
     • Recognition phase
     • Limitations
   • CNN layers
     • Convolution layers
     • Pooling layers
     • Reshaping layers
     • Upsampling layers
   • Advanced CNNs
     • LeNet-5
     • AlexNet
     • VGGNet
     • ResNet
3. Recurrent Neural Networks
   • Traditional sequence processing
     • Sequential and temporal data
     • Markov assumption
     • Limitations
   • Recurrent neural networks
     • Architecture
     • Forward propagation and applications
     • Backpropagation through time
   • Limitations
     • Vanishing and exploding gradients
     • Long-term dependency problem
     • Computational challenges
   • Advanced RNNs
     • Long Short-Term Memory (LSTM)
     • Gated Recurrent Unit (GRU)
     • Bidirectional and deep RNNs
4. Graph Neural Networks
   • Graph structured data
     • Graph-level tasks
     • Node-level tasks
     • Edge-level tasks
   • Iterative GNNs
     • Message Passing Formalism
     • Aggregation methods
     • Limitations
   • Deep GNNs
     • Layer-Based Architecture
     • Graph Convolutional Networks (GCN)
     • Graph Attention Networks (GAT)
5. Unsupervised Deep Learning
   • Autoencoders
     • Denoising autoencoder
     • Sparse autoencoder
     • Variational autoencoder
   • Energy-based generative models
     • Boltzmann Machines
     • Restricted Boltzmann Machines
     • Deep Boltzmann Machines
   • Generative Adversarial Networks (GANs)
     • Adversarial learning framework
     • Sequence-based generator
     • Conditional GANs (cGANs)
6. Attention
   • Sequence models
     • Sequential data
     • Sequence to sequence model
     • Limitations
   • Attention mechanism
     • Formalism
     • Attention functions
     • Self-attention
     • Multi-head attention
   • Transformers
     • Architecture
     • Encoder
     • Decoder
   • Pre-trained models
     • Full-transformer
     • Encoder based
     • Decoder based

Licence

Copyright (C) 2021 Abdelkrime Aries

Attribution 4.0 International (CC BY 4.0)

https://creativecommons.org/licenses/by/4.0/