A-CNN-Roadmap

A reading roadmap of CNN and Deep Learning basics.

Table of Contents

• Books
• Courses
• Tutorials
• Blogs
  • Understanding CNN
  • Understanding Backpropagation
  • Deep Learning
• Papers
  • CNN Architectures
  • Training Techniques
  • Object Detection

Books

• Neural Networks and Deep Learning by Michael Nielsen, Determination Press, 2015.
• Deep Learning by Yoshua Bengio, Ian Goodfellow and Aaron Courville, MIT Press, 2016.
• Hands-On Machine Learning with Scikit-Learn and TensorFlow by Aurélien Geron, O'Reilly Media, 2016.

Courses

• CS231n: Convolutional Neural Networks for Visual Recognition by y Fei-Fei Li, Andrej Karpathy, Justin Johnson.
• CS224d: Deep Learning for Natural Language Processing by Richard Socher.
• NVIDIA Self-Paced Courses for Deep Learning
• Udacity - Deep Learning by Google by Vincent Vanhoucke and Arpan Chakraborty.
• Neural networks class by Hugo Larochelle.
• Bay Area Deep Learning School, Stanford 2016.
• Deep Learning Summer School, Montreal 2016.

Tutorials

• UFLDL Tutorial
• VGG Convolutional Neural Networks Practical by Andrea Vedaldi and Andrew Zisserman.
• A Tutorial on Deep Learning by Quoc Le.
  • Part 1: Nonlinear Classifiers and The Backpropagation Algorithm
  • Part 2: Autoencoders, Convolutional Neural Networks and Recurrent Neural Networks
• A Deep Learning Tutorial: From Perceptrons to Deep Networks by Ivan Vasilev.
• ICML 2016 Tutorial on Deep Residual Networks by Kaiming He.

Blogs

Understanding CNN

• A Beginner's Guide To Understanding Convolutional Neural Networks Series, by Adit Deshpande.
  • Part 1, Part 2, Part 3
• Understanding Convolution in Deep Learning by Tim Dettmers.
• Understanding Convolutions by Christopher Olah.
• [Understanding Convolutional Neural Networks for NLP] (http://www.wildml.com/2015/11/understanding-convolutional-neural-networks-for-nlp/) by Denny Britz (WildML).
• Machine Learning is Fun! Series, by Adam Geitgey.
  • Part 1: The world’s easiest introduction to Machine Learning
  • Part 2: Using Machine Learning to generate Super Mario Maker levels
  • Part 3: Deep Learning and Convolutional Neural Networks
  • Part 4: Modern Face Recognition with Deep Learning

Understanding Backpropagation

• [How the backpropagation algorithm works] (http://neuralnetworksanddeeplearning.com/chap2.html) by Michael Nielsen.
• Yes you should understand backprop by Andrej Karpathy.
• A Visual Explanation of the Back Propagation Algorithm for Neural Networks

Deep Learning

• The Neural Network Zoo.
• Neural Network Architectures by Eugenio Culurciello.
• The major advancements in Deep Learning in 2016
• Tombone's Series
  • Deep down the rabbit hole: CVPR 2015 and beyond
  • ICCV 2015: Twenty one hottest research papers
  • From feature descriptors to deep learning: 20 years of computer vision
  • Deep Learning Trends @ ICLR 2016
  • Deep Learning vs Machine Learning vs Pattern Recognition

Papers

CNN Architectures

• ResNet [Paper][Project]
  • Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, Deep Residual Learning for Image Recognition, CVPR, 2016.
• VGG-Net [Paper][Project]
  • Karen Simonyan and Andrew Zisserman, Very Deep Convolutional Networks for Large-Scale Visual Recognition, ICLR, 2015.
• GoogLeNet [Paper]
  • Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, Andrew Rabinovich, Going deeper with convolutions, CVPR, 2015.
• ZF Net [Paper]
  • Matthew Zeiler and Rob Fergus, Visualizing and Understanding Convolutional Networks, ECCV, 2014.
• AlexNet [Paper]
  • Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton, ImageNet Classification with Deep Convolutional Neural Networks, NIPS, 2012.

Training Techniques

• Batch normalization [Paper]
  • Sergey Ioffe and Christian Szegedy, Batch normalization: Accelerating deep network training by reducing internal covariate shift, ICML, 2015.
• Dropout [Paper]
  • Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov, Dropout: A Simple Way to Prevent Neural Networks from Overfitting, JMLR, 2014.

Object Detection

• R-FCN [Paper][Code][PyCode]
  • Jifeng Dai, Yi Li, Kaiming He, Jian Sun, R-FCN: Object Detection via Region-based Fully Convolutional Networks, NIPS, 2016.
• SSD [Paper][Code]
  • Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, Alexander C. Berg, SSD: Single Shot MultiBox Detector, ECCV, 2016.
• YOLO [Paper][Code]
  • Joseph Redmon, Santosh Divvala, Ross Girshick, Ali Farhadi, You Only Look Once: Unified, Real-Time Object Detection, ECCV, 2016.
• Faster R-CNN [Paper][Code] [[PyCode]] (https://github.com/rbgirshick/py-faster-rcnn)
  • Shaoqing Ren, Kaiming He, Ross Girshick, Jian Sun, Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks, NIPS, 2015.
• Fast R-CNN [Paper][Code]
  • Ross Girshick, Fast R-CNN, ICCV, 2015.
• SPP [Paper][Code]
  • Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun, Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition, ECCV, 2014.
• R-CNN [Paper][Code]
  • Ross Girshick, Jeff Donahue, Trevor Darrell, Jitendra Malik, Rich feature hierarchies for accurate object detection and semantic segmentation, CVPR, 2014.