Techniques and insights for applied deep learning (computer vision) from papers published at NeurIPS 2020

As mentioned in part 1— the most important thing:) — I went through all the titles of NeurIPS 2020 papers (more than 1900!) and read abstracts of 175 papers, and extracted DL engineer relevant insights from the following papers.

This is part 2. See the part 1 below.

Rethinking Pre-training and Self-training

Using other…

Techniques and insights for applied deep learning from papers published at NeurIPS 2020

Advances in Deep Learning research are of great utility for a Deep Learning engineer working on real-world problems as most of the Deep Learning research is empirical with validation of new techniques and theories done on datasets that closely resemble real-world datasets/tasks (ImageNet pre-trained weights are still useful!).

But, churning…

The latest advances and insights for Neural Network model compression with Knowledge Distillation

Knowledge Distillation is a process where a smaller/less complex model is trained to imitate the behavior of a larger/more complex model.

Particularly when deploying NN models on mobiles or edge devices, Pruning, and model compression in general, is desirable and often the only plausible way to deploy as the memory…

The latest advances and insights for Neural Network model compression with Quantization

Neural network quantization is a process of reducing the precision of the weights in the neural network, thus reducing the memory, computation, and energy bandwidths.

Particularly when deploying NN models on mobile or edge devices, quantization, and model compression in general, is desirable and often the only plausible way to…

The latest advances and insights towards Neural Network model compression with Pruning

Neural Network (NN) Pruning is a task of reducing the size of a Neural Network by removing some of its parameters/weights.

Pruning is often performed with the objective of reducing the memory, computational, and energy bandwidths required for training and deploying NN models which are notorious for their large model…

Prabhu Prakash Kagitha

Storyteller of art and science. Deep learning & Cognitive science. Reach me at prakashkagitha@gmail.com to discuss research or collaboration or consulting.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store