NYU Deep Learning (Spring 2020), worked through in PyTorch

My notebooks, slides and notes from working through NYU DS-GA 1008 Deep Learning taught by Yann LeCun and Alfredo Canziani, organised by course week.

The notebooks come from the course's practica in Alfredo Canziani's NYU-DLSP20 repository. Here they are run end to end, with outputs saved, and with my solutions to the exercises where the notebooks leave them open (for example the logic-neuron exercises in Week 1).

Course links

• Course website: atcold.github.io/NYU-DLSP20
• Original notebooks and slides: Atcold/NYU-DLSP20
• Lecture videos: NYU Deep Learning SP20 playlist

Contents

Week	Topics	Notebooks	Slides
1	History and motivation, evolution of DL, neural nets	00 Logic neuron programming 01 Tensor tutorial 02 Space stretching	Intro
2	SGD and backpropagation, backprop in practice, NN training	04 Spiral classification 05 Regression	Backprop, Spiral classification
3	Parameter transformations, ConvNets, properties of natural signals	06 ConvNet	ConvNets, CNN
4	Linear algebra and 1D convolutions	07 Listening to kernels
5	Optimisation, autograd, convolutions revisited	03 Autograd tutorial Custom gradients	Optimisation
6	ConvNet applications, RNNs and attention, training RNNs	08 Sequence classification 09 Echo data	ConvNets II, Architectures, RNN
7	Energy-based models, self-supervised learning, autoencoders	10 Autoencoder	EBM I, Generative models
8	Contrastive methods, regularised latent variables, variational autoencoders	11 VAE	EBM II, Generative models
9	Sparsity, world models, GANs	DCGAN script	EBM III, Generative models
10	Self-supervised learning in vision, predictive control	14 Truck backer-upper	SSL, Controller learning
11	Activations, losses, prediction and policy learning under uncertainty (PPUU)		EBM IV, Activations, Losses, PPUU
12	Deep learning for NLP, attention and transformers	15 Transformer	Language, Attention & transformer
13	Graph convolutional networks	16 Gated GCN	GCN, GCN practicum
14	Structured prediction, graphical methods, regularisation and Bayesian NNs	12 Regularisation 13 Bayesian NN	Structured prediction, Regularisation, Bayesian NN
15	Inference and training for latent-variable EBMs		EBM

Notebook numbers follow the course repo, so they don't always line up with the week they were covered in.

Running the notebooks

Locally (recommended, since several notebooks import helper files that sit next to them):

git clone https://github.com/Anri-Lombard/Deep-Learning-NYU.git
cd Deep-Learning-NYU
pip install torch torchvision matplotlib numpy jupyter
jupyter notebook

Some notebooks need extra packages: librosa (Week 4), torchviz (Week 5), opencv-python (Week 7), scikit-learn (Week 8), tqdm (Week 10), torchtext (Weeks 12 and 14), dgl and networkx (Week 13). The notebooks were written against 2020-era PyTorch, so newer torchtext and dgl releases may need small API changes. MNIST is downloaded by torchvision on first run.

On Colab, use the badges above. Notebooks that import plot_lib or sequential_tasks, or show images from res/, need those files uploaded to the Colab session first (they're in the same Notebooks/ folder, or in res/).

Credit and license

All course material (lectures, slides, and the original notebooks) is by Yann LeCun, Alfredo Canziani and the NYU Center for Data Science, released under CC BY-NC-SA 4.0 in Atcold/NYU-DLSP20. The Week 9 DCGAN script is from pytorch/examples. This repository is shared under the same licence, CC BY-NC-SA 4.0 (see LICENSE).

I'm not affiliated with NYU, Yann LeCun or Alfredo Canziani; this is just a student's record of learning from the material they generously made public. Any mistakes in the solutions are mine, so feel free to open an issue if you spot one.