deep-learning

Building makemore Part 3: Activations & Gradients, BatchNorm

0
We dive into some of the internals of MLPs with multiple layers and scrutinize the statistics of the forward pass activations, backward pass gradients, and some of the pitfalls when they are improperly scaled. We also look at the typical diagnostic tools and visualizations you’d want to use to understand the health of your deep network. We learn why training deep neural nets can be fragile and introduce the first modern innovation that made doing so much easier: Batch Normalization.
CS197 Harvard: AI Research Experiences

CS197 Harvard: AI Research Experiences

0
Once we go from training one model to training hundreds of different models with different hyperparameters, we need to start organizing. We’re going to break down our organization into three pieces: experiment tracking, hyperparameter search, and configuration setup. Link
How diffusion models work: the math from scratch

How diffusion models work: the math from scratch

0
Diffusion models are a new class of state-of-the-art generative models that generate diverse high-resolution images. They have already attracted a lot of attention after OpenAI, Nvidia and Google managed to train large-scale models. Example architectures that are based on diffusion models are GLIDE, DALLE-2, Imagen, and the full open-source stable diffusion. Link
labml.ai Annotated PyTorch Paper Implementations

labml.ai Annotated PyTorch Paper Implementations

0
This is a collection of simple PyTorch implementations of neural networks and related algorithms. These implementations are documented with explanations, and the website renders these as side-by-side formatted notes. We believe these would help you understand these algorithms better. Link
12 Most Popular NLP Projects of 2022 So Far

12 Most Popular NLP Projects of 2022 So Far

0
Natural Language Processing remains one of the hottest topics of 2022. By using GitHub stars (albeit certainly not the only measure) as a proxy for popularity, we took a look at what NLP projects are getting the most traction so far this year, just as we recently did with machine learning projects. It’s a list with some familiar names but there are plenty of surprises also! Link
Best Practices for ML Engineering

Best Practices for ML Engineering

0
This document is intended to help those with a basic knowledge of machine learning get the benefit of Google’s best practices in machine learning. It presents a style for machine learning, similar to the Google C++ Style Guide and other popular guides to practical programming. If you have taken a class in machine learning, or built or worked on a machine­-learned model, then you have the necessary background to read this document.