Posts

PyTorch

PyTorch

deep-learning, python
0
PyTorch Lightning & Lightning Hydra Template are neat. Tensor Puzzles is great intro. TorchScale is nice library for making Transformers efficiently. BackPACK is interesting too. Link
QUOTE MAKER TOOLS

QUOTE MAKER TOOLS

apps
0
QuotesCover.com helps you create nice-looking quotes images for any purposes. Please pick a start point below Link
TRL - Transformer Reinforcement Learning

TRL - Transformer Reinforcement Learning

deep-learning
0
With trl you can train transformer language models with Proximal Policy Optimization (PPO). The library is built on top of the transformers library by 🤗 Hugging Face. Therefore, pre-trained language models can be directly loaded via transformers. At this point most of decoder architectures and encoder-decoder architectures are supported. Link
Writing a tokenizer with ChatGPT

Writing a tokenizer with ChatGPT

deep-learning
0
This morning I decided to test how good ChatGPT is at generating a non-trivial piece of code. I want to write a complete interpreter along the lines of Robert Nystrom’s excellent book Crafting Interpreters. Link
Illustrating Reinforcement Learning from Human Feedback (RLHF)

Illustrating Reinforcement Learning from Human Feedback (RLHF)

deep-learning
0
Language models have shown impressive capabilities in the past few years by generating diverse and compelling text from human input prompts. However, what makes a “good” text is inherently hard to define as it is subjective and context dependent. There are many applications such as writing stories where you want creativity, pieces of informative text which should be truthful, or code snippets that we want to be executable. Link Github