Let's build GPT: from scratch, in code, spelled out
Advanced
1 views
Understanding LLMs
WorkMagic Team
Published on 11/8/2024
We build a Generatively Pre-trained Transformer (GPT), following the paper "Attention is All You Need" and OpenAI's GPT-2 / GPT-3. We talk about connections to ChatGPT, which has taken the world by storm. We watch GitHub Copilot, itself a GPT, help us write a GPT (meta :D!) . I recommend people watch the earlier make more videos to get comfortable with the autoregressive language modeling framework and basics of tensors and PyTorch nn, which we take for granted in this video.