ChatGPT from Scratch

Los Angeles, CA

ChatGPT is all over the news today as a major breakthrough in artificial intelligence. Some are saying it's the most important new technology since the invention of the computer or internet. If this is true, then it would probably be a good use of time and maybe even fun to try to understand exactly how it works. What better way to understand it then to create it from scratch? (Well, a version similar to it). Andrej Karpathy, one of the founders of OpenAI (the company that created ChatGPT) and former head of artificial intelligence for Tesla's autonomous driving division, has created a 7 part lecture series ("Neural Networks: Zero to Hero") in which he shows us how to create a GPT artificial intelligence, step by step, starting from the basics. We will do a series of seven meetups, one for each lecture. Prior to each meetup, please watch the Youtube lecture and try writing and executing the code from the Jupyter notebooks on your computer. The prerequisites are proficiency in the programming language python and a vague recollection of basic calculus. Expect to spend about three times the length of each video to properly absorb the material and try the code. Each lecture is about a 2 hour video, so expect to spend about 6 hours preparing for each meetup. We will meet per the following schedule: Tue 5/16/23: Lecture 1: Introduction to neural networks and backpropagationTue 6/13/23: Lecture 2: Introduction to language modelingTue 7/11/23: Lecture 3: Implementing a multilayer perceptron (MLP) character-level language modelTue 8/15/23: Lecture 4: Forward pass activations, backward pass gradients, and batch normalizationTue 9/12/23: Lecture 5: Cross entropy loss and tensorsTue 10/10/23: Lecture 6: Convolutional neural network architectureTue 11/7/23: Lecture 7: Building a Generatively Pretrained Transformer (GPT) All the needed course material is here: https://github.com/karpathy/nn-zero-to-hero Note from the organizer Viraj: I'm new to AI so I will be learning from the basics just like you might be. If you get stuck and want help, a good resource will be the course's discord channel: https://discord.gg/3zy8kqD9Cp