TECH 77 — Modern AI: Transformers, Language Models, and Agents
Quarter: Winter
Instructor(s): Ishaani Priyadarshini
Date(s): Jan 14—Mar 4
Class Recording Available: Yes
Class Meeting Day: Thursdays
Class Meeting Time: 6:00—7:30 pm (PT)
Tuition: $590
Refund Deadline: Jan 16
Unit(s): 1
Status: Registration opens Nov 30, 8:30 am (PT)
Artificial intelligence did not emerge fully formed with ChatGPT. It evolved through a series of architectural breakthroughs that reshaped how machines process language and learn from data. This course traces that technical evolution, offering a structured framework for understanding how modern language models came to be.
Designed for students with prior exposure to machine learning, the course examines five pivotal shifts in AI architecture, beginning with the limitations of recurrent neural networks (RNNs) and the emergence of encoder-decoder models for sequence-to-sequence learning. From there, we explore attention mechanisms, the transformer architecture, and the rise of large-scale pretraining and fine-tuning, including developments such as ULMFiT, BERT, and GPT. Throughout, technical concepts are examined at the architectural level, linking design decisions to the capabilities and constraints of today’s chatbots and generative systems. By the end, participants gain a deeper understanding of the models shaping contemporary AI and the trade-offs driving future innovation.
Designed for students with prior exposure to machine learning, the course examines five pivotal shifts in AI architecture, beginning with the limitations of recurrent neural networks (RNNs) and the emergence of encoder-decoder models for sequence-to-sequence learning. From there, we explore attention mechanisms, the transformer architecture, and the rise of large-scale pretraining and fine-tuning, including developments such as ULMFiT, BERT, and GPT. Throughout, technical concepts are examined at the architectural level, linking design decisions to the capabilities and constraints of today’s chatbots and generative systems. By the end, participants gain a deeper understanding of the models shaping contemporary AI and the trade-offs driving future innovation.
Students should have prior exposure to core machine learning concepts, including supervised learning, model training, and evaluation. Familiarity with neural networks, particularly feedforward and recurrent models, is recommended. Proficiency in Python is required, as the course includes hands-on coding using libraries such as TensorFlow or PyTorch.