TECH 77 — Modern AI Architectures: From RNNs to Chatbots
Quarter: Summer
Instructor(s): Ishaani Priyadarshini
Date(s): Jul 8—Aug 26
Class Recording Available: Yes
Class Meeting Day: Wednesdays
Class Meeting Time: 6:00—7:30 pm (PT)
Tuition: $470
Refund Deadline: Jul 10
Unit(s): 1
Status: Registration opens May 18, 8:30 am (PT)
Artificial intelligence did not emerge fully formed with ChatGPT. It evolved through a series of architectural breakthroughs that reshaped how machines process language and learn from data. This course traces that technical evolution, offering a structured framework for understanding how modern language models came to be.
Designed for students with prior exposure to machine learning, the course examines five pivotal shifts in AI architecture, beginning with the limitations of recurrent neural networks (RNNs) and the emergence of encoder-decoder models for sequence-to-sequence learning. From there, we explore attention mechanisms, the transformer architecture, and the rise of large-scale pretraining and fine-tuning, including developments such as ULMFiT, BERT, and GPT.
Throughout, technical concepts are examined at the architectural level, linking design decisions to the capabilities and constraints of today’s chatbots and generative systems. By the end, participants gain a deeper understanding of the models shaping contemporary AI and the trade-offs driving future innovation.
Designed for students with prior exposure to machine learning, the course examines five pivotal shifts in AI architecture, beginning with the limitations of recurrent neural networks (RNNs) and the emergence of encoder-decoder models for sequence-to-sequence learning. From there, we explore attention mechanisms, the transformer architecture, and the rise of large-scale pretraining and fine-tuning, including developments such as ULMFiT, BERT, and GPT.
Throughout, technical concepts are examined at the architectural level, linking design decisions to the capabilities and constraints of today’s chatbots and generative systems. By the end, participants gain a deeper understanding of the models shaping contemporary AI and the trade-offs driving future innovation.
Students should have prior exposure to core machine learning concepts, including supervised learning, model training, and evaluation. Familiarity with neural networks, particularly feedforward and recurrent models, is recommended. Proficiency in Python is required, as the course includes hands-on coding using libraries such as TensorFlow or PyTorch.