CS886 - Multiagent Systems - Winter 2025 Schedule & Reading List Jan 6 Course Logistics / Introduction I Course Logistics Introduction to Multiagent Systems How to read a research paper Cooperative AI: Machines Must Learn to Find Common Ground Open Problems in Cooperative AI Paper bids due Jan 12 Jan 13 Introduction II / Presentation Advice / Background Introduction to Game Theory Multiagent Reinforcement Learning Resources for Preparing Presentations: 1, 2, 3, 4, 5 Jan 20 Cooperation Social Norm Complexity and Past Reputations in the Evolution of CooperationSantos et al. [Nature 2018] Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM AgentsPiatti et al. [NeurIPS 2024 version] Jan 27 Learning and Social Dilemmas Learning with Opponent Learning AwarenessFoerster et al. [AAMAS 2018 version] Inequity aversion improves cooperation in intertemporal social dilemmasHughes et al. [NeurIPS 2018] Feb 3 Complex Groups Theory of Minds: Understanding Behavior in Groups through Inverse PlanningShum et al. [AAAI 2019] The AI Economist: Taxation policy design via two-level deep multiagent reinforcement learningZheng et al. [Science 2021] Feb 10 Strategic Reasoning Human-level play in the game of Diplomacy by combining language models with strategic reasoning Bakhtin et al. [Science 2022] STEER: Assessing the Economic Rationality of Large Language ModelsRaman et al. [ICML 2024] Project proposals due Feb 14 Feb 17 No class due to Reading Week Feb 24 Mechanism Design The Good Shepherd: An Oracle Agent for Mechanism DesignBalaguer et al [arXiv 2022] Mechanism Design for Large Language ModelsDuetting et al. [WWW 2024] March 3 Social Choice and Voting Generative Social ChoiceFish et al. [ACM EC 2024] AI can Help Humans Find Common Ground in Democratic DeliberationTessler et al. [Science 2024] March 10 Alignment Constitutional AI: Harmlessness from AI FeedbackBai et al. [2022] Fine-Tuning Language Models from Human PreferencesZiegler et al [2020] Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafailov et al [2023]] March 17 RLHF Distributional Preference Learning: Understanding and Accounting for Hidden Context in RLHFSiththaranjan et al. [ICLR 2024] Nash Learning from Human FeedbackMunos et al. [ICML 2024] Looking Forward 24 Miscellaneous Topics A Roadmap to Pluralistic AlignmentSorensen et al. [ICML 2024] Social Choice Should Guide AI Alignment in Dealing with Diverse Human FeedbackConitzer et al. [ICML 2024] March 31 Project Presentations Final project report due April 11
