Special Topics in Large Language Models

Course Information

Course Description: This graduate level special topics course examines emerging frontiers in large language models and their expanding roles across cognitive science, human AI interaction, and the social sciences. Students will explore state of the art research in areas such as cognitive architectures, reasoning and planning, compositionality, social cognition, and test time scaling, as well as applications of language models in domains including law, medicine, journalism, and scientific discovery.

Each student or team will select a focused topic, conduct a comprehensive literature review, and lead a seminar style lecture and discussion. The course culminates in a semester long research or implementation project, presented as a final paper and in class presentation. Projects should emphasize novel failure modes, under explored behaviors, or emerging risks rather than incremental performance gains.

Assessment

Student learning is assessed based on how effectively students identify, define, and solve research problems in their chosen topic.

• Research Project (50%) originality, rigor, and clarity in identifying and addressing a novel research question
• Topic Presentation (30%) analytical depth, synthesis of literature, and ability to communicate complex ideas effectively
• Participation and Discussion (20%) quality of contributions to peer learning and critical engagement with readings

Participation and Discussion

Participation is evaluated based on the quality of contributions to seminar discussion and critical engagement with readings. Lecture recordings support class participation.

• Come prepared to discuss the assigned readings
• Ask precise questions and challenge assumptions constructively
• Use Slack for clarifications and ongoing discussion

Course Schedule

We will cover a variety of machine-centered topics and human-centered topics. Based on knowledge you gain during the class, your team will conduct semester-long "research" project. Pay attention to due dates for projects.

Reading

Topic Presentation (30m talk and 15m discussion)

Reading for Presentation is organized around topic blocks, with four papers assigned per block and typically two papers discussed in each class session. Each student is required to present a total of two to three papers over the course of the term. Students may indicate their topic and paper preferences using the provided interest form. Presenters should select papers across perspectives, with at least one paper from the human centered track and one from the machine centered track.

Class Projects

The course includes a semester long research or implementation project. Projects should go beyond surface level performance gains and instead focus on novel failure modes, under explored behaviors, or emerging risks of large language models or agentic systems. Incremental leaderboard gains or prompt only tweaks are discouraged. Projects may be completed individually or in small teams, maximum 2 members. Each project must be DK confirmed during DK Office Hour week (Feb 10, 12) and grounded in a clear research question, supported by relevant literature and empirical analysis.

Project Scope

Students are encouraged to produce one of the following research artifacts:

• A benchmark or evaluation suite
• A dataset capturing non trivial behaviors or failure modes
• A measurement or diagnostic framework
• A failure taxonomy or behavioral analysis
• A mitigation, repair, or control algorithm

Deliverables

Each team must submit a written report, reproducible code or data, and presentation materials via Canvas. Reports should follow a standard conference paper format, ACL style preferred, using templates via Overleaf or GitHub.

Milestones and Timeline

The following milestones are aligned with the course schedule. Canvas submission links will be provided.

• Team formation and project idea Due: early February
• Project proposal (problem statement, related work, plan) Due: mid February (Canvas link TBD)
• Midterm presentation In class: March 17 to 19
• Final presentation In class: April 28 to 30
• Final report and code submission Due: May 8 (Friday) (Canvas link TBD)

Projects are expected to be reproducible, clearly scoped, and analytically grounded. Strong projects typically combine careful problem formulation with diagnostic evaluation, stress testing, or controlled ablation studies. Evaluation emphasis: originality, rigor, and insight into model behavior under realistic, adversarial, or long horizon conditions rather than raw performance.

See evaluation rubric for final reports.

Selected Past Projects

Reference examples. Links will be added when available.

• Simulating Everyone's Voice: Exploring ChatGPT's Ability to Simulate Human Annotators (report and poster TBD)
• Vision and Language guided Generalized Object Grasping (report and poster TBD)
• Generating Controllable Long dialogue with Coherence (Published in AAAI 2024, link TBD)
• Understanding Narrative Transportation in Fantasy Fanfiction (ACL Workshop on Narrative Understanding, link TBD)