11-697: Introduction to Question Answering with Large Language Models

Fall 2025: MW 12:30-1:50P, DH 1211

Instructors: Eric Nyberg and Teruko Mitamura

Teaching Assistant: Kimihiro Hasegawa

Course Description: This course is designed to be accessible to Masters and advanced undergraduate students who seek the basic skills necessary to implement practical Question Answering (QA) applications using Large Language Models (LLMs) in specific information domains. The syllabus includes learning materials on the core concepts of QA and LLMs, and how they are applied in closed commercial systems (e.g. ChatGPT) as well as open systems (e.g. Llama, T5). Students complete a set of hands-on exercises in Python that develop skills in applying LLMs for various open-source QA datasets. The course is also a prerequisite for 11-797 Question Answering (an advanced project-oriented course).

Prerequisite Knowledge: A course in Statistics and Probability and at least intermediate Python programming skills

Course Goals: Students acquire basic knowledge of QA approaches and tasks, including Data Analysis, Solution Design, Metrics, Evaluation and Error Analysis.

Grading:

• Quizzes = 24% (8 x 3%)
• Homeworks = 66% (5 x 10% + 1 x 16%)
• Attendance/Class Participation = 10%

Outline of Learning Materials:

1. Foundations (Course Prerequisites, Definitions, Concepts, etc.)
2. A First Example: LLMs for QA (e.g. ChatGPT)
   • What are LLMs? How can LLMs be incorporated into QA systems?
   • What happens when ChatGPT is evaluated as a QA system?
3. Survey of tasks, domains, methods
   • Tasks: factoid, list factoid, summary, yes/no, etc.
   • Domains: media collections, Q/A datasets, languages & distributions
   • Classic Methods and Pipelines: retrieval-based, NLP-based; multi-strategy architecture
4. Task Curation & Evaluation
   • Curation (Dataset design, sourcing, preliminary analysis, bias)
   • Evaluation (Metrics & significance, overlap / error analysis & prioritization)
5. Sparse and Dense Retrieval for QA
6. Extractive vs. Generative QA
7. Retrieval Augmented Generation and Prompt Engineering
8. Supervised Fine-Tuning and Parameter-Efficient Fine-Tuning
9. Putting it All Together: Practical Engineering of QA Systems
10. Advanced Topics
    • Multi-Hop QA
    • Conversational QA
    • Multimodal QA
    • Generative LLMs
11. Wrap-Up
    • Discuss learning objectives / outcomes, material covered, feedback
    • b. Discuss open challenges, possible project topics for 11-797

Last Updated: June 26, 2025