DSC 261: Responsible Data Science

Trustworthiness is a claim that must survive contact with a real system.

DSC 261 is an evidence-first technical course on evaluating, attacking, defending, inspecting, monitoring, and controlling deployed machine-learning systems. Centered on primary research from leading venues and hands-on experiments with modern language models and agents, it teaches a broader discipline: define the system boundary, state a falsifiable trust claim, choose valid evidence, and produce an executable artifact—such as a metric, attack, defense, monitor, or control protocol—that supports an accountable decision.

Module 1: Course opening and compact foundations

Module 1 spans the first two meetings. It begins with concrete failures that motivate trustworthy machine learning, then develops only the model foundations needed to reason about modern language models and deployed LLM systems.

Module 2: Uncertainty

Lecture materials

Reading list

Papers covered in the uncertainty lecture, in lecture order.

Module 3: Grounding

Lecture materials

Grounding reading list

Core Grounding lecture papers, in lecture order.

Optional extension

Data Valuation

Which data should we keep, clean, acquire, or prioritize for a chosen task? This lecture connects data valuation to data-centric trustworthy AI: model influence, LLM data selection, distributional similarity, KAIROS, and learning across possible repairs of uncertain data.

Lecture materials

Topics covered

• Using data values for selection, cleaning, acquisition, and reweighting.
• Influence functions and valuation across the LLM development pipeline.
• Wasserstein distance, MMD, Gaussian kernels, and the intuition behind bandwidth.
• KAIROS: reference-based, model-agnostic valuation and distributional influence.
• Learning from uncertain data: possible worlds, possible models, and data repairs.

Core reading list

Background and further reading

Optional readings for the valuation methods, distribution comparisons, and LLM applications introduced in the lecture.

Module 4: Alignment and Post-training

Does alignment make a model trustworthy, or better at pleasing its evaluator? The lectures connect reward overoptimization, sycophancy, preference-model biases, and the integrity of oversight.

Core reading list

Three papers taught in depth, in lecture order.

Opening context and case reports

Original source screenshots link to the complete reports. Incident accounts and instructional context are distinct from the three core research papers.

Building Reasoning Models from the Ground Up: A Full Video CourseSebastian Raschka · Instructional context

Teaching Claude whyAnthropic · Training and behavioral generalization

Training a Misaligned Reward SeekerAnthropic · Deliberately pessimistic training experiment

Investigating unintended model actions in our evaluations and internal useAnthropic · Task completion and authorized behavior

Suppressing misalignment monitoring alerts and disabling misalignment monitoringOpenAI · Internal deployment incident report

Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incidentMETR · Independent investigation with a bounded scope

Optional extension

Course topics: evaluating concrete safety claims

Nine modules and a Data Valuation lecture connect evaluation and grounding to data-centric methods, post-training, adversarial robustness, agent security, oversight, privacy, and control. Internal-model techniques are introduced where they help evaluate a concrete safety claim.

Fall 2026 meetings

Grounding is scheduled for October 20 and 22, followed by two Alignment and Post-training meetings. Later topics are planned as below; combined coverage and supporting material are identified explicitly.

Earlier topic labels summarize module coverage; session-level boundaries are approximate. September 24 and 29 retain the original published lecture labels. The October 6 lecture was canceled, and its makeup date is not yet confirmed. Planned pacing may be adjusted as the course progresses.

The project is the spine of the course.

Small teams will make and test one technically precise trust claim about a real LLM system. A successful project produces evidence—not a survey—and ends with a reproducible artifact, a concise report, and an in-person project presentation.

Grading emphasizes execution and evidence.

Canvas is authoritative for exact deadlines, rubrics, and any instructor-announced adjustments.

Instructional team

Technical ambition with responsible boundaries

Resources

• Canvas — announcements, slides, submissions, and authoritative deadlines
• UC San Diego enrollment and course logistics
• UC San Diego 2026–27 academic calendar
• HDSI graduate course offerings