Please note: This master’s thesis presentation will take place online.
Alireza Arbabi, Master’s candidate
David R. Cheriton School of Computer Science
Supervisor: Professor Xi He
Large language models are increasingly capable on specialized legal NLP tasks, but their direct use incurs recurring inference costs and may require proprietary or confidential data to be shared with external providers, creating practical barriers for sensitive legal applications. In this setting, knowledge distillation provides a natural approach for building small and specialized language models. However, conventional distillation methods are limited when only a sparse set of real task examples is available, a common challenge in the legal domain.
To address this challenge, this thesis presents LegalSynth, a framework that uses synthetic data generation followed by attribution-based data selection as an alternative channel for transferring knowledge from the teacher to the student. Given a task description and a small set of real examples, LegalSynth first prompts a teacher model to infer the task structure from the available task information and then leverages this understanding to generate a large and diverse pool of synthetic training examples beyond the initial seed set. Because synthetic examples vary in quality and utility, LegalSynth then applies data attribution to filter the synthetic pool and retain the examples that are most useful for training, improving the performance of the student model. Experiments across four low-resource, multi-way legal classification tasks from LegalBench demonstrate the effectiveness of this framework, enabling Qwen3.5 models with up to 4B parameters to compete with, and in several settings outperform, frontier language models including GPT-5.6-Terra, Gemini-3.7-Flash, and Qwen3.5-397B.