Please note: This PhD seminar will take place in DC 3301 and online.
Amy Tai, PhD candidate
David R. Cheriton School of Computer Science
Supervisors: Professors Lukasz Golab, Alexander Wong
Large language models (LLMs) are increasingly used in clinical trial eligibility screening, where patients’ medical data are checked against free-text eligibility criteria such as “to participate in the trial, the patient must not have a history of alcohol abuse”. Because patient records can be very long, the big data problem here is context selection: which fragments of the patient data should be given to the LLM? This is an important problem due to token cost and inference latency, and even long-context models can be confused by unnecessary inputs. Existing context selection approaches use information retrieval techniques such as syntactic or embedding similarity between eligibility criteria and chunks of patient records.
We propose PassRL-X: the first reinforcement learning solution. PassRL-X learns a policy for selecting decisive sentences from patient records based on training data labelled with whether the patient met the trial criteria. We compare our method with existing approaches and a distantly-supervised baseline trained on the same labelled data to isolate the impact of reinforcement learning from supervised learning. We demonstrate the inference time, prompt length, and accuracy advantages of PassRL-X on six open-source LLMs and 12 real-world (anonymized) clinical trial recruitment tasks.
To attend this PhD seminar in person, please go to DC 3301. You can also attend virtually on Zoom.