Professor Shvartzshnaider presented at the recent HotPETS 2025
In LLMs We Trust? A Contextual Integrity Perspective
Yan Shvartzshnaider (York University), Vasisht Duddu (University of Waterloo)
The talk broadly discuss recent results in LLM-CI: Assessing Contextual Integrity Norms in Language Models
"Large language models (LLMs), while memorizing parts of their training data scraped from the Internet, may also inadvertently encode societal preferences and norms. As these models are integrated into sociotechnical systems, it is crucial that the norms they encode align with societal expectations. These norms could vary across models, hyperparameters, optimization techniques, and datasets. This is especially challenging due to prompt sensitivity–small variations in prompts yield different responses, rendering existing assessment methodologies unreliable. There is a need for a comprehensive framework covering various models, optimization, and datasets, along with a reliable methodology to assess encoded norms."