Evaluating Temporal Persistence of Information Retrieval Models at CLEF 2024 LongEval

Evaluating Temporal Persistence of Information Retrieval Models at CLEF 2024 LongEval-Retrieval Track
Petra Galuščáková, University of Stavanger, Norway

Talk at the Search Engines course, A.Y. 2023/2024 (https://iiia.dei.unipd.it/education/search-engines/)
MD in Computer Engineering (https://degrees.dei.unipd.it/master-degrees/computer-engineering/)
MD in Data Science (https://datascience.math.unipd.it/)
University of Padua, Italy

----------------------------------------------------------------------
Evaluating Temporal Persistence of Information Retrieval Models at CLEF 2024 LongEval-Retrieval Track

Abstract:
Recent research suggests that model performance declines over time as test data diverges from training data temporally. This raises several important questions for web search engines, such as how they perform as the queries and the collection of documents evolve, and when the search engine updates are necessary to keep up with these changes. In this talk, I introduce LongEval-Retrieval, a CLEF 2024 track focused on evaluating how information retrieval systems handle evolving collections. I will first describe the LongEval-Retrieval collection, which was designed specifically to evaluate behaviour of information retrieval systems over time. This collection uses data from the privacy-preserving French web search engine Qwant, including user queries and Web documents. Afterwards, I will discuss observations from the 2023 edition of the LongEval-Retrieval track.

Bio:
Petra Galuščáková is an associate professor at the University of Stavanger, working on an intersection of information retrieval and natural language processing. She completed her Ph.D. at Charles University in Prague. Afterwards, she worked as a postdoctoral researcher at the University of Maryland, College Park, and at the University Grenoble Alpes. Petra’s research primarily concentrates on providing robust access strategies for complex search scenarios, such as cross-language retrieval and speech retrieval and on and leveraging multiple diverse models for problems with a high degree of uncertainty.