Publications
Preprints
-
Simple Linear Corrections for Compression in Late Interaction
arXiv:2610.14196, October 2026. [pdf] [bibtex]Summary
Late-interaction (multi-vector) models store a vector per token, which makes their indices expensive, and the pooling and pruning methods used to compress them lose noticeable quality when compression is aggressive. We ask whether a lightweight change to the scoring function can recover that quality. For each encoder, compressor and vector budget we learn a single global linear correction by distilling the uncompressed model's score distributions over candidate documents; the encoder and compressor stay frozen, and the correction can be applied per query at search time or once, offline, to the stored vectors. Across ColBERTv2 and Jina v4 on MS MARCO and BEIR, and ColPali2 on ViDoRe v3, the in-domain correction improves over baseline compression in 154 of 160 BEIR settings, while transfer from MS MARCO is more variable.
-
Retrieval Needs Multivectors: An Exponential Separation
arXiv:2608.21494, August 2026. [pdf] [bibtex]Summary
Multi-vector embeddings consistently outperform single-vector ones in retrieval, but the representational gap between them was poorly understood. Following Jayaram's work, we give the first explicit family of query and document sets, together with their relevance matrices, for which single-vector embeddings that rank all relevant documents above irrelevant ones require exponential size, whereas polynomial-size multi-vector embeddings suffice. This establishes an exponential separation in expressive power for the task of ranking documents. Motivated by the construction, we introduce ANDOR, a benchmark that naturally instantiates these hard examples: state-of-the-art single-vector models perform poorly zero-shot and improve only marginally with fine-tuning, while multi-vector models consistently do better and improve substantially with fine-tuning, closely matching our theoretical predictions.