A query and its ranked list
The ranker returns the best documents for the query. We show the first . ExCoR explains this order: the whole list, not one document at a time.
Chunks, encoded once
Each document is cut into non-overlapping chunks of tokens, each encoded independently and cached.
With MaxP, a document's score is the similarity of its best chunk (outlined) to the query.
Hover a chunk to read it.
Topics shared across documents
k-means clusters the chunks of the documents into k = groups, local to this query and shared across documents. An LLM names each group from its most central chunks.
In colour: the groups that turn out most important. In grey: the other groups.
Remove groups, rescore exactly
A coalition is the set of groups we keep. Here we drop and rescore each document from its remaining chunks.
Since chunks are encoded independently, deleting text is the same as dropping cached vectors. The new scores are exact, with no re-encoding, and the list reorders.
How much of the order survives?
The value v(S) of a coalition is the NDCG of the new order against the original one: 1 when the order is intact.
Without these two groups, v(S) = . With no group at all, every document is empty and tied: v(∅) = .
100,000 coalitions, one Shapley value per group
Each coalition costs a few vector operations, so ExCoR scores 100,000 random coalitions per query. KernelSHAP turns them into a Shapley value φ per group: its average contribution to keeping the order.
The values add up: v(∅) + Σφ = v(full) = 1. This explanation took s on one GPU.
Read the explanation
The groups with the largest φ explain the order of the list. A pairwise question, , is read from the same attributions: compare the groups present in one document and absent from the other.
A reading of the listwise attributions, not a separate pairwise method.