AdarshSingh7647/TabRankMultiTableCoTGen

TabRank — Single + Multi Table, Standard SFT

Part of the TabRank family: six Qwen3-8B checkpoints for single-call generative listwise table reranking. Given a question and a list of candidate tables, the model reads them all in one prompt and returns the full ranking in a single generation — no pairwise scoring, no cross-encoder passes.

This checkpoint is the Standard SFT variant: standard chain-of-thought distillation, trained with loss over the full reasoning trace plus the final ranking. It writes its own <think>...</think> reasoning at inference time. It is trained on NQ Tables + MultiTabQA, adding multi-table retrieval questions on top of the single-table split.

Related checkpoints: TabRankMultiTableCoTCond (TabRank, our best-performing method, same data mix) and TabRankSingleTableCoTGen (same objective, single-table data only). Full family: TabRankSingleTableNaive · TabRankSingleTableCoTGen · TabRankSingleTableCoTCond · TabRankMultiTableNaive · TabRankMultiTableCoTGen (this model) · TabRankMultiTableCoTCond.

How it works

TabRank is trained on 6,728 chain-of-thought reasoning traces distilled from a teacher model reasoning about table relevance. This variant trains loss over the entire trace, so the model must reproduce reasoning close to the teacher's before ranking. In our results this generalizes less reliably out-of-distribution than the reasoning-conditioned method (TabRank) — see the paper for the full comparison and ablations.

Input / output format

Input — a chat message with the question followed by each candidate table, labeled ### Table 1, ### Table 2, ...:

Question: Which table shows 2022 quarterly revenue by region?

### Table 1
| Region | Q1 2022 | Q2 2022 | Q3 2022 | Q4 2022 |
|---|---|---|---|---|
| North America | 120 | 134 | 128 | 145 |
| Europe | 88 | 91 | 95 | 102 |

### Table 2
| Product | Units Sold | Year |
|---|---|---|
| Widget A | 4200 | 2021 |

### Table 3
| Region | Headcount |
|---|---|
| North America | 340 |

Output — a <think> block with the model's reasoning, followed by a single JSON object with the ranked, one-indexed candidate positions, best first:

<think>
Table 1 has quarterly revenue by region for 2022, which is exactly what the question asks
for. Table 3 has region data but no revenue. Table 2 has neither region nor 2022 data.
</think>
{"ranked_tables": [1, 3, 2]}

Map the numbers back to your own table ids to get the reranked list — position 1 in the output is ### Table 1 from the input, etc.

Evaluation — this checkpoint's results

Scored as a listwise reranker reordering a first-stage top-25 candidate list on 5 in-distribution benchmarks (SQA, TAT-QA, HybridQA, TabFact, and NQ-Tables — the actual LoRA training split) and 7 out-of-distribution benchmarks from the IBM table-text-ir-evaluation suite (OpenWikiTables, OTT-QA, MultiHiertt, AIT-QA, FeTaQA, StatCanDialogue, WatsonxDocsQA).

This checkpoint (Standard SFT), ndcg@10:

	SQA	TAT-QA	HybridQA	TabFact	NQ-Tables	OpenWikiTables	OTT-QA	MultiHiertt	AIT-QA	FeTaQA	StatCanDialogue	WatsonxDocsQA	Mean
Standard SFT	0.736	0.540	0.791	0.670	0.735	0.903	0.832	0.537	0.506	0.881	0.585	0.679	0.700

Compared against Base Qwen3-8B and TabRank (our method) on the same data mix:

Model	SQA	TAT-QA	HybridQA	TabFact	NQ-Tables	OpenWikiTables	OTT-QA	MultiHiertt	AIT-QA	FeTaQA	StatCanDialogue	WatsonxDocsQA	Mean
Base Qwen3-8B	—	—	0.735	0.656	0.723	0.887	0.813	0.521	0.495	0.896	0.615	0.756	0.710
Standard SFT (this model)	0.736	0.540	0.791	0.670	0.735	0.903	0.832	0.537	0.506	0.881	0.585	0.679	0.700
TabRank (our method)	0.741	0.519	0.783	0.688	0.747	0.938	0.903	0.599	0.536	0.919	0.580	0.690	0.720

The first 5 columns (SQA through NQ-Tables) are in-distribution; the remaining 7 are out-of-distribution. This checkpoint sits between Base and TabRank on mean ndcg@10 — modest gains from CoT distillation, but weaker generalization than the reasoning-conditioned TabRank method. See TabRankMultiTableCoTCond for the full breakdown. Source eval code and logs: GitHub repo.

Usage with vLLM

from vllm import LLM, SamplingParams
from transformers import AutoTokenizer

repo = "AdarshSingh7647/TabRankMultiTableCoTGen"
tok = AutoTokenizer.from_pretrained(repo)
llm = LLM(model=repo, dtype="bfloat16", max_model_len=32768)

system = ("You are a table relevance expert. Given a question and a set of candidate tables "
          "rank them from most to least useful for answering the question. Reason in a "
          "<think>...</think> block then output exactly JSON with key ranked_tables.")
user = "Question: ...\n\n### Table 1\n...\n\n### Table 2\n...\n"

msgs = [{"role": "system", "content": system}, {"role": "user", "content": user}]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
# the model writes a <think> block then the ranking json
out = llm.generate([text], SamplingParams(temperature=0.6, top_p=0.95, max_tokens=8192))
print(out[0].outputs[0].text)

Usage with Transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "AdarshSingh7647/TabRankMultiTableCoTGen"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype=torch.bfloat16, device_map="auto")

system = ("You are a table relevance expert. Given a question and a set of candidate tables "
          "rank them from most to least useful for answering the question. Reason in a "
          "<think>...</think> block then output exactly JSON with key ranked_tables.")
user = "Question: ...\n\n### Table 1\n...\n\n### Table 2\n...\n"

msgs = [{"role": "system", "content": system}, {"role": "user", "content": user}]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
inputs = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=8192, temperature=0.6, top_p=0.95, do_sample=True)
print(tok.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

For full training and evaluation code, dataset builders, and the reasoning-trace dataset, see the TabRanker GitHub repo.

Model details

• Base model: Qwen3-8B
• Method: LoRA rank 16 fine-tuning, merged into the base weights so it loads directly
• Precision: bfloat16, single-file safetensors, ~16 GB
• Training data: NQ Tables + MultiTabQA (single- and multi-table retrieval)
• Family: six TabRank checkpoints span three objectives (Answer-Only, Standard SFT, TabRank) across two training mixes (Single Table, Single + Multi Table)

Citation

If you use these models, please cite the TabRank paper:

@misc{singh2026tabrank,
      title={TabRank: Chain-of-Thought Distillation for Table Re-Rankers},
      author={Adarsh Singh and Kushal Raj Bhandari and Jianxi Gao and Soham Dan and Vivek Gupta},
      year={2026},
      eprint={2607.25182},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2607.25182}
}

The MultiTabQA data in this checkpoint's training mix comes from RAG over Tables. If your usage relies specifically on the multi-table data, please also cite:

@misc{zou2025ragtableshierarchicalmemory,
      title={RAG over Tables: Hierarchical Memory Index, Multi-Stage Retrieval, and Benchmarking},
      author={Jiaru Zou and Dongqi Fu and Sirui Chen and Xinrui He and Zihao Li and Yada Zhu and Jiawei Han and Jingrui He},
      year={2025},
      eprint={2504.01346},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2504.01346}
}