pei39/iol-ai-qwen25-7b-rag-v1

Qwen2.5-7B-AWQ book-RAG submission

This is a training-free IOL-AI submission. It retrieves only from the public, string-only extraction of Vlad A. Neacșu's Linguistics Olympiad: Training guide, prompts Qwen2.5-7B-Instruct-AWQ to reason step by step, parses reasoning separately from the final answer block, and writes the required submission.csv.

Private curated training data are not present, indexed, imported, or needed at runtime.

Upload layout

Upload this whole directory as the Hugging Face model repository:

<repository root>/
├── script.py
├── README.md
├── config.json                 # AWQ model config (add with model snapshot)
├── generation_config.json      # add with model snapshot
├── tokenizer files             # add with model snapshot
├── AWQ safetensor shards       # add with model snapshot
└── rag_resources/
    ├── config.json
    ├── system_prompt.txt
    ├── retriever.py
    ├── book_methods.jsonl
    ├── book_examples.jsonl
    └── ATTRIBUTION.md

The large model files are deliberately not duplicated in this project working copy. Before uploading, place the complete official Qwen/Qwen2.5-7B-Instruct-AWQ snapshot in this directory. From the parent directory:

huggingface-cli download Qwen/Qwen2.5-7B-Instruct-AWQ \
  --local-dir qwen2_5_7b_awq_rag_submission \
  --exclude README.md .gitattributes

Retain the upstream Qwen license files. The script never downloads anything during evaluation and loads the model from the repository root with local_files_only=True.

Challenge execution

python script.py

The script reads /tmp/data/test.csv and writes submission.csv in the current directory. The output has exactly the columns id,pred,explanation. id is copied unchanged, pred is a JSON-encoded list of answer strings, and explanation is a short model-generated summary for the optional human-evaluation track. The accepted raw reasoning is used internally but is never written to the CSV.

For a local file:

python script.py --input path/to/test.csv --output submission.csv

Test parsing, retrieval, and CSV serialization without loading the model:

python script.py --self-test

Runtime and quantization

This version targets the challenge's exact stack: Python 3.10, Torch 2.4.0, Transformers 4.44.1, AutoAWQ 0.2.7.post3, AutoAWQ kernels 0.0.9, and one 16 GB T4.

Qwen2.5 uses the qwen2 Transformers architecture and requires Transformers 4.37 or newer, so version 4.44.1 is compatible. The script verifies that the shipped model config declares both model_type=qwen2 and quant_method=awq.

The checkpoint is already AWQ 4-bit. It is loaded directly with AutoModelForCausalLM.from_pretrained(...) and FP16 computation. Do not add a BitsAndBytesConfig to this AWQ model: that would attempt to combine two quantization routes. Bitsandbytes remains a valid alternative only when shipping the unquantized Qwen checkpoint and quantizing it during loading; that alternative requires much larger model files and more startup work.

Transformers 4.44.1 computes and FP32-upcasts Qwen2 logits for every prompt token during prefill. The submission installs an inference-only pre-hook on lm_head so it receives only the final hidden-state token, which is the only position used by autoregressive generation. This avoids the otherwise multi-gigabyte temporary logits allocation on a T4 without changing the generated-token distribution. PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True is also enabled to reduce allocator fragmentation.

Sampling follows the Qwen2.5 Instruct generation configuration:

• temperature=0.7
• top_p=0.8
• top_k=20
• repetition_penalty=1.05
• sampling enabled

The model context is conservatively capped at 32,768 tokens and max_new_tokens at 4,096 for the 30-minute T4 evaluation. Retrieved text is shortened first when necessary; the current problem is never truncated.

Hybrid retrieval

Retrieval uses only the current row's context + query and the two book JSONL files. It combines:

• dependency-free BM25 word retrieval;
• scikit-learn character char_wb TF-IDF over 3–5-grams;
• task-family inference from the actual problem text; and
• book topic and method-link boosts.

The default character TF-IDF contribution is 3.0 × cosine_similarity. It improves recognition of spelling, diacritics, linguistic forms, and differently formatted versions of a historical problem. All 94 string-only book examples remain eligible, including exact matches.

The model must emit FINAL ANSWERS: followed by one bare answer per line. Everything before the marker is retained internally as reasoning but is not submitted verbatim. Missing or empty answer blocks are rejected. A rejected sample is regenerated from a fresh conversation with a stricter format reminder, for at most five total attempts. After an answer is accepted, a deterministic second generation condenses its reasoning into at most 400 new tokens for the explanation field.

Settings are in rag_resources/config.json. Environment overrides are available for IOL_TOP_METHODS, IOL_TOP_EXAMPLES, IOL_CHAR_TFIDF_WEIGHT, IOL_RAG_MAX_CHARS, IOL_MAX_NEW_TOKENS, IOL_EXPLANATION_MAX_NEW_TOKENS, IOL_MAX_ATTEMPTS, and IOL_SEED.

Sources

• IOL-AI 2026 challenge
• Official workshop notebook
• Qwen2.5-7B-Instruct-AWQ model card
• Language Science Press book source

See rag_resources/ATTRIBUTION.md for book licensing and attribution.