Instructions to use 1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use 1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0 with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.6-27B")
model = PeftModel.from_pretrained(base_model, "1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0")

Transformers

How to use 1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use 1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0

SGLang

How to use 1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use 1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0 with Docker Model Runner:
```
docker model run hf.co/1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0
```

1jamesthompson1 commited on 5 days ago

Commit

e6e6f76

verified ·

1 Parent(s): ef18eca

Upload folder using huggingface_hub

Browse files

Files changed (13) hide show

.gitattributes +1 -0
README.md +179 -0
adapter_config.json +50 -0
adapter_model.safetensors +3 -0
chat_template.jinja +154 -0
environment.json +82 -0
finetune_config.json +22 -0
processor_config.json +60 -0
tokenizer.json +3 -0
tokenizer_config.json +32 -0
training.log +85 -0
training_args.bin +3 -0
training_log.json +761 -0

.gitattributes CHANGED Viewed

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text

 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md ADDED Viewed

	@@ -0,0 +1,179 @@

+---
+base_model: Qwen/Qwen3.6-27B
+library_name: peft
+model_name: cluster_0
+tags:
+  - base_model:adapter:Qwen/Qwen3.6-27B
+  - lora
+  - sft
+  - transformers
+  - trl
+  - single_sample
+  - cluster_0
+license: cc-by-sa-4.0
+pipeline_tag: text-generation
+datasets:
+  - 1jamesthompson1/wvs-nz-value-alignment
+---
+# Qwen3.6-27B LoRA — Single Sample, Cluster 0
+This model is a LoRA fine-tune of [Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B)  as part of the
+[AIML589 project](https://github.com/1jamesthompson1/AIML589).
+This adapter is licensed under CC BY-SA 4.0.
+## Dataset
+Fine-tuned on the **single_sample** config of the
+[wvs-nz-value-alignment](https://huggingface.co/datasets/1jamesthompson1/wvs-nz-value-alignment)
+dataset, **cluster_0** subpopulation.
+Part of the [wvs-nz-value-alignment](https://huggingface.co/collections/wvs-nz-value-alignment) collection.
+**GPU:** NVIDIA RTX PRO 6000 Blackwell Workstation Edition · **Training time:** 53m 37s
+## Training hyperparameters
+| Parameter | Value |
+|-----------|-------|
+| LoRA rank | 256 |
+| LoRA alpha | 512 |
+| LoRA dropout | 0.05 |
+| DoRA | False |
+| Learning rate | 0.0002 |
+| Batch size | 4 |
+| Gradient accumulation | 4 |
+| Epochs | 10 |
+| Max seq length | 1024 |
+| Warmup ratio | 0.1 |
+| Dtype | bf16 |
+## Training log
+```
+{"loss": 2.341543960571289, "grad_norm": 2.112253189086914, "learning_rate": 2.4e-05, "entropy": 0.9884634539484978, "mean_token_accuracy": 0.5556907095015049, "num_tokens": 25909.0, "epoch": 0.13377926421404682, "step": 10}
+{"loss": 0.3863301038742065, "grad_norm": 2.2011115550994873, "learning_rate": 5.0666666666666674e-05, "entropy": 0.4089386660605669, "mean_token_accuracy": 0.9011316433548927, "num_tokens": 52070.0, "epoch": 0.26755852842809363, "step": 20}
+{"loss": 0.17976584434509277, "grad_norm": 0.5207965970039368, "learning_rate": 7.733333333333333e-05, "entropy": 0.18403972331434487, "mean_token_accuracy": 0.9493107855319977, "num_tokens": 78405.0, "epoch": 0.4013377926421405, "step": 30}
+{"loss": 0.10954103469848633, "grad_norm": 0.3419843316078186, "learning_rate": 0.00010400000000000001, "entropy": 0.11044432241469622, "mean_token_accuracy": 0.9671209067106247, "num_tokens": 104414.0, "epoch": 0.5351170568561873, "step": 40}
+{"loss": 0.09990320205688477, "grad_norm": 0.33741509914398193, "learning_rate": 0.00013066666666666668, "entropy": 0.09925312716513872, "mean_token_accuracy": 0.9688541933894157, "num_tokens": 130830.0, "epoch": 0.6688963210702341, "step": 50}
+{"loss": 0.0990227460861206, "grad_norm": 0.43753594160079956, "learning_rate": 0.00015733333333333333, "entropy": 0.0994369624182582, "mean_token_accuracy": 0.9684862270951271, "num_tokens": 156575.0, "epoch": 0.802675585284281, "step": 60}
+{"loss": 0.0771427571773529, "grad_norm": 0.3096464276313782, "learning_rate": 0.00018400000000000003, "entropy": 0.07597130816429853, "mean_token_accuracy": 0.9754018127918244, "num_tokens": 182787.0, "epoch": 0.9364548494983278, "step": 70}
+{"loss": 0.08177689909934997, "grad_norm": 1.9745781421661377, "learning_rate": 0.00019881481481481483, "entropy": 0.07385796203445165, "mean_token_accuracy": 0.9744954705238342, "num_tokens": 208722.0, "epoch": 1.0668896321070234, "step": 80}
+{"loss": 0.08656259775161743, "grad_norm": 0.24118545651435852, "learning_rate": 0.00019585185185185187, "entropy": 0.0961530750617385, "mean_token_accuracy": 0.9735429108142852, "num_tokens": 234820.0, "epoch": 1.2006688963210703, "step": 90}
+{"loss": 0.07904126048088074, "grad_norm": 1.4924163818359375, "learning_rate": 0.0001928888888888889, "entropy": 0.07621442535892128, "mean_token_accuracy": 0.9752445220947266, "num_tokens": 261255.0, "epoch": 1.334448160535117, "step": 100}
+{"loss": 0.1028050422668457, "grad_norm": 0.22435367107391357, "learning_rate": 0.00018992592592592593, "entropy": 0.07814982496201991, "mean_token_accuracy": 0.9725448682904243, "num_tokens": 287420.0, "epoch": 1.468227424749164, "step": 110}
+{"loss": 0.06647659540176391, "grad_norm": 0.16586251556873322, "learning_rate": 0.00018696296296296297, "entropy": 0.06741584213450551, "mean_token_accuracy": 0.9774157494306565, "num_tokens": 313385.0, "epoch": 1.6020066889632107, "step": 120}
+{"loss": 0.057019442319869995, "grad_norm": 0.13507577776908875, "learning_rate": 0.00018400000000000003, "entropy": 0.05611985893920064, "mean_token_accuracy": 0.9796610698103905, "num_tokens": 340048.0, "epoch": 1.7357859531772575, "step": 130}
+{"loss": 0.061611056327819824, "grad_norm": 0.2060474306344986, "learning_rate": 0.00018103703703703706, "entropy": 0.06344100441783666, "mean_token_accuracy": 0.97853914052248, "num_tokens": 365876.0, "epoch": 1.8695652173913042, "step": 140}
+{"loss": 0.061782336235046385, "grad_norm": 0.3134608268737793, "learning_rate": 0.0001780740740740741, "entropy": 0.05643900770407457, "mean_token_accuracy": 0.978085016592955, "num_tokens": 390738.0, "epoch": 2.0, "step": 150}
+{"loss": 0.057867032289505, "grad_norm": 0.33745983242988586, "learning_rate": 0.00017511111111111113, "entropy": 0.05852913670241833, "mean_token_accuracy": 0.9795649841427803, "num_tokens": 416830.0, "epoch": 2.1337792642140467, "step": 160}
+{"loss": 0.058108162879943845, "grad_norm": 0.1164756640791893, "learning_rate": 0.00017214814814814816, "entropy": 0.0567542714998126, "mean_token_accuracy": 0.9790374860167503, "num_tokens": 443309.0, "epoch": 2.2675585284280935, "step": 170}
+{"loss": 0.05357828140258789, "grad_norm": 0.12977160513401031, "learning_rate": 0.0001691851851851852, "entropy": 0.05269257873296738, "mean_token_accuracy": 0.9791603803634643, "num_tokens": 469381.0, "epoch": 2.4013377926421406, "step": 180}
+{"loss": 0.06209459900856018, "grad_norm": 5.890456676483154, "learning_rate": 0.00016622222222222223, "entropy": 0.05242626103572547, "mean_token_accuracy": 0.9787786990404129, "num_tokens": 495823.0, "epoch": 2.5351170568561874, "step": 190}
+{"loss": 0.052966177463531494, "grad_norm": 0.1879248023033142, "learning_rate": 0.00016325925925925926, "entropy": 0.05595421427860856, "mean_token_accuracy": 0.9809569671750069, "num_tokens": 521936.0, "epoch": 2.668896321070234, "step": 200}
+{"loss": 0.1533456563949585, "grad_norm": 2.4737038612365723, "learning_rate": 0.0001602962962962963, "entropy": 0.09764697654172778, "mean_token_accuracy": 0.9594999879598618, "num_tokens": 547808.0, "epoch": 2.802675585284281, "step": 210}
+{"loss": 0.14215301275253295, "grad_norm": 0.21627607941627502, "learning_rate": 0.00015733333333333333, "entropy": 0.09831915665417909, "mean_token_accuracy": 0.96669692248106, "num_tokens": 573896.0, "epoch": 2.936454849498328, "step": 220}
+{"loss": 0.07142106890678405, "grad_norm": 0.24911274015903473, "learning_rate": 0.0001543703703703704, "entropy": 0.08219717662685956, "mean_token_accuracy": 0.975415417781243, "num_tokens": 599295.0, "epoch": 3.0668896321070234, "step": 230}
+{"loss": 0.06142540574073792, "grad_norm": 2.818284273147583, "learning_rate": 0.00015140740740740742, "entropy": 0.06195763573050499, "mean_token_accuracy": 0.978814198076725, "num_tokens": 625594.0, "epoch": 3.20066889632107, "step": 240}
+{"loss": 0.09106510877609253, "grad_norm": 0.3751987814903259, "learning_rate": 0.00014844444444444445, "entropy": 0.07343985894694924, "mean_token_accuracy": 0.9736507266759873, "num_tokens": 651379.0, "epoch": 3.334448160535117, "step": 250}
+{"loss": 0.06325957775115967, "grad_norm": 3.0620086193084717, "learning_rate": 0.0001454814814814815, "entropy": 0.06504379613325, "mean_token_accuracy": 0.9780099034309387, "num_tokens": 677883.0, "epoch": 3.468227424749164, "step": 260}
+{"loss": 0.06386491656303406, "grad_norm": 0.13123036921024323, "learning_rate": 0.00014251851851851852, "entropy": 0.06299111396074294, "mean_token_accuracy": 0.9776420131325722, "num_tokens": 703942.0, "epoch": 3.6020066889632107, "step": 270}
+{"loss": 0.07105348110198975, "grad_norm": 0.16725099086761475, "learning_rate": 0.00013955555555555558, "entropy": 0.06523667583242058, "mean_token_accuracy": 0.9771258398890496, "num_tokens": 730355.0, "epoch": 3.7357859531772575, "step": 280}
+{"loss": 0.05892015695571899, "grad_norm": 0.8018252849578857, "learning_rate": 0.00013659259259259261, "entropy": 0.06290404153987765, "mean_token_accuracy": 0.9795312806963921, "num_tokens": 756288.0, "epoch": 3.869565217391304, "step": 290}
+{"loss": 0.05513965487480164, "grad_norm": 0.23648729920387268, "learning_rate": 0.00013362962962962965, "entropy": 0.05575119321927046, "mean_token_accuracy": 0.980286703659938, "num_tokens": 781476.0, "epoch": 4.0, "step": 300}
+{"loss": 0.049145272374153136, "grad_norm": 0.23902687430381775, "learning_rate": 0.00013066666666666668, "entropy": 0.047657360602170226, "mean_token_accuracy": 0.9813551798462867, "num_tokens": 807995.0, "epoch": 4.133779264214047, "step": 310}
+{"loss": 0.052027386426925656, "grad_norm": 0.08278259634971619, "learning_rate": 0.00012770370370370371, "entropy": 0.04914208557456732, "mean_token_accuracy": 0.9807460099458695, "num_tokens": 834211.0, "epoch": 4.2675585284280935, "step": 320}
+{"loss": 0.04958886206150055, "grad_norm": 0.06415274739265442, "learning_rate": 0.00012474074074074075, "entropy": 0.05262458473443985, "mean_token_accuracy": 0.9813323050737381, "num_tokens": 860320.0, "epoch": 4.40133779264214, "step": 330}
+{"loss": 0.0506575882434845, "grad_norm": 0.08827455341815948, "learning_rate": 0.0001217777777777778, "entropy": 0.049570453260093926, "mean_token_accuracy": 0.9807545498013497, "num_tokens": 886260.0, "epoch": 4.535117056856187, "step": 340}
+{"loss": 0.04992738664150238, "grad_norm": 0.1842905879020691, "learning_rate": 0.00011881481481481483, "entropy": 0.049180435854941605, "mean_token_accuracy": 0.9801862895488739, "num_tokens": 912192.0, "epoch": 4.668896321070234, "step": 350}
+{"loss": 0.04837550222873688, "grad_norm": 0.07146720588207245, "learning_rate": 0.00011585185185185186, "entropy": 0.04882028605788946, "mean_token_accuracy": 0.9817976549267768, "num_tokens": 938443.0, "epoch": 4.802675585284281, "step": 360}
+{"loss": 0.047449523210525514, "grad_norm": 0.07254461199045181, "learning_rate": 0.0001128888888888889, "entropy": 0.04849126562476158, "mean_token_accuracy": 0.9812644407153129, "num_tokens": 964704.0, "epoch": 4.936454849498328, "step": 370}
+{"loss": 0.047966784238815306, "grad_norm": 0.05580724775791168, "learning_rate": 0.00010992592592592593, "entropy": 0.04855830528033085, "mean_token_accuracy": 0.9817173633819971, "num_tokens": 989807.0, "epoch": 5.066889632107023, "step": 380}
+{"loss": 0.04565880596637726, "grad_norm": 0.06281863152980804, "learning_rate": 0.00010696296296296297, "entropy": 0.04446020070463419, "mean_token_accuracy": 0.98235524892807, "num_tokens": 1016432.0, "epoch": 5.20066889632107, "step": 390}
+{"loss": 0.047007548809051516, "grad_norm": 0.05037130042910576, "learning_rate": 0.00010400000000000001, "entropy": 0.046955365035682914, "mean_token_accuracy": 0.9810686483979225, "num_tokens": 1042595.0, "epoch": 5.334448160535117, "step": 400}
+{"loss": 0.04774223566055298, "grad_norm": 0.05550120770931244, "learning_rate": 0.00010103703703703704, "entropy": 0.04687619395554066, "mean_token_accuracy": 0.9813349410891533, "num_tokens": 1068499.0, "epoch": 5.468227424749164, "step": 410}
+{"loss": 0.04721843302249908, "grad_norm": 0.07114529609680176, "learning_rate": 9.807407407407407e-05, "entropy": 0.048110970109701154, "mean_token_accuracy": 0.9819222643971444, "num_tokens": 1094712.0, "epoch": 5.602006688963211, "step": 420}
+{"loss": 0.04641242623329163, "grad_norm": 0.054782308638095856, "learning_rate": 9.511111111111112e-05, "entropy": 0.045364385657012465, "mean_token_accuracy": 0.9820315048098565, "num_tokens": 1121039.0, "epoch": 5.735785953177258, "step": 430}
+{"loss": 0.04601489901542664, "grad_norm": 0.05270834639668465, "learning_rate": 9.214814814814815e-05, "entropy": 0.046417954843491316, "mean_token_accuracy": 0.9826092898845673, "num_tokens": 1147476.0, "epoch": 5.869565217391305, "step": 440}
+{"loss": 0.047896665334701535, "grad_norm": 0.07369400560855865, "learning_rate": 8.918518518518519e-05, "entropy": 0.04759720369027211, "mean_token_accuracy": 0.9811742412738311, "num_tokens": 1172214.0, "epoch": 6.0, "step": 450}
+{"loss": 0.04460802972316742, "grad_norm": 0.043926533311605453, "learning_rate": 8.622222222222222e-05, "entropy": 0.0450520227663219, "mean_token_accuracy": 0.983108639717102, "num_tokens": 1198481.0, "epoch": 6.133779264214047, "step": 460}
+{"loss": 0.04490154683589935, "grad_norm": 0.049713313579559326, "learning_rate": 8.325925925925925e-05, "entropy": 0.043880753125995396, "mean_token_accuracy": 0.981929925084114, "num_tokens": 1225270.0, "epoch": 6.2675585284280935, "step": 470}
+{"loss": 0.04616124629974365, "grad_norm": 0.04489952698349953, "learning_rate": 8.02962962962963e-05, "entropy": 0.04594219285063446, "mean_token_accuracy": 0.9817362502217293, "num_tokens": 1251191.0, "epoch": 6.40133779264214, "step": 480}
+{"loss": 0.04496528506278992, "grad_norm": 0.04445449262857437, "learning_rate": 7.733333333333333e-05, "entropy": 0.045685965567827225, "mean_token_accuracy": 0.9831607535481452, "num_tokens": 1277832.0, "epoch": 6.535117056856187, "step": 490}
+{"loss": 0.04584149122238159, "grad_norm": 0.05437646806240082, "learning_rate": 7.437037037037038e-05, "entropy": 0.04615831598639488, "mean_token_accuracy": 0.9816597178578377, "num_tokens": 1304203.0, "epoch": 6.668896321070234, "step": 500}
+{"loss": 0.04616081714630127, "grad_norm": 0.04138358682394028, "learning_rate": 7.140740740740741e-05, "entropy": 0.04780284911394119, "mean_token_accuracy": 0.9815138846635818, "num_tokens": 1330059.0, "epoch": 6.802675585284281, "step": 510}
+{"loss": 0.047184526920318604, "grad_norm": 0.05079558491706848, "learning_rate": 6.844444444444445e-05, "entropy": 0.04507539812475443, "mean_token_accuracy": 0.9809173971414566, "num_tokens": 1355654.0, "epoch": 6.936454849498328, "step": 520}
+{"loss": 0.045189327001571654, "grad_norm": 0.04889153689146042, "learning_rate": 6.54814814814815e-05, "entropy": 0.04632591761839695, "mean_token_accuracy": 0.9823385003285531, "num_tokens": 1380740.0, "epoch": 7.066889632107023, "step": 530}
+{"loss": 0.046108204126358035, "grad_norm": 0.06250233948230743, "learning_rate": 6.251851851851853e-05, "entropy": 0.047147559747099874, "mean_token_accuracy": 0.9815558344125748, "num_tokens": 1406229.0, "epoch": 7.20066889632107, "step": 540}
+{"loss": 0.04495341181755066, "grad_norm": 0.04576072841882706, "learning_rate": 5.9555555555555554e-05, "entropy": 0.04380973177030682, "mean_token_accuracy": 0.9825871646404266, "num_tokens": 1432718.0, "epoch": 7.334448160535117, "step": 550}
+{"loss": 0.04445241391658783, "grad_norm": 0.03827248513698578, "learning_rate": 5.6592592592592594e-05, "entropy": 0.04404275622218847, "mean_token_accuracy": 0.9829956665635109, "num_tokens": 1459141.0, "epoch": 7.468227424749164, "step": 560}
+{"loss": 0.04570685029029846, "grad_norm": 0.03917380049824715, "learning_rate": 5.362962962962963e-05, "entropy": 0.04583866978064179, "mean_token_accuracy": 0.9810639604926109, "num_tokens": 1485265.0, "epoch": 7.602006688963211, "step": 570}
+{"loss": 0.04584062099456787, "grad_norm": 0.04370535910129547, "learning_rate": 5.0666666666666674e-05, "entropy": 0.04647259535267949, "mean_token_accuracy": 0.9812271296977997, "num_tokens": 1511176.0, "epoch": 7.735785953177258, "step": 580}
+{"loss": 0.045942744612693785, "grad_norm": 0.049611467868089676, "learning_rate": 4.770370370370371e-05, "entropy": 0.045514973998069766, "mean_token_accuracy": 0.9814796030521393, "num_tokens": 1537083.0, "epoch": 7.869565217391305, "step": 590}
+{"loss": 0.04436405599117279, "grad_norm": 0.044528670608997345, "learning_rate": 4.474074074074075e-05, "entropy": 0.04354679699127491, "mean_token_accuracy": 0.9819874656506073, "num_tokens": 1562952.0, "epoch": 8.0, "step": 600}
+{"loss": 0.044945243000984195, "grad_norm": 0.045649204403162, "learning_rate": 4.177777777777778e-05, "entropy": 0.04686344414949417, "mean_token_accuracy": 0.9817773103713989, "num_tokens": 1588851.0, "epoch": 8.133779264214047, "step": 610}
+{"loss": 0.04427960515022278, "grad_norm": 0.043687306344509125, "learning_rate": 3.8814814814814814e-05, "entropy": 0.04569701394066215, "mean_token_accuracy": 0.9820200949907303, "num_tokens": 1615149.0, "epoch": 8.267558528428093, "step": 620}
+{"loss": 0.0447018563747406, "grad_norm": 0.04362299293279648, "learning_rate": 3.5851851851851854e-05, "entropy": 0.04486168110743165, "mean_token_accuracy": 0.9820230007171631, "num_tokens": 1641174.0, "epoch": 8.40133779264214, "step": 630}
+{"loss": 0.04458645284175873, "grad_norm": 0.04046420380473137, "learning_rate": 3.2888888888888894e-05, "entropy": 0.044725227542221545, "mean_token_accuracy": 0.9823110207915307, "num_tokens": 1667583.0, "epoch": 8.535117056856187, "step": 640}
+{"loss": 0.044426333904266355, "grad_norm": 0.036781083792448044, "learning_rate": 2.992592592592593e-05, "entropy": 0.04480462055653334, "mean_token_accuracy": 0.9823192492127418, "num_tokens": 1694027.0, "epoch": 8.668896321070234, "step": 650}
+{"loss": 0.045354658365249635, "grad_norm": 0.03921104222536087, "learning_rate": 2.696296296296296e-05, "entropy": 0.04497002325952053, "mean_token_accuracy": 0.9815022855997085, "num_tokens": 1719962.0, "epoch": 8.80267558528428, "step": 660}
+{"loss": 0.04457056820392609, "grad_norm": 0.03931435942649841, "learning_rate": 2.4e-05, "entropy": 0.044097188767045735, "mean_token_accuracy": 0.9824405118823052, "num_tokens": 1746249.0, "epoch": 8.936454849498327, "step": 670}
+{"loss": 0.04478446841239929, "grad_norm": 0.032005202025175095, "learning_rate": 2.1037037037037037e-05, "entropy": 0.045094484893175274, "mean_token_accuracy": 0.9815951723318833, "num_tokens": 1771309.0, "epoch": 9.066889632107024, "step": 680}
+{"loss": 0.04368177354335785, "grad_norm": 0.03807147592306137, "learning_rate": 1.8074074074074074e-05, "entropy": 0.04654490072280169, "mean_token_accuracy": 0.9825977519154548, "num_tokens": 1797647.0, "epoch": 9.200668896321071, "step": 690}
+{"loss": 0.043600788712501524, "grad_norm": 0.03561871498823166, "learning_rate": 1.5111111111111112e-05, "entropy": 0.04407275160774589, "mean_token_accuracy": 0.9828053265810013, "num_tokens": 1824009.0, "epoch": 9.334448160535118, "step": 700}
+{"loss": 0.04528459906578064, "grad_norm": 0.03744061291217804, "learning_rate": 1.2148148148148149e-05, "entropy": 0.04444531872868538, "mean_token_accuracy": 0.9812339514493942, "num_tokens": 1849648.0, "epoch": 9.468227424749164, "step": 710}
+{"loss": 0.0440999835729599, "grad_norm": 0.0362723171710968, "learning_rate": 9.185185185185186e-06, "entropy": 0.04479610063135624, "mean_token_accuracy": 0.9827097550034523, "num_tokens": 1875926.0, "epoch": 9.602006688963211, "step": 720}
+{"loss": 0.044123375415802, "grad_norm": 0.03960822895169258, "learning_rate": 6.222222222222222e-06, "entropy": 0.04520800011232495, "mean_token_accuracy": 0.9823104426264763, "num_tokens": 1902227.0, "epoch": 9.735785953177258, "step": 730}
+{"loss": 0.04401821792125702, "grad_norm": 0.03371009603142738, "learning_rate": 3.259259259259259e-06, "entropy": 0.04439304880797863, "mean_token_accuracy": 0.9820376738905907, "num_tokens": 1928651.0, "epoch": 9.869565217391305, "step": 740}
+{"loss": 0.044789040088653566, "grad_norm": 0.04102063924074173, "learning_rate": 2.962962962962963e-07, "entropy": 0.045203205484610334, "mean_token_accuracy": 0.9820073674886655, "num_tokens": 1953690.0, "epoch": 10.0, "step": 750}
+{"train_runtime": 3217.5242, "train_samples_per_second": 3.714, "train_steps_per_second": 0.233, "total_flos": 3.662638954482125e+17, "train_loss": 0.09439072036743164, "epoch": 10.0, "step": 750}
+```
+## Environment
+| Package | Version |
+|---------|---------|
+| torch | 2.13.0 |
+| transformers | 5.14.1 |
+| trl | 1.9.2 |
+| datasets | 5.0.1 |
+| accelerate | 1.14.0 |
+| python-dotenv | 1.2.2 |
+| peft | 0.20.0 |
+| bitsandbytes | 0.50.0 |
+| huggingface-hub | ? |
+| jinja2 | ? |
+| torchvision | 0.28.0 |
+| pillow | 12.3.0 |
+## Intended use
+This adapter is intended for **research purposes only** as part of the
+[AIML589 project](https://github.com/1jamesthompson1/AIML589), which
+investigates value alignment of LLMs with New Zealand population
+distributions from the World Values Survey.
+### Out-of-scope
+This model has not been safety-tuned for general-purpose deployment.
+It should not be used in production systems, for making decisions about
+people, or in contexts where reliability and safety are critical.
+## Limitations and biases
+- Fine-tuned on a single WVS wave (Wave 7) for New Zealand only.
+- The training data reflects the values of those who responded to the
+  survey and may not represent all New Zealanders.
+- LoRA adapters are subject to the limitations and biases of the base
+  model ([Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B)).

adapter_config.json ADDED Viewed

	@@ -0,0 +1,50 @@

+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.6-27B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 512,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.20.0",
+  "qalora_group_size": 16,
+  "r": 256,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "k_proj",
+    "v_proj",
+    "gate_proj",
+    "up_proj",
+    "o_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

adapter_model.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c80158e5e8c49405abbc9337b9717fe3ca759ba3d8c5e6da1a318312fef45fd4
+size 5100343888

chat_template.jinja ADDED Viewed

	@@ -0,0 +1,154 @@

+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if (preserve_thinking is defined and preserve_thinking is true) or (loop.index0 > ns.last_query_index) %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | string if args_value is string else args_value | tojson | safe %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

environment.json ADDED Viewed

	@@ -0,0 +1,82 @@

+{
+  "Jinja2": "3.1.6",
+  "MarkupSafe": "3.0.3",
+  "PyYAML": "6.0.3",
+  "Pygments": "2.20.0",
+  "accelerate": "1.14.0",
+  "aiohappyeyeballs": "2.7.1",
+  "aiohttp": "3.14.3",
+  "aiosignal": "1.4.0",
+  "annotated-doc": "0.0.5",
+  "anyio": "4.14.2",
+  "attrs": "26.1.0",
+  "bitsandbytes": "0.50.0",
+  "certifi": "2026.7.22",
+  "charset-normalizer": "3.4.9",
+  "click": "8.4.2",
+  "cuda-bindings": "13.3.1",
+  "cuda-pathfinder": "1.6.0",
+  "cuda-toolkit": "13.0.3.0",
+  "datasets": "5.0.1",
+  "dill": "0.4.1",
+  "filelock": "3.32.0",
+  "frozenlist": "1.8.0",
+  "fsspec": "2026.6.0",
+  "h11": "0.16.0",
+  "hf-xet": "1.5.2",
+  "httpcore": "1.0.9",
+  "httpx": "0.28.1",
+  "huggingface_hub": "1.25.1",
+  "idna": "3.18",
+  "markdown-it-py": "4.2.0",
+  "mdurl": "0.1.2",
+  "mpmath": "1.3.0",
+  "multidict": "6.7.1",
+  "multiprocess": "0.70.19",
+  "networkx": "3.6.1",
+  "numpy": "2.5.1",
+  "nvidia-cublas": "13.1.1.3",
+  "nvidia-cuda-cupti": "13.0.85",
+  "nvidia-cuda-nvrtc": "13.0.88",
+  "nvidia-cuda-runtime": "13.0.96",
+  "nvidia-cudnn-cu13": "9.20.0.48",
+  "nvidia-cufft": "12.0.0.61",
+  "nvidia-cufile": "1.15.1.6",
+  "nvidia-curand": "10.4.0.35",
+  "nvidia-cusolver": "12.0.4.66",
+  "nvidia-cusparse": "12.6.3.3",
+  "nvidia-cusparselt-cu13": "0.8.1",
+  "nvidia-nccl-cu13": "2.29.7",
+  "nvidia-nvjitlink": "13.3.33",
+  "nvidia-nvshmem-cu13": "3.4.5",
+  "nvidia-nvtx": "13.0.85",
+  "packaging": "26.2",
+  "pandas": "3.0.5",
+  "peft": "0.20.0",
+  "pillow": "12.3.0",
+  "propcache": "0.5.2",
+  "psutil": "7.2.2",
+  "pyarrow": "25.0.0",
+  "python-dateutil": "2.9.0.post0",
+  "python-dotenv": "1.2.2",
+  "regex": "2026.7.19",
+  "requests": "2.34.2",
+  "rich": "15.0.0",
+  "safetensors": "0.8.0",
+  "setuptools": "83.0.0",
+  "shellingham": "1.5.4",
+  "six": "1.17.0",
+  "sympy": "1.14.0",
+  "tokenizers": "0.22.2",
+  "torch": "2.13.0",
+  "torchvision": "0.28.0",
+  "tqdm": "4.70.0",
+  "transformers": "5.14.1",
+  "triton": "3.7.1",
+  "trl": "1.9.2",
+  "typer": "0.27.0",
+  "typing_extensions": "4.16.0",
+  "urllib3": "2.7.0",
+  "xxhash": "3.8.1",
+  "yarl": "1.24.5"
+}

finetune_config.json ADDED Viewed

	@@ -0,0 +1,22 @@

+{
+  "model": "Qwen/Qwen3.6-27B",
+  "dataset": "single_sample",
+  "subpopulation": "cluster_0",
+  "library_name": "peft",
+  "pipeline_tag": "text-generation",
+  "timestamp": "2026-07-30T23:39:22.150075",
+  "hyperparameters": {
+    "lora_r": 256,
+    "lora_alpha": 512,
+    "lora_dropout": 0.05,
+    "dora": false,
+    "quantization": null,
+    "lr": 0.0002,
+    "num_epochs": 10,
+    "batch_size": 4,
+    "gradient_accumulation_steps": 4,
+    "max_seq_length": 1024,
+    "warmup_ratio": 0.1,
+    "dtype": "bf16"
+  }
+}

processor_config.json ADDED Viewed

	@@ -0,0 +1,60 @@

+{
+  "image_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "image_mean": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "image_processor_type": "Qwen2VLImageProcessor",
+    "image_std": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "merge_size": 2,
+    "patch_size": 16,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "size": {
+      "longest_edge": 16777216,
+      "shortest_edge": 65536
+    },
+    "temporal_patch_size": 2
+  },
+  "processor_class": "Qwen3VLProcessor",
+  "video_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "do_sample_frames": true,
+    "fps": 2,
+    "image_mean": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "image_std": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "max_frames": 768,
+    "merge_size": 2,
+    "min_frames": 4,
+    "patch_size": 16,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "return_metadata": false,
+    "size": {
+      "longest_edge": 25165824,
+      "shortest_edge": 4096
+    },
+    "temporal_patch_size": 2,
+    "video_processor_type": "Qwen3VLVideoProcessor"
+  }
+}

tokenizer.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523
+size 19989325

tokenizer_config.json ADDED Viewed

	@@ -0,0 +1,32 @@

+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

training.log ADDED Viewed

	@@ -0,0 +1,85 @@

+logging to output/single_sample/cluster_0/training.log
+[sft] loading tokenizer...
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/config.json "HTTP/1.1 307 Temporary Redirect"
+HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/Qwen/Qwen3.6-27B/6a9e13bd6fc8f0983b9b99948120bc37f49c13e9/config.json "HTTP/1.1 200 OK"
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/tokenizer_config.json "HTTP/1.1 307 Temporary Redirect"
+HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/Qwen/Qwen3.6-27B/6a9e13bd6fc8f0983b9b99948120bc37f49c13e9/tokenizer_config.json "HTTP/1.1 200 OK"
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/tokenizer_config.json "HTTP/1.1 307 Temporary Redirect"
+HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/Qwen/Qwen3.6-27B/6a9e13bd6fc8f0983b9b99948120bc37f49c13e9/tokenizer_config.json "HTTP/1.1 200 OK"
+HTTP Request: GET https://huggingface.co/api/models/Qwen/Qwen3.6-27B/tree/main/additional_chat_templates?recursive=false&expand=false "HTTP/1.1 404 Not Found"
+HTTP Request: GET https://huggingface.co/api/models/Qwen/Qwen3.6-27B/tree/main?recursive=true&expand=false "HTTP/1.1 200 OK"
+HTTP Request: GET https://huggingface.co/api/models/Qwen/Qwen3.6-27B "HTTP/1.1 200 OK"
+[sft] loading model...
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/config.json "HTTP/1.1 307 Temporary Redirect"
+HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/Qwen/Qwen3.6-27B/6a9e13bd6fc8f0983b9b99948120bc37f49c13e9/config.json "HTTP/1.1 200 OK"
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/config.json "HTTP/1.1 307 Temporary Redirect"
+HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/Qwen/Qwen3.6-27B/6a9e13bd6fc8f0983b9b99948120bc37f49c13e9/config.json "HTTP/1.1 200 OK"
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/model.safetensors "HTTP/1.1 404 Not Found"
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/model.safetensors.index.json "HTTP/1.1 307 Temporary Redirect"
+HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/Qwen/Qwen3.6-27B/6a9e13bd6fc8f0983b9b99948120bc37f49c13e9/model.safetensors.index.json "HTTP/1.1 200 OK"
+HTTP Request: GET https://huggingface.co/api/models/Qwen/Qwen3.6-27B/revision/main "HTTP/1.1 200 OK"
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/generation_config.json "HTTP/1.1 307 Temporary Redirect"
+HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/Qwen/Qwen3.6-27B/6a9e13bd6fc8f0983b9b99948120bc37f49c13e9/generation_config.json "HTTP/1.1 200 OK"
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/custom_generate/generate.py "HTTP/1.1 404 Not Found"
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/processor_config.json "HTTP/1.1 404 Not Found"
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/preprocessor_config.json "HTTP/1.1 307 Temporary Redirect"
+HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/Qwen/Qwen3.6-27B/6a9e13bd6fc8f0983b9b99948120bc37f49c13e9/preprocessor_config.json "HTTP/1.1 200 OK"
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/processor_config.json "HTTP/1.1 404 Not Found"
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/preprocessor_config.json "HTTP/1.1 307 Temporary Redirect"
+HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/Qwen/Qwen3.6-27B/6a9e13bd6fc8f0983b9b99948120bc37f49c13e9/preprocessor_config.json "HTTP/1.1 200 OK"
+HTTP Request: GET https://huggingface.co/api/models/Qwen/Qwen3.6-27B/tree/main/additional_chat_templates?recursive=false&expand=false "HTTP/1.1 404 Not Found"
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/processor_config.json "HTTP/1.1 404 Not Found"
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/chat_template.json "HTTP/1.1 404 Not Found"
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/chat_template.jinja "HTTP/1.1 307 Temporary Redirect"
+HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/Qwen/Qwen3.6-27B/6a9e13bd6fc8f0983b9b99948120bc37f49c13e9/chat_template.jinja "HTTP/1.1 200 OK"
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/audio_tokenizer_config.json "HTTP/1.1 404 Not Found"
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/processor_config.json "HTTP/1.1 404 Not Found"
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/preprocessor_config.json "HTTP/1.1 307 Temporary Redirect"
+HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/Qwen/Qwen3.6-27B/6a9e13bd6fc8f0983b9b99948120bc37f49c13e9/preprocessor_config.json "HTTP/1.1 200 OK"
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/processor_config.json "HTTP/1.1 404 Not Found"
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/preprocessor_config.json "HTTP/1.1 307 Temporary Redirect"
+HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/Qwen/Qwen3.6-27B/6a9e13bd6fc8f0983b9b99948120bc37f49c13e9/preprocessor_config.json "HTTP/1.1 200 OK"
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/config.json "HTTP/1.1 307 Temporary Redirect"
+HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/Qwen/Qwen3.6-27B/6a9e13bd6fc8f0983b9b99948120bc37f49c13e9/config.json "HTTP/1.1 200 OK"
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/tokenizer_config.json "HTTP/1.1 307 Temporary Redirect"
+HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/Qwen/Qwen3.6-27B/6a9e13bd6fc8f0983b9b99948120bc37f49c13e9/tokenizer_config.json "HTTP/1.1 200 OK"
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/tokenizer_config.json "HTTP/1.1 307 Temporary Redirect"
+HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/Qwen/Qwen3.6-27B/6a9e13bd6fc8f0983b9b99948120bc37f49c13e9/tokenizer_config.json "HTTP/1.1 200 OK"
+HTTP Request: GET https://huggingface.co/api/models/Qwen/Qwen3.6-27B/tree/main/additional_chat_templates?recursive=false&expand=false "HTTP/1.1 404 Not Found"
+HTTP Request: GET https://huggingface.co/api/models/Qwen/Qwen3.6-27B/tree/main?recursive=true&expand=false "HTTP/1.1 200 OK"
+HTTP Request: GET https://huggingface.co/api/models/Qwen/Qwen3.6-27B "HTTP/1.1 200 OK"
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/processor_config.json "HTTP/1.1 404 Not Found"
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/video_preprocessor_config.json "HTTP/1.1 307 Temporary Redirect"
+HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/Qwen/Qwen3.6-27B/6a9e13bd6fc8f0983b9b99948120bc37f49c13e9/video_preprocessor_config.json "HTTP/1.1 200 OK"
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/preprocessor_config.json "HTTP/1.1 307 Temporary Redirect"
+HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/Qwen/Qwen3.6-27B/6a9e13bd6fc8f0983b9b99948120bc37f49c13e9/preprocessor_config.json "HTTP/1.1 200 OK"
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/processor_config.json "HTTP/1.1 404 Not Found"
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/video_preprocessor_config.json "HTTP/1.1 307 Temporary Redirect"
+HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/Qwen/Qwen3.6-27B/6a9e13bd6fc8f0983b9b99948120bc37f49c13e9/video_preprocessor_config.json "HTTP/1.1 200 OK"
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/preprocessor_config.json "HTTP/1.1 307 Temporary Redirect"
+HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/Qwen/Qwen3.6-27B/6a9e13bd6fc8f0983b9b99948120bc37f49c13e9/preprocessor_config.json "HTTP/1.1 200 OK"
+[sft] starting training...
+HTTP Request: HEAD https://huggingface.co/api/telemetry/trl/SFTTrainer "HTTP/1.1 200 OK"
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/config.json "HTTP/1.1 307 Temporary Redirect"
+HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/Qwen/Qwen3.6-27B/6a9e13bd6fc8f0983b9b99948120bc37f49c13e9/config.json "HTTP/1.1 200 OK"
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/config.json "HTTP/1.1 307 Temporary Redirect"
+HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/Qwen/Qwen3.6-27B/6a9e13bd6fc8f0983b9b99948120bc37f49c13e9/config.json "HTTP/1.1 200 OK"
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/config.json "HTTP/1.1 307 Temporary Redirect"
+HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/Qwen/Qwen3.6-27B/6a9e13bd6fc8f0983b9b99948120bc37f49c13e9/config.json "HTTP/1.1 200 OK"
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/config.json "HTTP/1.1 307 Temporary Redirect"
+HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/Qwen/Qwen3.6-27B/6a9e13bd6fc8f0983b9b99948120bc37f49c13e9/config.json "HTTP/1.1 200 OK"
+[sft] saving...
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/config.json "HTTP/1.1 307 Temporary Redirect"
+HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/Qwen/Qwen3.6-27B/6a9e13bd6fc8f0983b9b99948120bc37f49c13e9/config.json "HTTP/1.1 200 OK"
+HTTP Request: HEAD https://huggingface.co/Qwen/Qwen3.6-27B/resolve/main/config.json "HTTP/1.1 307 Temporary Redirect"
+HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/Qwen/Qwen3.6-27B/6a9e13bd6fc8f0983b9b99948120bc37f49c13e9/config.json "HTTP/1.1 200 OK"
+    saved to output/single_sample/cluster_0
+    gpu: NVIDIA RTX PRO 6000 Blackwell Workstation Edition  time: 3218s
+    training log -> output/single_sample/cluster_0/training_log.json
+    run metadata -> output/single_sample/cluster_0/finetune_config.json
+    environment -> output/single_sample/cluster_0/environment.json
+    README -> output/single_sample/cluster_0/README.md
+HTTP Request: POST https://huggingface.co/api/repos/create "HTTP/1.1 200 OK"
+[upload] uploading output/single_sample/cluster_0 to https://huggingface.co/1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0...
+HTTP Request: POST https://huggingface.co/api/validate-yaml "HTTP/1.1 200 OK"
+HTTP Request: POST https://huggingface.co/api/models/1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0/preupload/main "HTTP/1.1 200 OK"

training_args.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:dd4c327a3e85fbbe3eac78baaf879b393914c659f7bf5becfee75442dabb097c
+size 5713

training_log.json ADDED Viewed

	@@ -0,0 +1,761 @@

+[
+  {
+    "loss": 2.341543960571289,
+    "grad_norm": 2.112253189086914,
+    "learning_rate": 2.4e-05,
+    "entropy": 0.9884634539484978,
+    "mean_token_accuracy": 0.5556907095015049,
+    "num_tokens": 25909.0,
+    "epoch": 0.13377926421404682,
+    "step": 10
+  },
+  {
+    "loss": 0.3863301038742065,
+    "grad_norm": 2.2011115550994873,
+    "learning_rate": 5.0666666666666674e-05,
+    "entropy": 0.4089386660605669,
+    "mean_token_accuracy": 0.9011316433548927,
+    "num_tokens": 52070.0,
+    "epoch": 0.26755852842809363,
+    "step": 20
+  },
+  {
+    "loss": 0.17976584434509277,
+    "grad_norm": 0.5207965970039368,
+    "learning_rate": 7.733333333333333e-05,
+    "entropy": 0.18403972331434487,
+    "mean_token_accuracy": 0.9493107855319977,
+    "num_tokens": 78405.0,
+    "epoch": 0.4013377926421405,
+    "step": 30
+  },
+  {
+    "loss": 0.10954103469848633,
+    "grad_norm": 0.3419843316078186,
+    "learning_rate": 0.00010400000000000001,
+    "entropy": 0.11044432241469622,
+    "mean_token_accuracy": 0.9671209067106247,
+    "num_tokens": 104414.0,
+    "epoch": 0.5351170568561873,
+    "step": 40
+  },
+  {
+    "loss": 0.09990320205688477,
+    "grad_norm": 0.33741509914398193,
+    "learning_rate": 0.00013066666666666668,
+    "entropy": 0.09925312716513872,
+    "mean_token_accuracy": 0.9688541933894157,
+    "num_tokens": 130830.0,
+    "epoch": 0.6688963210702341,
+    "step": 50
+  },
+  {
+    "loss": 0.0990227460861206,
+    "grad_norm": 0.43753594160079956,
+    "learning_rate": 0.00015733333333333333,
+    "entropy": 0.0994369624182582,
+    "mean_token_accuracy": 0.9684862270951271,
+    "num_tokens": 156575.0,
+    "epoch": 0.802675585284281,
+    "step": 60
+  },
+  {
+    "loss": 0.0771427571773529,
+    "grad_norm": 0.3096464276313782,
+    "learning_rate": 0.00018400000000000003,
+    "entropy": 0.07597130816429853,
+    "mean_token_accuracy": 0.9754018127918244,
+    "num_tokens": 182787.0,
+    "epoch": 0.9364548494983278,
+    "step": 70
+  },
+  {
+    "loss": 0.08177689909934997,
+    "grad_norm": 1.9745781421661377,
+    "learning_rate": 0.00019881481481481483,
+    "entropy": 0.07385796203445165,
+    "mean_token_accuracy": 0.9744954705238342,
+    "num_tokens": 208722.0,
+    "epoch": 1.0668896321070234,
+    "step": 80
+  },
+  {
+    "loss": 0.08656259775161743,
+    "grad_norm": 0.24118545651435852,
+    "learning_rate": 0.00019585185185185187,
+    "entropy": 0.0961530750617385,
+    "mean_token_accuracy": 0.9735429108142852,
+    "num_tokens": 234820.0,
+    "epoch": 1.2006688963210703,
+    "step": 90
+  },
+  {
+    "loss": 0.07904126048088074,
+    "grad_norm": 1.4924163818359375,
+    "learning_rate": 0.0001928888888888889,
+    "entropy": 0.07621442535892128,
+    "mean_token_accuracy": 0.9752445220947266,
+    "num_tokens": 261255.0,
+    "epoch": 1.334448160535117,
+    "step": 100
+  },
+  {
+    "loss": 0.1028050422668457,
+    "grad_norm": 0.22435367107391357,
+    "learning_rate": 0.00018992592592592593,
+    "entropy": 0.07814982496201991,
+    "mean_token_accuracy": 0.9725448682904243,
+    "num_tokens": 287420.0,
+    "epoch": 1.468227424749164,
+    "step": 110
+  },
+  {
+    "loss": 0.06647659540176391,
+    "grad_norm": 0.16586251556873322,
+    "learning_rate": 0.00018696296296296297,
+    "entropy": 0.06741584213450551,
+    "mean_token_accuracy": 0.9774157494306565,
+    "num_tokens": 313385.0,
+    "epoch": 1.6020066889632107,
+    "step": 120
+  },
+  {
+    "loss": 0.057019442319869995,
+    "grad_norm": 0.13507577776908875,
+    "learning_rate": 0.00018400000000000003,
+    "entropy": 0.05611985893920064,
+    "mean_token_accuracy": 0.9796610698103905,
+    "num_tokens": 340048.0,
+    "epoch": 1.7357859531772575,
+    "step": 130
+  },
+  {
+    "loss": 0.061611056327819824,
+    "grad_norm": 0.2060474306344986,
+    "learning_rate": 0.00018103703703703706,
+    "entropy": 0.06344100441783666,
+    "mean_token_accuracy": 0.97853914052248,
+    "num_tokens": 365876.0,
+    "epoch": 1.8695652173913042,
+    "step": 140
+  },
+  {
+    "loss": 0.061782336235046385,
+    "grad_norm": 0.3134608268737793,
+    "learning_rate": 0.0001780740740740741,
+    "entropy": 0.05643900770407457,
+    "mean_token_accuracy": 0.978085016592955,
+    "num_tokens": 390738.0,
+    "epoch": 2.0,
+    "step": 150
+  },
+  {
+    "loss": 0.057867032289505,
+    "grad_norm": 0.33745983242988586,
+    "learning_rate": 0.00017511111111111113,
+    "entropy": 0.05852913670241833,
+    "mean_token_accuracy": 0.9795649841427803,
+    "num_tokens": 416830.0,
+    "epoch": 2.1337792642140467,
+    "step": 160
+  },
+  {
+    "loss": 0.058108162879943845,
+    "grad_norm": 0.1164756640791893,
+    "learning_rate": 0.00017214814814814816,
+    "entropy": 0.0567542714998126,
+    "mean_token_accuracy": 0.9790374860167503,
+    "num_tokens": 443309.0,
+    "epoch": 2.2675585284280935,
+    "step": 170
+  },
+  {
+    "loss": 0.05357828140258789,
+    "grad_norm": 0.12977160513401031,
+    "learning_rate": 0.0001691851851851852,
+    "entropy": 0.05269257873296738,
+    "mean_token_accuracy": 0.9791603803634643,
+    "num_tokens": 469381.0,
+    "epoch": 2.4013377926421406,
+    "step": 180
+  },
+  {
+    "loss": 0.06209459900856018,
+    "grad_norm": 5.890456676483154,
+    "learning_rate": 0.00016622222222222223,
+    "entropy": 0.05242626103572547,
+    "mean_token_accuracy": 0.9787786990404129,
+    "num_tokens": 495823.0,
+    "epoch": 2.5351170568561874,
+    "step": 190
+  },
+  {
+    "loss": 0.052966177463531494,
+    "grad_norm": 0.1879248023033142,
+    "learning_rate": 0.00016325925925925926,
+    "entropy": 0.05595421427860856,
+    "mean_token_accuracy": 0.9809569671750069,
+    "num_tokens": 521936.0,
+    "epoch": 2.668896321070234,
+    "step": 200
+  },
+  {
+    "loss": 0.1533456563949585,
+    "grad_norm": 2.4737038612365723,
+    "learning_rate": 0.0001602962962962963,
+    "entropy": 0.09764697654172778,
+    "mean_token_accuracy": 0.9594999879598618,
+    "num_tokens": 547808.0,
+    "epoch": 2.802675585284281,
+    "step": 210
+  },
+  {
+    "loss": 0.14215301275253295,
+    "grad_norm": 0.21627607941627502,
+    "learning_rate": 0.00015733333333333333,
+    "entropy": 0.09831915665417909,
+    "mean_token_accuracy": 0.96669692248106,
+    "num_tokens": 573896.0,
+    "epoch": 2.936454849498328,
+    "step": 220
+  },
+  {
+    "loss": 0.07142106890678405,
+    "grad_norm": 0.24911274015903473,
+    "learning_rate": 0.0001543703703703704,
+    "entropy": 0.08219717662685956,
+    "mean_token_accuracy": 0.975415417781243,
+    "num_tokens": 599295.0,
+    "epoch": 3.0668896321070234,
+    "step": 230
+  },
+  {
+    "loss": 0.06142540574073792,
+    "grad_norm": 2.818284273147583,
+    "learning_rate": 0.00015140740740740742,
+    "entropy": 0.06195763573050499,
+    "mean_token_accuracy": 0.978814198076725,
+    "num_tokens": 625594.0,
+    "epoch": 3.20066889632107,
+    "step": 240
+  },
+  {
+    "loss": 0.09106510877609253,
+    "grad_norm": 0.3751987814903259,
+    "learning_rate": 0.00014844444444444445,
+    "entropy": 0.07343985894694924,
+    "mean_token_accuracy": 0.9736507266759873,
+    "num_tokens": 651379.0,
+    "epoch": 3.334448160535117,
+    "step": 250
+  },
+  {
+    "loss": 0.06325957775115967,
+    "grad_norm": 3.0620086193084717,
+    "learning_rate": 0.0001454814814814815,
+    "entropy": 0.06504379613325,
+    "mean_token_accuracy": 0.9780099034309387,
+    "num_tokens": 677883.0,
+    "epoch": 3.468227424749164,
+    "step": 260
+  },
+  {
+    "loss": 0.06386491656303406,
+    "grad_norm": 0.13123036921024323,
+    "learning_rate": 0.00014251851851851852,
+    "entropy": 0.06299111396074294,
+    "mean_token_accuracy": 0.9776420131325722,
+    "num_tokens": 703942.0,
+    "epoch": 3.6020066889632107,
+    "step": 270
+  },
+  {
+    "loss": 0.07105348110198975,
+    "grad_norm": 0.16725099086761475,
+    "learning_rate": 0.00013955555555555558,
+    "entropy": 0.06523667583242058,
+    "mean_token_accuracy": 0.9771258398890496,
+    "num_tokens": 730355.0,
+    "epoch": 3.7357859531772575,
+    "step": 280
+  },
+  {
+    "loss": 0.05892015695571899,
+    "grad_norm": 0.8018252849578857,
+    "learning_rate": 0.00013659259259259261,
+    "entropy": 0.06290404153987765,
+    "mean_token_accuracy": 0.9795312806963921,
+    "num_tokens": 756288.0,
+    "epoch": 3.869565217391304,
+    "step": 290
+  },
+  {
+    "loss": 0.05513965487480164,
+    "grad_norm": 0.23648729920387268,
+    "learning_rate": 0.00013362962962962965,
+    "entropy": 0.05575119321927046,
+    "mean_token_accuracy": 0.980286703659938,
+    "num_tokens": 781476.0,
+    "epoch": 4.0,
+    "step": 300
+  },
+  {
+    "loss": 0.049145272374153136,
+    "grad_norm": 0.23902687430381775,
+    "learning_rate": 0.00013066666666666668,
+    "entropy": 0.047657360602170226,
+    "mean_token_accuracy": 0.9813551798462867,
+    "num_tokens": 807995.0,
+    "epoch": 4.133779264214047,
+    "step": 310
+  },
+  {
+    "loss": 0.052027386426925656,
+    "grad_norm": 0.08278259634971619,
+    "learning_rate": 0.00012770370370370371,
+    "entropy": 0.04914208557456732,
+    "mean_token_accuracy": 0.9807460099458695,
+    "num_tokens": 834211.0,
+    "epoch": 4.2675585284280935,
+    "step": 320
+  },
+  {
+    "loss": 0.04958886206150055,
+    "grad_norm": 0.06415274739265442,
+    "learning_rate": 0.00012474074074074075,
+    "entropy": 0.05262458473443985,
+    "mean_token_accuracy": 0.9813323050737381,
+    "num_tokens": 860320.0,
+    "epoch": 4.40133779264214,
+    "step": 330
+  },
+  {
+    "loss": 0.0506575882434845,
+    "grad_norm": 0.08827455341815948,
+    "learning_rate": 0.0001217777777777778,
+    "entropy": 0.049570453260093926,
+    "mean_token_accuracy": 0.9807545498013497,
+    "num_tokens": 886260.0,
+    "epoch": 4.535117056856187,
+    "step": 340
+  },
+  {
+    "loss": 0.04992738664150238,
+    "grad_norm": 0.1842905879020691,
+    "learning_rate": 0.00011881481481481483,
+    "entropy": 0.049180435854941605,
+    "mean_token_accuracy": 0.9801862895488739,
+    "num_tokens": 912192.0,
+    "epoch": 4.668896321070234,
+    "step": 350
+  },
+  {
+    "loss": 0.04837550222873688,
+    "grad_norm": 0.07146720588207245,
+    "learning_rate": 0.00011585185185185186,
+    "entropy": 0.04882028605788946,
+    "mean_token_accuracy": 0.9817976549267768,
+    "num_tokens": 938443.0,
+    "epoch": 4.802675585284281,
+    "step": 360
+  },
+  {
+    "loss": 0.047449523210525514,
+    "grad_norm": 0.07254461199045181,
+    "learning_rate": 0.0001128888888888889,
+    "entropy": 0.04849126562476158,
+    "mean_token_accuracy": 0.9812644407153129,
+    "num_tokens": 964704.0,
+    "epoch": 4.936454849498328,
+    "step": 370
+  },
+  {
+    "loss": 0.047966784238815306,
+    "grad_norm": 0.05580724775791168,
+    "learning_rate": 0.00010992592592592593,
+    "entropy": 0.04855830528033085,
+    "mean_token_accuracy": 0.9817173633819971,
+    "num_tokens": 989807.0,
+    "epoch": 5.066889632107023,
+    "step": 380
+  },
+  {
+    "loss": 0.04565880596637726,
+    "grad_norm": 0.06281863152980804,
+    "learning_rate": 0.00010696296296296297,
+    "entropy": 0.04446020070463419,
+    "mean_token_accuracy": 0.98235524892807,
+    "num_tokens": 1016432.0,
+    "epoch": 5.20066889632107,
+    "step": 390
+  },
+  {
+    "loss": 0.047007548809051516,
+    "grad_norm": 0.05037130042910576,
+    "learning_rate": 0.00010400000000000001,
+    "entropy": 0.046955365035682914,
+    "mean_token_accuracy": 0.9810686483979225,
+    "num_tokens": 1042595.0,
+    "epoch": 5.334448160535117,
+    "step": 400
+  },
+  {
+    "loss": 0.04774223566055298,
+    "grad_norm": 0.05550120770931244,
+    "learning_rate": 0.00010103703703703704,
+    "entropy": 0.04687619395554066,
+    "mean_token_accuracy": 0.9813349410891533,
+    "num_tokens": 1068499.0,
+    "epoch": 5.468227424749164,
+    "step": 410
+  },
+  {
+    "loss": 0.04721843302249908,
+    "grad_norm": 0.07114529609680176,
+    "learning_rate": 9.807407407407407e-05,
+    "entropy": 0.048110970109701154,
+    "mean_token_accuracy": 0.9819222643971444,
+    "num_tokens": 1094712.0,
+    "epoch": 5.602006688963211,
+    "step": 420
+  },
+  {
+    "loss": 0.04641242623329163,
+    "grad_norm": 0.054782308638095856,
+    "learning_rate": 9.511111111111112e-05,
+    "entropy": 0.045364385657012465,
+    "mean_token_accuracy": 0.9820315048098565,
+    "num_tokens": 1121039.0,
+    "epoch": 5.735785953177258,
+    "step": 430
+  },
+  {
+    "loss": 0.04601489901542664,
+    "grad_norm": 0.05270834639668465,
+    "learning_rate": 9.214814814814815e-05,
+    "entropy": 0.046417954843491316,
+    "mean_token_accuracy": 0.9826092898845673,
+    "num_tokens": 1147476.0,
+    "epoch": 5.869565217391305,
+    "step": 440
+  },
+  {
+    "loss": 0.047896665334701535,
+    "grad_norm": 0.07369400560855865,
+    "learning_rate": 8.918518518518519e-05,
+    "entropy": 0.04759720369027211,
+    "mean_token_accuracy": 0.9811742412738311,
+    "num_tokens": 1172214.0,
+    "epoch": 6.0,
+    "step": 450
+  },
+  {
+    "loss": 0.04460802972316742,
+    "grad_norm": 0.043926533311605453,
+    "learning_rate": 8.622222222222222e-05,
+    "entropy": 0.0450520227663219,
+    "mean_token_accuracy": 0.983108639717102,
+    "num_tokens": 1198481.0,
+    "epoch": 6.133779264214047,
+    "step": 460
+  },
+  {
+    "loss": 0.04490154683589935,
+    "grad_norm": 0.049713313579559326,
+    "learning_rate": 8.325925925925925e-05,
+    "entropy": 0.043880753125995396,
+    "mean_token_accuracy": 0.981929925084114,
+    "num_tokens": 1225270.0,
+    "epoch": 6.2675585284280935,
+    "step": 470
+  },
+  {
+    "loss": 0.04616124629974365,
+    "grad_norm": 0.04489952698349953,
+    "learning_rate": 8.02962962962963e-05,
+    "entropy": 0.04594219285063446,
+    "mean_token_accuracy": 0.9817362502217293,
+    "num_tokens": 1251191.0,
+    "epoch": 6.40133779264214,
+    "step": 480
+  },
+  {
+    "loss": 0.04496528506278992,
+    "grad_norm": 0.04445449262857437,
+    "learning_rate": 7.733333333333333e-05,
+    "entropy": 0.045685965567827225,
+    "mean_token_accuracy": 0.9831607535481452,
+    "num_tokens": 1277832.0,
+    "epoch": 6.535117056856187,
+    "step": 490
+  },
+  {
+    "loss": 0.04584149122238159,
+    "grad_norm": 0.05437646806240082,
+    "learning_rate": 7.437037037037038e-05,
+    "entropy": 0.04615831598639488,
+    "mean_token_accuracy": 0.9816597178578377,
+    "num_tokens": 1304203.0,
+    "epoch": 6.668896321070234,
+    "step": 500
+  },
+  {
+    "loss": 0.04616081714630127,
+    "grad_norm": 0.04138358682394028,
+    "learning_rate": 7.140740740740741e-05,
+    "entropy": 0.04780284911394119,
+    "mean_token_accuracy": 0.9815138846635818,
+    "num_tokens": 1330059.0,
+    "epoch": 6.802675585284281,
+    "step": 510
+  },
+  {
+    "loss": 0.047184526920318604,
+    "grad_norm": 0.05079558491706848,
+    "learning_rate": 6.844444444444445e-05,
+    "entropy": 0.04507539812475443,
+    "mean_token_accuracy": 0.9809173971414566,
+    "num_tokens": 1355654.0,
+    "epoch": 6.936454849498328,
+    "step": 520
+  },
+  {
+    "loss": 0.045189327001571654,
+    "grad_norm": 0.04889153689146042,
+    "learning_rate": 6.54814814814815e-05,
+    "entropy": 0.04632591761839695,
+    "mean_token_accuracy": 0.9823385003285531,
+    "num_tokens": 1380740.0,
+    "epoch": 7.066889632107023,
+    "step": 530
+  },
+  {
+    "loss": 0.046108204126358035,
+    "grad_norm": 0.06250233948230743,
+    "learning_rate": 6.251851851851853e-05,
+    "entropy": 0.047147559747099874,
+    "mean_token_accuracy": 0.9815558344125748,
+    "num_tokens": 1406229.0,
+    "epoch": 7.20066889632107,
+    "step": 540
+  },
+  {
+    "loss": 0.04495341181755066,
+    "grad_norm": 0.04576072841882706,
+    "learning_rate": 5.9555555555555554e-05,
+    "entropy": 0.04380973177030682,
+    "mean_token_accuracy": 0.9825871646404266,
+    "num_tokens": 1432718.0,
+    "epoch": 7.334448160535117,
+    "step": 550
+  },
+  {
+    "loss": 0.04445241391658783,
+    "grad_norm": 0.03827248513698578,
+    "learning_rate": 5.6592592592592594e-05,
+    "entropy": 0.04404275622218847,
+    "mean_token_accuracy": 0.9829956665635109,
+    "num_tokens": 1459141.0,
+    "epoch": 7.468227424749164,
+    "step": 560
+  },
+  {
+    "loss": 0.04570685029029846,
+    "grad_norm": 0.03917380049824715,
+    "learning_rate": 5.362962962962963e-05,
+    "entropy": 0.04583866978064179,
+    "mean_token_accuracy": 0.9810639604926109,
+    "num_tokens": 1485265.0,
+    "epoch": 7.602006688963211,
+    "step": 570
+  },
+  {
+    "loss": 0.04584062099456787,
+    "grad_norm": 0.04370535910129547,
+    "learning_rate": 5.0666666666666674e-05,
+    "entropy": 0.04647259535267949,
+    "mean_token_accuracy": 0.9812271296977997,
+    "num_tokens": 1511176.0,
+    "epoch": 7.735785953177258,
+    "step": 580
+  },
+  {
+    "loss": 0.045942744612693785,
+    "grad_norm": 0.049611467868089676,
+    "learning_rate": 4.770370370370371e-05,
+    "entropy": 0.045514973998069766,
+    "mean_token_accuracy": 0.9814796030521393,
+    "num_tokens": 1537083.0,
+    "epoch": 7.869565217391305,
+    "step": 590
+  },
+  {
+    "loss": 0.04436405599117279,
+    "grad_norm": 0.044528670608997345,
+    "learning_rate": 4.474074074074075e-05,
+    "entropy": 0.04354679699127491,
+    "mean_token_accuracy": 0.9819874656506073,
+    "num_tokens": 1562952.0,
+    "epoch": 8.0,
+    "step": 600
+  },
+  {
+    "loss": 0.044945243000984195,
+    "grad_norm": 0.045649204403162,
+    "learning_rate": 4.177777777777778e-05,
+    "entropy": 0.04686344414949417,
+    "mean_token_accuracy": 0.9817773103713989,
+    "num_tokens": 1588851.0,
+    "epoch": 8.133779264214047,
+    "step": 610
+  },
+  {
+    "loss": 0.04427960515022278,
+    "grad_norm": 0.043687306344509125,
+    "learning_rate": 3.8814814814814814e-05,
+    "entropy": 0.04569701394066215,
+    "mean_token_accuracy": 0.9820200949907303,
+    "num_tokens": 1615149.0,
+    "epoch": 8.267558528428093,
+    "step": 620
+  },
+  {
+    "loss": 0.0447018563747406,
+    "grad_norm": 0.04362299293279648,
+    "learning_rate": 3.5851851851851854e-05,
+    "entropy": 0.04486168110743165,
+    "mean_token_accuracy": 0.9820230007171631,
+    "num_tokens": 1641174.0,
+    "epoch": 8.40133779264214,
+    "step": 630
+  },
+  {
+    "loss": 0.04458645284175873,
+    "grad_norm": 0.04046420380473137,
+    "learning_rate": 3.2888888888888894e-05,
+    "entropy": 0.044725227542221545,
+    "mean_token_accuracy": 0.9823110207915307,
+    "num_tokens": 1667583.0,
+    "epoch": 8.535117056856187,
+    "step": 640
+  },
+  {
+    "loss": 0.044426333904266355,
+    "grad_norm": 0.036781083792448044,
+    "learning_rate": 2.992592592592593e-05,
+    "entropy": 0.04480462055653334,
+    "mean_token_accuracy": 0.9823192492127418,
+    "num_tokens": 1694027.0,
+    "epoch": 8.668896321070234,
+    "step": 650
+  },
+  {
+    "loss": 0.045354658365249635,
+    "grad_norm": 0.03921104222536087,
+    "learning_rate": 2.696296296296296e-05,
+    "entropy": 0.04497002325952053,
+    "mean_token_accuracy": 0.9815022855997085,
+    "num_tokens": 1719962.0,
+    "epoch": 8.80267558528428,
+    "step": 660
+  },
+  {
+    "loss": 0.04457056820392609,
+    "grad_norm": 0.03931435942649841,
+    "learning_rate": 2.4e-05,
+    "entropy": 0.044097188767045735,
+    "mean_token_accuracy": 0.9824405118823052,
+    "num_tokens": 1746249.0,
+    "epoch": 8.936454849498327,
+    "step": 670
+  },
+  {
+    "loss": 0.04478446841239929,
+    "grad_norm": 0.032005202025175095,
+    "learning_rate": 2.1037037037037037e-05,
+    "entropy": 0.045094484893175274,
+    "mean_token_accuracy": 0.9815951723318833,
+    "num_tokens": 1771309.0,
+    "epoch": 9.066889632107024,
+    "step": 680
+  },
+  {
+    "loss": 0.04368177354335785,
+    "grad_norm": 0.03807147592306137,
+    "learning_rate": 1.8074074074074074e-05,
+    "entropy": 0.04654490072280169,
+    "mean_token_accuracy": 0.9825977519154548,
+    "num_tokens": 1797647.0,
+    "epoch": 9.200668896321071,
+    "step": 690
+  },
+  {
+    "loss": 0.043600788712501524,
+    "grad_norm": 0.03561871498823166,
+    "learning_rate": 1.5111111111111112e-05,
+    "entropy": 0.04407275160774589,
+    "mean_token_accuracy": 0.9828053265810013,
+    "num_tokens": 1824009.0,
+    "epoch": 9.334448160535118,
+    "step": 700
+  },
+  {
+    "loss": 0.04528459906578064,
+    "grad_norm": 0.03744061291217804,
+    "learning_rate": 1.2148148148148149e-05,
+    "entropy": 0.04444531872868538,
+    "mean_token_accuracy": 0.9812339514493942,
+    "num_tokens": 1849648.0,
+    "epoch": 9.468227424749164,
+    "step": 710
+  },
+  {
+    "loss": 0.0440999835729599,
+    "grad_norm": 0.0362723171710968,
+    "learning_rate": 9.185185185185186e-06,
+    "entropy": 0.04479610063135624,
+    "mean_token_accuracy": 0.9827097550034523,
+    "num_tokens": 1875926.0,
+    "epoch": 9.602006688963211,
+    "step": 720
+  },
+  {
+    "loss": 0.044123375415802,
+    "grad_norm": 0.03960822895169258,
+    "learning_rate": 6.222222222222222e-06,
+    "entropy": 0.04520800011232495,
+    "mean_token_accuracy": 0.9823104426264763,
+    "num_tokens": 1902227.0,
+    "epoch": 9.735785953177258,
+    "step": 730
+  },
+  {
+    "loss": 0.04401821792125702,
+    "grad_norm": 0.03371009603142738,
+    "learning_rate": 3.259259259259259e-06,
+    "entropy": 0.04439304880797863,
+    "mean_token_accuracy": 0.9820376738905907,
+    "num_tokens": 1928651.0,
+    "epoch": 9.869565217391305,
+    "step": 740
+  },
+  {
+    "loss": 0.044789040088653566,
+    "grad_norm": 0.04102063924074173,
+    "learning_rate": 2.962962962962963e-07,
+    "entropy": 0.045203205484610334,
+    "mean_token_accuracy": 0.9820073674886655,
+    "num_tokens": 1953690.0,
+    "epoch": 10.0,
+    "step": 750
+  },
+  {
+    "train_runtime": 3217.5242,
+    "train_samples_per_second": 3.714,
+    "train_steps_per_second": 0.233,
+    "total_flos": 3.662638954482125e+17,
+    "train_loss": 0.09439072036743164,
+    "epoch": 10.0,
+    "step": 750
+  }
+]