Instructions to use 1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use 1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0 with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.6-27B")
model = PeftModel.from_pretrained(base_model, "1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0")

Transformers

How to use 1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use 1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0

SGLang

How to use 1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use 1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0 with Docker Model Runner:
```
docker model run hf.co/1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0
```

Qwen3.6-27B LoRA — Single Sample, Cluster 0

This model is a LoRA fine-tune of Qwen/Qwen3.6-27B as part of the AIML589 project.

This adapter is licensed under CC BY-SA 4.0.

Dataset

Fine-tuned on the single_sample config of the wvs-nz-value-alignment dataset, cluster_0 subpopulation.

Part of the wvs-nz-value-alignment collection.

GPU: NVIDIA RTX PRO 6000 Blackwell Workstation Edition · Training time: 53m 37s

Training hyperparameters

Parameter	Value
LoRA rank	256
LoRA alpha	512
LoRA dropout	0.05
DoRA	False
Learning rate	0.0002
Batch size	4
Gradient accumulation	4
Epochs	10
Max seq length	1024
Warmup ratio	0.1
Dtype	bf16

Training log

{"loss": 2.341543960571289, "grad_norm": 2.112253189086914, "learning_rate": 2.4e-05, "entropy": 0.9884634539484978, "mean_token_accuracy": 0.5556907095015049, "num_tokens": 25909.0, "epoch": 0.13377926421404682, "step": 10}
{"loss": 0.3863301038742065, "grad_norm": 2.2011115550994873, "learning_rate": 5.0666666666666674e-05, "entropy": 0.4089386660605669, "mean_token_accuracy": 0.9011316433548927, "num_tokens": 52070.0, "epoch": 0.26755852842809363, "step": 20}
{"loss": 0.17976584434509277, "grad_norm": 0.5207965970039368, "learning_rate": 7.733333333333333e-05, "entropy": 0.18403972331434487, "mean_token_accuracy": 0.9493107855319977, "num_tokens": 78405.0, "epoch": 0.4013377926421405, "step": 30}
{"loss": 0.10954103469848633, "grad_norm": 0.3419843316078186, "learning_rate": 0.00010400000000000001, "entropy": 0.11044432241469622, "mean_token_accuracy": 0.9671209067106247, "num_tokens": 104414.0, "epoch": 0.5351170568561873, "step": 40}
{"loss": 0.09990320205688477, "grad_norm": 0.33741509914398193, "learning_rate": 0.00013066666666666668, "entropy": 0.09925312716513872, "mean_token_accuracy": 0.9688541933894157, "num_tokens": 130830.0, "epoch": 0.6688963210702341, "step": 50}
{"loss": 0.0990227460861206, "grad_norm": 0.43753594160079956, "learning_rate": 0.00015733333333333333, "entropy": 0.0994369624182582, "mean_token_accuracy": 0.9684862270951271, "num_tokens": 156575.0, "epoch": 0.802675585284281, "step": 60}
{"loss": 0.0771427571773529, "grad_norm": 0.3096464276313782, "learning_rate": 0.00018400000000000003, "entropy": 0.07597130816429853, "mean_token_accuracy": 0.9754018127918244, "num_tokens": 182787.0, "epoch": 0.9364548494983278, "step": 70}
{"loss": 0.08177689909934997, "grad_norm": 1.9745781421661377, "learning_rate": 0.00019881481481481483, "entropy": 0.07385796203445165, "mean_token_accuracy": 0.9744954705238342, "num_tokens": 208722.0, "epoch": 1.0668896321070234, "step": 80}
{"loss": 0.08656259775161743, "grad_norm": 0.24118545651435852, "learning_rate": 0.00019585185185185187, "entropy": 0.0961530750617385, "mean_token_accuracy": 0.9735429108142852, "num_tokens": 234820.0, "epoch": 1.2006688963210703, "step": 90}
{"loss": 0.07904126048088074, "grad_norm": 1.4924163818359375, "learning_rate": 0.0001928888888888889, "entropy": 0.07621442535892128, "mean_token_accuracy": 0.9752445220947266, "num_tokens": 261255.0, "epoch": 1.334448160535117, "step": 100}
{"loss": 0.1028050422668457, "grad_norm": 0.22435367107391357, "learning_rate": 0.00018992592592592593, "entropy": 0.07814982496201991, "mean_token_accuracy": 0.9725448682904243, "num_tokens": 287420.0, "epoch": 1.468227424749164, "step": 110}
{"loss": 0.06647659540176391, "grad_norm": 0.16586251556873322, "learning_rate": 0.00018696296296296297, "entropy": 0.06741584213450551, "mean_token_accuracy": 0.9774157494306565, "num_tokens": 313385.0, "epoch": 1.6020066889632107, "step": 120}
{"loss": 0.057019442319869995, "grad_norm": 0.13507577776908875, "learning_rate": 0.00018400000000000003, "entropy": 0.05611985893920064, "mean_token_accuracy": 0.9796610698103905, "num_tokens": 340048.0, "epoch": 1.7357859531772575, "step": 130}
{"loss": 0.061611056327819824, "grad_norm": 0.2060474306344986, "learning_rate": 0.00018103703703703706, "entropy": 0.06344100441783666, "mean_token_accuracy": 0.97853914052248, "num_tokens": 365876.0, "epoch": 1.8695652173913042, "step": 140}
{"loss": 0.061782336235046385, "grad_norm": 0.3134608268737793, "learning_rate": 0.0001780740740740741, "entropy": 0.05643900770407457, "mean_token_accuracy": 0.978085016592955, "num_tokens": 390738.0, "epoch": 2.0, "step": 150}
{"loss": 0.057867032289505, "grad_norm": 0.33745983242988586, "learning_rate": 0.00017511111111111113, "entropy": 0.05852913670241833, "mean_token_accuracy": 0.9795649841427803, "num_tokens": 416830.0, "epoch": 2.1337792642140467, "step": 160}
{"loss": 0.058108162879943845, "grad_norm": 0.1164756640791893, "learning_rate": 0.00017214814814814816, "entropy": 0.0567542714998126, "mean_token_accuracy": 0.9790374860167503, "num_tokens": 443309.0, "epoch": 2.2675585284280935, "step": 170}
{"loss": 0.05357828140258789, "grad_norm": 0.12977160513401031, "learning_rate": 0.0001691851851851852, "entropy": 0.05269257873296738, "mean_token_accuracy": 0.9791603803634643, "num_tokens": 469381.0, "epoch": 2.4013377926421406, "step": 180}
{"loss": 0.06209459900856018, "grad_norm": 5.890456676483154, "learning_rate": 0.00016622222222222223, "entropy": 0.05242626103572547, "mean_token_accuracy": 0.9787786990404129, "num_tokens": 495823.0, "epoch": 2.5351170568561874, "step": 190}
{"loss": 0.052966177463531494, "grad_norm": 0.1879248023033142, "learning_rate": 0.00016325925925925926, "entropy": 0.05595421427860856, "mean_token_accuracy": 0.9809569671750069, "num_tokens": 521936.0, "epoch": 2.668896321070234, "step": 200}
{"loss": 0.1533456563949585, "grad_norm": 2.4737038612365723, "learning_rate": 0.0001602962962962963, "entropy": 0.09764697654172778, "mean_token_accuracy": 0.9594999879598618, "num_tokens": 547808.0, "epoch": 2.802675585284281, "step": 210}
{"loss": 0.14215301275253295, "grad_norm": 0.21627607941627502, "learning_rate": 0.00015733333333333333, "entropy": 0.09831915665417909, "mean_token_accuracy": 0.96669692248106, "num_tokens": 573896.0, "epoch": 2.936454849498328, "step": 220}
{"loss": 0.07142106890678405, "grad_norm": 0.24911274015903473, "learning_rate": 0.0001543703703703704, "entropy": 0.08219717662685956, "mean_token_accuracy": 0.975415417781243, "num_tokens": 599295.0, "epoch": 3.0668896321070234, "step": 230}
{"loss": 0.06142540574073792, "grad_norm": 2.818284273147583, "learning_rate": 0.00015140740740740742, "entropy": 0.06195763573050499, "mean_token_accuracy": 0.978814198076725, "num_tokens": 625594.0, "epoch": 3.20066889632107, "step": 240}
{"loss": 0.09106510877609253, "grad_norm": 0.3751987814903259, "learning_rate": 0.00014844444444444445, "entropy": 0.07343985894694924, "mean_token_accuracy": 0.9736507266759873, "num_tokens": 651379.0, "epoch": 3.334448160535117, "step": 250}
{"loss": 0.06325957775115967, "grad_norm": 3.0620086193084717, "learning_rate": 0.0001454814814814815, "entropy": 0.06504379613325, "mean_token_accuracy": 0.9780099034309387, "num_tokens": 677883.0, "epoch": 3.468227424749164, "step": 260}
{"loss": 0.06386491656303406, "grad_norm": 0.13123036921024323, "learning_rate": 0.00014251851851851852, "entropy": 0.06299111396074294, "mean_token_accuracy": 0.9776420131325722, "num_tokens": 703942.0, "epoch": 3.6020066889632107, "step": 270}
{"loss": 0.07105348110198975, "grad_norm": 0.16725099086761475, "learning_rate": 0.00013955555555555558, "entropy": 0.06523667583242058, "mean_token_accuracy": 0.9771258398890496, "num_tokens": 730355.0, "epoch": 3.7357859531772575, "step": 280}
{"loss": 0.05892015695571899, "grad_norm": 0.8018252849578857, "learning_rate": 0.00013659259259259261, "entropy": 0.06290404153987765, "mean_token_accuracy": 0.9795312806963921, "num_tokens": 756288.0, "epoch": 3.869565217391304, "step": 290}
{"loss": 0.05513965487480164, "grad_norm": 0.23648729920387268, "learning_rate": 0.00013362962962962965, "entropy": 0.05575119321927046, "mean_token_accuracy": 0.980286703659938, "num_tokens": 781476.0, "epoch": 4.0, "step": 300}
{"loss": 0.049145272374153136, "grad_norm": 0.23902687430381775, "learning_rate": 0.00013066666666666668, "entropy": 0.047657360602170226, "mean_token_accuracy": 0.9813551798462867, "num_tokens": 807995.0, "epoch": 4.133779264214047, "step": 310}
{"loss": 0.052027386426925656, "grad_norm": 0.08278259634971619, "learning_rate": 0.00012770370370370371, "entropy": 0.04914208557456732, "mean_token_accuracy": 0.9807460099458695, "num_tokens": 834211.0, "epoch": 4.2675585284280935, "step": 320}
{"loss": 0.04958886206150055, "grad_norm": 0.06415274739265442, "learning_rate": 0.00012474074074074075, "entropy": 0.05262458473443985, "mean_token_accuracy": 0.9813323050737381, "num_tokens": 860320.0, "epoch": 4.40133779264214, "step": 330}
{"loss": 0.0506575882434845, "grad_norm": 0.08827455341815948, "learning_rate": 0.0001217777777777778, "entropy": 0.049570453260093926, "mean_token_accuracy": 0.9807545498013497, "num_tokens": 886260.0, "epoch": 4.535117056856187, "step": 340}
{"loss": 0.04992738664150238, "grad_norm": 0.1842905879020691, "learning_rate": 0.00011881481481481483, "entropy": 0.049180435854941605, "mean_token_accuracy": 0.9801862895488739, "num_tokens": 912192.0, "epoch": 4.668896321070234, "step": 350}
{"loss": 0.04837550222873688, "grad_norm": 0.07146720588207245, "learning_rate": 0.00011585185185185186, "entropy": 0.04882028605788946, "mean_token_accuracy": 0.9817976549267768, "num_tokens": 938443.0, "epoch": 4.802675585284281, "step": 360}
{"loss": 0.047449523210525514, "grad_norm": 0.07254461199045181, "learning_rate": 0.0001128888888888889, "entropy": 0.04849126562476158, "mean_token_accuracy": 0.9812644407153129, "num_tokens": 964704.0, "epoch": 4.936454849498328, "step": 370}
{"loss": 0.047966784238815306, "grad_norm": 0.05580724775791168, "learning_rate": 0.00010992592592592593, "entropy": 0.04855830528033085, "mean_token_accuracy": 0.9817173633819971, "num_tokens": 989807.0, "epoch": 5.066889632107023, "step": 380}
{"loss": 0.04565880596637726, "grad_norm": 0.06281863152980804, "learning_rate": 0.00010696296296296297, "entropy": 0.04446020070463419, "mean_token_accuracy": 0.98235524892807, "num_tokens": 1016432.0, "epoch": 5.20066889632107, "step": 390}
{"loss": 0.047007548809051516, "grad_norm": 0.05037130042910576, "learning_rate": 0.00010400000000000001, "entropy": 0.046955365035682914, "mean_token_accuracy": 0.9810686483979225, "num_tokens": 1042595.0, "epoch": 5.334448160535117, "step": 400}
{"loss": 0.04774223566055298, "grad_norm": 0.05550120770931244, "learning_rate": 0.00010103703703703704, "entropy": 0.04687619395554066, "mean_token_accuracy": 0.9813349410891533, "num_tokens": 1068499.0, "epoch": 5.468227424749164, "step": 410}
{"loss": 0.04721843302249908, "grad_norm": 0.07114529609680176, "learning_rate": 9.807407407407407e-05, "entropy": 0.048110970109701154, "mean_token_accuracy": 0.9819222643971444, "num_tokens": 1094712.0, "epoch": 5.602006688963211, "step": 420}
{"loss": 0.04641242623329163, "grad_norm": 0.054782308638095856, "learning_rate": 9.511111111111112e-05, "entropy": 0.045364385657012465, "mean_token_accuracy": 0.9820315048098565, "num_tokens": 1121039.0, "epoch": 5.735785953177258, "step": 430}
{"loss": 0.04601489901542664, "grad_norm": 0.05270834639668465, "learning_rate": 9.214814814814815e-05, "entropy": 0.046417954843491316, "mean_token_accuracy": 0.9826092898845673, "num_tokens": 1147476.0, "epoch": 5.869565217391305, "step": 440}
{"loss": 0.047896665334701535, "grad_norm": 0.07369400560855865, "learning_rate": 8.918518518518519e-05, "entropy": 0.04759720369027211, "mean_token_accuracy": 0.9811742412738311, "num_tokens": 1172214.0, "epoch": 6.0, "step": 450}
{"loss": 0.04460802972316742, "grad_norm": 0.043926533311605453, "learning_rate": 8.622222222222222e-05, "entropy": 0.0450520227663219, "mean_token_accuracy": 0.983108639717102, "num_tokens": 1198481.0, "epoch": 6.133779264214047, "step": 460}
{"loss": 0.04490154683589935, "grad_norm": 0.049713313579559326, "learning_rate": 8.325925925925925e-05, "entropy": 0.043880753125995396, "mean_token_accuracy": 0.981929925084114, "num_tokens": 1225270.0, "epoch": 6.2675585284280935, "step": 470}
{"loss": 0.04616124629974365, "grad_norm": 0.04489952698349953, "learning_rate": 8.02962962962963e-05, "entropy": 0.04594219285063446, "mean_token_accuracy": 0.9817362502217293, "num_tokens": 1251191.0, "epoch": 6.40133779264214, "step": 480}
{"loss": 0.04496528506278992, "grad_norm": 0.04445449262857437, "learning_rate": 7.733333333333333e-05, "entropy": 0.045685965567827225, "mean_token_accuracy": 0.9831607535481452, "num_tokens": 1277832.0, "epoch": 6.535117056856187, "step": 490}
{"loss": 0.04584149122238159, "grad_norm": 0.05437646806240082, "learning_rate": 7.437037037037038e-05, "entropy": 0.04615831598639488, "mean_token_accuracy": 0.9816597178578377, "num_tokens": 1304203.0, "epoch": 6.668896321070234, "step": 500}
{"loss": 0.04616081714630127, "grad_norm": 0.04138358682394028, "learning_rate": 7.140740740740741e-05, "entropy": 0.04780284911394119, "mean_token_accuracy": 0.9815138846635818, "num_tokens": 1330059.0, "epoch": 6.802675585284281, "step": 510}
{"loss": 0.047184526920318604, "grad_norm": 0.05079558491706848, "learning_rate": 6.844444444444445e-05, "entropy": 0.04507539812475443, "mean_token_accuracy": 0.9809173971414566, "num_tokens": 1355654.0, "epoch": 6.936454849498328, "step": 520}
{"loss": 0.045189327001571654, "grad_norm": 0.04889153689146042, "learning_rate": 6.54814814814815e-05, "entropy": 0.04632591761839695, "mean_token_accuracy": 0.9823385003285531, "num_tokens": 1380740.0, "epoch": 7.066889632107023, "step": 530}
{"loss": 0.046108204126358035, "grad_norm": 0.06250233948230743, "learning_rate": 6.251851851851853e-05, "entropy": 0.047147559747099874, "mean_token_accuracy": 0.9815558344125748, "num_tokens": 1406229.0, "epoch": 7.20066889632107, "step": 540}
{"loss": 0.04495341181755066, "grad_norm": 0.04576072841882706, "learning_rate": 5.9555555555555554e-05, "entropy": 0.04380973177030682, "mean_token_accuracy": 0.9825871646404266, "num_tokens": 1432718.0, "epoch": 7.334448160535117, "step": 550}
{"loss": 0.04445241391658783, "grad_norm": 0.03827248513698578, "learning_rate": 5.6592592592592594e-05, "entropy": 0.04404275622218847, "mean_token_accuracy": 0.9829956665635109, "num_tokens": 1459141.0, "epoch": 7.468227424749164, "step": 560}
{"loss": 0.04570685029029846, "grad_norm": 0.03917380049824715, "learning_rate": 5.362962962962963e-05, "entropy": 0.04583866978064179, "mean_token_accuracy": 0.9810639604926109, "num_tokens": 1485265.0, "epoch": 7.602006688963211, "step": 570}
{"loss": 0.04584062099456787, "grad_norm": 0.04370535910129547, "learning_rate": 5.0666666666666674e-05, "entropy": 0.04647259535267949, "mean_token_accuracy": 0.9812271296977997, "num_tokens": 1511176.0, "epoch": 7.735785953177258, "step": 580}
{"loss": 0.045942744612693785, "grad_norm": 0.049611467868089676, "learning_rate": 4.770370370370371e-05, "entropy": 0.045514973998069766, "mean_token_accuracy": 0.9814796030521393, "num_tokens": 1537083.0, "epoch": 7.869565217391305, "step": 590}
{"loss": 0.04436405599117279, "grad_norm": 0.044528670608997345, "learning_rate": 4.474074074074075e-05, "entropy": 0.04354679699127491, "mean_token_accuracy": 0.9819874656506073, "num_tokens": 1562952.0, "epoch": 8.0, "step": 600}
{"loss": 0.044945243000984195, "grad_norm": 0.045649204403162, "learning_rate": 4.177777777777778e-05, "entropy": 0.04686344414949417, "mean_token_accuracy": 0.9817773103713989, "num_tokens": 1588851.0, "epoch": 8.133779264214047, "step": 610}
{"loss": 0.04427960515022278, "grad_norm": 0.043687306344509125, "learning_rate": 3.8814814814814814e-05, "entropy": 0.04569701394066215, "mean_token_accuracy": 0.9820200949907303, "num_tokens": 1615149.0, "epoch": 8.267558528428093, "step": 620}
{"loss": 0.0447018563747406, "grad_norm": 0.04362299293279648, "learning_rate": 3.5851851851851854e-05, "entropy": 0.04486168110743165, "mean_token_accuracy": 0.9820230007171631, "num_tokens": 1641174.0, "epoch": 8.40133779264214, "step": 630}
{"loss": 0.04458645284175873, "grad_norm": 0.04046420380473137, "learning_rate": 3.2888888888888894e-05, "entropy": 0.044725227542221545, "mean_token_accuracy": 0.9823110207915307, "num_tokens": 1667583.0, "epoch": 8.535117056856187, "step": 640}
{"loss": 0.044426333904266355, "grad_norm": 0.036781083792448044, "learning_rate": 2.992592592592593e-05, "entropy": 0.04480462055653334, "mean_token_accuracy": 0.9823192492127418, "num_tokens": 1694027.0, "epoch": 8.668896321070234, "step": 650}
{"loss": 0.045354658365249635, "grad_norm": 0.03921104222536087, "learning_rate": 2.696296296296296e-05, "entropy": 0.04497002325952053, "mean_token_accuracy": 0.9815022855997085, "num_tokens": 1719962.0, "epoch": 8.80267558528428, "step": 660}
{"loss": 0.04457056820392609, "grad_norm": 0.03931435942649841, "learning_rate": 2.4e-05, "entropy": 0.044097188767045735, "mean_token_accuracy": 0.9824405118823052, "num_tokens": 1746249.0, "epoch": 8.936454849498327, "step": 670}
{"loss": 0.04478446841239929, "grad_norm": 0.032005202025175095, "learning_rate": 2.1037037037037037e-05, "entropy": 0.045094484893175274, "mean_token_accuracy": 0.9815951723318833, "num_tokens": 1771309.0, "epoch": 9.066889632107024, "step": 680}
{"loss": 0.04368177354335785, "grad_norm": 0.03807147592306137, "learning_rate": 1.8074074074074074e-05, "entropy": 0.04654490072280169, "mean_token_accuracy": 0.9825977519154548, "num_tokens": 1797647.0, "epoch": 9.200668896321071, "step": 690}
{"loss": 0.043600788712501524, "grad_norm": 0.03561871498823166, "learning_rate": 1.5111111111111112e-05, "entropy": 0.04407275160774589, "mean_token_accuracy": 0.9828053265810013, "num_tokens": 1824009.0, "epoch": 9.334448160535118, "step": 700}
{"loss": 0.04528459906578064, "grad_norm": 0.03744061291217804, "learning_rate": 1.2148148148148149e-05, "entropy": 0.04444531872868538, "mean_token_accuracy": 0.9812339514493942, "num_tokens": 1849648.0, "epoch": 9.468227424749164, "step": 710}
{"loss": 0.0440999835729599, "grad_norm": 0.0362723171710968, "learning_rate": 9.185185185185186e-06, "entropy": 0.04479610063135624, "mean_token_accuracy": 0.9827097550034523, "num_tokens": 1875926.0, "epoch": 9.602006688963211, "step": 720}
{"loss": 0.044123375415802, "grad_norm": 0.03960822895169258, "learning_rate": 6.222222222222222e-06, "entropy": 0.04520800011232495, "mean_token_accuracy": 0.9823104426264763, "num_tokens": 1902227.0, "epoch": 9.735785953177258, "step": 730}
{"loss": 0.04401821792125702, "grad_norm": 0.03371009603142738, "learning_rate": 3.259259259259259e-06, "entropy": 0.04439304880797863, "mean_token_accuracy": 0.9820376738905907, "num_tokens": 1928651.0, "epoch": 9.869565217391305, "step": 740}
{"loss": 0.044789040088653566, "grad_norm": 0.04102063924074173, "learning_rate": 2.962962962962963e-07, "entropy": 0.045203205484610334, "mean_token_accuracy": 0.9820073674886655, "num_tokens": 1953690.0, "epoch": 10.0, "step": 750}
{"train_runtime": 3217.5242, "train_samples_per_second": 3.714, "train_steps_per_second": 0.233, "total_flos": 3.662638954482125e+17, "train_loss": 0.09439072036743164, "epoch": 10.0, "step": 750}

Environment

Package	Version
torch	2.13.0
transformers	5.14.1
trl	1.9.2
datasets	5.0.1
accelerate	1.14.0
python-dotenv	1.2.2
peft	0.20.0
bitsandbytes	0.50.0
huggingface-hub	?
jinja2	?
torchvision	0.28.0
pillow	12.3.0

Intended use

This adapter is intended for research purposes only as part of the AIML589 project, which investigates value alignment of LLMs with New Zealand population distributions from the World Values Survey.

Out-of-scope

This model has not been safety-tuned for general-purpose deployment. It should not be used in production systems, for making decisions about people, or in contexts where reliability and safety are critical.

Limitations and biases

Fine-tuned on a single WVS wave (Wave 7) for New Zealand only.
The training data reflects the values of those who responded to the survey and may not represent all New Zealanders.
LoRA adapters are subject to the limitations and biases of the base model (Qwen/Qwen3.6-27B).

Downloads last month: 21

Model tree for 1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0

Base model

Qwen/Qwen3.6-27B

Adapter

(386)

this model

Dataset used to train 1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0

Collection including 1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-cluster_0

wvs-nz-value-alignment

Collection

Datasets and LoRA adapters for LLM alignment project using World Value Survey NZ data. • 10 items • Updated about 3 hours ago