Banaxi Inc. PRO

Banaxi-Tech

Banaxi-Tech

AI & ML interests

SLMs, training from scratch, LoRA, TTS

Recent Activity

new activity 11 minutes ago

Banaxi-Larp/supralarps-50m-base:🚩 Report: Illegal or restricted content

repliedto their post about 1 hour ago

Hello AI Community! 👋 We currently have a new AI Model and we are currently training it. We are training it on 27B tokens and are currently 8% done. Follow us to be notified when it releases 🚀 Some Info: Parameters 75M GPU: RTX Pro 6000 We expect to be able to release it in the coming days EDIT: We are now at step 40000/274000 expect a Preview Model coming at about 100K

repliedto their post about 2 hours ago

View all activity

Organizations

replied to their post about 1 hour ago

Tasks	Version	Filter	Metric		Value		Stderr
arc_challenge	1	none	acc	↑	0.2073	±	0.0118
		none	acc_norm	↑	0.2381	±	0.0124
arc_easy	1	none	acc	↑	0.4882	±	0.0103
		none	acc_norm	↑	0.4234	±	0.0101
hellaswag	1	none	acc	↑	0.2880	±	0.0045
		none	acc_norm	↑	0.3038	±	0.0046
piqa	1	none	acc	↑	0.6153	±	0.0114
		none	acc_norm	↑	0.5968	±	0.0114
winogrande	1	none	acc	↑	0.5107	±	0.0140

=== ArithMark-2.0 checkpoint-150000 ===
Average: 0.2540  (635/2500)
Random chance: 0.2500

By difficulty:
easy: 0.2528  (316/1250)
hard: 0.2320  (116/500)
medium: 0.2707  (203/750)

By operator_count:
1: 0.2528  (316/1250)
2: 0.2707  (203/750)
3: 0.2320  (116/500)

By topic:
addition: 0.1543  (83/538)
division: 0.5000  (65/130)
mixed_three_ops: 0.2479  (60/242)
mixed_two_ops: 0.2456  (97/395)
multiplication: 0.4375  (63/144)
parentheses_three_ops: 0.2171  (56/258)
parentheses_two_ops: 0.2986  (106/355)
subtraction: 0.2397  (105/438)

replied to their post about 2 hours ago

Im at 150K and the benchmarks arent that good, did this also happen to you with Supra 50M Base?

posted an update about 23 hours ago

Post

733

Hello AI Community! 👋
We currently have a new AI Model and we are currently training it.
We are training it on 27B tokens and are currently 8% done.
Follow us to be notified when it releases 🚀
Some Info:
Parameters 75M
GPU: RTX Pro 6000
We expect to be able to release it in the coming days

EDIT: We are now at step 40000/274000 expect a Preview Model coming at about 100K

7 replies

posted an update 2 days ago

Post

100

We are excited to release MiniBananaMind-v2-9M

MiniBananaMind-v2-9M is a tiny causal language model trained fully from scratch on FineWeb-Edu.

It has only ~8.9M parameters, but was trained for ~3.54B tokens after retokenization using a custom 8k byte-level BPE tokenizer.
We trained it on a RTX 5070 Ti in just 4h 34m!
Go check it out at BananaMind/MiniBananaMind-v2-9M

Banaxi Inc. PRO

AI & ML interests

Recent Activity

Organizations

Banaxi-Tech's activity