OpenMOSE

/

RWKV-Reka-Flash-Gen2

like 3

its L12 Hybrid.

first release "Gen1" uses L6 GQA + L38 RWKV
second release "Gen2" uses L12 GQA + L32 RWKV

Gen1 could 40k NIAH.
Gen2 reached 65k NIAH(more ctx len, stable reasoning, fine-tuning capability)

will write description :)