🚀 Crucible 4.5 out now
Loom
textilelabs
·
AI & ML interests
Independent builder training small/efficient models from scratch on consumer hardware.
Recent Activity
liked a Space about 1 hour ago
huggingface/moe-blogpost reacted to gustavecortal's post with 🚀 about 8 hours ago
Releasing PhiloLM (https://huggingface.co/gustavecortal/PhiloLM), a 112M base model trained from scratch on open philosophy texts.
PhiloLM was trained on PhiloText (https://huggingface.co/datasets/gustavecortal/PhiloText): 0.5B tokens of open philosophy encyclopedias (SEP, IEP), public-domain books, reviews, textbooks and educational web pages.
Trained on (only!) 1.5B tokens in 15 hours on a $350 GPU (Intel Arc A770). PhiloLM builds on efficient methods: Muon optimizer, bigram embeddings, sliding-window attention, GQA.
PhiloLM is optimized for efficient inference, reaching ~500 tokens/s on a $150 CPU. Try it in your browser: https://gustavecortal.com/philosophy-lm/
Organizations
None yet