Article
Writing Down the Line Between Luck and Skill
FINAL-Bench
• • 10
Warecube Korea
Warecube
AI & ML interests
None yet
Recent Activity
upvoted an article about 8 hours ago
Writing Down the Line Between Luck and Skill reacted to SeaWolf-AI's post with 🔥 about 8 hours ago
Can AI beat the market? Nobody has actually measured it.
We opened a 122-day public experiment to find out. $2,000 in prizes.
Here is the problem with every trading result you have ever read. Someone returns 30% in a month. Skill or luck? There has never been a way to tell, because nobody measured how far a player with zero skill could have gone over the same window.
So we measured it first. Twenty thousand random players, per asset, charged the same fees.
Bitcoin +86.6%. NVIDIA +51.7%. Crude oil +26.9%. Gold +9.2%.
That is the luck ceiling. A return below it is not evidence of skill, and every row on our leaderboard shows where it sits against that line.
How you compete: submit one number between −1.0 and +1.0. It holds until you replace it, traded against live prices with real execution costs. Leverage is fixed at 1, so betting bigger is not a way to win. The answer lives in the future — the world writes it after you submit, which means fitting the past cannot help you.
Humans move a slider. Agents attach an MCP server and gain four tools, then you tell them "enter the challenge."
We already found something before the season began. Thirteen well-known rules, run from 1 January through the same scorer: Stochastic 14/3 finishes 1st on NVIDIA at +43% and 12th on Bitcoin at −25%. Donchian breakout does the exact opposite — last on NVIDIA, first on Bitcoin. The ranking inverts. "Which indicator is good" turns out not to be a well-posed question; the character of the market decides.
Four assets: NVIDIA, Bitcoin, Gold, Crude Oil. $500 to the top return in each. 24 August to 24 December 2026.
The organisers do not compete. Three baselines — buy and hold, volatility targeting, random — sit in the same table instead, because a leaderboard without a scale cannot be read.
The scoring code is public. Read what it does before you enter.
https://huggingface.co/spaces/FINAL-Bench/finchal
https://huggingface.co/blog/FINAL-Bench/financial-forecast-challenge liked a Space about 8 hours ago
FINAL-Bench/finchalOrganizations
None yet
upvoted an article about 8 hours ago
reacted to SeaWolf-AI's post with 🔥 about 8 hours ago
Post
514
Can AI beat the market? Nobody has actually measured it.
We opened a 122-day public experiment to find out. $2,000 in prizes.
Here is the problem with every trading result you have ever read. Someone returns 30% in a month. Skill or luck? There has never been a way to tell, because nobody measured how far a player with zero skill could have gone over the same window.
So we measured it first. Twenty thousand random players, per asset, charged the same fees.
Bitcoin +86.6%. NVIDIA +51.7%. Crude oil +26.9%. Gold +9.2%.
That is the luck ceiling. A return below it is not evidence of skill, and every row on our leaderboard shows where it sits against that line.
How you compete: submit one number between −1.0 and +1.0. It holds until you replace it, traded against live prices with real execution costs. Leverage is fixed at 1, so betting bigger is not a way to win. The answer lives in the future — the world writes it after you submit, which means fitting the past cannot help you.
Humans move a slider. Agents attach an MCP server and gain four tools, then you tell them "enter the challenge."
We already found something before the season began. Thirteen well-known rules, run from 1 January through the same scorer: Stochastic 14/3 finishes 1st on NVIDIA at +43% and 12th on Bitcoin at −25%. Donchian breakout does the exact opposite — last on NVIDIA, first on Bitcoin. The ranking inverts. "Which indicator is good" turns out not to be a well-posed question; the character of the market decides.
Four assets: NVIDIA, Bitcoin, Gold, Crude Oil. $500 to the top return in each. 24 August to 24 December 2026.
The organisers do not compete. Three baselines — buy and hold, volatility targeting, random — sit in the same table instead, because a leaderboard without a scale cannot be read.
The scoring code is public. Read what it does before you enter.
FINAL-Bench/finchal
https://huggingface.co/blog/FINAL-Bench/financial-forecast-challenge
We opened a 122-day public experiment to find out. $2,000 in prizes.
Here is the problem with every trading result you have ever read. Someone returns 30% in a month. Skill or luck? There has never been a way to tell, because nobody measured how far a player with zero skill could have gone over the same window.
So we measured it first. Twenty thousand random players, per asset, charged the same fees.
Bitcoin +86.6%. NVIDIA +51.7%. Crude oil +26.9%. Gold +9.2%.
That is the luck ceiling. A return below it is not evidence of skill, and every row on our leaderboard shows where it sits against that line.
How you compete: submit one number between −1.0 and +1.0. It holds until you replace it, traded against live prices with real execution costs. Leverage is fixed at 1, so betting bigger is not a way to win. The answer lives in the future — the world writes it after you submit, which means fitting the past cannot help you.
Humans move a slider. Agents attach an MCP server and gain four tools, then you tell them "enter the challenge."
We already found something before the season began. Thirteen well-known rules, run from 1 January through the same scorer: Stochastic 14/3 finishes 1st on NVIDIA at +43% and 12th on Bitcoin at −25%. Donchian breakout does the exact opposite — last on NVIDIA, first on Bitcoin. The ranking inverts. "Which indicator is good" turns out not to be a well-posed question; the character of the market decides.
Four assets: NVIDIA, Bitcoin, Gold, Crude Oil. $500 to the top return in each. 24 August to 24 December 2026.
The organisers do not compete. Three baselines — buy and hold, volatility targeting, random — sit in the same table instead, because a leaderboard without a scale cannot be read.
The scoring code is public. Read what it does before you enter.
FINAL-Bench/finchal
https://huggingface.co/blog/FINAL-Bench/financial-forecast-challenge
- 1 reply
liked a
Space about 8 hours ago
liked a
Space 2 days ago
upvoted an article 2 days ago
Article
We changed one line and the benchmark score moved 0.21 AUROC
FINAL-Bench
• • 10 reacted to SeaWolf-AI's post with ❤️ 2 days ago
Post
3622
We opened a benchmark for drug property prediction tools. LEADBOARD: 21 boards across 7 disciplines, 18,382 held-out compounds, labels we never hand out.
Two numbers we hit while building it are the reason it exists.
First. Split the hERG cardiotoxicity data at random and you get AUROC 0.818. Split it by first-report year instead and you get 0.606. Same molecules, same fingerprints, same learner, same hyperparameters. The only thing that changed was where the line went, and the score moved 0.211. That is a wider gap than you will find between most competing methods in the literature.
Second. On 7 of our 19 regression boards, predicting the training mean for everything has a lower MAE than a trained gradient-boosted model. hERG is one of them, 0.599 against 0.589. The trained model loses.
So every board publishes its homework before anyone submits. Three untrained baselines, the measured experimental noise floor from compounds that appear in two or more papers, and exactly how the test set was cut. A gap smaller than the noise floor is not a difference in skill, and you should be able to see that without guessing.
Entering is simple. Download a test set that contains structures and nothing else, predict with whatever you like, upload a two-column CSV of compound_id and prediction. Trained model, physics engine, LLM, rule of thumb. We do not care what is inside. We measure the output.
Post: https://huggingface.co/blog/FINAL-Bench/leadboard-drug
Leaderboard: FINAL-Bench/leadboard
Two numbers we hit while building it are the reason it exists.
First. Split the hERG cardiotoxicity data at random and you get AUROC 0.818. Split it by first-report year instead and you get 0.606. Same molecules, same fingerprints, same learner, same hyperparameters. The only thing that changed was where the line went, and the score moved 0.211. That is a wider gap than you will find between most competing methods in the literature.
Second. On 7 of our 19 regression boards, predicting the training mean for everything has a lower MAE than a trained gradient-boosted model. hERG is one of them, 0.599 against 0.589. The trained model loses.
So every board publishes its homework before anyone submits. Three untrained baselines, the measured experimental noise floor from compounds that appear in two or more papers, and exactly how the test set was cut. A gap smaller than the noise floor is not a difference in skill, and you should be able to see that without guessing.
Entering is simple. Download a test set that contains structures and nothing else, predict with whatever you like, upload a two-column CSV of compound_id and prediction. Trained model, physics engine, LLM, rule of thumb. We do not care what is inside. We measure the output.
Post: https://huggingface.co/blog/FINAL-Bench/leadboard-drug
Leaderboard: FINAL-Bench/leadboard
- 6 replies
reacted to SeaWolf-AI's post with 🚀🔥 3 days ago
Post
4006
🔋 Open Materials Challenge, Season 1 — Solid-State Battery Electrolytes
A solid-state battery replaces the liquid electrolyte of a lithium-ion cell with a solid. It does not catch fire, it lasts longer, and it can hold more. What has not been solved is finding a material that is solid and still lets lithium through.
Such a material has to do four things at once: give lithium a path to move along, block electrons, hold up at the charging voltage, and survive contact with the lithium-metal anode without decomposing. Plenty of materials manage three. Very few manage all four.
This challenge looks for candidates, together. You submit one composition — for example Li3YCl6. We score it computationally and place it on the board. There is no prize.
Scoring (100 points)
Oxidation stability 40 does it resist decomposing as the voltage rises
Lithium-metal stability 35 does it survive contact with the anode
Use novelty 25 higher if it has not been reported as an electrolyte
Entry condition a percolating path for lithium must exist
Ionic conductivity is not a scored axis this season. Every value is a computational estimate and implies nothing about real performance or safety.
The board also carries seven electrolytes in actual use — LGPS, argyrodite, LLZO, LATP and others. They are scored but hold no rank. They are there so you can see where materials people already build with happen to land.
Compositions are private by default. Nothing is disclosed unless you choose to publish it, and each entry is recorded with its timestamp. If a third party asks to discuss a particular entry, we pass the request along — never the submitter's identity, unless they agree to it.
Season 1 runs 2026-08-21 to 11-30. A participation guide and a set of prompts are included.
👉 FINAL-Bench/open-materials-challenge
Our sister challenge🧪https://huggingface.co/spaces/FINAL-Bench/open-discovery-challenge
A solid-state battery replaces the liquid electrolyte of a lithium-ion cell with a solid. It does not catch fire, it lasts longer, and it can hold more. What has not been solved is finding a material that is solid and still lets lithium through.
Such a material has to do four things at once: give lithium a path to move along, block electrons, hold up at the charging voltage, and survive contact with the lithium-metal anode without decomposing. Plenty of materials manage three. Very few manage all four.
This challenge looks for candidates, together. You submit one composition — for example Li3YCl6. We score it computationally and place it on the board. There is no prize.
Scoring (100 points)
Oxidation stability 40 does it resist decomposing as the voltage rises
Lithium-metal stability 35 does it survive contact with the anode
Use novelty 25 higher if it has not been reported as an electrolyte
Entry condition a percolating path for lithium must exist
Ionic conductivity is not a scored axis this season. Every value is a computational estimate and implies nothing about real performance or safety.
The board also carries seven electrolytes in actual use — LGPS, argyrodite, LLZO, LATP and others. They are scored but hold no rank. They are there so you can see where materials people already build with happen to land.
Compositions are private by default. Nothing is disclosed unless you choose to publish it, and each entry is recorded with its timestamp. If a third party asks to discuss a particular entry, we pass the request along — never the submitter's identity, unless they agree to it.
Season 1 runs 2026-08-21 to 11-30. A participation guide and a set of prompts are included.
👉 FINAL-Bench/open-materials-challenge
Our sister challenge🧪https://huggingface.co/spaces/FINAL-Bench/open-discovery-challenge
liked a
Space 3 days ago
reacted to SeaWolf-AI's post with 🔥 5 days ago
Post
3955
3,631 candidate molecules arrived in five days, from 83 accounts — roughly 700 a day. Far more than we expected. Thank you.
Yesterday we opened the third season and 224 arrived within a day: Chagas disease.
Why this disease
Around 6 million people live with it, mostly in Latin America (WHO). Many carry it for decades without knowing, while the heart is slowly damaged. There are two drugs and both date from the 1960s, hard enough to tolerate that many patients cannot finish the two-month course.
Sixty years without a new drug is not only a scientific problem. Most patients live where development costs cannot be recovered, which is why WHO calls this a neglected tropical disease.
But the cost of proposing a candidate and filtering it has changed. So it seemed worth asking whether work nobody funds could be done by many people sharing it out.
The problem this season
The target is CYP51, the enzyme T. cruzi uses to build its membrane sterols. Block it and the parasite cannot survive. The difficulty is that we carry the same enzyme.
Scoring: binding 30 · selectivity 30 · ADMET 15 · whole-cell 10 · novelty 10 · synthesis 5
Selectivity carries 30 points because nobody has solved it. Among the approved azoles on the board as reference compounds, some score 0 on selectivity — not a scorer fault, but the measurement.
Taking part
Design with any model, submit a SMILES, scored within minutes. Five ready-to-paste prompts per season, and the full rubric is published. Your molecule stays yours; private submission is the default.
Prizes — 4,000 USD across three seasons
Malaria 30 Sep · 1,000 | Tuberculosis 31 Oct · 2,000 | Chagas 30 Nov · 1,000
We know this does not cover the time you spend. It is a way of saying the work had worth.
FINAL-Bench/open-discovery-challenge
Yesterday we opened the third season and 224 arrived within a day: Chagas disease.
Why this disease
Around 6 million people live with it, mostly in Latin America (WHO). Many carry it for decades without knowing, while the heart is slowly damaged. There are two drugs and both date from the 1960s, hard enough to tolerate that many patients cannot finish the two-month course.
Sixty years without a new drug is not only a scientific problem. Most patients live where development costs cannot be recovered, which is why WHO calls this a neglected tropical disease.
But the cost of proposing a candidate and filtering it has changed. So it seemed worth asking whether work nobody funds could be done by many people sharing it out.
The problem this season
The target is CYP51, the enzyme T. cruzi uses to build its membrane sterols. Block it and the parasite cannot survive. The difficulty is that we carry the same enzyme.
Scoring: binding 30 · selectivity 30 · ADMET 15 · whole-cell 10 · novelty 10 · synthesis 5
Selectivity carries 30 points because nobody has solved it. Among the approved azoles on the board as reference compounds, some score 0 on selectivity — not a scorer fault, but the measurement.
Taking part
Design with any model, submit a SMILES, scored within minutes. Five ready-to-paste prompts per season, and the full rubric is published. Your molecule stays yours; private submission is the default.
Prizes — 4,000 USD across three seasons
Malaria 30 Sep · 1,000 | Tuberculosis 31 Oct · 2,000 | Chagas 30 Nov · 1,000
We know this does not cover the time you spend. It is a way of saying the work had worth.
FINAL-Bench/open-discovery-challenge
reacted to SeaWolf-AI's post with ❤️ 8 days ago
Post
4275
🧬 Your AI can design a malaria drug candidate. Can it tell you whether it's any good?
Open Discovery Challenge #1 — Malaria is live. Design a molecule with any model — OpenAI, Claude, Gemini, Qwen, KIMI, DeepSeek, open weights, or by hand — submit it as SMILES, and it's scored in minutes on whole-cell activity, target binding, selectivity over the human enzyme, ADMET, novelty and synthesisability.
You can check the scoring instead of trusting it. Approved drugs sit on the same leaderboard as the entries: DSM265, a clinical-stage antimalarial, scores 50.9. Teriflunomide — approved, but it hits the human enzyme — scores 2.8. Caffeine scores 1.8. If the clinical candidate lands on top and coffee lands at the bottom, the scorer discriminates.
We caught 14 defects before opening — conventional toxicity cutoffs rejected all three approved antimalarials and coffee. All written up, along with the rule we now hold everything to: a gate that rejects an approved drug is a broken gate.
Your molecule stays yours. No patent interest, nothing into our pipeline. You choose whether it's published — and publishing can cost you patentability, so we say so.
USD 1,000 to the top entry when Season #1 closes 30 September 2026 — not payment for your tokens, but a way of saying the work had worth.
Malaria killed ~597,000 people in 2023, three quarters of them children under five. Not for want of chemistry — for want of a market.
No chemistry needed: the guide ships five prompts you can paste straight into your model, and the full rubric is published.
📖 https://huggingface.co/blog/FINAL-Bench/open-discovery-challenge
🚀 FINAL-Bench/open-discovery-challenge
Computational assessments of candidates — not measurements, not claims of efficacy.
Open Discovery Challenge #1 — Malaria is live. Design a molecule with any model — OpenAI, Claude, Gemini, Qwen, KIMI, DeepSeek, open weights, or by hand — submit it as SMILES, and it's scored in minutes on whole-cell activity, target binding, selectivity over the human enzyme, ADMET, novelty and synthesisability.
You can check the scoring instead of trusting it. Approved drugs sit on the same leaderboard as the entries: DSM265, a clinical-stage antimalarial, scores 50.9. Teriflunomide — approved, but it hits the human enzyme — scores 2.8. Caffeine scores 1.8. If the clinical candidate lands on top and coffee lands at the bottom, the scorer discriminates.
We caught 14 defects before opening — conventional toxicity cutoffs rejected all three approved antimalarials and coffee. All written up, along with the rule we now hold everything to: a gate that rejects an approved drug is a broken gate.
Your molecule stays yours. No patent interest, nothing into our pipeline. You choose whether it's published — and publishing can cost you patentability, so we say so.
USD 1,000 to the top entry when Season #1 closes 30 September 2026 — not payment for your tokens, but a way of saying the work had worth.
Malaria killed ~597,000 people in 2023, three quarters of them children under five. Not for want of chemistry — for want of a market.
No chemistry needed: the guide ships five prompts you can paste straight into your model, and the full rubric is published.
📖 https://huggingface.co/blog/FINAL-Bench/open-discovery-challenge
🚀 FINAL-Bench/open-discovery-challenge
Computational assessments of candidates — not measurements, not claims of efficacy.
- 12 replies
upvoted a collection 8 days ago
liked a
Space 9 days ago
upvoted an article 9 days ago
Article
Who Tells You Whether the Molecule Your AI Just Designed Is Any Good?
FINAL-Bench
• • 11 liked a
dataset 10 days ago
liked a
Space 10 days ago
upvoted an article 10 days ago
Article
AX-Ray, Finding Causal-Leakage Defects in Two General-Purpose Public Models
FINAL-Bench
• • 13 upvoted an article 16 days ago
Article
Model Genome: Fingerprinting Whether an LLM Was Trained From Scratch or Derived
mayafree
• • 18 liked a
Space 16 days ago
reacted to mayafree's post with 🔥 16 days ago
Post
2401
🧬 Architecture lineage of Korea's sovereign-AI foundation models — checked with public data
In late July 2026, as Korea released self-developed foundation models competing with DeepSeek and Qwen (e.g. LG K-EXAONE 2.0, 750B), interest grew — including a Zhihu thread with 2.7M+ views (→ https://www.zhihu.com/question/2067512422555029717 ) — over whether these models are trained from scratch or built on foreign open-weights.
Sharing a tool that answers this with public data rather than opinion.
🔗 Model Genome Korea → mayafree/Model-Genome-Korea
It classifies the public models of 9 Korean organizations that released "self-developed, from-scratch foundation models" on HuggingFace — 3 large enterprises (LG, NAVER, Kakao), 2 telcos (SKT, KT), 2 mid-size firms (NCSOFT, Upstage), 2 startups (Motif, VIDRAFT) — on two axes measured from public config.json + model weights:
• Architecture fingerprint — does model_type + (hidden·intermediate·layers) match a foreign open-weight model
• Weight fingerprint — embedding similarity (from-scratch vs continued-pretraining)
Genotypes: 🟢 Native · 🔵 Adapted · 🟡 Mixed · 🔴 Ported
The results are not uniform. Some models match foreign architectures (Qwen, Llama, …) exactly; others use self-built architectures and weights with no foreign match. Which company/model falls where is shown per model in the Space, along with attention originality, license, and reproducible open-source status.
This is a neutral transparency tool, not an accusation — building foundation models on open-weight bases is a legitimate, industry-standard practice. The exact same yardstick is applied to every model, without exception.
Features a 3D lineage graph, search, EN / 中文 / 한국어, and dark mode. Corrections are welcome via the Community tab.
Articles: https://huggingface.co/blog/mayafree/model-dna
#KoreanAI #LLM #ModelLineage #OpenSource #SovereignAI
In late July 2026, as Korea released self-developed foundation models competing with DeepSeek and Qwen (e.g. LG K-EXAONE 2.0, 750B), interest grew — including a Zhihu thread with 2.7M+ views (→ https://www.zhihu.com/question/2067512422555029717 ) — over whether these models are trained from scratch or built on foreign open-weights.
Sharing a tool that answers this with public data rather than opinion.
🔗 Model Genome Korea → mayafree/Model-Genome-Korea
It classifies the public models of 9 Korean organizations that released "self-developed, from-scratch foundation models" on HuggingFace — 3 large enterprises (LG, NAVER, Kakao), 2 telcos (SKT, KT), 2 mid-size firms (NCSOFT, Upstage), 2 startups (Motif, VIDRAFT) — on two axes measured from public config.json + model weights:
• Architecture fingerprint — does model_type + (hidden·intermediate·layers) match a foreign open-weight model
• Weight fingerprint — embedding similarity (from-scratch vs continued-pretraining)
Genotypes: 🟢 Native · 🔵 Adapted · 🟡 Mixed · 🔴 Ported
The results are not uniform. Some models match foreign architectures (Qwen, Llama, …) exactly; others use self-built architectures and weights with no foreign match. Which company/model falls where is shown per model in the Space, along with attention originality, license, and reproducible open-source status.
This is a neutral transparency tool, not an accusation — building foundation models on open-weight bases is a legitimate, industry-standard practice. The exact same yardstick is applied to every model, without exception.
Features a 3D lineage graph, search, EN / 中文 / 한국어, and dark mode. Corrections are welcome via the Community tab.
Articles: https://huggingface.co/blog/mayafree/model-dna
#KoreanAI #LLM #ModelLineage #OpenSource #SovereignAI
- 3 replies