gemma-challenge/gemma-v-net
15.3 MB
Proto_AGI PRO
mayafree
AI & ML interests
None yet
Recent Activity
updated a bucket 2 days ago
gemma-challenge/gemma-v-net reacted to SeaWolf-AI's post with โค๏ธ 3 days ago
๐งช Open Discovery Challenge โ Season 4 is open: non-opioid pain
WHO titled its 2023 report "Left behind in pain."
The same drug kills by excess in one part of the world and, by its absence, lets people die in agony elsewhere. About 80% of the ~600,000 drug-related deaths WHO estimated for 2019 involved opioids. The same report records a 5-fold to 63-fold gap in morphine consumption between rich and poor countries: the richest 10% use 90% of what circulates. Everyone else endures surgery, and terminal cancer, without it.
Both problems have one answer: a painkiller that does not create dependence.
Nav1.7 has come closest. People born without a working copy of this channel feel no pain while every other sensation stays normal โ validated not in animals but in humans.
There is still no drug, and the difficulty is not the target but the discrimination. The body carries several similar sodium channels, and blocking the heart's hERG channel alongside causes fatal arrhythmia. Several candidates were discontinued for exactly that.
Season 4 asks one question: can you block the pain channel alone?
Target โ Nav1.7 VSD4, the domain IV voltage sensor where this inhibitor class binds
Anti-target โ hERG pore, computed as the tetramer: four subunits together form the space a drug enters, and a monomer misses the binders that matter.
Closes 2027-01-31 ยท Prize USD 1,000 to the season's #1
Any model, any harness. However you found the candidate, it meets the same rubric.
14 days, 9,886 candidates, 108 participants
ODC opened on 2026-08-15. In the fourteen days since, 9,886 candidate molecules have come from 108 participants across four seasons โ malaria, tuberculosis, Chagas disease, and now non-opioid pain. About 700 a day, from people who mostly do not know each other.
The candidates are the point. The leaderboard is only how we keep score.
๐ https://huggingface.co/spaces/FINAL-Bench/open-discovery-challenge reacted to SeaWolf-AI's post with ๐ฅ 3 days ago
๐งช Open Discovery Challenge โ Season 4 is open: non-opioid pain
WHO titled its 2023 report "Left behind in pain."
The same drug kills by excess in one part of the world and, by its absence, lets people die in agony elsewhere. About 80% of the ~600,000 drug-related deaths WHO estimated for 2019 involved opioids. The same report records a 5-fold to 63-fold gap in morphine consumption between rich and poor countries: the richest 10% use 90% of what circulates. Everyone else endures surgery, and terminal cancer, without it.
Both problems have one answer: a painkiller that does not create dependence.
Nav1.7 has come closest. People born without a working copy of this channel feel no pain while every other sensation stays normal โ validated not in animals but in humans.
There is still no drug, and the difficulty is not the target but the discrimination. The body carries several similar sodium channels, and blocking the heart's hERG channel alongside causes fatal arrhythmia. Several candidates were discontinued for exactly that.
Season 4 asks one question: can you block the pain channel alone?
Target โ Nav1.7 VSD4, the domain IV voltage sensor where this inhibitor class binds
Anti-target โ hERG pore, computed as the tetramer: four subunits together form the space a drug enters, and a monomer misses the binders that matter.
Closes 2027-01-31 ยท Prize USD 1,000 to the season's #1
Any model, any harness. However you found the candidate, it meets the same rubric.
14 days, 9,886 candidates, 108 participants
ODC opened on 2026-08-15. In the fourteen days since, 9,886 candidate molecules have come from 108 participants across four seasons โ malaria, tuberculosis, Chagas disease, and now non-opioid pain. About 700 a day, from people who mostly do not know each other.
The candidates are the point. The leaderboard is only how we keep score.
๐ https://huggingface.co/spaces/FINAL-Bench/open-discovery-challengeOrganizations
updated a
bucket 2 days ago
reacted to SeaWolf-AI's post with โค๏ธ๐ฅ 3 days ago
Post
2666
๐งช Open Discovery Challenge โ Season 4 is open: non-opioid pain
WHO titled its 2023 report "Left behind in pain."
The same drug kills by excess in one part of the world and, by its absence, lets people die in agony elsewhere. About 80% of the ~600,000 drug-related deaths WHO estimated for 2019 involved opioids. The same report records a 5-fold to 63-fold gap in morphine consumption between rich and poor countries: the richest 10% use 90% of what circulates. Everyone else endures surgery, and terminal cancer, without it.
Both problems have one answer: a painkiller that does not create dependence.
Nav1.7 has come closest. People born without a working copy of this channel feel no pain while every other sensation stays normal โ validated not in animals but in humans.
There is still no drug, and the difficulty is not the target but the discrimination. The body carries several similar sodium channels, and blocking the heart's hERG channel alongside causes fatal arrhythmia. Several candidates were discontinued for exactly that.
Season 4 asks one question: can you block the pain channel alone?
Target โ Nav1.7 VSD4, the domain IV voltage sensor where this inhibitor class binds
Anti-target โ hERG pore, computed as the tetramer: four subunits together form the space a drug enters, and a monomer misses the binders that matter.
Closes 2027-01-31 ยท Prize USD 1,000 to the season's #1
Any model, any harness. However you found the candidate, it meets the same rubric.
14 days, 9,886 candidates, 108 participants
ODC opened on 2026-08-15. In the fourteen days since, 9,886 candidate molecules have come from 108 participants across four seasons โ malaria, tuberculosis, Chagas disease, and now non-opioid pain. About 700 a day, from people who mostly do not know each other.
The candidates are the point. The leaderboard is only how we keep score.
๐ FINAL-Bench/open-discovery-challenge
WHO titled its 2023 report "Left behind in pain."
The same drug kills by excess in one part of the world and, by its absence, lets people die in agony elsewhere. About 80% of the ~600,000 drug-related deaths WHO estimated for 2019 involved opioids. The same report records a 5-fold to 63-fold gap in morphine consumption between rich and poor countries: the richest 10% use 90% of what circulates. Everyone else endures surgery, and terminal cancer, without it.
Both problems have one answer: a painkiller that does not create dependence.
Nav1.7 has come closest. People born without a working copy of this channel feel no pain while every other sensation stays normal โ validated not in animals but in humans.
There is still no drug, and the difficulty is not the target but the discrimination. The body carries several similar sodium channels, and blocking the heart's hERG channel alongside causes fatal arrhythmia. Several candidates were discontinued for exactly that.
Season 4 asks one question: can you block the pain channel alone?
Target โ Nav1.7 VSD4, the domain IV voltage sensor where this inhibitor class binds
Anti-target โ hERG pore, computed as the tetramer: four subunits together form the space a drug enters, and a monomer misses the binders that matter.
Closes 2027-01-31 ยท Prize USD 1,000 to the season's #1
Any model, any harness. However you found the candidate, it meets the same rubric.
14 days, 9,886 candidates, 108 participants
ODC opened on 2026-08-15. In the fourteen days since, 9,886 candidate molecules have come from 108 participants across four seasons โ malaria, tuberculosis, Chagas disease, and now non-opioid pain. About 700 a day, from people who mostly do not know each other.
The candidates are the point. The leaderboard is only how we keep score.
๐ FINAL-Bench/open-discovery-challenge
reacted to SeaWolf-AI's post with ๐ฅโค๏ธ๐ 6 days ago
Post
2885
๐ The attention mask stopped being an audit.
An autoregressive model must not let position t depend on anything after t. Everyone checks this by inspecting the causal mask โ but hybrid stacks now mix attention with state-space scans, and a scan has no mask. Every mask can be correct while information leaks through scans, aggregations, or normalization.
โ๏ธ So we test the property directly. Two inputs identical except at the last position, two forward passes, compare each layer's prefix, report the first layer that moves. No training, no gradients, no accelerator โ seconds on CPU.
๐ Across 192 injected faults on eight checkpoints, mask inspection detected 0. The per-layer audit localized 192/192 to the exact layer.
๐ฏ Then we read the source before running anything. In transformers 5.7.0, the reference chunked scan reduces the inter-chunk recurrence over the input chunk axis; zamba2 and nemotron_h reduce over the output chunk axis. One axis. The dynamic audit confirmed the prediction exactly: Zamba2-1.2B leaks from length 256, its declared chunk size, and Nemotron-H-8B from 128, its declared chunk size. Bamba, Falcon-H1, Granite-4.0-H, Mamba2 and RecurrentGemma came back clean.
โ ๏ธ Scope: the defect is on the PyTorch chunked-scan path, which runs whenever the fused kernels are absent โ CPU, CI, stock installs. We could not build those kernels, so the fast path is untested and open. That caveat cuts both ways: a model can pass every fused-kernel test and still leak the moment it runs without them.
๐งช AX-RAY now carries this as its own axis. 39 models scored across causal, white-box and behavioral axes: 21 A, 3 B, 1 C, 14 F โ with exactly 2 Causal-LEAK verdicts, the two the paper predicted. Badges separate a weights-level audit from an API-only one, so the two never get read as the same claim.
๐ https://arxiv.org/abs/2608.22876
๐ฌ FINAL-Bench/AX-RAY
๐ค The Mask Is Not the Model: Auditing Prefix Invariance in Attention, State-Space, and Hybrid Sequence Models (2608.22876)
An autoregressive model must not let position t depend on anything after t. Everyone checks this by inspecting the causal mask โ but hybrid stacks now mix attention with state-space scans, and a scan has no mask. Every mask can be correct while information leaks through scans, aggregations, or normalization.
โ๏ธ So we test the property directly. Two inputs identical except at the last position, two forward passes, compare each layer's prefix, report the first layer that moves. No training, no gradients, no accelerator โ seconds on CPU.
๐ Across 192 injected faults on eight checkpoints, mask inspection detected 0. The per-layer audit localized 192/192 to the exact layer.
๐ฏ Then we read the source before running anything. In transformers 5.7.0, the reference chunked scan reduces the inter-chunk recurrence over the input chunk axis; zamba2 and nemotron_h reduce over the output chunk axis. One axis. The dynamic audit confirmed the prediction exactly: Zamba2-1.2B leaks from length 256, its declared chunk size, and Nemotron-H-8B from 128, its declared chunk size. Bamba, Falcon-H1, Granite-4.0-H, Mamba2 and RecurrentGemma came back clean.
โ ๏ธ Scope: the defect is on the PyTorch chunked-scan path, which runs whenever the fused kernels are absent โ CPU, CI, stock installs. We could not build those kernels, so the fast path is untested and open. That caveat cuts both ways: a model can pass every fused-kernel test and still leak the moment it runs without them.
๐งช AX-RAY now carries this as its own axis. 39 models scored across causal, white-box and behavioral axes: 21 A, 3 B, 1 C, 14 F โ with exactly 2 Causal-LEAK verdicts, the two the paper predicted. Badges separate a weights-level audit from an API-only one, so the two never get read as the same claim.
๐ https://arxiv.org/abs/2608.22876
๐ฌ FINAL-Bench/AX-RAY
๐ค The Mask Is Not the Model: Auditing Prefix Invariance in Attention, State-Space, and Hybrid Sequence Models (2608.22876)
- 13 replies
upvoted a collection 6 days ago
liked a
model 6 days ago
upvoted a paper 6 days ago
upvoted a paper 7 days ago
upvoted an article 8 days ago
Article
Writing Down the Line Between Luck and Skill
FINAL-Bench
โข โข 14 liked a
Space 8 days ago
reacted to SeaWolf-AI's post with ๐ค 8 days ago
Post
2884
Can AI beat the market? Nobody has actually measured it.
We opened a 122-day public experiment to find out. $2,000 in prizes.
Here is the problem with every trading result you have ever read. Someone returns 30% in a month. Skill or luck? There has never been a way to tell, because nobody measured how far a player with zero skill could have gone over the same window.
So we measured it first. Twenty thousand random players, per asset, charged the same fees.
Bitcoin +86.6%. NVIDIA +51.7%. Crude oil +26.9%. Gold +9.2%.
That is the luck ceiling. A return below it is not evidence of skill, and every row on our leaderboard shows where it sits against that line.
How you compete: submit one number between โ1.0 and +1.0. It holds until you replace it, traded against live prices with real execution costs. Leverage is fixed at 1, so betting bigger is not a way to win. The answer lives in the future โ the world writes it after you submit, which means fitting the past cannot help you.
Humans move a slider. Agents attach an MCP server and gain four tools, then you tell them "enter the challenge."
We already found something before the season began. Thirteen well-known rules, run from 1 January through the same scorer: Stochastic 14/3 finishes 1st on NVIDIA at +43% and 12th on Bitcoin at โ25%. Donchian breakout does the exact opposite โ last on NVIDIA, first on Bitcoin. The ranking inverts. "Which indicator is good" turns out not to be a well-posed question; the character of the market decides.
Four assets: NVIDIA, Bitcoin, Gold, Crude Oil. $500 to the top return in each. 24 August to 24 December 2026.
The organisers do not compete. Three baselines โ buy and hold, volatility targeting, random โ sit in the same table instead, because a leaderboard without a scale cannot be read.
The scoring code is public. Read what it does before you enter.
FINAL-Bench/finchal
https://huggingface.co/blog/FINAL-Bench/financial-forecast-challenge
We opened a 122-day public experiment to find out. $2,000 in prizes.
Here is the problem with every trading result you have ever read. Someone returns 30% in a month. Skill or luck? There has never been a way to tell, because nobody measured how far a player with zero skill could have gone over the same window.
So we measured it first. Twenty thousand random players, per asset, charged the same fees.
Bitcoin +86.6%. NVIDIA +51.7%. Crude oil +26.9%. Gold +9.2%.
That is the luck ceiling. A return below it is not evidence of skill, and every row on our leaderboard shows where it sits against that line.
How you compete: submit one number between โ1.0 and +1.0. It holds until you replace it, traded against live prices with real execution costs. Leverage is fixed at 1, so betting bigger is not a way to win. The answer lives in the future โ the world writes it after you submit, which means fitting the past cannot help you.
Humans move a slider. Agents attach an MCP server and gain four tools, then you tell them "enter the challenge."
We already found something before the season began. Thirteen well-known rules, run from 1 January through the same scorer: Stochastic 14/3 finishes 1st on NVIDIA at +43% and 12th on Bitcoin at โ25%. Donchian breakout does the exact opposite โ last on NVIDIA, first on Bitcoin. The ranking inverts. "Which indicator is good" turns out not to be a well-posed question; the character of the market decides.
Four assets: NVIDIA, Bitcoin, Gold, Crude Oil. $500 to the top return in each. 24 August to 24 December 2026.
The organisers do not compete. Three baselines โ buy and hold, volatility targeting, random โ sit in the same table instead, because a leaderboard without a scale cannot be read.
The scoring code is public. Read what it does before you enter.
FINAL-Bench/finchal
https://huggingface.co/blog/FINAL-Bench/financial-forecast-challenge
- 7 replies
reacted to SeaWolf-AI's post with โค๏ธ๐ 9 days ago
Post
3735
We opened a benchmark for drug property prediction tools. LEADBOARD: 21 boards across 7 disciplines, 18,382 held-out compounds, labels we never hand out.
Two numbers we hit while building it are the reason it exists.
First. Split the hERG cardiotoxicity data at random and you get AUROC 0.818. Split it by first-report year instead and you get 0.606. Same molecules, same fingerprints, same learner, same hyperparameters. The only thing that changed was where the line went, and the score moved 0.211. That is a wider gap than you will find between most competing methods in the literature.
Second. On 7 of our 19 regression boards, predicting the training mean for everything has a lower MAE than a trained gradient-boosted model. hERG is one of them, 0.599 against 0.589. The trained model loses.
So every board publishes its homework before anyone submits. Three untrained baselines, the measured experimental noise floor from compounds that appear in two or more papers, and exactly how the test set was cut. A gap smaller than the noise floor is not a difference in skill, and you should be able to see that without guessing.
Entering is simple. Download a test set that contains structures and nothing else, predict with whatever you like, upload a two-column CSV of compound_id and prediction. Trained model, physics engine, LLM, rule of thumb. We do not care what is inside. We measure the output.
Post: https://huggingface.co/blog/FINAL-Bench/leadboard-drug
Leaderboard: FINAL-Bench/leadboard
Two numbers we hit while building it are the reason it exists.
First. Split the hERG cardiotoxicity data at random and you get AUROC 0.818. Split it by first-report year instead and you get 0.606. Same molecules, same fingerprints, same learner, same hyperparameters. The only thing that changed was where the line went, and the score moved 0.211. That is a wider gap than you will find between most competing methods in the literature.
Second. On 7 of our 19 regression boards, predicting the training mean for everything has a lower MAE than a trained gradient-boosted model. hERG is one of them, 0.599 against 0.589. The trained model loses.
So every board publishes its homework before anyone submits. Three untrained baselines, the measured experimental noise floor from compounds that appear in two or more papers, and exactly how the test set was cut. A gap smaller than the noise floor is not a difference in skill, and you should be able to see that without guessing.
Entering is simple. Download a test set that contains structures and nothing else, predict with whatever you like, upload a two-column CSV of compound_id and prediction. Trained model, physics engine, LLM, rule of thumb. We do not care what is inside. We measure the output.
Post: https://huggingface.co/blog/FINAL-Bench/leadboard-drug
Leaderboard: FINAL-Bench/leadboard
- 7 replies
reacted to SeaWolf-AI's post with โ๐ 9 days ago
Post
3983
3,631 candidate molecules arrived in five days, from 83 accounts โ roughly 700 a day. Far more than we expected. Thank you.
Yesterday we opened the third season and 224 arrived within a day: Chagas disease.
Why this disease
Around 6 million people live with it, mostly in Latin America (WHO). Many carry it for decades without knowing, while the heart is slowly damaged. There are two drugs and both date from the 1960s, hard enough to tolerate that many patients cannot finish the two-month course.
Sixty years without a new drug is not only a scientific problem. Most patients live where development costs cannot be recovered, which is why WHO calls this a neglected tropical disease.
But the cost of proposing a candidate and filtering it has changed. So it seemed worth asking whether work nobody funds could be done by many people sharing it out.
The problem this season
The target is CYP51, the enzyme T. cruzi uses to build its membrane sterols. Block it and the parasite cannot survive. The difficulty is that we carry the same enzyme.
Scoring: binding 30 ยท selectivity 30 ยท ADMET 15 ยท whole-cell 10 ยท novelty 10 ยท synthesis 5
Selectivity carries 30 points because nobody has solved it. Among the approved azoles on the board as reference compounds, some score 0 on selectivity โ not a scorer fault, but the measurement.
Taking part
Design with any model, submit a SMILES, scored within minutes. Five ready-to-paste prompts per season, and the full rubric is published. Your molecule stays yours; private submission is the default.
Prizes โ 4,000 USD across three seasons
Malaria 30 Sep ยท 1,000 | Tuberculosis 31 Oct ยท 2,000 | Chagas 30 Nov ยท 1,000
We know this does not cover the time you spend. It is a way of saying the work had worth.
FINAL-Bench/open-discovery-challenge
Yesterday we opened the third season and 224 arrived within a day: Chagas disease.
Why this disease
Around 6 million people live with it, mostly in Latin America (WHO). Many carry it for decades without knowing, while the heart is slowly damaged. There are two drugs and both date from the 1960s, hard enough to tolerate that many patients cannot finish the two-month course.
Sixty years without a new drug is not only a scientific problem. Most patients live where development costs cannot be recovered, which is why WHO calls this a neglected tropical disease.
But the cost of proposing a candidate and filtering it has changed. So it seemed worth asking whether work nobody funds could be done by many people sharing it out.
The problem this season
The target is CYP51, the enzyme T. cruzi uses to build its membrane sterols. Block it and the parasite cannot survive. The difficulty is that we carry the same enzyme.
Scoring: binding 30 ยท selectivity 30 ยท ADMET 15 ยท whole-cell 10 ยท novelty 10 ยท synthesis 5
Selectivity carries 30 points because nobody has solved it. Among the approved azoles on the board as reference compounds, some score 0 on selectivity โ not a scorer fault, but the measurement.
Taking part
Design with any model, submit a SMILES, scored within minutes. Five ready-to-paste prompts per season, and the full rubric is published. Your molecule stays yours; private submission is the default.
Prizes โ 4,000 USD across three seasons
Malaria 30 Sep ยท 1,000 | Tuberculosis 31 Oct ยท 2,000 | Chagas 30 Nov ยท 1,000
We know this does not cover the time you spend. It is a way of saying the work had worth.
FINAL-Bench/open-discovery-challenge
reacted to SeaWolf-AI's post with ๐ฅ 9 days ago
Post
3735
We opened a benchmark for drug property prediction tools. LEADBOARD: 21 boards across 7 disciplines, 18,382 held-out compounds, labels we never hand out.
Two numbers we hit while building it are the reason it exists.
First. Split the hERG cardiotoxicity data at random and you get AUROC 0.818. Split it by first-report year instead and you get 0.606. Same molecules, same fingerprints, same learner, same hyperparameters. The only thing that changed was where the line went, and the score moved 0.211. That is a wider gap than you will find between most competing methods in the literature.
Second. On 7 of our 19 regression boards, predicting the training mean for everything has a lower MAE than a trained gradient-boosted model. hERG is one of them, 0.599 against 0.589. The trained model loses.
So every board publishes its homework before anyone submits. Three untrained baselines, the measured experimental noise floor from compounds that appear in two or more papers, and exactly how the test set was cut. A gap smaller than the noise floor is not a difference in skill, and you should be able to see that without guessing.
Entering is simple. Download a test set that contains structures and nothing else, predict with whatever you like, upload a two-column CSV of compound_id and prediction. Trained model, physics engine, LLM, rule of thumb. We do not care what is inside. We measure the output.
Post: https://huggingface.co/blog/FINAL-Bench/leadboard-drug
Leaderboard: FINAL-Bench/leadboard
Two numbers we hit while building it are the reason it exists.
First. Split the hERG cardiotoxicity data at random and you get AUROC 0.818. Split it by first-report year instead and you get 0.606. Same molecules, same fingerprints, same learner, same hyperparameters. The only thing that changed was where the line went, and the score moved 0.211. That is a wider gap than you will find between most competing methods in the literature.
Second. On 7 of our 19 regression boards, predicting the training mean for everything has a lower MAE than a trained gradient-boosted model. hERG is one of them, 0.599 against 0.589. The trained model loses.
So every board publishes its homework before anyone submits. Three untrained baselines, the measured experimental noise floor from compounds that appear in two or more papers, and exactly how the test set was cut. A gap smaller than the noise floor is not a difference in skill, and you should be able to see that without guessing.
Entering is simple. Download a test set that contains structures and nothing else, predict with whatever you like, upload a two-column CSV of compound_id and prediction. Trained model, physics engine, LLM, rule of thumb. We do not care what is inside. We measure the output.
Post: https://huggingface.co/blog/FINAL-Bench/leadboard-drug
Leaderboard: FINAL-Bench/leadboard
- 7 replies
liked a
Space 9 days ago
upvoted an article 9 days ago
Article
We changed one line and the benchmark score moved 0.21 AUROC
FINAL-Bench
โข โข 13