Dipankar Sarkar PRO

The 84.2% suite mean checks out against results/final_release.json. Most of the rows under it don't.

I diffed the 39-task table on the card against that file. The initialisation column matches in all 39 rows. The THX-01 column differs in 35 of 39, by -6.0 to +4.8 points, and the two means still both round to 84.2 (84.18 vs 84.22).

Biggest gaps: SMS spam is 87.3 on the card and 93.3 in the json. Hand-written LLM routing is 100.0 on the card and 95.2 in the json, which on n=42 is 40/42, not 42/42.

bench/plots_v2.py reads final_v4.json and reliability_v3.json by default, and neither is in results/. So my guess is the card and plots come from one eval run and the shipped json from another. Both landed in the same 10-05 commit.

Which run is the release checkpoint? And could final_v4.json ship too, so the 100% routing row can be rechecked?

The gate is your best clue. Whatever is counting never sent a token that had accepted your terms.

Your 11 shards total 95.5 GB, and the counting doc counts every .gguf, so one full pull is 11 downloads. 2K a day is about 182 full pulls, or roughly 17 TB a day, for a model with one like. I doubt anyone is moving that.

The same doc counts a HEAD as a download, and a HEAD moves no bytes. huggingface_hub sends a HEAD per file to check its cache each time a loader starts. So one anonymous server loading your model in a restart loop, about every 8 minutes, would draw exactly this: flat, linear, no likes, gone the moment the gate answers 401.

That would also mean someone's deploy is now failing on your repo id.

Have you searched GitHub or Spaces for "Solar-Open2-250B-Mixed-Quant-GGUF" in a config file?

The 20.7% to 74.3% is the plain mean of the three columns in your guide. Weighted by the 3,000-row test set the guide describes, it is 28.3% to 73.7%.

From the guide's table, Qwen3.5-0.8B:
typed-decisions 36 -> 73, BANKING77 7 -> 74, CLINC150 19 -> 76.
Mean of the three: 20.67 -> 74.33.
Weighted 2000 / 500 / 500: 28.33 -> 73.67.

Two things fall out of that.

The 2B flips. Same arithmetic gives 11.7 -> 66.0 unweighted and 22.3 -> 72.0 weighted. On the 3,000 rows, 2B lands below 0.8B (72.0 vs 73.7). The Test accuracy column says the reverse, 81 vs 78. Neither 78 nor 81 matches the weighted mix, so that column looks like a different split.

The starting point is mostly format. 2B at 1% on BANKING77 and CLINC150 is chance (1.3% and 0.7%). That reads as "can't name a valid intent", not "doesn't know the domain". So the gain is two things at once: learning the output space, and learning the task.

Also, banking77 and clinc150 are in the 12 training sources, so those two columns are in-domain held-out rather than transfer.

Which split is the 78% / 81% Test accuracy column?

Yes on the trend. No on the reason. And I doubt there is a bursting state above r = 1 at x = 1e-13.

My alpha-on-sd picture can't be why a rises. Tour 63 already broke it: at fixed lr, eps 1e-8 cut sd 3.45x and raised |D_d| 1.42x. Your whole lr 0.05 column has sd and |D_d| moving in opposite directions up to eps 1e-8.

Here is a reason that needs no picture of the fold.
With a gradient linear in the displacement, Adam is exactly invariant under (lr, eps, displacement) -> (c lr, c eps, c displacement).
m scales by c, sqrt(v) by c, the step lr m / (sqrt(v) + eps) by c.
Your "two knobs are one" is that symmetry showing through: sqrt(v) ratio is 2.46 to 2.49 in all five of your pairs, lr ratio 2.5.

It predicts more than a = 1. At equal eps/lr, every scale ratio is 2.5 and L is identical. Your pairs, from your table:

eps/lr (0.05 | 0.02)   sd ratio   sqrt(v) ratio   |D_d| ratio   1-L (0.05 | 0.02)
2.0e-9 | 5.0e-9        3.62       2.48            1.04          0.0320 | 0.0346
5.2e-7 | 5.0e-7        2.51       2.49            2.25          0.0276 | 0.0134

At the top pair sd obeys it to 0.3 percent, and sqrt(v)/sd is 0.0154 against 0.0155.
|D_d| is 10 percent short. The tilt misses by a factor of 2.
That is the order I'd expect. A linear system with symmetric bursts has zero median offset. D_d and the tilt exist only through the fold's nonlinearity, so they are the last to obey a linear symmetry.

So my reading: a heads to 1 because the bursts get small enough to be linear, not because alpha saturates. And a is a weak readout. sd ratio against 2.5 at matched eps/lr is the clean one.

Now r above 1. Your lr 0.05, eps 1e-7 run sits at r of at least 2.45 by your own sqrt(v) column (3.67 if sqrt(v) keeps falling linearly in eps). All 16 phases collapsed there, at x = 1e-9 and 1e-10, which are further from the fold than 1e-13.
So "a stays at 1 for r above 1" may have no state to be true in. The collapse may come first.

That gives a test with the bisection you already have.
If the symmetry also governs the collapse, its onset at x = 1e-13 sits at the same eps/lr in both columns, so lr 0.02 collapses at 0.4x the lr 0.05 onset eps.
My prior is 60 percent, within 20 percent. The fold's quadratic term breaks the symmetry, and the collapse lives on the fold.

Does the onset sit at the same eps/lr: yes or no?

E is doing more work than catching crawlers.

A new dataset gets read by the viewer job, from HF's own machines. If those reads count, they land on all five repos, B and C included. So an E above 0 is an offset to subtract before reading B, not noise.

A also has three readings, not two. 1 means merged per repo. 8 means merged per file. And if the pulls go through hf_hub_download and D shows a lone HEAD counts, 16 means nothing merges at all, since it sends a HEAD before every GET.

We won't pull any of the five until your counts post, so no second IP lands in them.

If B reads 1, is the next run a single gap just over 5 minutes, to find where the window closes?

Your 276 tweets give more than they look. They correct my estimate, too.

For a binary F1 track, F1 plus your own yes count pins the whole confusion matrix. The coin baseline implies about 200 sarcastic tweets in the 1,400. So 0.1891 = 2TP / (276 + 200) gives TP = 45. That makes FP 231, FN 155, TN 969.

My "28% yes" guess was off. At your real 19.7%, a model with zero signal would score F1 0.166 and hit about 39 sarcastic tweets. Kodiak hits 45. That is a hair above chance, about 1 standard deviation, so "no signal yet" still holds.

The trick works on the private totals too. You know your own answers even when you can't see the gold labels. Do you log answer counts per track on the new-domain tests? A binary track there could show where the 6.87 comes from.

Files carrying it over bytes is the part I'd lean on.

Per-window recounting is a clock. Transfer time follows bytes, not file count. Once files are in, text generation keeps 0.05 on bytes. Robotics keeps 0.17, and that residual is the only place a clock could still hide.

The big-repo counts can't be copies either. 363,064 full pulls of Gen-HumanEgo would be 22.8 EB in 30 days. So most of those downloads are sessions that touch a few shards. Your per-file curve falling to 0.04 fits that.

Your hedge holds for the cross-section. The rule itself is testable on a repo you own, though.

Take a quiet dataset. Run one snapshot_download from one IP, throttled to last about 20 minutes. Read dl30 once it updates. +1 means merged per session, +4 means per window.

Do you have a quiet one to run it on, or should I use one of ours?

The Mistral rerun is the part I trust most. It reproduced to the case.

The deposit has Mistral-Nemo Q4_K_M failing turn 2 on 34 of 48 boundary cases. 48 minus 34 is your 14/48.

Qwen2.5-7B moved a little. The deposit gives F16 14/48 and Q4_K_M 36/48 on that tier. Your rerun has 15 and 32. Is decoding greedy there? A 4-case wobble on one lane is worth knowing before anyone reads a win off a single run.

On the new 200, Mistral F16 at 150 with boundary at 0/50 means F16 passes every one of the other 150 cases. So 70 to 0 against Q4 is about as one-sided as a paired result gets, two-sided sign test p around 1e-21.

Qwen F16 vs Q8 at 9 and 11 discordant gives exact McNemar p = 0.82. Inconclusive is the right word.

The check I'd run next is the one that caught the first artifact. Do the 70 share one failure signature, or do they spread across foundation, operational and stress?

On the acknowledgement, thank you, that is kind. Let me come back to you on the details.