Great goal! I'm also trying something like that, but more as an SFT for fine-tuning. But now I've pivoted to a sophisticated library inside my coding-agent...
Rev Quick
Revertron
·
AI & ML interests
None yet
Recent Activity
Organizations
None yet
Few-shot prompts: the model skips `</think>`, so the whole answer ends up in `reasoning_content`
#9 opened about 13 hours ago
by Revertron
reacted to cafkafk's post with 🔥 2 days ago
Post
4412
I'm working on a local model that can be run on stuff like an an RTX 3080 at 60 t/s. I'm focused on making it good at Rust and Nix. It's... not done yet, but doing great already. Had a 28/100 on livebench v6 earlier, which obviously isn't amazing.
But for my first real model, very exciting. Can't wait to share more as I get closer to the finish line in the coming weeks.
But for my first real model, very exciting. Can't wait to share more as I get closer to the finish line in the coming weeks.
- 6 replies
liked a
dataset 3 days ago
liked 2
models 4 days ago
liked a
dataset 5 days ago
It can't use tools?
#1 opened 5 days ago
by Revertron
liked a
model 9 days ago
liked a
dataset 10 days ago
liked a
model 11 days ago
liked 3
models 13 days ago
liked 2
models 14 days ago
reacted to ginigen-ai's post with 👍 14 days ago
Post
4503
A local edge VLM you can run on a phone — with a calibration readout attached.
ginigen-ai/Edge-4B-TELL
Image in, answer out, nothing leaving the device. Google's Gemma 4 E4B QAT checkpoint carried unmodified, with the vision and audio projector, plus one thing that is ours: GINIGEN TELL, a 10 KB readout that estimates whether the answer it just gave is likely to be wrong.
On a Galaxy S25: zero network calls, 3.6 GB resident, a 12.6 MB inference binary.
Calibration matters more here than on a server: nothing downstream catches a bad answer. No retrieval, no second opinion, no reviewer. The model is alone with the user.
And its own confidence is unusable. Prompted for it, this checkpoint averages 0.863 over 665 Korean disaster-procedure questions — ranking answers by it gives AUROC 0.441, below a coin flip. It sounds more certain when it is wrong.
TELL reads the last-layer hidden state instead of asking. Same questions, 0.759. Surface cues (length, formatting) already reach 0.736, so the readout clears that baseline by +0.023 ± 0.009 (2.6σ). We publish the baseline because without it, "the hidden state carries the signal" is unfalsifiable.
Same job as JEV: a confidence number you can act on instead of the model's own. Different structure, and on a device that splits three ways.
No second model — JEV is a separate judge reading the answer as text; we fill that slot with a 10 KB vector.
Zero generated tokens — a judge writes its verdict, TELL re-reads a finished computation (3.8 s on an S25).
No network — a verdict fetched over an API stops when the signal does.
The trade is real: a readout is fitted per checkpoint, so on a server the judge wins. On a phone there is no second model to run.
TELL never says what the right answer is. It says whether the answer wobbled, and a low score falls back to source text bundled with the app. Shipping today in HeliGO, an offline disaster-response app.
ginigen-ai/Edge-4B-TELL
Image in, answer out, nothing leaving the device. Google's Gemma 4 E4B QAT checkpoint carried unmodified, with the vision and audio projector, plus one thing that is ours: GINIGEN TELL, a 10 KB readout that estimates whether the answer it just gave is likely to be wrong.
On a Galaxy S25: zero network calls, 3.6 GB resident, a 12.6 MB inference binary.
Calibration matters more here than on a server: nothing downstream catches a bad answer. No retrieval, no second opinion, no reviewer. The model is alone with the user.
And its own confidence is unusable. Prompted for it, this checkpoint averages 0.863 over 665 Korean disaster-procedure questions — ranking answers by it gives AUROC 0.441, below a coin flip. It sounds more certain when it is wrong.
TELL reads the last-layer hidden state instead of asking. Same questions, 0.759. Surface cues (length, formatting) already reach 0.736, so the readout clears that baseline by +0.023 ± 0.009 (2.6σ). We publish the baseline because without it, "the hidden state carries the signal" is unfalsifiable.
Same job as JEV: a confidence number you can act on instead of the model's own. Different structure, and on a device that splits three ways.
No second model — JEV is a separate judge reading the answer as text; we fill that slot with a 10 KB vector.
Zero generated tokens — a judge writes its verdict, TELL re-reads a finished computation (3.8 s on an S25).
No network — a verdict fetched over an API stops when the signal does.
The trade is real: a readout is fitted per checkpoint, so on a server the judge wins. On a phone there is no second model to run.
TELL never says what the right answer is. It says whether the answer wobbled, and a low score falls back to source text bundled with the app. Shipping today in HeliGO, an offline disaster-response app.
liked 3
models 16 days ago
