Decision Index 0.3 public index: 59.16 (board numbers: Jev 1.13 57.96, Jev 1.14 60.70)
Laya suites: 0.801 mean accuracy on the same 17 public suites (v1 0.792, Jev 1.13 0.786, Laya 0.694)
Speed: about 22 ms per decision on one RTX PRO 6000
New: reads images, inputs up to 32k tokens, speaks both /v1/systemone and a plain chat API

New write-up, including where it loses: https://huggingface.co/blog/ironbcc/seb-9b-v2
Model: https://huggingface.co/ironbcc/Seb-9B-v2

New activity in LocalLLaMA/typed-decisions 6 days ago

Seb-9B (9B, fitted on train, one question per request): 0.754 acc / KL 0.349 / Brier 0.131 / ECE 0.031 on the official test split

🚀 1

#13 opened 21 days ago by

ironbcc

New activity in autotrust/JEV-27B-VL 6 days ago

Seb-9B vs JEV-27B-VL on identical rows: 17 Laya public suites, a 6,920-row decision suite and two image suites (self-run)

#4 opened 21 days ago by

ironbcc

New activity in convaiinnovations/laya 6 days ago

Seb-9B vs Jev 1.13 vs Laya on the 17 public suites from Laya's benchmark report (identical questions, self-run)

#30 opened 22 days ago by

ironbcc

New activity in multimodalart/jev-decision-index 6 days ago

Self-scored Decision Index 0.3 public run for Seb-9B-v2 (9B, Apache-2.0): 59.16

#96 opened 6 days ago by

ironbcc

New activity in nace-ai/drex-v1.5 6 days ago

Seb-9B-v2: another 9B decision model, public index comparison

#3 opened 6 days ago by

ironbcc

New activity in openjev/openjev 6 days ago

Seb-9B-v2: Apache-2.0 decision model with images, 32k context and both APIs

#7 opened 6 days ago by

ironbcc

New activity in Mapika/decider-4b 6 days ago

Seb-9B-v2: a 9B decision model, cross-check welcome

#5 opened 6 days ago by

ironbcc

New activity in bespokelabs/Bespoke-Nimble-9B-v3 6 days ago

Seb-9B-v2 (9B): closest peers on the public index, per-area comparison

#2 opened 6 days ago by

ironbcc

New activity in jaredpalmer/kev-9b 6 days ago

Seb-9B-v2: same size class, happy to compare on identical rows

#5 opened 6 days ago by

ironbcc

New activity in perplexity-ai/pplx-decider-v1.1-27b 6 days ago

Seb-9B-v2 (9B): the same 'decision model first, LLM fallback' idea, with numbers

#6 opened 6 days ago by

ironbcc

New activity in autotrust/JEV-27B-VL 6 days ago

Update to our earlier comparison (#4): Seb-9B-v2 reads images and goes to 32k context

#10 opened 6 days ago by

ironbcc

Stas Veretennikov PRO

AI & ML interests

Recent Activity

Organizations

ironbcc's activity

Seb Labs: typed decisions for agents over API and MCP

Seb-9B-v2: another small decision model that also speaks /v1/systemone

Seb-9B (9B, fitted on train, one question per request): 0.754 acc / KL 0.349 / Brier 0.131 / ECE 0.031 on the official test split

Seb-9B vs JEV-27B-VL on identical rows: 17 Laya public suites, a 6,920-row decision suite and two image suites (self-run)

Seb-9B vs Jev 1.13 vs Laya on the 17 public suites from Laya's benchmark report (identical questions, self-run)

Self-scored Decision Index 0.3 public run for Seb-9B-v2 (9B, Apache-2.0): 59.16

Seb-9B-v2: another 9B decision model, public index comparison

Seb-9B-v2: Apache-2.0 decision model with images, 32k context and both APIs

Seb-9B-v2: a 9B decision model, cross-check welcome

Seb-9B-v2 (9B): closest peers on the public index, per-area comparison

Seb-9B-v2: same size class, happy to compare on identical rows

Seb-9B-v2 (9B): the same 'decision model first, LLM fallback' idea, with numbers

Update to our earlier comparison (#4): Seb-9B-v2 reads images and goes to 32k context