🤏 quecto mode

appvoid

49 4 176

https://ko-fi.com/appvoid

AI & ML interests

singularity through byte-level tokens and small language models

Recent Activity

reacted to Banaxi-Tech's post with 😎 about 17 hours ago

We're releasing Overfitter 1.0. Its a completly useless overfitted model trained on 202 epochs of SWE Bench Verified, SWE Bench Pro, Terminal Bench 2.1, DeepSWE. It gets 100% on SWE Bench Verified, 98.6% on SWE bench Pro, 100% on Terminal Bench 2.1 and 100% on DeepSWE! https://huggingface.co/BananaMind/Overfitter-1.0

repliedto their post about 19 hours ago

I hope that after this OpenAI disaster on Plus users, more people start realizing why Open Weights were always the only way.

posted an update about 20 hours ago

I hope that after this OpenAI disaster on Plus users, more people start realizing why Open Weights were always the only way.

View all activity

Organizations

Posts 42

Post

898

I hope that after this OpenAI disaster on Plus users, more people start realizing why Open Weights were always the only way.

Post

974

Byte-level state-space models. That sounded pretty scary for a scientist decades ago. Now we have:

1. Knowledge that deeper layers train smoothly.
2. Knowledge that Transformers work but is quadratic on sequence length.
3. Knowledge that SSMs work even better. Numerically unstable sometimes.
4. Speculative-decoding.
5. Open high-quality data.
6. Knowledge that KD works.

It slowly feels like is no longer a bad idea.