jbduran/bartholomew-sft

BART SFT

The instruction-tuned BART: a 2.8B-parameter model whose knowledge ends in 1930, fine-tuned to hold a conversation in period-appropriate voice.

📝 Read the write-up · 🌐 Unbounded Labs

Run Think.Unbounded-d32-v2mix-cont-pre1930-curriculum-c3-robust-v2, branched from BART at base step 9,600.

Specifications

Base model	bart @ step 9,600
Parameters	2,818,575,450 (~2.8B, including embeddings)
Final step	42
Validation bpb	0.7037
Recipe	curriculum (C3Rv2, staged)

Contents

checkpoints/model_000042.pt    final weights (8.99 GB)
checkpoints/optim_000042_rank0.pt
checkpoints/meta_000042.json
checkpoints/eval_metrics.json
config.json                    full SFT configuration
run.json                       lineage record
evals/chatcore.json

The curriculum

Training runs in three staged passes over the task routes, with a 30% noise rate throughout, then a robustness pass.

Stage	Routes	Authentic data
1	knowledge_qa	single-turn
2	reasoning_qa, stem_reasoning, how_to_qa, opinion_qa, composition_qa, verse_qa	calibration QA
3	multiturn_qa, narrative_grounded, narrative_fiction	multi-turn

The robustness pass (1 epoch) adds conversation_qa, unparseable_qa, typo_qa, era_qa, and conversation_multiturn — teaching the model to cope with malformed input, typos, and questions about its own era.

Training data

Dataset	Role
synthetic-pre1930-sft	the task routes (~416K rows, eleven routes)
authentic-pre1930-sft-conversational	authentic single- and multi-turn rows from 27 public-domain texts
vintage-sft-robustness	the robustness pass (7,338 rows, five categories)

Evaluation

ChatCORE (karpathy suite) at step 42:

Task	Score
ARC-Easy	0.2458
ARC-Challenge	0.2304
MMLU	0.2314
GSM8K	0.0000
HumanEval	0.0000
ChatCORE	-0.0113

These are modern benchmarks, and the multiple-choice scores sit at chance. That is the expected result for a 2.8B model that has never seen a word written after 1930 — GSM8K and HumanEval in particular test knowledge and formats that simply are not in the training data. The period-adapted suites (vintage-core, vintage-gsm8k, history-event-reconstruction) exist precisely because these numbers don't measure what the model was built to do.

Related

bart (base) · bart-experiments (all 39 runs, including five other SFT variants) · bart-dataset-v3 · bart-midtrain

License

MIT

---

Built by Unbounded Labs.