Instructions to use CompassioninMachineLearning/OLMo-3-7B-CPT-BF16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use CompassioninMachineLearning/OLMo-3-7B-CPT-BF16 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="CompassioninMachineLearning/OLMo-3-7B-CPT-BF16")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("CompassioninMachineLearning/OLMo-3-7B-CPT-BF16")
model = AutoModelForCausalLM.from_pretrained("CompassioninMachineLearning/OLMo-3-7B-CPT-BF16", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use CompassioninMachineLearning/OLMo-3-7B-CPT-BF16 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "CompassioninMachineLearning/OLMo-3-7B-CPT-BF16"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CompassioninMachineLearning/OLMo-3-7B-CPT-BF16",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/CompassioninMachineLearning/OLMo-3-7B-CPT-BF16

SGLang

How to use CompassioninMachineLearning/OLMo-3-7B-CPT-BF16 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "CompassioninMachineLearning/OLMo-3-7B-CPT-BF16" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CompassioninMachineLearning/OLMo-3-7B-CPT-BF16",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "CompassioninMachineLearning/OLMo-3-7B-CPT-BF16" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CompassioninMachineLearning/OLMo-3-7B-CPT-BF16",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use CompassioninMachineLearning/OLMo-3-7B-CPT-BF16 with Docker Model Runner:
```
docker model run hf.co/CompassioninMachineLearning/OLMo-3-7B-CPT-BF16
```

OLMo-3-7B-CPT-BF16

Standalone BF16 (16-bit) continued-pretraining model. This revision contains the complete model from epoch 2, step 750, in exactly eight safetensors weight shards. Load directly with Transformers.

Epochs

main contains epoch 2, the best training-time validation checkpoint. Each completed epoch is independently loadable using its epoch-N revision. Both training runs stopped after four epochs; there is no epoch-five export.

Revision	Training step	Training-time validation loss
epoch-1	375	1.3267499
epoch-2	750	1.3001196
epoch-3	1125	1.3246491
epoch-4	1500	1.3742925

These losses were measured during adapter training, not by a new evaluation of the BF16 exports. Evaluate the four revisions separately to measure behavior changes across epochs.

Load a specific epoch

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "CompassioninMachineLearning/OLMo-3-7B-CPT-BF16"
revision = "main"
tokenizer = AutoTokenizer.from_pretrained(model_id, revision=revision)
model = AutoModelForCausalLM.from_pretrained(
    model_id, revision=revision, dtype=torch.bfloat16, device_map="auto"
)

Merge provenance and validation

Original base: allenai/Olmo-3-1025-7B at a81bae42db3975be1671e27b9c9a56da1a9f980f.
Source adapter: checkpoint 750.
Adapter repository revision: edba4e91735a37b3c886961e707355f3e541ef61.
Adapter SHA-256: c1486c306d6a1a9b5a77d6abd8151f0a4f0f4a4051397181db754dda4700f04d.
Merged 224 rsLoRA layers with PEFT's safe merge.
The separately trained embed_tokens and lm_head matrices are included.

Training used a 4-bit base; these exports merge the trained adapter into the pinned original BF16 base. The merge was performed using PEFT's safe merge, not Unsloth's export helper. Every tensor was checked for finite BF16 values and the complete architecture's names and shapes. Every epoch passed a standalone load, finite-logit forward pass and short greedy generation. No separate adapter is required, and optimizer/trainer checkpoint files are not included.

merge_manifest.json records source and output checksums. validation.json records the inference smoke test. training_manifest.json preserves the training settings. Shard count controls packaging; BF16 specifies the 16-bit precision.

Downloads last month: 217

Safetensors

Model size

7B params

Tensor type

BF16

Model tree for CompassioninMachineLearning/OLMo-3-7B-CPT-BF16

Base model

allenai/Olmo-3-1025-7B

Finetuned

(66)

this model