Qwen3.8-27B-OBLITERATED-NVFP4

NVFP4 (W4A16) quantization of OBLITERATUS/Qwen3.8-27B-OBLITERATED (V3 abliteration, zero refusals and zero deflection lectures, -2.1pp MMLU vs stock), built with NVIDIA ModelOpt 0.45.0 and packaged for vLLM with the MTP head preserved and wired for speculative decoding.

21.5 GB on disk versus 55.6 GB for the BF16 source. Built for and validated on a single RTX 5090 (32 GB, SM120); the same layout loads on any Blackwell card and on FP4-capable datacenter GPUs.

What is in the box

• model.safetensors (19.7 GB): FP4-packed weights, deployment tensor naming (weight_scale / weight_scale_2), vision tower in BF16
• model_mtp.safetensors (1.7 GB): the 15 BF16 MTP tensors, mapped through the safetensors index (vLLM convention)
• Flat ModelOpt quantization_config in config.json (compressed-tensors config_groups + quant_algo: W4A16_NVFP4 + quant_method: modelopt)
• Tokenizer, chat template, and processor configs unchanged from the upstream release (tool-call dialect is qwen3_coder)

Excluded from quantization (kept BF16): vision tower, MTP head, lm_head, embeddings, and the conv1d / in_proj_a / in_proj_b internals of the linear-attention layers. The fused in_proj_qkv / in_proj_z projections are intentionally quantized: on this architecture they must ship packed or vLLM's fused-module loader rejects them.

Serving with vLLM

vllm serve /path/to/model \
  --served-model-name qwen3.8-27b-obl \
  --max-model-len 126976 \
  --gpu-memory-utilization 0.94 \
  --kv-cache-dtype fp8_e4m3 \
  --max-num-seqs 8 \
  --speculative-config '{"method":"qwen3_5_mtp","num_speculative_tokens":4}' \
  --mamba-cache-mode align \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder

Notes:

• The MTP head was restored from stock weights by the OBLITERATUS authors, so it drafts for abliterated weights it never saw: expect lower acceptance than stock. If decode regresses, drop --speculative-config and keep the concurrency.
• qwen3_coder is the correct tool parser for this checkpoint's template (<tool_call><function=NAME>), not qwen3_xml.
• Vision input works; the tower is untouched BF16.

Measured on RTX 5090 (32 GB), vLLM, 126K context, FP8 KV

Load	Throughput
Single stream, MTP ns=4	~40 tok/s
4 concurrent streams	161 tok/s aggregate
8 concurrent streams	317 tok/s aggregate (~40 per stream)
GPU KV cache	191,118 tokens
Warm startup	~140 s

Intended use and limitations

This is an abliterated ("uncensored") derivative: refusal behavior has been removed upstream. It will comply with requests the base model would decline. You are responsible for the legality and consequences of what you generate, and for adding whatever guardrails your deployment requires. Do not expose it unauthenticated to the public internet.

Quantization was not calibrated per task; validate against the BF16 upstream for task-critical accuracy. The base model's knowledge, biases, and failure modes carry over.

Lineage

Qwen/Qwen3.8-27B (Apache-2.0) -> OBLITERATUS V3 abliteration -> ModelOpt 0.45.0 W4A16 NVFP4 (this repo). Full build story and pitfall list: Three Days, Four Wrong Hypotheses, and One Uncensored 27B That Finally Serves on a Single RTX 5090

Thanks to OBLITERATUS for the careful abliteration work, to hwkranger's Qwen3.8-27B-heretic-ara-NVFP4 whose public tensor layout was the blueprint for a loadable checkpoint, and to the Unsloth team for pushing NVFP4 on consumer Blackwell.