---
license: apache-2.0
base_model:
  - OBLITERATUS/Qwen3.8-27B-OBLITERATED
  - Qwen/Qwen3.8-27B
tags:
  - nvfp4
  - vllm
  - mtp
  - speculative-decoding
  - abliterated
  - uncensored
  - rtx-5090
  - blackwell
  - quantization
  - modelopt
library_name: vllm
pipeline_tag: image-text-to-text
---

# Qwen3.8-27B-OBLITERATED-NVFP4

NVFP4 (W4A16) quantization of [OBLITERATUS/Qwen3.8-27B-OBLITERATED](https://huggingface.co/OBLITERATUS/Qwen3.8-27B-OBLITERATED)
(V3 abliteration, zero refusals and zero deflection lectures, -2.1pp MMLU vs stock),
built with NVIDIA ModelOpt 0.45.0 and packaged for **vLLM** with the **MTP head preserved
and wired** for speculative decoding.

21.5 GB on disk versus 55.6 GB for the BF16 source. Built for and validated on a single
**RTX 5090 (32 GB, SM120)**; the same layout loads on any Blackwell card and on FP4-capable
datacenter GPUs.

## What is in the box

- `model.safetensors` (19.7 GB): FP4-packed weights, deployment tensor naming
  (`weight_scale` / `weight_scale_2`), vision tower in BF16
- `model_mtp.safetensors` (1.7 GB): the 15 BF16 MTP tensors, mapped through the
  safetensors index (vLLM convention)
- Flat ModelOpt `quantization_config` in `config.json` (compressed-tensors
  `config_groups` + `quant_algo: W4A16_NVFP4` + `quant_method: modelopt`)
- Tokenizer, chat template, and processor configs unchanged from the upstream release
  (tool-call dialect is `qwen3_coder`)

Excluded from quantization (kept BF16): vision tower, MTP head, `lm_head`, embeddings,
and the `conv1d` / `in_proj_a` / `in_proj_b` internals of the linear-attention layers.
The fused `in_proj_qkv` / `in_proj_z` projections are intentionally **quantized**: on
this architecture they must ship packed or vLLM's fused-module loader rejects them.

## Serving with vLLM

```bash
vllm serve /path/to/model \
  --served-model-name qwen3.8-27b-obl \
  --max-model-len 126976 \
  --gpu-memory-utilization 0.94 \
  --kv-cache-dtype fp8_e4m3 \
  --max-num-seqs 8 \
  --speculative-config '{"method":"qwen3_5_mtp","num_speculative_tokens":4}' \
  --mamba-cache-mode align \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder
```

Notes:

- The MTP head was restored from stock weights by the OBLITERATUS authors, so it drafts
  for abliterated weights it never saw: expect lower acceptance than stock. If decode
  regresses, drop `--speculative-config` and keep the concurrency.
- `qwen3_coder` is the correct tool parser for this checkpoint's template
  (`<tool_call><function=NAME>`), not `qwen3_xml`.
- Vision input works; the tower is untouched BF16.

## Measured on RTX 5090 (32 GB), vLLM, 126K context, FP8 KV

| Load | Throughput |
|---|---|
| Single stream, MTP ns=4 | ~40 tok/s |
| 4 concurrent streams | 161 tok/s aggregate |
| 8 concurrent streams | 317 tok/s aggregate (~40 per stream) |
| GPU KV cache | 191,118 tokens |
| Warm startup | ~140 s |

## Intended use and limitations

This is an abliterated ("uncensored") derivative: refusal behavior has been removed
upstream. It will comply with requests the base model would decline. You are responsible
for the legality and consequences of what you generate, and for adding whatever guardrails
your deployment requires. Do not expose it unauthenticated to the public internet.

Quantization was not calibrated per task; validate against the BF16 upstream for
task-critical accuracy. The base model's knowledge, biases, and failure modes carry over.

## Lineage

`Qwen/Qwen3.8-27B` (Apache-2.0) -> OBLITERATUS V3 abliteration -> ModelOpt 0.45.0
W4A16 NVFP4 (this repo). Full build story and pitfall list:
[Three Days, Four Wrong Hypotheses, and One Uncensored 27B That Finally Serves on a Single RTX 5090](https://stondo.github.io/posts/obliteratus-qwen38-27b-nvfp4-rtx5090/)

Thanks to OBLITERATUS for the careful abliteration work, to hwkranger's
Qwen3.8-27B-heretic-ara-NVFP4 whose public tensor layout was the blueprint for a
loadable checkpoint, and to the Unsloth team for pushing NVFP4 on consumer Blackwell.