Qwen3.6-35B-A3B-MXFP4xMXFP8

MXFP4-weight / MXFP8-activation quantization of Qwen/Qwen3.6-35B-A3B, produced for development purposes with llm-compressor's model-free PTQ: round-to-nearest (RTN) weights, no calibration data, no model forward.

Quantization scheme (compressed-tensors)

	element	bits	strategy	scales
weights	fp4 E2M1	4	group-32, symmetric	E8M0 (uint8), static, RTN via memoryless_minmax
activations	fp8 E4M3	8	group-32, symmetric	E8M0, dynamic (computed at runtime)

Ignored (kept in bf16): ['lm_head', 're:.*embed_tokens.*', 're:model\\.visual\\..*', 're:mtp\\..*', 're:.*conv1d.*', 're:.*linear_attn\\.in_proj_a.*', 're:.*linear_attn\\.in_proj_b.*', 're:.*mlp\\.gate$', 're:.*shared_expert_gate$'] — embeddings/lm_head, the vision tower, the MTP head, the linear-attention conv, the tiny delta-rule gate projections (in_proj_a/in_proj_b), MoE router gates (mlp.gate) and shared_expert_gate stay full-precision. Fused 3-D MoE expert tensors are split and quantized per expert.

Provenance

• Script: pytorch_runtime/scripts/quantize_mxfp4_mxfp8.py (dram-computing repo), following llm-compressor's experimental/model_free_ptq/glm_52_mxfp4_mxfp8.py
• Date: 2026-08-24
• llm-compressor 0.13.0, compressed-tensors 0.18.0, torch 2.13.0+cpu

Usage

Stock vLLM (GPU) loads it through the compressed-tensors backend. The dram-computing vllm_runtime plugin runs the decode-time linears of this checkpoint bit-exactly inside DRAM on compute-in-memory silicon:

python vllm_runtime/scripts/run_qwen35_vllm.py \
    --model casperhansen/Qwen3.6-35B-A3B-MXFP4xMXFP8 --quantization dram_mxfp4