Instructions to use NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF:IQ3_XXS
# Run inference directly in the terminal:
llama cli -hf NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF:IQ3_XXS

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF:IQ3_XXS
# Run inference directly in the terminal:
llama cli -hf NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF:IQ3_XXS

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF:IQ3_XXS
# Run inference directly in the terminal:
./llama-cli -hf NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF:IQ3_XXS

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF:IQ3_XXS
# Run inference directly in the terminal:
./build/bin/llama-cli -hf NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF:IQ3_XXS

Use Docker

docker model run hf.co/NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF:IQ3_XXS

LM Studio
Jan

vLLM

How to use NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF:IQ3_XXS

Ollama
How to use NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF with Ollama:
```
ollama run hf.co/NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF:IQ3_XXS
```

Unsloth Studio

How to use NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF to start chatting

How to use NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF:IQ3_XXS

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF:IQ3_XXS"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF with Docker Model Runner:
```
docker model run hf.co/NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF:IQ3_XXS
```

Lemonade

How to use NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF:IQ3_XXS

Run and chat with the model

lemonade run user.Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF-IQ3_XXS

List all available models

lemonade list

Hermes Agent

How to use NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF:IQ3_XXS

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF:IQ3_XXS

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF:IQ3_XXS

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF:IQ3_XXS" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Quality-first GGUF quants of Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP

Source model: https://huggingface.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP

This model excels in writing, storytelling, roleplaying, chat.

If you are after agentic/coding/hard logic download this one (much better): https://huggingface.co/NikiKrutan/GRM-2.6-Plus-0628-MTP-GGUF

These quants were produced by my quality-first experimental tool, niki-allocator.

The main difference of my method is per-tensor type allocation (not per tensor groups, layers, community lore etc.)

Each tensor gets what is estimated to be best for it.

The tool is under active development and testing, and has not been published yet.

Quantization quality comparison

To prove my method I've performed comparison with quants from other providers:

DavidAU

mradermacher

Quants for comparison were downloaded on August 8, 2026.

Lower is better

Main metrics

Tail metrics

p99.9 are extremely corpus-driven with usual test corpus sizes of ~300K tokens

Same top probability

useful for temperature 0, otherwise RMS Δp should be considered

Perplexity (PPL)

very crude metric, cheaper to compute though, that's why so popular

Sizes

I've generated 37 quants to fill 8500-23000 MiB with geometric progression and rounded to 100 MiB. 19 of them were chosen for publication. Most neighbor sizes were dropped but some nearest to foreign quants were kept for better comparison. (I believe we don't need 37 quants, and even 19 is extreme, but they are all good).

niki-allocator is able to produce allocation with ~1 MiB precision.

Which one to choose?

If you are after quality: the bigger - the better. So it is always a compromise of what you can afford considering context size and other constraints. Choose the biggest you can fit.

I haven't measured speed since it is highly hardware and configuration dependable. What is faster for me may be slower for you. If speed is the only that matters and you have Blackwell - get some NVFP4 quant (but quality is considerably worse for the same quant size). Otherwise you should measure speed yourself on your hardware and your configuration to make real decision. By the way, on my setup ubatch is parameter that has huge impact on speed. On 27B models 384 was found to be optimal in my case (2x5060Ti), 512 or 256 are much slower. So probably you should test this first.

Quant names

Hugging Face requires quant type in the filename to properly show it. But all my quants are mixed by design. I decided to include dominating quant type (by size, not by tensor count) to the name therefore. It may easily be less than 50% of the model though. So choose by size, ignore listed quant type. For my quants: bigger --> better quality. Always.

Methodology

niki-allocator

It is quality-first per-tensor quantization allocator for llama.cpp llama-quantize.

Allocator decides, per tensor, which quant type to use so that the resulting GGUF hits an exact target file size with the best estimated quality.

It is not direct measurement of KLD-metrics, it is only estimation. But as you can see from real KLD evaluations estimation is good enough.

Raw results are already good. But I added some calibration levers on top to make it better (for Qwen3.5-0.8B I used uncalibrated estimation). These levers need series of KLD evaluations to move estimation closer to reality.

Corpora

imatrix: niki-small corpus - ~330K tokens from my own agentic sessions (English, Russian, code, reasoning). Deduped, cleaned, filtered by simhash similarity to include only really different turns, short turns (less than 16K chars) dropped. Used for imatrix only.
Calibration of niki-allocator: randomly chosen 1/4 of niki-small-plus for faster crude KLD calibration (full corpus is too slow on my hardware).
Evaluation 1: niki-small-plus corpus - ~405K tokens from my own agentic sessions (English, Russian, code, reasoning). Deduped, cleaned, filtered by simhash similarity to include only really different turns, short turns (less than 12K chars) dropped. Does not include any exact or similar (by simhash) agentic turns from niki-small. So while it is from the same domain as imatrix-corpus, it is different.
Evaluation 2: wikitext-2 test corpus - ~290 K tokens standard corpus. It wasn't seen during calibration or imatrix at all. Only for final evaluation before publishing. So my quants are not anyhow tuned for this dataset.

Notes:

While wikitext-2 test may be considered as independent evaluation corpus for my quants. It easily may not be true for others quants (wikitext-2 is the most popular corpus somehow for such jobs). They may have seen it (or part of it) during imatrix creation or their methodology calibration. So for really independent evaluation there should be corpus that wasn't seen anyhow by any quants and even better if it wasn't seen during base model creation. So for now I use my own corpus because I need these quants to be good in my tasks. And wikitext2 as a "standard".
The main difference from wikitext-2 test and niki-small-plus is that wiki is mostly homogenous (not very easy, but there are very few hard tokens and chunks). My corpus is everything from "walk in the park" to "high mountain climbing". Its tail metrics are much higher. So it is actually more representative for the real model work (if crawling wiki is not your main use case of course).

Chunk sizes

imatrix (niki-small): 8064 tokens (384 x 21). To catch reasoning and long turn traces, not just basic chat.
niki-small-plus evaluation: 6144 tokens (384 x 16). 66 chunks total. To catch reasoning and long turns, but to keep chunks amount somewhat reasonable (that's why lower than for imatrix).
wikitext-2 test evaluation: 768 tokens (384 x 2). 386 chunks total. It is not directly comparable with popular 512 tokens chunks!

Why 384 magic number?

My hardware is fastest with ubatch 384, so I use it. Sorry for inconvenience.

Metrics

All metrics were evaluated with llama-perplexity against Q8_0 standard quant (without imatrix) due to hardware constraints.

Quantization was performed with patched llama-quantize from BF_16 source. Fully compatible with original llama.cpp.

All my and foreign quants were evaluated against the same base.

llama.cpp commit 876a4321163249c43ca4e986818fab5ab081f282.

The main metrics during calibration of niki-allocator were mean KLD and RMS Δp.

Better PPL doesn't always mean better other more important metrics. So it can't be used as a main metric (some people still use). It is just fast indicator (no need for logits base) to check for big problems. It can't be main quality metric. Don't make or accept comparisons done only by PPL. It is noise.

Notes on p99 KLD, p99.9 KLD, max KLD.

With such a small corpus (well, almost everybody seems to use wikitext-2 test with only 290K tokens) these metrics are not very robust. While p99 KLD can give some plausible comparison for the allocation method as a whole, p99.9 is extremely corpus-driven (too little data), max KLD is just random at any corpus size (one hardest token of the whole dataset, saying nothing about other hard tokens). Please correct me if I am wrong.

Evaluation results

niki-small-plus corpus

Warning! Huge tables inside!

Source	Label	Size, MiB	Mean KLD	RMS Δp	KLD p95	KLD p99	KLD p99.9	Same top p, %	PPL
NikiKrutan	8700	8700	0.14436 ± 0.00168	10.466 ± 0.052	0.3791	0.8701	14.3400	86.09 ± 0.08	3.857 ± 0.018
NikiKrutan	9200	9200	0.12842 ± 0.00173	9.504 ± 0.055	0.3188	0.7440	15.1161	87.54 ± 0.07	3.796 ± 0.018
NikiKrutan	9800	9800	0.09947 ± 0.00162	8.228 ± 0.056	0.2253	0.5231	13.7616	89.29 ± 0.07	3.720 ± 0.017
NikiKrutan	10300	10300	0.08827 ± 0.00159	7.758 ± 0.057	0.1886	0.4247	13.7025	90.11 ± 0.07	3.670 ± 0.017
mradermacher	Q2_K	10361	0.13567 ± 0.00180	9.830 ± 0.055	0.3426	0.8027	15.2329	87.28 ± 0.07	3.818 ± 0.018
NikiKrutan	10900	10900	0.07912 ± 0.00154	7.264 ± 0.058	0.1561	0.3590	13.5491	90.97 ± 0.06	3.659 ± 0.016
DavidAU	MTP-IQ2_M	11563	0.14638 ± 0.00173	10.455 ± 0.053	0.3871	0.8967	14.5856	86.26 ± 0.08	3.873 ± 0.018
NikiKrutan	11800	11800	0.06773 ± 0.00143	6.805 ± 0.060	0.1232	0.2879	12.2526	92.06 ± 0.06	3.654 ± 0.016
mradermacher	IQ3_S	12019	0.07450 ± 0.00150	7.058 ± 0.064	0.1350	0.3298	12.5772	91.71 ± 0.06	3.697 ± 0.017
mradermacher	IQ3_M	12177	0.07770 ± 0.00155	7.295 ± 0.065	0.1379	0.3394	13.4378	91.69 ± 0.06	3.710 ± 0.017
NikiKrutan	12200	12200	0.06403 ± 0.00147	6.318 ± 0.064	0.1122	0.2736	12.3046	92.52 ± 0.06	3.652 ± 0.017
NikiKrutan	12900	12900	0.05337 ± 0.00145	5.587 ± 0.069	0.0769	0.1988	12.1633	93.68 ± 0.05	3.610 ± 0.017
NikiKrutan	13600	13600	0.04566 ± 0.00138	5.154 ± 0.075	0.0540	0.1391	11.7627	94.59 ± 0.05	3.612 ± 0.017
DavidAU	MTP-IQ3_M	13859	0.07345 ± 0.00148	7.073 ± 0.061	0.1368	0.3276	12.2153	91.82 ± 0.06	3.684 ± 0.017
NikiKrutan	14400	14400	0.03860 ± 0.00128	4.772 ± 0.076	0.0417	0.1063	10.2557	95.14 ± 0.05	3.584 ± 0.016
DavidAU	LOW-MTP-IQ4_XS	14438	0.04176 ± 0.00134	4.965 ± 0.076	0.0460	0.1111	11.6181	94.86 ± 0.05	3.585 ± 0.016
mradermacher	IQ4_XS	14600	0.04078 ± 0.00135	4.814 ± 0.077	0.0429	0.1077	11.4960	95.15 ± 0.05	3.588 ± 0.016
NikiKrutan	14800	14800	0.03722 ± 0.00128	4.609 ± 0.078	0.0375	0.0969	9.9734	95.46 ± 0.05	3.579 ± 0.016
mradermacher	Q4_K_S	15092	0.04142 ± 0.00136	4.843 ± 0.077	0.0438	0.1109	11.6744	95.10 ± 0.05	3.589 ± 0.016
mradermacher	Q4_K_M	16032	0.03882 ± 0.00132	4.697 ± 0.079	0.0382	0.0972	10.8362	95.36 ± 0.05	3.579 ± 0.016
NikiKrutan	16100	16100	0.03160 ± 0.00117	4.224 ± 0.081	0.0259	0.0702	9.4424	96.16 ± 0.04	3.585 ± 0.016
DavidAU	MTP-IQ4_XS	16245	0.04057 ± 0.00134	4.830 ± 0.078	0.0423	0.1073	11.5971	95.30 ± 0.05	3.580 ± 0.016
DavidAU	MTP-Q4_K_S	16725	0.04204 ± 0.00139	4.838 ± 0.077	0.0433	0.1092	11.7071	95.20 ± 0.05	3.588 ± 0.016
DavidAU	MTP-IQ4_NL	16931	0.04025 ± 0.00134	4.833 ± 0.078	0.0416	0.1067	11.5804	95.27 ± 0.05	3.587 ± 0.016
NikiKrutan	17000	17000	0.02911 ± 0.00116	3.967 ± 0.081	0.0204	0.0567	9.1943	96.52 ± 0.04	3.566 ± 0.016
DavidAU	MTP-Q4_K_M	17642	0.03977 ± 0.00136	4.681 ± 0.079	0.0380	0.0974	11.3667	95.47 ± 0.05	3.589 ± 0.016
NikiKrutan	17900	17900	0.02830 ± 0.00118	3.899 ± 0.085	0.0162	0.0453	9.0268	96.83 ± 0.04	3.578 ± 0.016
mradermacher	Q5_K_S	18093	0.03356 ± 0.00130	4.228 ± 0.085	0.0197	0.0569	10.7425	96.49 ± 0.04	3.597 ± 0.017
mradermacher	Q5_K_M	18631	0.03024 ± 0.00122	3.997 ± 0.084	0.0181	0.0525	10.2840	96.68 ± 0.04	3.578 ± 0.016
NikiKrutan	19000	19000	0.02258 ± 0.00103	3.559 ± 0.085	0.0113	0.0318	7.3814	97.26 ± 0.04	3.578 ± 0.016
DavidAU	MTP-Q5_K_S	19675	0.03491 ± 0.00134	4.316 ± 0.087	0.0190	0.0545	11.1909	96.70 ± 0.04	3.601 ± 0.017
NikiKrutan	20000	20000	0.01974 ± 0.00095	3.349 ± 0.088	0.0085	0.0250	6.4874	97.77 ± 0.03	3.581 ± 0.016
DavidAU	MTP-Q5_K_M	20201	0.03113 ± 0.00125	4.033 ± 0.087	0.0171	0.0493	10.1559	96.92 ± 0.04	3.585 ± 0.016
NikiKrutan	21200	21200	0.01687 ± 0.00088	3.086 ± 0.088	0.0058	0.0181	5.4929	98.14 ± 0.03	3.569 ± 0.016
mradermacher	Q6_K	21392	0.01935 ± 0.00096	3.324 ± 0.089	0.0069	0.0195	6.2846	97.80 ± 0.03	3.591 ± 0.017
DavidAU	LOW-MTP-Q6_K	21784	0.01837 ± 0.00093	3.251 ± 0.088	0.0062	0.0189	5.8547	98.05 ± 0.03	3.585 ± 0.016
NikiKrutan	21800	21800	0.01602 ± 0.00086	3.056 ± 0.089	0.0049	0.0149	5.2329	98.24 ± 0.03	3.578 ± 0.016
DavidAU	MTP-Q6_K	22920	0.01836 ± 0.00093	3.249 ± 0.088	0.0062	0.0189	5.8578	98.06 ± 0.03	3.585 ± 0.016
NikiKrutan	23000	23000	0.01334 ± 0.00077	2.835 ± 0.088	0.0035	0.0111	4.0122	98.52 ± 0.03	3.581 ± 0.016

wikitext-2 test corpus

Warning! Huge tables inside!

Source	Label	Size, MiB	Mean KLD	RMS Δp	KLD p95	KLD p99	KLD p99.9	Same top p, %	PPL
NikiKrutan	8700	8700	0.16727 ± 0.00095	12.651 ± 0.069	0.5773	1.6347	4.4061	82.54 ± 0.10	6.897 ± 0.043
NikiKrutan	9200	9200	0.12242 ± 0.00078	10.484 ± 0.064	0.4192	1.2206	3.6900	85.17 ± 0.09	6.653 ± 0.041
NikiKrutan	9800	9800	0.08896 ± 0.00061	8.897 ± 0.059	0.3009	0.8869	2.8462	87.20 ± 0.09	6.448 ± 0.040
NikiKrutan	10300	10300	0.07676 ± 0.00055	8.264 ± 0.057	0.2511	0.7601	2.4618	88.12 ± 0.08	6.401 ± 0.039
mradermacher	Q2_K	10361	0.12382 ± 0.00083	10.660 ± 0.065	0.4132	1.2537	3.8754	84.74 ± 0.09	6.630 ± 0.042
NikiKrutan	10900	10900	0.06186 ± 0.00047	7.372 ± 0.051	0.1970	0.5539	1.9423	89.18 ± 0.08	6.293 ± 0.038
DavidAU	MTP-IQ2_M	11563	0.13774 ± 0.00086	11.450 ± 0.067	0.4616	1.4094	4.0348	84.12 ± 0.10	6.744 ± 0.042
NikiKrutan	11800	11800	0.04916 ± 0.00038	6.626 ± 0.048	0.1530	0.4512	1.5811	90.63 ± 0.08	6.245 ± 0.038
mradermacher	IQ3_S	12019	0.04802 ± 0.00045	6.474 ± 0.050	0.1499	0.4493	1.6264	90.70 ± 0.08	6.224 ± 0.038
mradermacher	IQ3_M	12177	0.04881 ± 0.00044	6.561 ± 0.049	0.1515	0.4448	1.6430	90.67 ± 0.08	6.233 ± 0.038
NikiKrutan	12200	12200	0.04001 ± 0.00040	5.763 ± 0.049	0.1314	0.3922	1.5520	91.52 ± 0.07	6.206 ± 0.038
NikiKrutan	12900	12900	0.02772 ± 0.00038	4.734 ± 0.045	0.0893	0.2630	1.1323	92.89 ± 0.07	6.147 ± 0.038
NikiKrutan	13600	13600	0.01977 ± 0.00029	4.027 ± 0.042	0.0621	0.1860	0.7510	93.92 ± 0.06	6.133 ± 0.038
DavidAU	MTP-IQ3_M	13859	0.04806 ± 0.00042	6.507 ± 0.049	0.1498	0.4570	1.6447	90.89 ± 0.07	6.236 ± 0.038
NikiKrutan	14400	14400	0.01563 ± 0.00027	3.557 ± 0.039	0.0488	0.1469	0.5912	94.55 ± 0.06	6.106 ± 0.038
DavidAU	LOW-MTP-IQ4_XS	14438	0.01607 ± 0.00022	3.652 ± 0.041	0.0503	0.1469	0.5514	94.33 ± 0.06	6.112 ± 0.038
mradermacher	IQ4_XS	14600	0.01510 ± 0.00023	3.507 ± 0.039	0.0484	0.1459	0.5774	94.77 ± 0.06	6.101 ± 0.038
NikiKrutan	14800	14800	0.01397 ± 0.00022	3.344 ± 0.039	0.0440	0.1343	0.5699	95.02 ± 0.06	6.096 ± 0.038
mradermacher	Q4_K_S	15092	0.01519 ± 0.00026	3.528 ± 0.040	0.0476	0.1472	0.6046	94.77 ± 0.06	6.097 ± 0.038
mradermacher	Q4_K_M	16032	0.01264 ± 0.00023	3.201 ± 0.038	0.0398	0.1168	0.5010	95.22 ± 0.06	6.080 ± 0.037
NikiKrutan	16100	16100	0.01012 ± 0.00021	2.845 ± 0.040	0.0311	0.0933	0.4084	95.73 ± 0.05	6.069 ± 0.037
DavidAU	MTP-IQ4_XS	16245	0.01471 ± 0.00022	3.491 ± 0.042	0.0470	0.1446	0.5588	94.87 ± 0.06	6.100 ± 0.038
DavidAU	MTP-Q4_K_S	16725	0.01483 ± 0.00025	3.489 ± 0.040	0.0466	0.1430	0.5978	94.90 ± 0.06	6.096 ± 0.038
DavidAU	MTP-IQ4_NL	16931	0.01456 ± 0.00025	3.444 ± 0.041	0.0458	0.1413	0.5862	94.89 ± 0.06	6.097 ± 0.038
NikiKrutan	17000	17000	0.00813 ± 0.00021	2.540 ± 0.037	0.0247	0.0730	0.3379	96.12 ± 0.05	6.053 ± 0.037
DavidAU	MTP-Q4_K_M	17642	0.01251 ± 0.00025	3.181 ± 0.041	0.0387	0.1165	0.5239	95.28 ± 0.06	6.081 ± 0.037
NikiKrutan	17900	17900	0.00641 ± 0.00021	2.317 ± 0.040	0.0187	0.0572	0.2493	96.53 ± 0.05	6.050 ± 0.037
mradermacher	Q5_K_S	18093	0.00668 ± 0.00022	2.317 ± 0.043	0.0190	0.0570	0.2481	96.60 ± 0.05	6.049 ± 0.037
mradermacher	Q5_K_M	18631	0.00594 ± 0.00023	2.172 ± 0.039	0.0167	0.0493	0.2274	96.73 ± 0.05	6.043 ± 0.037
NikiKrutan	19000	19000	0.00448 ± 0.00015	1.930 ± 0.040	0.0129	0.0392	0.1617	97.05 ± 0.04	6.053 ± 0.037
DavidAU	MTP-Q5_K_S	19675	0.00637 ± 0.00020	2.270 ± 0.040	0.0185	0.0568	0.2806	96.75 ± 0.05	6.055 ± 0.037
NikiKrutan	20000	20000	0.00305 ± 0.00009	1.592 ± 0.039	0.0090	0.0268	0.1299	97.75 ± 0.04	6.049 ± 0.037
DavidAU	MTP-Q5_K_M	20201	0.00576 ± 0.00023	2.144 ± 0.044	0.0162	0.0491	0.2450	96.96 ± 0.04	6.052 ± 0.037
NikiKrutan	21200	21200	0.00217 ± 0.00009	1.359 ± 0.044	0.0061	0.0188	0.0892	98.07 ± 0.04	6.045 ± 0.037
mradermacher	Q6_K	21392	0.00241 ± 0.00014	1.413 ± 0.041	0.0060	0.0172	0.0909	97.87 ± 0.04	6.040 ± 0.037
DavidAU	LOW-MTP-Q6_K	21784	0.00202 ± 0.00012	1.303 ± 0.045	0.0053	0.0164	0.0816	98.14 ± 0.04	6.040 ± 0.037
NikiKrutan	21800	21800	0.00186 ± 0.00010	1.255 ± 0.038	0.0049	0.0156	0.0747	98.24 ± 0.03	6.042 ± 0.037
DavidAU	MTP-Q6_K	22920	0.00200 ± 0.00012	1.299 ± 0.046	0.0053	0.0164	0.0821	98.15 ± 0.04	6.039 ± 0.037
NikiKrutan	23000	23000	0.00157 ± 0.00012	1.149 ± 0.051	0.0036	0.0114	0.0631	98.48 ± 0.03	6.041 ± 0.037

MTP

I've tested MTP performance and discovered that there is no point to keep MTP head higher than IQ4_XS. Q3_K is near but slightly worse (used for lower quants). Exact allocation of IQ4_XS/Q3_K is chosen by allocator (not pinned). Lower than that there is considerable quality drop, so not used in my quants. Higher is just waste of size.

For numbers check my previous GRM-2.6-Plus-0628 quant's MTP section.

I've done fast test on this model (same architecture). Same conclusions.

Also I've additionally checked fast if there is any degradation on IQ4_XS vs Q8_0 with longer context (tested up to 28K), none found. And temperature = 0.6, still no evidence of degradation. But I must say that doing this test properly and scientifically much time and effort is needed. Since not 0 temperature gives different results: many runs needed to stabilize. Reasonably long context (100K+ from my opinion) is very slow on my hardware. So there actually may be some degradation. But that should be proven and not taken by "it seems" or "everybody knows". If there is such comprehensive analysis already, please let me know.

Basically I use these parameters for MTP: --spec-draft-n-max 5 --spec-draft-p-min 0.8. Primary reason of poor results with MTP is ignoring --spec-draft-p-min. Don't put obvious crap in your drafts. But let drafter do more if it is sure enough. MTP performance speed-up is highly dependable on hardware, configuration and specific context. So to decide what is best for your case you should test yourself.

Also don't use other llama.cpp drafters like ngram-mod with MTP. MTP wastes time even if ngram-mod already done draft. That is how llama.cpp is programmed. Not obvious. I have created experimental fork to overcome this issue (and added much better ngram-mod-v2 + minor fixes): https://github.com/NikiKrutan/niki-llama.cpp. I use it myself on a daily basis, but it is more like crude draft than real working fork. It breaks some llama.cpp behavior. But it gives another ~1.5x speed-up on top of MTP for my use cases.

Chat template

Chat template included with gguf's is chat template from original model. No changes.

Multimodality

Tested and confirmed with this mmproj: https://huggingface.co/DAXZEIT/Qwen3.6-27B-mmproj-hybrid-Q8_0-F16-gguf

This one is smaller than F16, but quality stays high, recommended.

Any standard Qwen3.6-27B mmproj should work as well.

Acknowledgments

DavidAU - great source model (not so great quants, sorry).
nightmedia - big contribution to the source model. Very interesting creator.
Qwen - great base model which we all are tinkering with.
mradermacher - quants for comparison.
llama.cpp - well, it is nice, but shouldn't we RIIR already? 😉
michaelw9999 - some ideas to start with.

Downloads last month: 8,882

GGUF

Model size

27B params

Architecture

qwen35

Hardware compatibility

2-bit

3-bit

4-bit

5-bit

6-bit

Model tree for NikiKrutan/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-GGUF

Base model

Qwen/Qwen3.6-27B

Finetuned

DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP

Quantized

(61)

this model