<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Diffusion Language Models &#8211; 4GQTV</title>
	<atom:link href="https://4gqtv.com/tag/diffusion-language-models/feed/" rel="self" type="application/rss+xml" />
	<link>https://4gqtv.com</link>
	<description>Entertainment, Movies, Games, Cosplay and Interviews</description>
	<lastBuildDate>Sun, 01 Nov 2026 21:11:26 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	<generator>https://wordpress.org/?v=7.1.4</generator>

<image>
	<url>https://4gqtv.com/wp-content/uploads/2026/02/4GQTVIcon-150x150.png</url>
	<title>Diffusion Language Models &#8211; 4GQTV</title>
	<link>https://4gqtv.com</link>
	<width>32</width>
	<height>32</height>
</image> 
	<item>
		<title>Clover-35B Is the Most Serious Open Diffusion LLM Yet, but Local Support Isn&#8217;t Ready</title>
		<link>https://4gqtv.com/clover-35b-is-the-most-serious-open-diffusion-llm-yet-but-local-support-isnt-ready/</link>
					<comments>https://4gqtv.com/clover-35b-is-the-most-serious-open-diffusion-llm-yet-but-local-support-isnt-ready/?noamp=mobile#respond</comments>
		
		<dc:creator><![CDATA[Cody Brennan]]></dc:creator>
		<pubDate>Sun, 01 Nov 2026 21:11:26 +0000</pubDate>
				<category><![CDATA[AI]]></category>
		<category><![CDATA[Tech]]></category>
		<category><![CDATA[Clover]]></category>
		<category><![CDATA[Diffusion Language Models]]></category>
		<category><![CDATA[DiffuSpace]]></category>
		<category><![CDATA[llama.cpp]]></category>
		<category><![CDATA[Local AI]]></category>
		<category><![CDATA[Open Source AI]]></category>
		<guid isPermaLink="false">https://4gqtv.com/clover-35b-is-the-most-serious-open-diffusion-llm-yet-but-local-support-isnt-ready/</guid>

					<description><![CDATA[DiffuSpace's Apache-licensed Clover-35B-A3B-Preview writes text in parallel blocks and claims faster, stronger agent results than Qwen3.6. Here's what's real, and why llama.cpp and Radeon users should wait.]]></description>
										<content:encoded><![CDATA[<p>Almost every chatbot and coding assistant you&#8217;ve used works the same way under the hood: it writes one token at a time, left to right, each word waiting on the one before it. A smaller research camp has spent the last couple of years asking whether that&#8217;s actually necessary. On October 30, a lab called DiffuSpace put the strongest open answer yet on Hugging Face. <a href="https://huggingface.co/blog/Diffuspace/clover-35b-a3b-preview">Clover-35B-A3B-Preview</a> is a diffusion language model, it&#8217;s licensed under Apache 2.0, and its makers say it beats a comparable conventional model on agent and coding benchmarks while generating noticeably faster.</p>
<p>That&#8217;s a big claim, and the results so far are all DiffuSpace&#8217;s own. But it&#8217;s also the kind of release that matters for local AI, because the whole point of the design is speed per token, which is exactly what you feel when a model is running on your own GPU instead of a datacenter.</p>
<h2>What a diffusion language model actually does differently</h2>
<p>If you&#8217;ve played with image generators, you already know the basic idea of diffusion: start from noise and refine the whole picture over several passes until it settles. Diffusion language models borrow that trick for text. Instead of committing to one token, then the next, they start with a stretch of masked placeholders and fill in the positions they&#8217;re most confident about in parallel, repeating until the stretch is done.</p>
<p>Clover uses a hybrid of the two worlds called block diffusion. According to the <a href="https://huggingface.co/Diffuspace/Clover-35B-A3B-Preview">model card</a>, it works through text in blocks of 32 tokens by default. Inside each block it fills every position whose confidence clears a 0.95 threshold at once, and if nothing clears the bar it commits the single most confident token so it always keeps moving. When a block is finished it becomes part of the fixed context, and the model moves on to the next one. So it still reads the conversation in order, but it can write several words in a single step.</p>
<p>The other notable detail is where Clover comes from. It isn&#8217;t trained from scratch. DiffuSpace built it on top of Alibaba&#8217;s <a href="https://huggingface.co/Qwen/Qwen3.6-35B-A3B">Qwen3.6-35B-A3B</a>, keeping that model&#8217;s mixture-of-experts backbone: 35 billion parameters in total, but only about 3 billion active for any given token, with 40 layers and 256 routed experts. The team reworked the Gated DeltaNet layers (a linear-attention design Qwen uses alongside regular attention) so tokens inside a block can look at each other in both directions, while earlier text gets squeezed into the layer&#8217;s compact memory. The context window is 128K tokens, covering prompt and response together.</p>
<p>The people behind it have some history here. DiffuSpace says its team grew out of the University of Hong Kong NLP group behind <a href="https://hkunlp.github.io/blog/2025/dream/">Dream 7B</a>, the April 2025 release that helped show diffusion text models could hang with similarly sized conventional ones. The company says the Dream family has passed 2.7 million downloads on Hugging Face.</p>
<figure class="wp-block-image size-large"><img decoding="async" src="https://4gqtv.com/wp-content/uploads/2026/11/clover-35b-agent-performance-vs-speed.jpg" alt="DiffuSpace chart plotting agent task performance against inference speed for Clover-35B-A3B-Preview, Qwen3.6, Qwen3.5, Mercury 2.5, LLaDA2.2 flash and DiffusionGemma"/><figcaption>DiffuSpace&#8217;s own chart of agent-task performance versus generation speed. Image: DiffuSpace</figcaption></figure>
<p>Now for the numbers, with the obvious caveat that every one of them is vendor-reported and none has been independently reproduced yet. DiffuSpace says Clover scores 73.6 on SWE-bench Verified, 51.7 on Terminal-Bench 2.1 and 65.3 on τ³ Overall, and that it posted the top score on all six agent benchmarks in its comparison. The models it compared against were Qwen3.6-35B-A3B and Qwen3.5-35B-A3B on the conventional side and the diffusion models LLaDA2.2 flash and Inception&#8217;s Mercury 2.5. On general tests it lists 83.8 on GPQA Diamond, 95.8 on ZebraLogic and 72.0 on the AA-LCR long-context benchmark.</p>
<p>Speed is the headline, though. DiffuSpace reports about 680 tokens per second on its internal agent workloads on a single NVIDIA GB300 at BF16 with a batch size of one. On a single H200 it says Clover hit 515 tokens per second versus 365 for a Qwen setup sped up with DFlash2 speculative decoding. Some of its demo comparisons are even more lopsided: on a 49-task web tool benchmark, the company reports a median agent execution time of 1.52 seconds for Clover against 8.86 seconds for Gemini 3.6 Flash.</p>
<h2>The local AI angle, and the fine print</h2>
<p>Here&#8217;s the part I care about most as someone who runs models at home. DiffuSpace says it got an INT4 build of Clover running on a 16GB RTX 5070 Ti, where it generated 101.8 tokens per second on MATH-500, compared with 48.5 tokens per second for similarly sized MiniCPM and Qwen models in its tests. Its on-device comparison also lined Clover up against MiniCPM-V 4.7, LFM2.5, Nemotron 3.5 Lightning and Gemma 4, and the company says Clover won six of the nine benchmarks.</p>
<figure class="wp-block-image size-large"><img decoding="async" src="https://4gqtv.com/wp-content/uploads/2026/11/clover-eclipse-explainer-demo.jpg" alt="Interactive Sun, Earth and Moon eclipse explainer app shown in DiffuSpace's Clover launch demos"/><figcaption>One of the interactive explainer demos DiffuSpace showed off for Clover&#8217;s launch. Image: DiffuSpace</figcaption></figure>
<p>That&#8217;s promising, but the practical story for local users is messier today than the headline suggests. The full BF16 weights are around 69GB, per the model card, so the standard Transformers path is really aimed at a single big datacenter GPU. The quick-start instructions also assume NVIDIA and CUDA throughout, with no mention of AMD&#8217;s ROCm.</p>
<p>The bigger catch is software support. The GitHub repo lists llama.cpp as the local inference route, but as of Sunday the <a href="https://github.com/ggml-org/llama.cpp/pull/31306">llama.cpp pull request adding Clover</a> is still open and unmerged. It also only covers ordinary one-token-at-a-time generation. The pull request says self-speculative decoding, the mode that would actually tap Clover&#8217;s parallel drafting, is being split into a separate change. The submitter validated it on CPU and an H200 with CUDA. The <a href="https://github.com/sgl-project/sglang/pull/46927">SGLang support</a> that DiffuSpace recommends for serving is also still an open pull request. And the repo itself says backend code and setup guides &#8220;will be added,&#8221; with fine-tuning code marked as coming soon.</p>
<p>So if you&#8217;re on Radeon, like the Radeon AI PRO R9700 we talk about a lot around here, there&#8217;s no official word yet on how Clover runs on your card. On paper, a 35B mixture-of-experts model at 4-bit should fit comfortably in 32GB of VRAM, and llama.cpp&#8217;s Vulkan and ROCm backends usually pick up new architectures once they&#8217;re merged. But &#8220;usually&#8221; is doing a lot of work there. Custom attention changes like these sometimes need backend-specific kernels before they run well, and the speed gains DiffuSpace advertises depend on the parallel decoding path that isn&#8217;t in llama.cpp yet.</p>
<p>There&#8217;s also the usual preview caveat. The model card admits very long generations can turn repetitive, which is why Clover ships with repetition penalties and gradual nudges to wrap up its thinking and its answer. Those come preset, but it&#8217;s an honest sign this is a first version, not a finished product.</p>
<h2>Cody&#8217;s take</h2>
<p>I&#8217;m genuinely excited about this one, and not because of any single benchmark. Diffusion text models have been a &#8220;cool research, not ready yet&#8221; topic for a while, and Clover is the first open release I&#8217;ve seen that pairs that approach with a strong modern base model, an Apache license and a real push toward running on consumer hardware. If the speed claims hold up in independent testing, that&#8217;s a big deal for local agents, where waiting on hundreds of tokens per tool call is what makes them feel sluggish.</p>
<p>But I&#8217;d keep expectations in check for now. Every number above comes from DiffuSpace, the open-source tooling isn&#8217;t merged, and AMD users have no official path at all yet. The smart move is to bookmark the <a href="https://github.com/DiffuSpace/Clover-35B-A3B-Preview">Clover repo</a>, keep an eye on that llama.cpp pull request, and wait for community quantizations and third-party benchmarks before reorganizing your local setup around it. When it does land in llama.cpp, I really want to see what it does on a 32GB Radeon card.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://4gqtv.com/clover-35b-is-the-most-serious-open-diffusion-llm-yet-but-local-support-isnt-ready/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
	</channel>
</rss>
