Clarification on baselines: Discrepancies between `dLLM-Cache` (Figure 1) and `Prefix-Cache` (Table 2)

[bxuanz]

Hi authors,

Thanks for the amazing and inspiring work on Streaming-dLLM!

While reading the paper, I noticed a few confusing points regarding the baseline methods and their results in Figure 1 and Table 2, specifically for the LLaDA-1.5 model on the GSM8K dataset (Generation Length 512). I would highly appreciate it if you could clarify:

1. Relation between dLLM-Cache and Prefix-Cache: Figure 1 includes a baseline named dLLM-Cache, but the main evaluation in Table 2 uses Prefix-Cache instead. Additionally, Prefix-Cache does not seem to be formally introduced or cited in the main text. Could you clarify what Prefix-Cache specifically refers to and how it differs from dLLM-Cache?

2. Throughput mismatch: For the LLaDA-1.5 model on GSM8K (length 512), the throughputs for Vanilla (2.5), Fast-dLLM (25.8), and Streaming-dLLM (69.8) perfectly match between Figure 1 and Table 2. However, the throughput for dLLM-Cache in Figure 1 is 12.2 tokens/s, while the throughput for Prefix-Cache in Table 2 is 8.2 tokens/s. Was Figure 1 perhaps generated using an older set of experimental logs, or is this a typo?

3. Identical Accuracy: Interestingly, dLLM-Cache (Fig 1), Prefix-Cache (Table 2), and the Vanilla baseline all report exactly 81.0% accuracy. Given that prefix caching in dLLMs usually involves lossy approximation (which is evident from the accuracy drops on HumanEval and MATH for Prefix-Cache in Table 2), is it a statistical coincidence that they all achieved exactly 81.0% on GSM8K, or was there an accidental copy-paste error regarding the accuracy numbers?

Thanks again for your time and contribution to the community. Looking forward to your insights!

[xiaoshideta]

1.For a detailed comparison, please refer to the two papers above. Briefly, Prefix-Cache accelerates inference by fixing all KV caches before the current generation block.

2.The results in Figure 1 and Table 2 are not typos; they correspond to two different baselines.

dllm-cache: Accelerating diffusion large language models with adaptive caching and Fast-dllm: Training-free acceleration of diffusion llm by enabling kv cache and parallel decoding are two different baselines, rather than a typo.

[xiaoshideta]

3.The identical 81.0% accuracy values are not caused by a copy-paste error. These results were independently obtained from separate runs of the three methods.

We agree that prefix caching may introduce approximation errors and can sometimes lead to accuracy degradation, as observed on HumanEval and MATH. However, on GSM8K, we empirically found that the affected tokens rarely changed the final answer, resulting in identical overall accuracy across these methods (same result as length of 256).