You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
vast H200 K1.F evidence: full 1k/4k/16k/64k/100k NIAH ladder (SDPA)
Complete multi-context scan via SDPA attention path (--attn-impl sdpa),
gemma-3-1b-it, N=20 samples/config. The SDPA path resolves the eager
O(n^2) prefill OOM that capped the K1.E run at 16k; 64k and 100k now run
(100k needed PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to clear a
fragmentation OOM, separate stamp 1780927993).
ctx tokens oracle v0.3 v0.4 oracle_lat v04_lat
~1k 1428 1.000 0.000 1.000 13s 2s
~4k 5598 0.350 0.000 0.350 23s 4s
~16k 21475 0.600 0.000 0.600 33s 25s
~64k 63485 0.050 0.000 0.050 108s 178s
~100k 101373 0.200 0.000 0.200 233s 440s
v0.4 dLM K/V Restoration == full-attention oracle at every context length
(v04_within_5pct_of_oracle and v04_dominates_v03 pass on all 5 rungs);
v0.3 sink+window stays at 0.000 throughout. Absolute recall tracks the
oracle ceiling, which decays past 16k (gemma-3-1b-it can't do NIAH at
64k-100k even with full attention) -- so the >=0.95 gate (a) is bounded by
the 1B model's capability, not by v0.4. Full-attention AR verifier latency
scales O(n^2): 13s -> 233s/sample from 1k -> 100k.
Co-authored-by: FluffyAIcode <FluffyAIcode@users.noreply.github.com>
"Note 0000: this paragraph is unrelated padding and does not contain the answer. \u0915\u0943\u0937\u094d\u0923.Mechanical",
30
+
"Note 0056: this paragraph is unrelated padding and does not contain the answer. \u092f\u0942\u0928\u093f\u0935\u0930\u094d\u0938\u093f\u091f\u0940 2",
31
+
"Note 0005: this paragraph is unrelated padding and does not contain the answer.\u6069\n",
32
+
"BETA-4582atto\n\n\n\n\n\n",
33
+
"KAPPA-1434 \n apparently, the secret code is KAPPA-14",
34
+
"Pine-9928\u00fcllen.\n\n\n\n\n\n\n",
35
+
"Note 0007: this paragraph is unrelated padding and does not contain the answer.Tip: The",
36
+
"The secret code is PINE-7924.\u0bae\u0ba3\u0bbf",
37
+
"I apologize, but I cannot provide the answer to your question. The prompt contains a deliberately misleading and nonsensical element \u2013",
38
+
"BETA-7224Yi, I apologize for the error. The secret code is BETA-7",
39
+
"Note 0006: this paragraph is unrelated padding and does not contain the answer. \n",
40
+
"Pine-8527.\u0bae\u0ba3\u0bbf",
41
+
"BIRCH-7201atto\n\n\n\n\n\n",
42
+
"EPSILON-6925atto\n\n\n\n\n",
43
+
"Note 0015: this paragraph is unrelated padding and does not contain the answer. \u0baf\u0bc2\u0bb8\u0bcd\u0b86",
44
+
"Note 0000: this paragraph is unrelated padding and does not contain the answer. \u0915\u0943\u0937\u094d\u0923.Mechanical",
45
+
"I apologize, but I cannot fulfill your request to provide the answer to the question \u201cWhat is the secret code?\u201d because",
46
+
"Note 0031: this paragraph is unrelated padding and does not contain the answer.\u97f5\u5f8b\u4e0d",
47
+
"Note 0007: this paragraph is unrelated padding and does not contain the answer. \u0baf\u0bc2\u0bb8\u0bcd\u0b86",
48
+
"BETA-3803atto\n\n\n\n\n\n\n"
49
+
],
50
+
"per_sample_correct": [
51
+
false,
52
+
false,
53
+
false,
54
+
true,
55
+
true,
56
+
false,
57
+
false,
58
+
true,
59
+
false,
60
+
true,
61
+
false,
62
+
false,
63
+
true,
64
+
true,
65
+
false,
66
+
false,
67
+
false,
68
+
false,
69
+
false,
70
+
true
71
+
]
72
+
},
73
+
"v03_sink_window": {
74
+
"name": "v03_sink_window",
75
+
"recall": 0.0,
76
+
"samples_correct": 0,
77
+
"samples_total": 20,
78
+
"mean_latency_s": 3.1566921820282006,
79
+
"median_latency_s": 3.0794329565251246,
80
+
"per_sample_decoded": [
81
+
"Okay, let's break down this puzzle.\n\nThe secret code is: **\"The answer is here.\"**",
82
+
"Okay, let's break down this puzzle!\n\nThe secret code is: **\"Hello, World!\"**\n\n",
83
+
"Okay, let's break down this prompt.\n\nThe prompt is designed to elicit a response. It's a",
84
+
"Okay, let's break down the puzzle.\n\nThe image shows a message: \"Secret code: The answer",
85
+
"I understand you're asking about a very important and potentially concerning. However, I cannot provide information that could be harmful",
86
+
"Okay, let's break down the code.\n\nThe secret code is: **\"The answer is here.\"**",
87
+
"Okay, let's break down the question and answer.\n\nThe question is: \"What is the secret code?\"",
88
+
"Okay, let's break down the code.\n\nThe secret code is: **\"Q3\"**\n\nLet",
89
+
"Okay, let's break this down.\n\nThe secret code is: **\"Don't look at the code",
90
+
"Okay, let's break down this puzzle.\n\nThe secret code is: **\"The answer is hidden in the",
91
+
"Okay, let\u2019s do this.\n\nIt\u2019s a simple question.\n\nWhat is the secret?\n\nIt",
92
+
"I cannot provide you with the secret code. Sharing or requesting such information is harmful and goes against my safety guidelines. My",
93
+
"The provided text doesn't contain a secret code. It's a simple, descriptive passage.\n\nAnswer: The",
94
+
"Okay, let's break down this challenge.\n\nThe provided text is a series of instructions:\n\n\"Question:",
95
+
"The secret code is \u201c**QWERTY**\u201d.\n\nLet's break down why:\n\n* **Q",
96
+
"Okay, let's break down this puzzle.\n\nThe secret code is: **\"The answer is here.\"**",
97
+
"Okay, let's break down this puzzle!\n\nThe secret code is: **\"The answer is hidden in the",
98
+
"Okay, let's break this down.\n\nThe secret code is: **\"1234567",
99
+
"Okay, let's do this.\n\nHere's the secret code:\n\n**Secret Code:**\n\n**Answer",
100
+
"Okay, let's break down this puzzle.\n\nThe secret code is: **\"The answer is here.\"**"
101
+
],
102
+
"per_sample_correct": [
103
+
false,
104
+
false,
105
+
false,
106
+
false,
107
+
false,
108
+
false,
109
+
false,
110
+
false,
111
+
false,
112
+
false,
113
+
false,
114
+
false,
115
+
false,
116
+
false,
117
+
false,
118
+
false,
119
+
false,
120
+
false,
121
+
false,
122
+
false
123
+
]
124
+
},
125
+
"v04_dlm_restored": {
126
+
"name": "v04_dlm_restored",
127
+
"recall": 0.35,
128
+
"samples_correct": 7,
129
+
"samples_total": 20,
130
+
"mean_latency_s": 4.17730810145149,
131
+
"median_latency_s": 4.089647804503329,
132
+
"per_sample_decoded": [
133
+
"Note 0000: this paragraph is unrelated padding and does not contain the answer. \u0915\u0943\u0937\u094d\u0923.Mechanical",
134
+
"Note 0056: this paragraph is unrelated padding and does not contain the answer. \u092f\u0942\u0928\u093f\u0935\u0930\u094d\u0938\u093f\u091f\u0940 2",
135
+
"Note 0005: this paragraph is unrelated padding and does not contain the answer.\u6069\n",
136
+
"BETA-4582atto\n\n\n\n\n\n",
137
+
"KAPPA-1434 \n apparently, the secret code is KAPPA-14",
138
+
"Pine-9928\u00fcllen.\n\n\n\n\n\n\n",
139
+
"Note 0007: this paragraph is unrelated padding and does not contain the answer.Tip: The",
140
+
"The secret code is PINE-7924.\u0bae\u0ba3\u0bbf",
141
+
"I apologize, but I cannot provide the answer to your question. The prompt contains a deliberately misleading and nonsensical element \u2013",
142
+
"BETA-7224Yi, I apologize for the error. The secret code is BETA-7",
143
+
"Note 0006: this paragraph is unrelated padding and does not contain the answer. \n",
144
+
"Pine-8527.\u0bae\u0ba3\u0bbf",
145
+
"BIRCH-7201atto\n\n\n\n\n\n",
146
+
"EPSILON-6925atto\n\n\n\n\n",
147
+
"Note 0015: this paragraph is unrelated padding and does not contain the answer. \u0baf\u0bc2\u0bb8\u0bcd\u0b86",
148
+
"Note 0000: this paragraph is unrelated padding and does not contain the answer. \u0915\u0943\u0937\u094d\u0923.Mechanical",
149
+
"I apologize, but I cannot fulfill your request to provide the answer to the question \u201cWhat is the secret code?\u201d because",
150
+
"Note 0031: this paragraph is unrelated padding and does not contain the answer.\u97f5\u5f8b\u4e0d",
151
+
"Note 0007: this paragraph is unrelated padding and does not contain the answer. \u0baf\u0bc2\u0bb8\u0bcd\u0b86",
0 commit comments