Skip to content

Commit 3ad9f04

Browse files
a.palmasa.palmas
authored andcommitted
Update webpage
1 parent e5868f0 commit 3ad9f04

1 file changed

Lines changed: 9 additions & 9 deletions

File tree

webpage/index.html

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -323,11 +323,12 @@ <h3 class="title is-4">Are models just biased toward the perturbed sentence?
323323
</h3>
324324
<div class="content has-text-justified">
325325
<p>
326-
One concern is that the perturbation simply steers the model to pick whichever
327-
sentence was perturbed, regardless of the question. To rule this out, we present the
328-
model with two sentences on different topics (<em>e.g.</em>, one about animals, one about cities),
329-
perturb one, and ask a simple comprehension question (<em>e.g.</em>, <em>"Which sentence was about
330-
animals?"</em>).
326+
One hypothesis that might have accounted for the performance on this
327+
localization task is that the perturbation simply steers the model to pick
328+
whichever sentence was perturbed, regardless of the question. To rule this out,
329+
we present the model with two sentences on different topics (<em>e.g.</em>, one
330+
about animals, one about cities), perturb one, and ask a simple comprehension
331+
question (<em>e.g.</em>, <em>"Which sentence was about animals?"</em>).
331332
</p>
332333
</div>
333334

@@ -390,8 +391,7 @@ <h2 class="title is-3">Result 2: Zero-Shot Classification</h2>
390391

391392
<div class="content has-text-justified">
392393
<p>
393-
Setting itself apart from the other models, <b>Qwen3-32B</b> exhibits a notable pattern: <em>accuracy
394-
increases monotonically</em> with
394+
Setting itself apart from the other models, <b>Qwen3-32B</b> (left) exhibits a notable pattern: <em>accuracy increases monotonically</em> with
395395
perturbation strength. The model has a high prior to answer "dropout" (96.2% at the lowest
396396
tested rate), and yet it climbs to 99.2% at the highest rate. For noise, accuracy
397397
rises from 4.3% to 15.5% with the correct labels, and drastically to 89.6% when
@@ -460,7 +460,7 @@ <h3 class="title is-4">Do correct labels matter?</h3>
460460
the degree to which the demonstrations conflict with other learned prior.
461461
We therefore compare the difference between in-context learning with the correct labels, and flipped ones
462462
(<em>i.e.</em>, dropout labeled as "noise", and vice versa).
463-
We also run the same test with control labels.
463+
We also run the same test with control labels. The resulting heatmaps of accuracy as a function of both perturbation strengths are shown for Qwen3-32B below.
464464
</p>
465465
</div>
466466

@@ -567,4 +567,4 @@ <h2 class="title">BibTeX</h2>
567567

568568
</body>
569569

570-
</html>
570+
</html>

0 commit comments

Comments
 (0)