Request for Additional Code or Instructions to Reproduce Paper Experiments

[1171000712]

Thank you for releasing the code associated with your paper. I greatly appreciate your contribution to the field and the effort put into sharing your work with the community. I am currently studying your paper and trying to reproduce the experiments presented in the appendix, such as:

Detectability under perturbation,
Automatic quality evaluations,
Diversity effects, and others.

However, it seems the current repository only includes code for generating watermarked language and detecting scores. Would it be possible to provide additional code or instructions to help reproduce these experiments?

[RyanMullins]

Assigning to @sumedhghaisas2 for full triage and response.

@1171000712 this repo includes a Python Notebook example to this effect.

[posenhuang]

For "Detectability under perturbation",

• We have the details for paraphrasing in the paper.
• For random word deletion, here is an example code.

import random

def word_deletion(s, ratio=0.2):
  keep = [random.random() for _ in range(len(s))]
  res = [si for i, si in enumerate(s) if keep[i] > ratio]
  if not res:
    return [s[random.randint(0, len(s) - 1)]]
  return res
  
text = "A B C D E."
delete_text = ' '.join(word_deletion(text.split(' '), ratio=0.2))
print(text)
print(delete_text)