hello authors

[l76454540-maker]

Hi authors,I would like to ask whether the metrics in your paper, including PSR, FID, PSNR, and SSIM, were measured under the replacement model setting, that is, with --is_obfuscation set to False. Also, I think my way of calculating PSR might be different from yours. As I understand it, the test images are divided into four groups, each corresponding to a test image (choosing different outfits of the same target person). Then, for each test model like IR152, we evaluate the four groups separately and take the average of the four results. The same method is applied to the other three models (only on the CelebA-HQ dataset). Using this method, the metrics I get are much higher than yours. I would greatly appreciate it if you could clarify.

[parham1998]

Hi,

Thank you very much for your interest in our work.

First, regarding the evaluation setting: yes, your understanding is correct that all metrics reported in the paper (including PSR, FID, PSNR, and SSIM) are measured with --is_obfuscation set to False.

Regarding the PSR metric: This is also correct, but bear in mind that for each model, such as mobile_face, we consider the other 3 models (ir152, facenet, and IR152) during fine-tuning. You may use all four models during fine-tuning. If not, the result might be a little different, but not obviously different.

Use this setting, and it should work:
The CelebA-HQ test set contains 1,000 images, which are divided into four groups of 250 images each. For a given test model (e.g., MobileFaceNet), the evaluation should therefore be run four times using the same configuration, while changing the target_choice (1, 2, 3, 4) and the corresponding source_dir for each group. The final reported metric is obtained by averaging the results across these four runs.

parser.add_argument('--target_choice',
default='2',
type=str)
parser.add_argument("--test_model_name",
default=['mobile_face'])
parser.add_argument("--surrogate_model_names",
default=['ir152', 'facenet', 'irse50'])
parser.add_argument('--MTCNN_cropping',
default=True,
type=bool)
parser.add_argument('--is_obfuscation',
default=False,
type=bool)