Residual Connection

[ChuanyangZheng]

Nice work!
how do you align pruned output connected by residual connection? For example:
x=x+fo(MHA(x)) # fo is output project
x=x+fc2(fc1(x))
if you prune fc2 weight, you should prune fo in MHA as the same, or the dimension can not match.

[yaozhewei]

During training, we set the pruned output channel of FC2 as 0, so that the output dimension of FC2 is the same as the original x. During inference, we have this output mask: https://github.com/yaozhewei/MLPruning/blob/main/inference/emmental/modeling_bert_masked.py#L423. Only the remain output dimension will be added to the original x.