AttributeError: 'TransformerDecoderLayerOptimal' object has no attribute 'self_attn'

[agathebeucher]

When i try to train the model on MSCOCO2014, i have this error that i can't fix : AttributeError: 'TransformerDecoderLayerOptimal' object has no attribute 'self_attn'... what should i do ?

[DatesYuen]

I have meet the same question.

[DatesYuen]

It has been resolved when I change Pytorch version from 2.1.1 to 1.13.1.

[agathebeucher]

But when i do that i have the error : "ModuleNotFoundError : No module named 'torch._dynamo'...

[agathebeucher]

Did you solve that ?

[DatesYuen]

But when i do that i have the error : "ModuleNotFoundError : No module named 'torch._dynamo'...

I don't have that problem. I can run this project after change Pytorch version.

[fnauman]

Found a fix. Use vanilla nn.TransformerDecoderLayer instead of the custom TransformerDecoderLayerOptimal in Line 124 of ml_decoder.py:

ML_Decoder/src_files/ml_decoder/ml_decoder.py

Line 124 in 8a9e984

layer_decode = TransformerDecoderLayerOptimal(d_model=decoder_embedding,

layer_decode = nn.TransformerDecoderLayer(
    d_model=decoder_embedding, 
    nhead=8, 
    dim_feedforward=dim_feedforward, 
    dropout=decoder_dropout
)

Did you solve that ?

[Esmaeel-Hilal]

Install PyTorch 1.7 to solve the issue

[DrOncogene]

There has been an API change in later versions of PyTorch. I did not want to downgrade the PyTorch version. You can solve this by subclassing nn.TransformerDecoderLayer like this:

class TransformerDecoderLayerOptimal(nn.TransformerDecoderLayer):

and conforming the forward method to the new API:

def forward(
        self,
        tgt: Tensor,
        memory: Tensor,
        tgt_mask: Tensor | None = None,
        memory_mask: Tensor | None = None,
        tgt_key_padding_mask: Tensor | None = None,
        memory_key_padding_mask: Tensor | None = None,
        tgt_is_causal: bool = False,
        memory_is_causal: bool = False,
    ) 

That fixes the error, and I think doesn't make any difference to the functionality

[DrOncogene]

Found a fix. Use vanilla nn.TransformerDecoderLayer instead of the custom TransformerDecoderLayerOptimal in Line 124 of ml_decoder.py:

ML_Decoder/src_files/ml_decoder/ml_decoder.py

Line 124 in 8a9e984
layer_decode = TransformerDecoderLayerOptimal(d_model=decoder_embedding,

layer_decode = nn.TransformerDecoderLayer(
d_model=decoder_embedding,
nhead=8,
dim_feedforward=dim_feedforward,
dropout=decoder_dropout
)

Did you solve that?

This defeats one of the purposes of ML-Decoder because it reintroduces the self-attention that was removed as an optimisation. TransformerDecoderLayerOptimal does not have a self-attention block and is not used in its forward method.