Add pirate network

[GiovanniCanali]

Description

This PR fixes #603

Checklist

• Code follows the project’s Code Style Guidelines
• Tests have been added or updated
• Documentation has been updated if necessary
• Pull request is linked to an open issue

[dario-coscia]

Everything look fine to me! The only thing I am wondering is if we really need PirateNetBlock. The reason why I am saying this, is to avoid to populate too much the pina.model.block module. The block itself can just be put inside the PirateNet model, I doubt it can be used otherwise as standalone. This is the same reasoning we adopted for ResidualFeedForward where all the logic is inside the network since the block itself cannot be reused by other models.

[GiovanniCanali]

Everything look fine to me! The only thing I am wondering is if we really need PirateNetBlock. The reason why I am saying this, is to avoid to populate too much the pina.model.block module. The block itself can just be put inside the PirateNet model, I doubt it can be used otherwise as standalone. This is the same reasoning we adopted for ResidualFeedForward where all the logic is inside the network since the block itself cannot be reused by other models.

I agree with you, I can move the block definition to the same file of the PirateNet.

[dario-coscia]

Everything look fine to me! The only thing I am wondering is if we really need PirateNetBlock. The reason why I am saying this, is to avoid to populate too much the pina.model.block module. The block itself can just be put inside the PirateNet model, I doubt it can be used otherwise as standalone. This is the same reasoning we adopted for ResidualFeedForward where all the logic is inside the network since the block itself cannot be reused by other models.

I agree with you, I can move the block definition to the same file of the PirateNet.

Is it now ready for review?

[GiovanniCanali]

Hi @ndem0, @dario-coscia, @FilippoOlivo,
I have added the possibility to pass a custom embedding to the PirateNet. If None, the FourierFeatureEmbedding is used, with the parameters suggested by the paper.

I also update the doc for the forward of the block: U and V must have the same dimension as x.

I checked if the parameter alpha controlling the contribution of each block must be between 0 and 1. There is no such restriction in the original paper, so I did not modify this.