Can a 1×n convolution kernel be applied to a row vector

[achinastone]

The convolutional layer seems to use square matrices, providing services for image recognition.
Can convolutional layers have different FFeatureSizeX and FFeatureSizeY to enable convolution operations to be applied in non-image recognition fields?
constructor TNNetConvolutionAbstract.Create(pFeatureSize, pInputPadding, pStride: integer; pSuppressBias: integer = 0); begin inherited Create(); FFeatureSizeX := pFeatureSize; FFeatureSizeY := pFeatureSize; FPadding := pInputPadding; FStride := Max(pStride,1); FSuppressBias := pSuppressBias; if FPadding > 0 then begin FInputCopy := TNNetVolume.Create; end; FPrevLayerErrorPadded := TNNetVolume.Create; end;

[joaopauloschuler]

@achinastone , you can achieve this by using pointwise (1x1) convolutions (TNNetPointwiseConvLinear, TNNetPointwiseConvReLU) on rotated activation maps (matrices). Have a look at these layers:
TNNetTransposeXD = class(TNNetLayer)
TNNetTransposeYD = class(TNNetLayer)

In CAI, the self-attention mechanism has been coded using these layers.

[joaopauloschuler]

@achinastone , hhuuummm. Thinking well, it is not the same thing.
I will classify your request as a "feature request".

[achinastone]

That's great

[achinastone]

For example:
I want to try applying a convolutional neural network to the input.
The input contains such dimensional information : x = 100 y = 1 depth = 2.
So, it need such a convolutional kernel : x= 5 y = 1 depth = 2.

[joaopauloschuler]

@achinastone ,
I have great news to you.

As your y is 1, in CAI, if your kernel is 5x5, CAI will internally adjust the kernel to x=5, y=1, depth = 2. So, it should work out of the box.

[achinastone]

Thanks

[achinastone]

@joaopauloschuler
I don't know if my thinking is correct. When performing convolution on tensors with y equal to 1, if padding is required for the convolution kernel, it can only be done on X and should not be done on y. That means, in the situation I described above, a 5 * 5 kernel can not perform padding

[joaopauloschuler]

@achinastone , you are correct on your understanding. For padding, you will add padding only to the X dimension with:
TNNetPadXY.Create(PaddingX, PaddingY: integer). In your case, it will be:
TNNetPadXY.Create(2, 0).

What do you think?

[achinastone]

Thank you, that's very nice of you.
I thought I couldn't use padding in CAI anymore, but there's also TNNetPadXY.Create. That's great！