RuntimeError: Tensor sizes mismatch when training TasNet model on MUSDB18 dataset

[MCalafato]

I am trying to train a tasNet model to separate vocals from an audio track (songs)

I have adjusted the example code for the TasNet to take in the musdb18 dataset. However when I try to train the model, the following error occures:

File "D:\Matthew\Uni\FYP\Astroid\asteroid\asteroid\masknn\norms.py", line 43, in apply_gain_and_bias
return (self.gamma * normed_x.transpose(1, -1) + self.beta).transpose(1, -1)
RuntimeError: The size of tensor a (512) must match the size of tensor b (2) at non-singleton dimension 3

Does anyone have any idea what might be causing the problem please?

If it would be of any help, this is the code I am running:
scripts.zip

[mpariente]

Sounds like the input is stereo when the model is expecting mono.

[MCalafato]

Is there a simple way to convert the audio to mono in Asteroid please?

[mpariente]

It's independent of Asteroid, I guess tensor[:, 0] will do.

[MCalafato]

To make the model accept stereo, do I have to change the shapes in the mask network only or the encoder and decoders also?

This is the model:

# Create TasNet encoders and decoders (could use nn.Conv1D as well)
self.encoder_sig = Encoder(FreeFB(**fb_conf))
self.encoder_relu = Encoder(FreeFB(**fb_conf))
  self.decoder = Decoder(FreeFB(**fb_conf))
  self.bn_layer = GlobLN(fb_conf["n_filters"])

  # Create TasNet masker
  self.masker = nn.Sequential(
      SingleRNN(
          "lstm",
          fb_conf["n_filters"],
          hidden_size=mask_conf["n_units"],
          n_layers=mask_conf["n_layers"],
          bidirectional=True,
          dropout=mask_conf["dropout"],
      ),
      nn.Linear(2 * mask_conf["n_units"], self.n_src * self.n_filters),
      nn.Sigmoid(),
)

[faroit]

(b, c, t) becomes (b*c, t)

[MCalafato]

So I have to change the model itself and also the inputs of the model to the shape (b*c, t)?

[faroit]

No you won't have to change the (mono) model for this. You just split the stereo channels into the batch dimension as either part of the dataloader or system

[MCalafato]

So split the dataloader in 2 for the two channels of the signal?
I don't understand the part 'split the stereo channels into the batch dimension', and would it be possible to specify how to do it with the dataloader or system as I am new to this and don't really know how to manage my way around everything yet

[faroit]

Have a look at #660 (reply in thread) again. B is the batch dimension, C is the 2 channels, T is the time samples.