Great work and code! My question isn't entirely about the code. Has the author tested the accuracy of selecting the TAM module in inference stage? As the results in Table 1, choosing the right TAM (orcal) can bring huge performance gains. Does this indicate that TAMiL is not able to accurately select the correct TAM? I tried to make modifications to the code to count the accuracy of TAM selection during the training and testing phases.
the command is: python main.py --dataset seq-cifar100 --model tam --buffer_size 500 --load_best_args --tensorboard --pretext_task mse
for testing phase: I add some code in /TAMiL/models/tam.py line 157 to get confusion matrix of tam select
code:
for value in indices:
confusion_matrix[k][value] += 1
confusion matrix in testing phase:
[[1970 1 0 0 29]
[1979 0 0 0 21]
[1977 0 6 0 17]
[1984 0 0 4 12]
[1926 1 0 0 73]]
As a result, it looks like the model will select the first tam module for all samples.
And for training phase，I did the same thing with the buffer sample.
confusion matrix in training phase:
[[1968 1 0 0 5]
[2033 201 0 0 0]
[2175 0 60 0 0]
[1925 0 0 54 0]
[1589 0 0 0 5]]
Do I have a problem with the use of the code, can you explain why this behavior occurs?
Great work and code! My question isn't entirely about the code. Has the author tested the accuracy of selecting the TAM module in inference stage? As the results in Table 1, choosing the right TAM (orcal) can bring huge performance gains. Does this indicate that TAMiL is not able to accurately select the correct TAM? I tried to make modifications to the code to count the accuracy of TAM selection during the training and testing phases.
the command is: python main.py --dataset seq-cifar100 --model tam --buffer_size 500 --load_best_args --tensorboard --pretext_task mse
for testing phase: I add some code in /TAMiL/models/tam.py line 157 to get confusion matrix of tam select
code:
for value in indices:
confusion_matrix[k][value] += 1
confusion matrix in testing phase:
[[1970 1 0 0 29]
[1979 0 0 0 21]
[1977 0 6 0 17]
[1984 0 0 4 12]
[1926 1 0 0 73]]
As a result, it looks like the model will select the first tam module for all samples.
And for training phase，I did the same thing with the buffer sample.
confusion matrix in training phase:
[[1968 1 0 0 5]
[2033 201 0 0 0]
[2175 0 60 0 0]
[1925 0 0 54 0]
[1589 0 0 0 5]]
Do I have a problem with the use of the code, can you explain why this behavior occurs?