Mixture of Experts

[ZCCDL]

The network you wrote looks more like Mixture of Experts.
α 1 (W1 ∗x)+...+α n (Wn ∗x)
Using n times conv operations.

Has the author ever tried CondConv: (α1 W1 + ... + αn Wn ) ∗ x.

[chenxinhua]

This execution is inefficient

[prstrive]

Yea, this is true. But this is a mechanism limited by Tensorflow.