How to ensure that the output of a convolutional or BN layer meets Crelu's input requirements #32
Description
According to the discussion in the original article, the real and imaginary parts should be strictly positive or negative at the same time so that Crelu can satisfy the Coccy-Riemann equation, but neither the convolutional layer nor the BN layer can make the output satisfy the strictly positive or negative at the same time, please tell me how everyone solved this problem so that Crelu can satisfy the Coccy-Riemann equation.
My current predictions all converge to the average of all labeled values， I guess that it's because of the unsatisfied Cauchy Riemann equation leading to negative non-conductivity, which in turn affects the gradient solution and transfer.