Sören Dittmer, Emily J. King, Peter Maass
The paper introduces ReLU singular values and Gaussian mean width as tools to analyze how ReLU activations interact with linear layers, providing theoretical insights, experimental…