Step 1 reminds me of Principal Component Analysis in Machine Learning where we determine which input dimensions or parameters/weights of the input has the maximum normalized variance in data. This gives us an intuition on which weights carry most "information gain".
We arrange the weights in the decreasing order of variance. We then select enough weights to maximize variance (which simultaneously minimizes classification error) to achieve the compression ratio we want.
Step 1 reminds me of Principal Component Analysis in Machine Learning where we determine which input dimensions or parameters/weights of the input has the maximum normalized variance in data. This gives us an intuition on which weights carry most "information gain".
We arrange the weights in the decreasing order of variance. We then select enough weights to maximize variance (which simultaneously minimizes classification error) to achieve the compression ratio we want.