Understanding Straight-Through Estimator in Training Activation Quantized Neural Nets

This is an implication that poor STEs generate coarse gradients incompatible with the energy landscape, which is consistent with 
our theoretical finding about the identity STE.

 Since this unusual "gradient" is certainly not the gradient of loss function, the following question arises: why searching in its 
 negative direction minimizes the training loss?

We prove that if the STE is properly chosen, the expected coarse gradient correlates positively with the population gradient 
(not available for the training), and its negation is a descent direction for minimizing the population loss.

We further show the associated coarse gradient descent algorithm converges to a critical point of the population loss minimization 
problem. Moreover, we show that a poor choice of STE leads to instability of the training algorithm near certain local minima.

This proxy derivative used in the backward pass only is referred as the straight-through estimator (STE) (Bengio et al., 2013). 
In the same paper, Bengio et al. (2013) proposed an alternative approach based on stochastic neurons. In addition, Friesen & 
Domingos (2017) proposed the feasible target propagation algorithm for learning hard-threshold (or binary activated) networks 
(Lee et al., 2015) via convex combinatorial optimization.

The convergence guarantee for the coarse gradient descent is established under the assumption that there are infinite training 
samples. When there are only a few data, in a coarse scale, the empirical loss roughly descends along the direction of negative 
coarse gradient. As the sample size increases, the empirical loss gains monotonicity and smoothness.