On: Why Momentum Works

[halflings]

I assume that multiplying by a given factor shouldn't matter since you still have the learning rate as a factor (which is itself a factor of the gradient). This might just mean that the learning rate should be lower or higher with this method.

[im3w1l → halflings]

The question is then really about which method makes it easier to tune parameters or which helps intuition the most.

[gabrielgoh → halflings]

this is a good way to think about this.