Plain gradient descent takes a step against the gradient, no matter what happened before. Momentum keeps a running velocity so steps that agree build up speed and steps that zigzag cancel; RMSProp divides each direction by its recent gradient size, so steep and shallow directions get similar steps; Adam does both.