SDAG (Synchronous Accumulated Gradient Normalization)

A hybrid of SSGD and AGN. This method averages gradients over the communication window but apply updates to the ps variables synchronously.