HogWild!

The famous, lock-free approach to SGD. Have a bunch of workers and parameter server, then let the workers update the variables whenever they want.