YaFSDP from Yandex, 20% faster for pre-training LLMs

Link: https://github.com/yandex/YaFSDP
Discussion: https://news.ycombinator.com/item?id=40716701