记录最常用的shell命令
lumos:Efficient Performance Modeling and Estimation for Large-scale LLM Training
lumos模拟器论文记录
attention中张量并行与GQA
megatron中attention实现中tp与GQA参数的关系
pytorch Shard
pytorch中Shard实现
ubuntu搭建技术博客指南
web网页搭建实战记录
pytorch中TCPStore Rendezvous机制
pytorch中TCPStore Rendezvous实现机制记录
pytorch send and recv
pytorch中send和recv实现
pytorch devicemesh
pytorch中devicemesh实现