Decentralized learning in the presence of low-rank noise