In navigation infrastructure-limited scenarios such as GNSS denied area, cooperative localization is an important approach to improve the localization accuracy. Self and relative localization are two main applications of cooperative localization. However, existing methods usually conduct self and relative localization separately, which not only limits the practical applications, but also leads to the performance loss by neglecting the motion correlations between swarm and individuals. To solve this problem, this work proposes a robust cooperative simultaneous self and relative localization (RC-SSRL) for collaborative swarm. It models the probability relationships among cooperative self-localization (CSL) and relative-localization (CRL) as well as cooperative measurements with a probability graph model, which depicts the probability relationship between relative-motion and self-motion. The marginal distributions in the graph models are calculated via the Gaussian belief propagation (GaBP), which is computationally efficient in message passing. Moreover, a Huber factor is designed based on the Huber loss function to implement the robust estimation by down-weighting the abnormal measurements in Gaussian message passing. Experimental results show that the proposed methods can estimate CSL and CRL states simultaneously, whose accuracy outperforms the traditional methods. Meanwhile, Huber factor can handle the abnormal measurements effectively to guarantee the system′s robustness. The proposed RC-SSRL provides a new way for the fusion framework of swarm coop`erative localization.