The positioning of chromosomes in the nucleus of a eukaryotic cell is highly organized and has a complex and dynamic relationship with gene expression. In the human malaria parasite Plasmodium falciparum, the clustering of a family of virulence genes correlates with their coordinated silencing and has a strong influence on the overall organization of the genome. To identify conserved and species-specific principles of genome organization, we performed Hi-C experiments and generated 3D genome models for five Plasmodium species and two related apicomplexan parasites. Except for P. knowlesi, all Plasmodium species showed clustering of centromeres, telomeres and virulence genes. In P. falciparum, the heterochromatic virulence gene cluster had a strong repressive effect on the surrounding nuclear space, while this was less pronounced in P. vivax and P. berghei, and absent in P. yoelii. In P. knowlesi, telomeres and virulence genes were more dispersed throughout the nucleus, but its 3D genome showed a strong correlation with gene expression. The Babesia microti genome showed a classical Rabl organization with colocalization of subtelomeric virulence genes, while the Toxoplasma gondii genome was dominated by strong clustering of the centromeres and lacked virulence gene clustering. Collectively, our results demonstrate that spatial genome organization in most Plasmodium species is constrained by the colocalization of virulence genes. P. falciparum and P. knowlesi, the only two Plasmodium species with gene families involved in antigenic variation, are unique in the effect of these genes on chromosome folding, indicating a potential link between genome organization and gene expression in more virulent pathogens.