Scleractinian corals’ microbial symbionts influence host health, yet how these coral microbiomes assembled over evolution is not well understood. We survey bacterial and archaeal communities in phylogenetically diverse Australian corals representing more than 425 million years of diversification. We show that corals exhibit anatomical compartmentalization of the microbiome such that the coral surface mucus layer, tissue, and skeleton microbiomes show distinct modern microbial ecology and evolutionary assembly. In corals, these compartments differ greatly in microbial community composition, richness, and response to host vs. environmental drivers. We also find evidence of coral-microbe phylosymbiosis, in which coral microbiome composition and richness reflects coral phylogeny. Surprisingly, the coral skeleton represents the most biodiverse coral microbiome, and also shows the strongest evidence of phylosymbiosis. Together these results trace microbial symbiosis across anatomy during the evolution of a basal animal lineage.