Our knowledge of the role of the microbiome in fish health has been steadily increasing in the last decade, especially for species of commercial interest. Conversely, relatively few studies focus on the microbiome of wild fishes, especially apex predators like sharks, linked to lower economic interest and to greater difficulty in obtaining samples. Studies investigating microbiome differences between diverse anatomical locations of sharks are limited, and the majority of available studies is focused on the microbial diversity present on the shark teeth, with the aim of preventing infections due to the bite of these animals or evaluating the presence of certain pathogens in healthy or diseased specimens. We investigated the skin, mouth, gills, and cloaca microbiome of five individuals of two phylogenetically distant species of sharks (Prionace glauca, Somniosus rostratus) to get a better understanding of the diversity of the microbiome of these animals, how it changes throughout different body parts and how much it is influenced and determined by the ecology and evolutionary relationship host-microbiome. To confirm the taxonomy of the sharks under study, we barcoded the specimens by sequencing the mtDNA COI from a biopsy of their skin. Microbial diversity based on the 16S rRNA gene reveals that partially overlapping microbiomes inhabit different body parts of each shark species, while the communities are distinct between the two species. Our results reveal that sharks’ microbiome species-specific differences are controlled by the ecology of the shark species. This is the first study comparatively analyzing the microbiome diversity of different anatomical locations in two shark species of the Mediterranean Sea.