The ecosystems of the Red Sea are among the least-explored microbial habitats in the marine environment. In this study, we investigated the microbial communities in the water columns overlying the Atlantis II Deep and Discovery Deep in the Red Sea. Taxonomic classification of pyrosequencing reads of the 16S rRNA gene amplicons showed vertical stratification of microbial diversity from the surface water to 1500 m below the surface. Significant differences in both bacterial and archaeal diversity were observed in the upper (20 m and 50 m) and deeper layers (200 m and 1500 m). There were no obvious differences in community structure at the same depth for the two sampling stations, which are 7.4 km apart. The bacterial community in the upper layer was dominated by Cyanobacteria whereas the deeper layer harbored a large proportion of Proteobacteria. Among Archaea, Euryarchaeota, especially Halobacteriales, were dominant in the upper layer but diminished drastically in the deeper layer where Desulfurococcales belonging to Crenarchaeota became the dominant group. Our results indicate that the microbial communities sampled in this study are different from those identified in water columns in other parts of the world. The depth-wise compositional variation in the microbial communities is attributable to their adaptations to the various environments in the Red Sea.